What is SKEP?
SKEP is a self-supervised pre-training method designed for sentiment analysis. It uses automatically-mined knowledge to embed sentiment information into pre-trained sentiment representation. The method constructs three sentiment knowledge prediction objectives that enable sentiment information to be embedded at the word, polarity, and aspect level. Specifically, it predicts aspect-sentiment pairs using multi-label classification to capture the dependency between words in a pair.
How Does SKEP Work?
SKEP has two parts: Sentiment masking and Sentiment pre-training objectives. First, sentiment masking recognizes the sentiment information of an input sequence and produces a corrupted version by removing these sentiment-related details. Then, Sentiment pre-training objectives require the transformer to recover the removed information from the corrupted version. The three prediction objectives on top are jointly optimized: Sentiment Word (SW) prediction, Word Polarity (SP) prediction, and Aspect-Sentiment pairs (AP) prediction. Only the polarity is calculated on the original word, which is predicted in the pair prediction.
Why is SKEP Important for Sentiment Analysis?
SKEP plays a vital role in sentiment analysis as it is a self-supervised learning method that can learn efficiently from unannotated data. This approach significantly reduces the data and the effort required for supervised training, which is a significant benefit for institutions and researchers alike. Moreover, the use of Sentiment masking and Sentiment pre-training objectives used in SKEP help to provide a detailed representation of the sentiment information in a sentence up to the word, polarity, and aspect level, thus providing a holistic understanding of the sentence's sentiment.
