Manifold Traversal for Reversing the Sentiment of Text

Abstract

Natural language processing (NLP) is a heavily researched field within machine learning, connecting linguistics to computer science and artificial intelligence. One particular problem in NLP is sentiment classification, e.g determining if a sentence holds a positive or negative opinion. There exist many established methods for solving the sentiment classification problem but none for modifying a negatively classified input so that it receives a positive classification. In this paper we propose a method for reversing the sentiment of sentences through manifold traversal. The method utilizes a convolutional neural network (CNN) and pre-trained word vectors for encoding sentences in a continuous space. The sentence representations are traversed through optimization of a test statistic as to resemble the representations of sentences with the opposite sentiment. Finally a recurrent neural network (RNN) is used for decoding the vector representation and generating new sentences. The encoder in our model achieves 80% accuracy on the sentiment classification task and produces sentence representations in 300 dimensions. Visualizations of these representations, using PCA, shows clustering with respect to both sentiment and different topics, indicating that the representations hold information about both sentiment and textual content. Decoding the traversed feature vectors using our RNN language model produces, in most cases, understandable sentences where the sentiment has changed compared to the original sentence.