Image-Text Sentiment Analysis Model Based on Visual Aspect Attention

Abstract

Social network has become an integral part of our daily life. Sentiment analysis of social media information is helpful to understand people's views, attitudes and emotions on social networking sites. Traditional sentiment analysis mainly relies on text. With the rise of smart phones, information on the network is gradually diversified, including not only text, but also images. It is found that, in many cases, images can enhance the text rather than express emotions independently. We propose a novel image text sentiment analysis model (LSTM-VAA). Specifically, this model does not take the picture information as the direct input, but uses the VGG16 network to extract the image features, and then generates the visual aspect attention, and gives the core sentences in the document a higher weight, and get a document representation based on the visual aspect attention. In addition, we use the LSTM network to extract the text sentiment and get the document representation based on text only. Finally, we fuse the two groups of classification results to obtain the final classification label. On the yelp restaurant reviews data set, our model achieves an accuracy of 62.08%, which is 18.92% higher than BiGRU-m VGG, which verifies the effectiveness of using visual information as aspect attention assisted text for emotion classification; It is 0.32% higher than Vista-Net model, which proves that LSTM model can effectively make up for the defect that images in Vista-Net model cannot completely cover text.