Analyzing the sentiment of social media images in hospitality domain using object identification on caption generation

Abstract

"With the advancement of hardware and the stepping up of social media worldwide, the number of images and videos shared on these platforms has increased massively. Therefore, organizations that are interested in their brand image would invest in brand monitoring and reputation management through means of social listening services. Sentiment analysis takes a key role in the social listening domain as a metric to measure user opinion on the brand, product, or service provided by the organization. With the increased pace in publishing multimedia content on the world wide web, it is important to position an automated mechanism to monitor sentiment on such content. The research was conducted based on the scope of social media and by utilizing social listening services, the image data were extracted. The methodology introduced in the research involves a convolutional neural network to detect objects in an image, a long, short-term memory model for generating a caption for the image, and ultimately use text sentiment analysis tools to determine the image sentiment. The model training process involves a large image dataset with five captions written for each image to build a vocabulary that is used to generate the captions based on identified objects."