Personalized POI Recommender System: Combining Image Processing and NLP in Recommender Systems

Abstract

Due to the recent increase in check-ins at Location Based Social Networks, more data has been stored in these systems. This data was realized to be potentially useful in recommendation and so began the process of PoI recommender systems. The initial systems that were developed started adhering to four main influential factors (i.e. social, geographical, contextual and temporal) that affect PoI recommendation. These factors were then combined with modern recommendation techniques like collaborative filtering and content based filtering. However, recent researches have noted that the introduction of images in social media have also become a potential mine for user interests. Whilst processing images is a good attempt at figuring out interests of users, without textual context it could be meaningless. The culmination of the research process has led to the formulation of a gap. The proposed system in this project will address the gap by combining the use of textual context along with image processing to accurately provide personalized recommendations. The proposed system will take social media as a basis for user interests by gathering user likes on social media. The content of these liked posts will be split into images and text. The text will go through text processing like sentiment analysis and POS tagging to find out what the text means and what the user’s opinion about it is. The images will be processed to find out what the image represents. These results are combined to get better accuracy and reliability in recommendations by providing a meaningful context to the interest. These interests are then fed into a ranking mechanism that is used to finalize the interests that are of most importance to the user. These interests will be used to categorize the places that the system will recommend to the user.