Detection of Image-Based Spam in Social Media X Using Deep Learning

Abstract

"Social media has become an integral part in everyone’s lives, facilitating spreading of information, communication, and networking. However, one of the main challenges that all social media platforms are currently facing is the issue of spreading spam or misleading content. These malicious activities, aimed at exposing users to fraudulent, misleading, and illicit activities, pose a significant threat to the user’s trust and degrade the overall user experience. In this study, the author attempts to provide a solution to detect spam or misleading content available in images posted in the popular social media platform, X (formerly known as Twitter). Unlike other platforms that are popular for personal data sharing, X is recognized for its role in spreading information, news, and more formal updates. This research explores two distinct approaches to solve this issue: fine-tuning a VGG16 Convolutional Neural Network model and fine-tuning the BERT transformer model on text extracted from images using OCR. This research contributed towards bridging the existing gap of detecting spam or misleading images, thereby enhancing the integrity and reliability of social media. According to the tests carried out on both models, the fine-tuned VGG16 model achieved an accuracy of 76% while the fine-tuned BERT model achieved an accuracy of 93%. This suggests that the approach of extracting the text present in the image, classifying the text and determining whether the particular image is spam or legitimate is more promising. By using this methodology, it is possible to identify and mitigate the spam and misleading content in social media. "