ThreatSense: Real-Time Abnormal Situation Detection through Handheld Weapon Identification and Action Recognition using Deep Learning Techniques

Abstract

"Contemporary surveillance systems exhibit fundamental limitations in threat detection, relying predominantly on isolated weapon identification or behavioral analysis rather than integrated contextual assessment. Current automated approaches achieve merely 70-85% accuracy with substantial false positive rates, primarily due to the absence of spatial-temporal contextual intelligence and human-weapon proximity validation. This research addresses the critical accuracy gap in existing surveillance technology through the development of a novel Spatiotemporal Contextual Threat Inference (SCTI) algorithm. An innovative deep learning framework integrating YOLOv5-based weapon detection with 3D Convolutional Neural Networks for spatiotemporal action recognition was developed. The core methodological innovation lies in the SCTI algorithm, which employs human-weapon spatial relationship validation coupled with temporal behavioral consistency analysis across 3-5 second windows. The framework implements hybrid fusion strategies, combining early-stage feature integration with late-stage decision fusion. Enhanced datasets incorporating UCF101, HMDB51, and custom surveillance recordings underwent rigorous augmentation and transfer learning protocols to optimize model robustness. The SCTI-enhanced prototype achieves 88.4% overall accuracy with early fusion methodology, representing significant advancement over baseline approaches. Weapon detection attains 87% F1-score through YOLOv5 optimization, while spatiotemporal action recognition demonstrates 91% accuracy for critical suspicious behaviors. The system maintains real-time processing at 25 frames per second with 39 millisecond latency using conventional GPU hardware. User validation confirms 84% contextual relevance of generated alerts, indicating a 60-70% reduction in false positives compared to conventional systems Subject Descriptors: • Computing methodologies → Computer vision → Object detection • Computing methodologies → Machine learning → Machine learning approaches → Neural networks • Applied computing → Law, social and behavioral sciences → Surveillance systems "