Sign Language Learning & Translation with User Interaction

Abstract

The barriers to communication between Deaf or Hard-of-Hearing people and the general public continue to be a serious social concern, often due to a lack of use and understanding of sign language. While various studies have focused on static hand gesture detection, many current systems lack real-time responsiveness, interactive learning.

This project proposes a real-time recognition system for American Sign Language (ASL) that allows users to upload hand gesture images or carry out live gestures via a webcam. The system predicts individual letters using a Convolutional Neural Network (CNN) and a Bidirectional Long Short-Term Memory (BiLSTM) model, which allows for both spatial and temporal information extraction, enhancing accuracy and understanding.

Users can interactively form words from the predicted letters, supporting both communication and self-paced learning. The system also has a simple, user-friendly interface that encourages interaction and accessibility for non-signers. This technique not only improves ASL learning but also helps to close the communication gap in educational and social settings.