A Real-time Two-way Sinhala Sign Language Interpretation System Using Deep Learning

Abstract

"In Sri Lanka, the main source of communication between hearing/talking impaired people and natural language-speaking people is the Sri Lankan sign language (SSL). Sri Lankan sign being a low-resource language and the lack of qualified sign language interpreters in the country have become a major issue in this area. The aim of this project is to lower the above-mentioned communication barrier. This research provides an overview of minimizing the language barrier between the hearing/impaired community and the natural language-speaking community in Sri Lanka by using machine learning and deep learning technologies.

To develop the targeted Real-Time Two-Way Sinhala Sign Language Interpretation System, Agile development methodology was used. As for the implementation of the model, the Convolutional Neural Network (CNN) architecture was utilized. The results of the developed solution demonstrated that the detected signs are relevant to the recognized sign gestures. Considering how inadequately such systems are developed in Sri Lanka, the finding of the study would serve the needs of the hearing/talking impaired community in the country and would act as a basement for the development of more advanced models in the current research area. However, since SSL is a low-resource language, the resources are very limited. There are only a few datasets available, and those datasets are also not up to the standards. Thus, the current study highlights the recognition and detection of more complex sign gestures can be done by utilizing more precise datasets and adding more elements to the system."