Convolution Neural Network based Sri Lankan Sign Recognition system

Abstract

Communication is well needed factor in the community in order to convey messages. Hearing impaired people as part of the community suffers having a communication barrier between ordinary people. This issue has been addressed in many situations, but in a country like Sri Lanka the problem is still exists and there are no feasible solutions have been addressed. Even though there are some proposed solution they are not feasible enough and having some gaps related to these approaches. Considering the real-world scenario and in order to address above mentioned issues and limitation, the author proposes a vison-based approach for sign gesture recognition to fulfill the communication barrier. Based on Convolutional Neural Network, it is used to enable the real-world knowledge, how a human really sees the images and identify the reality. This CNN based system is a sub category of neural networks that have proven extremely effective for tasks such as image recognition and classification. This proposed solution was able to achieve a high accuracy of 99.9% in the image classification process with an error rate of 0.1%. The proposed system is based on desktop application and using a camera it will start getting inputs as performed gestures. From the input data these frames will be further processed and going through several image processing techniques like preprocessing and segmentation etc. As the final step the classification, prediction and language translation will be performed. The result of the project will clearly affect as a factor to overcome the communication barrier as addressed in this research and provide benefits to the community. The implemented system was tested and evaluated under both qualitative and quantitative approaches. The evaluation and testing on all required and essential stages, the overall system was completed in an efficient and effective manner.