Assistive translator for Aurally - Impaired and Mute

Abstract

There is a communication gap between aurally-impaired and mute people and normal human. Aurally impaired and mute people around the world communicate using sign language as a distinct form of spoken language in their everyday lives. This thesis studies the new possibilities to assistive sign language translation for aurally and speaking impaired. Gesture recognition has done using Leap Motion sensor. The Leap Motion is an innovative, 3D motion capturing device designed especially for hand and finger tracking with precision up to 0.01mm. The outcome of this thesis is the Assistive Translator system dedicated to the aurally and speaking impaired people for Leap Motion Controller that contains algorithms allowing to learn and recognize gestures. The author examined the data provided by the sensor in the context of recognition of hand poses (static gesture), hand movements (dynamic gestures) and in the task of a finger recognition. The Static and Dynamic gestures are recognized using Template Matching algorithms. The thesis contains the evaluation of different feature sets, which have a significant impact on the recognition rate. The chosen feature with set allowed recognizing a set of gestures with 99% accuracy and a set of gestures with 86%.