Object detection and notification for visually challenged people

Abstract

There are many manual and technological solutions that exist to support visually impaired people. But they have many drawbacks such as lack of information to user, unable to identity the obstacle correctly, Cost, not flexible to carry and some are not specifically design walking purposes on the road. This project is aimed to help visually challenge people with the help of deep learning and Image recognition technologies with voice notification about the obstacles in front of the person and the distance to it from the origin. It detects the object in front of the person and, based on the dimensions of the object, severity will be categorized. For example, if small bricks are found it will categories this object as “Low Risk” obstacle or minor impacted object and if a wall found it will categories as “High Risk” object. In this research, developed a solution which can be used by visually impaired people with their mobile. Mobile captures images every 5 seconds and give voice notification to the user. Based on notification he can decide whether he must move front or change the foot path. In the research, a Custom object detection model has been developed for the objects that are not captured by standard model and which was able to achieve overall 55% accuracy. The object classification model has been trained with 83% model accuracy and 62.5% accuracy with the prototype. Voice notification has been developed Azure Cognitive Service called Speech Service. Finally developed prototype has been evaluated by a user as well as by a subject matter expert. Users have ranked the overall solution with 4 / 5 and technical expert with 3 / 5 rate.