Nail Shape and Color Recommendations Through Hand Shape Detection and Skin Color Extraction with Machine Learning

Abstract

Problem: The fashion industry currently lacks the ability to provide personalized fashion tips based on individual body characteristics and skin tones. Nail decoration has become increasingly popular, but individuals often struggle to identify nail shapes and polish colors that suit their hand shape and skin tone. This project aims to address this gap by analyzing hand features and skin tone to generate personalized nail decoration recommendations. Methodology: Machine learning techniques are used to detect hand key points, determine hand shape, and identify skin tone. The system analyzes fifteen key points on the hand to calculate measurements like palm width, height, and finger lengths, which are then used to classify the hand shape. Skin tone is determined by analyzing a specific region of the hand based on key points. Recommendations for nail shape and polish color are generated based on this analysis, supplemented by user input related to preferences or special occasions. Initial Result: Currently, the accuracy of the models is also different. The hand verification model achieved 86% accuracy, while left and right finger keypoint models performed well at 94%. The right palm keypoint model reached 83%, but the left palm keypoint model struggled with just 4% accuracy. Hand shape recognition reached 90%, skin color recognition reached 95%. Although most models obtain a good performance, the left palm keypoint model suffers from a lower performance. Accuracy is anticipated to improve as training proceeds, dataset size grows, and modifications are made. In future work, attention will be paid to enhancing, among other things, key point detection, robustness to edge cases and the usefulness in generalizing, allowing the system to become increasingly accurate and robust.