Abstract:
"User Interface (UI) and User Experience (UX) design plays a major role in the development of
any application, however the technology used to design have always remained unchanged,
stuck on the traditional devices. While those devices may work well, designers may not realize their true capabilities with more advanced technology. Additionally, the traditional devices confine the user to designing using a single hand.
Proposed is a novel UI design tool that applies real-time hand gesture recognition (HGR) to
design interfaces. HGR is a popular Computer Vision (CV) field of knowledge that has been
applied in many industries. The application of real-time HGR in UI/UX design would cause an
immense advancement in efficiency. Moreover, the proposed HGR system aims capture the
gesture via a built-in laptop camera or webcam and utilize an innovative hybrid classification
technique, CNN-Transformer, that has not been used in an HGR system prior to this. The
research aspires to prove the effectiveness of using HGR to design interfaces in order to bridge the identified gap"
