CoGen: Creation of reusable UI components for building the foundation of user interface designs

Abstract

"Nowadays multiple tools have been created in the field of User Interface (UI) design. The problem however lies in the functionality of these designs where the granular UI components aren’t considered. This results in issues related to consistency and reusability of UI components throughout a design and the creation of an efficient design system utilising the atomic design concept. To address the process of automating UI designs, authors of existing systems have used a varied number of techniques. These techniques focus on the image generation part of whole UIs rather than the foundation and editable components of the UI design or the code generation for front-end developers. However, CoGen focuses on reusability and consistency, and has proposed an approach to develop a system by having two stages: description or prompt generation and JSON generation. The description generation part uses an ensemble approach with three models (LSTM and GRU based Seq2Seq models, fine-tuned T5 model), while the JSON generation part was executed using a comparison between a simple T5 model and a complex T5 with cross mechanism and BERT embeddings. As per the results of the fine-tuned T5 model was selected to generate the descriptions or prompts as its metrics proved better. The accuracy of identifying the component name was 98.0% and the BLEU score was 0.2668. For the JSON generation the simple fined tuned T5 model was chosen over the complex one with BERT as its’ BLEU score was 0.6071 while having high success rates for each of the prompt test cases. "