StarryStudioAI: Automating UI Design with Code-Based Generative AI and RAG
Publication Date
1-1-2025
Document Type
Conference Proceeding
Publication Title
2025 IEEE 15th Annual Computing and Communication Workshop and Conference Ccwc 2025
DOI
10.1109/CCWC62904.2025.10903712
First Page
937
Last Page
944
Abstract
Designing interactive and visually appealing web interfaces often requires extensive collaboration between designers and developers, making the process timeintensive and challenging. To address these limitations, the paper introduces StarryStudioAI, an innovative Large Language Model (LLM) powered plugin for Figma that translates natural language prompts into high-quality design outputs. Utilizing fine-tuned models, such as GPT-4o, Llama 3.2, and Gemini 1 Pro, enhanced with Retrieval-Augmented Generation (RAG), StarryStudioAI automates the generation of Hypertext Markup Language (HTML) and Cascading Style Sheets (CSS) code. It then seamlessly converts them into pixel-perfect layouts compatible with Figma. The system leverages a dataset of HTML/CSS samples derived from publicly available Figma-compatible templates and other open sources to ensure precision and reusability in generated code. Experimental results underscore StarryStudioAI's ability to achieve a CHaracterlevel F-score(ChrF) of 0.73, a BERTScore F1 of 0.79, and a Pass@1 rate of 0.33. To complement these metrics, a human evaluation involving 50 participants from technical and nontechnical backgrounds demonstrated high user satisfaction, with acceptability rate above 0.90 for component level designs (e.g., button) and satisfaction rate above 0.80 for layout consistency across UI components. These results validate the semantic and structural robustness of the generated designs while requiring minimal manual intervention. By streamlining design-to-development workflows, StarryStudioAI bridges the gap between textual specifications and visual outputs, enhancing productivity in web development through automated template generation and accelerated design-developer handoffs.
Keywords
Code-Based Design Generation, Generative AI, Large Language Models, Retrieval-Augmented Generation, Web Development Automation
Department
Applied Data Science
Recommended Citation
Bhavik Patel, Poojan Gagrani, Smeet Sheth, Kashish Thakur, Priya Varahan, and Simon Shim. "StarryStudioAI: Automating UI Design with Code-Based Generative AI and RAG" 2025 IEEE 15th Annual Computing and Communication Workshop and Conference Ccwc 2025 (2025): 937-944. https://doi.org/10.1109/CCWC62904.2025.10903712
