Recent Progress in Automated Code Generation from GUI Images Using Machine Learning Techniques
Daniel de Souza Baulé (Federal University of Santa Catarina, Brazil)
Christiane Gresse von Wangenheim (Federal University of Santa Catarina, Brazil)
Aldo von Wangenheim (Federal University of Santa Catarina, Brazil)
Jean C. R. Hauck (Federal University of Santa Catarina, Brazil)
Abstract: The manual transformation of a user interface design into code is a costly and time-consuming process. A solution can be the automation of the generation of code based on sketches or GUI design images. Recently, Machine Learning approaches have shown promising results in detecting GUI elements for such automation. Thus, to provide an overview of existing approaches, we performed a systematic mapping study. As a result, we identified and compared 20 approaches, that demonstrate good performance results being considered useful. These results can be used by researchers and practitioners in order to improve the efficiency of the GUI design process as well as continue to evolve and improve approaches for its support.
Keywords: machine learning, systematic mapping, user interface design
Categories: H.5.2, I.2.10, I.2.6