Sustainable Energy and Artificial Intelligence

Large Language Models as an Assistant to Interpret UML Models in Model-Based Engineering: An Exploratory Study

Document Type : Original Article

Authors

Hassan Bashiri 1
Alireza Khalilipour 2
Parsa Bakhtiari 1
Moharram Challenger 2

1 Department of Computer Engineering, Hamedan University of Technology, Hamedan, Iran

2 Department of Computer Science, University of Antwerp, and Flanders Make Strategic Research Center, Antwerp, Belgium

https://doi.org/10.61186/seai.2409-1009
Abstract
Creating a formal common language, beyond the ambiguities of natural languages, between different stakeholders, from analysts to test engineers, is one of the key goals of software modeling. Although notations are standard in software modeling languages such as UML, junior engineers’ interpretation of models varies. Model interpretation in the presence of experienced people increases the learning rate for junior engineers. One of the potentials of large language models is the ability to interpret images and models. This research aims to use large language models as an assistant to interpret UML models to increase junior engineers’ learning rate and understanding of the software models. We conducted an evaluation study to examine how helpful an LLM can be to help interpret the software models. Although large language models are still not very accurate in interpreting UML models, the experiment’s results showed that students’ learning rates increased by LLMs as model interpretation assistants. In other words, the large language model worked well as a teaching assistant. The detailed results of this exploratory study are reported in this paper.

Highlights

  • Explores the potential of LLMs, such as ChatGPT and Gemini, to enhance the interpretation of UML diagrams for training junior engineers.
  • Demonstrates LLMs' ability to act as effective teaching assistants by uncovering hidden details in complex engineering models.
  • Observed a significant improvement in students' understanding and explanation of UML diagrams after using LLMs.
  • LLMs not only clarify complex aspects of UML diagrams but also inspire broader, critical thinking among learners.
  • Plans to fine-tune LLMs for specialized applications in modeling and software engineering to further enhance their educational utility.

Keywords

Large Language Model
Model-Based Engineering
Software Modeling Language
UML
Model Interpretation Assistant
Subjects
  1. Brambilla, M., Cabot, J., & Wimmer, M. (2017). Model-driven software engineering in practice. Morgan & Claypool Publishers.
  2. Broy, M., Kirstan, S., Krcmar, H., & Schätz, B. (2012). What is the benefit of a model-based design of embedded software systems in the car industry?. In Emerging technologies for the evolution and maintenance of software models (pp. 343–369). IGI Global.
  3. Bruneliere, H., Eramo, R., Gomez, A., Besnard, V., Bruel, J. M., Gogolla, M., ... & Rutle, A. (2018). Model-Driven Engineering for Design-Runtime Interaction in Complex Systems: Scientific Challenges and Roadmap: Report on the MDE@ DeRun 2018 Workshop. In Software Technologies: Applications and Foundations: STAF 2018 Collocated Workshops, Toulouse, France, June 25-29, 2018, Revised Selected Papers(pp. 536-543). Springer International Publishing.
  4. Di Rocco, J., Di Ruscio, D., Iovino, L., Laemmel, R., & Pierantonio, A. (2017). MDE adoption—a three-legged chair. In Proceedings of the Workshop on Grand Challenges in Modeling at STAF.
  5. Wei, C., Wang, Y. C., Wang, B., & Kuo, C. C. J. (2023). An overview on language models: Recent developments and outlook. arXiv preprint arXiv:2303.05759.
  6. Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., ... & Mian, A. (2023). A comprehensive overview of large language models. arXiv preprint arXiv:2307.06435.
  7. Dai, W., Lin, J., Jin, H., Li, T., Tsai, Y. S., Gašević, D., & Chen, G. (2023, July). Can large language models provide feedback to students? A case study on ChatGPT. In 2023 IEEE International Conference on Advanced Learning Technologies (ICALT)(pp. 323-325). IEEE.
  8. Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., ... & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences, 103, 102274.
  9. Rane, N. (2023). Enhancing the quality of teaching and learning through ChatGPT and similar large language models: challenges. Future prospects, and ethical considerations in education (September 15, 2023).
  10. Young, J. C., & Shishido, M. (2023). Investigating OpenAI’s ChatGPT potentials in generating Chatbot's dialogue for English as a foreign language learning. International journal of advanced computer science and applications, 14(6).
  11. Conrardy, A., & Cabot, J. (2024). From Image to UML: First Results of Image Based UML Diagram Generation Using LLMs. arXiv preprint arXiv:2404.11376.
  12. Bozyigit, F., Bardakci, T., Khalilipour, A., Challenger, M., Ramackers, G., Babur, Ö., & Chaudron, M. R. (2024). Generating domain models from natural language text using NLP: a benchmark dataset and experimental comparison of tools. Software and Systems Modeling, 1-19.
  13. López, J. A. H., Cánovas Izquierdo, J. L., & Cuadrado, J. S. (2022). ModelSet: a dataset for machine learning in model-driven engineering. Software and Systems Modeling, 1-20.
  14. Rumbaugh, J., Jacobson, I., and Booch, G. The unified modeling language reference manual (2nd Edition). Pearson Education India, 2004.
Sustainable Energy and Artificial Intelligence
Volume 1, Issue 1
Winter 2025
Pages 45-50

  • Receive Date 25 September 2024
  • Revise Date 01 November 2024
  • Accept Date 17 November 2024
  • First Publish Date 17 November 2024

Home

Guide for Authors

Submit Manuscript

Reviewers

Contact Us