ChatGPT as support for architecture and requirements engineering

Welcome to our multi-part series on the use of modern AI tools in the world of software development! There is now evidence-based research on how developers can optimize experiences with the tools at all stages of the software development cycle. Whether developing new features or fixing bugs. In the coming weeks, we will show you exactly how this works in this series. The first part of the series can be found here.

LLMs in Software Engineering

The ongoing development of large language models (LLMs) such as ChatGPT and GitHub Copilot is having a lasting impact on software engineering. These models are not only capable of summarizing texts and composing emails, but are also able to generate source code.

At Accso, we have tested these technologies in various customer projects and present an analysis of the potential for the use of LLM-based tools in software development in this blog series. By using ChatGPT, we were not only able to increase the efficiency of software development, but also achieve concrete improvements in the areas of architecture and requirements. In this article, we share our best practices.

We also took a close look at GitHub Copilot. However, as this tool was only used for implementation tasks, it will be covered in a later blog post.

ChatGPT in Requirements Engineering

In the area of requirements engineering, we tested various aspects in a pilot study, with the following three proving useful:

1. The first involves the creation of personas, where the AI was able to assist in the development of sentence components.
   
2. The second was the understanding of technical requirements, using the chatbot as a knowledge resource.
   
3. Understanding implications and dependencies between requirements was also tested, with similarly good results.
   

The chatbot as a knowledge resource supports effective and efficient working methods throughout software engineering.

Chatbots as a Knowledge Resource for Architecture and Design

In the area of architecture and design, we looked at understanding operational documentation, architectural patterns and styles, and architectural documentation. The chatbot as a knowledge resource makes it possible to obtain knowledge more quickly and to understand and deepen new patterns. For project planning and architecture prior to the tasks of a software engineer, the application possibilities are currently more limited than for coding. Nevertheless, ChatGPT provides support here and leads to faster and better solutions.

Conclusion

In our qualitative study, we were unable to identify any patterns in which LLMs provide a particularly high level of support, as all application scenarios were very individualized. Nevertheless, the existing LLMs offer support with requirements and architecture, even if these topics are complex and cannot lead to a good solution with a short prompt by the LLM. Nevertheless, we were able to see the following: For the areas of requirements engineering and architecture, we can see that an "interactive mode" with the chatbot, where questions are asked and answered, is an efficient way to use an AI in customer projects. This approach helps to stay focused on solving the actual problem. Concrete use cases in which an LLM can be used sensibly are the development of component sections, the processing of trade-offs between different solutions and the creation of interface schemas in JSON format.

Limitations of the survey

Finally, we would like to make a few brief comments on the framework of our study. The best practices presented here are based on a pilot study in which we gathered knowledge and iteratively improved our skills in order to gain sound experience from our projects. Not all topics were tested and examined in the open pilot study. The study is based on the subjective experience of the test subjects, which could be distorted by a bias in self-assessment. The protection of secrecy also made it difficult to use chatbots, which is why we have only been able to successfully use a small number of the above-mentioned tasks in customer projects with ChatGPT.

In the next blog post, we will then go into many more specific applications when writing code. We will also be able to share our experiences from customer projects there.