23. Feb 2024

ChatGPT and GitHub Copilot in Software Development

The first part of our series on the use of modern AI tools in the world of software development is dedicated to the two tools GitHub Copilot and ChatGPT. In which areas of the software development process can they be used successfully?
Software Engineer


Jaid Heesen

Ein Mann im grünen Pullover sitzt mit dem Rücken zum Betrachter gewandt vor einem Bildschirm, auf dem Code zu sehen ist.

Welcome to a multi-part series on the usage of modern AI tools in the world of software engineering! Whether you're developing new features or fixing bugs, there is now evidence-based research on how to optimize your experience with the tools at all stages of the software development life cycle. In the coming weeks, we will show you exactly how this works in this series.

Opportunities and possibilities of ChatGPT/GitHub Pilot in the software development process

Better Understanding of Requirements

ChatGPT and GitHub Copilot can be used in order to aid understanding of requirements for new or existing software projects. By providing the chatbot with some background to the problem, the existing knowledge of engineers can be supplemented and further expanded through brainstorming, as the chatbot can offer new insights and ensure that no aspects have been overlooked. This includes considering tradeoffs between different solutions and understanding the architecture of the system.

Generating Code

Further on in the software development process, the tools can be used once the implementation stage has begun, and code is being written. One could provide the chatbot with text-based prompts to generate sections of code, or feed code into the tool and ask it to identify bugs or make adaptations.

Help with Testing

The tools can also be used in the testing phase. If tests require a lot of repetitive input data, then using a tool such as ChatGPT could be beneficial in terms of speeding up the process, as it can even produce CSV files of test data. However, internal studies have concluded that GitHub Copilot is the more effective tool here, as it can use the existing code base for context on what to generate.

Support with Meta-Tasks

Last, but certainly not least, ChatGPT and Github Copilot can be used for "meta"-programming tasks - things such as learning new programming languages, or figuring out the intricacies of setting up one's own dev environment, whether it be IDE's or virtual machines.

AI tools in software development: Limitations of ChatGPT and Github Copilot

There are two major things to be aware of when making use of the in this article discussed tools ChatGPT and Github Copilot:

  1. These tools are not infallible and there is potential for them to provide incorrect or misleading information.
  2. There can be data privacy concerns regarding the usage of company data as part of the input to these tools.

With these caveats in mind these tools have been proven to help software engineers. Other exciting research, will be explored in more depth in future blog posts.

AI-assisted Software Engineering

This blog article is based on a wide-ranging potential analysis that evaluates the use of LLM-based tools in software engineering today. The identified potentials were tested in real project scenarios by means of an empirical pilot study.


Jaid Heesen

Software Engineer
Your contact for AI-assisted software engineering with ChatGPT and GitHub Copilot
Jaid Heesen Raute