The Effects of AI Assisted Programming in Software Engineering

Abstract

The recent emergence of artificial intelligence (AI) learning algorithms has brought generative AI (GAI) tools to the market. In software engineering, an example of such a tool is GitHub Copilot (Copilot), which can generate code suggestions in real-time and through natural language input.

In contrast to contemporary studies, this report attempts to fill a knowledge gap by employing a qualitative study, gaining insights into professional software engineers’ opinions regarding GAI use in natural settings. While Copilot was the primary reference point, the study acknowledges the emergence of other GAI such as ChatGPT which also fit within the scope of the thesis.

The study was initially designed to let engineers use Copilot in their work for two weeks, followed by a semi-structured interview. However, hesitance from approached companies to use Copilot in their code due to legal and privacy concerns led to an alternative study design being used in tandem. Retaining the interview format and questions, participants were instead shown a demo showcasing Copilot’s features. In total, 13 professionals participated in the study.

Through thematic analysis, findings revealed that utilizing Copilot can increase efficiency through auto-completion specifically. A lack of conversational capabilities and disruptive elements of Copilot lead to hindrances in development and code analysis. Furthermore, GAI tools allow engineers to focus on higher-level problems and offer inspiration, enhancing end-product creativity. Engineers also emphasized the retention of base knowledge to criticize GAI output. Finally, widespread GAI integration can lower the profession’s entry barrier, and developer roles can shift to take advantage of the enhancements the tools provide. It is still evident that there are currently many concerns with the technology for trusted integration. Therefore, efforts should be made to address these issues, which in turn can make studies in natural settings more viable.