Prompt Engineering with Requirements in GitHub Copilot

Abstract

Practices within requirements engineering are essential for the success of software projects. In particular, document analysis enables the elicitation of requirements without engaging directly with the stakeholders. This thesis presents a method of scraping documents from English open-source software repositories hosted on GitHub and extracting requirements from these documents. This is done in order to supply GitHub Copilot with more context when prompting it to repay self-admitted technical debt expressed through TODO comments. Three different prompt tem plates were created to evaluate this approach. The first template had no require ments in the prompt, the second template had some relevant requirements, and the third one had all the relevant requirements. After prompting Copilot, prompts from the first template were shown to the best at repaying debt– amounting to a repayment rate of 72%. However, they also accounted for the most repayments that did not conform to all the requirements. While prompts from the third template showed the best results in regard to this, they displayed a lower debt repayment rate of 64%. Finally, it is noted that self-admitted debt is not truly repaid unless done in a way that conforms to the relevant requirements. As such, there is a need for future research to expand upon this particular problem through the development of a robust framework.