An Experiment on the Effectiveness of Unit Testing when Introducing Gamification

Abstract

Software testing has been associated with both technical and socio-technical difficulties. While software systems grow larger and are becoming more complex, there is simultaneously a problem with lack of motivation and interest in the software testing process. The purpose of this study is to research how the concept of Gamification, which has shown positive effects in varying areas of application, can be used to increase the effectiveness of unit testing, as well as increasing the motivation and interest among the software testers. The focus of the study was put on the number of defects found, as well as the percentage of requirements covered by tests when measuring effectiveness. The effect of Gamification in unit testing was researched through an experiment involving 24 subjects. The subjects were sampled into two groups, one group with Gamification applied to their testing process, and one group without Gamification applied to their testing process. The subjects conducted, during a one hour testing session, unit testing on a selected system, which had been seeded with a specific number of faults. The results show that the Gamification group found significantly more defects and received a significantly higher percentage of requirements covered by tests, than the Control group. Moreover, the results show that the Gamification group found the task significantly more interesting than the Control group. We conclude that, even if the significantly higher number of found defects and requirements covered by tests are hard to correlate to specific Gamification elements; Gamification might be used to improve the stated variables of effectiveness in software testing.