Generative scenario-based testing on a real-world system

Abstract

Testing is a vital part of the software development process and occupies nearly half of the time used for developing a product. Automating the testing process could result in less developing time required for testing; however, test automation brings many challenges. This thesis explores generative scenario-based testing, in a real-world system, using a QuickCheck implementation. We implement two test-suites adopting different variants of the method, and discuss advantages and disadvantages of each method, design choices we made, limitations of the QuickCheck tool we used, and obstacles we encountered. The implementation produced satisfying results and managed to reveal 11 bugs of varying severity. From this work we conclude that generative scenario-based testing is especially suitable for testing interfaces and random user scenarios. Moreover, we found that it could be beneficial to adopt the method already in an early phase of development, and that the method could potentially constitute a great complement to an already existing test-suite.