Search-based Test Generation Framework for Android Apps with Support for Multiobjective Generation: STGFA-SMOG
Typ
Examensarbete för masterexamen
Program
Publicerad
2022
Författare
Gebrekrstos, Selam Welu
Gereziher, Teklit Berihu
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Search-based test generation is an automated test generation technique that saves
time and energy which developers spend for a manual test generation. In searchbased
test generation, the test input creation is framed as a search problem, in
which search algorithms are used to automatically select the optimal solution based
on feedback from one or more fitness functions - scoring functions based on attainment
of selected test criteria, such as code coverage or number of crashes detected.
The effectiveness of test input generation depends on the selection of measurable test
goals and the selection of effective fitness functions. Search-based test generation
with support for multi-objective test generation has been applied in other domains
such as desktop software. However, automated test case generation for Android apps
is not a mature field, and existing approaches do not allow testing to be adapted
to specific goals, like identifying excessive CPU usage or maximizing code coverage.
In addition, multi-objective search-based test generation can target multiple fitness
functions at once, shaping a test suite that has multiple properties of interest (for
example, you could both maximize CPU usage and the number of crashes, and you
could shorten test case length, all during one test generation session). However,
we do not know which combinations of fitness functions are effective at triggering
crashes or meeting other testing goals during Android GUI-based testing.
In this study, we developed a search-based test generation framework for Android
GUI-based testing with support for multiple fitness functions, largely based on nonfunctional
properties such as maximizing CPU or battery usage. We compared
STGFA-SMOG with the Random test generation tool we developed as a baseline in
the intermediate and final evaluations to determine whether STGFA-SMOG generates
tests more adept at maximizing the properties of interest, and whether it generates
tests more effective at triggering crashes. We also assess which configurations
of STGFA-SMOG (fitness function combinations) trigger the most crashes, and the
impact of adjusting the search budget on fitness and crash detection. Based on the
intermediate evaluation STGFA-SMOG produced better fitness values than Random
test generation for all search budgets we applied. Furthermore, STGFA-SMOG with
both search budgets detected more crashes than Random test generation, and more
crashes were detected with a 30 generation search budget than with a 10 generation
budget. However, the comparison between the different configurations of fitness
functions only demonstrated minor differences between fitness function configurations.
The empirical results revealed that the best configuration is dependent on
the app-under-test. Ultimately, STGFA-SMOG can enable testers to generate test
cases for non-functional properties of interest, such as CPU, memory, battery, and
network usage, and is able to trigger crashes in Android applications.