Investigating the Applicability of the Bayesian Plackett-Luce Model in Software Engineering Problems
Ladda ner
Typ
Examensarbete för masterexamen
Program
Software engineering and technology (MPSOF), MSc
Publicerad
2021
Författare
Somayagi, Vallisha
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
he statistical ranking has been used for ordering the artifacts based on the importance or priority in different problems in software engineering (SE) research. While frequentist statistical ranking methods such as the Friedman test and area under the curve are commonly found in research, these methods face many limitations.
For instance, the Friedman test may lack power if the sample size is small and focuses on hypothesis testing rather than estimating effects. Similarly, the area under the curve method is inconsistent and unreliable in choosing confidence scales. Frequentist methods can lead to lower conclusion validity and interpretation pitfalls. To address these limitations, we introduce the Bayesian Plackett-Luce model and examine its applicability to SE research. Following a design science methodology, iteratively developed an R package for the BPL model. We examined the applicability of this package with three SE datasets and compared it with the other ranking models. Further evaluation with SE researchers confirms the suitability of the Bayesian Plackett-Luce model for ranking in SE. This thesis shows that: First, the Bayesian Plackett-Luce model is suitable for ranking software engineering problems. Second, the additional information about the data given by the density
plot in the Bayesian Plackett-Luce model is the advantage compared with other ranking models. The additional information is vital for making someone consider using the BPL model instead of other ranking models.
Beskrivning
Ämne/nyckelord
ranking , software engineering , SE , statistical , frequentist , Bayesian , Plackett-Luce model , researchers , datasets