Predicting annotation difficulty using Monte Carlo dropout
Ladda ner
Typ
Examensarbete för masterexamen
Master Thesis
Master Thesis
Program
Complex adaptive systems (MPCAS), MSc
Publicerad
2019
Författare
Wilhelmsson, Jens
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Developing products based on machine learning algorithms require relevant and accurate datasets. In particular when it comes to supervised learning algorithms whose performance is directly related to the quality and amount of training data. Within the field of computer vision, classification is a task that require training data in the form of annotated images. Annotating images is a manual task and I propose that the annotation difficulty of an image should be interpreted as the likelihood of someone else annotating an image differently. Knowing in advance which images are hard to annotate would facilitate the distribution of work between annotators with varying experience. In this thesis, it is shown that the uncertainty derived from Monte Carlo dropout resembles the variance of a group of persons annotations of the same image. This finding indicates that the level of agreement between persons can be predicted, and thus enable for better distribution of work between annotators. Furthermore, the finding could also be used to order images during training by prioritizing harder images higher.
Beskrivning
Ämne/nyckelord
Fysik , Physical Sciences