Predicting annotation difficulty using Monte Carlo dropout

Examensarbete för masterexamen

Please use this identifier to cite or link to this item: https://hdl.handle.net/20.500.12380/256747
Download file(s):
File Description SizeFormat 
256747.pdfFulltext18.37 MBAdobe PDFView/Open
Type: Examensarbete för masterexamen
Master Thesis
Title: Predicting annotation difficulty using Monte Carlo dropout
Authors: Wilhelmsson, Jens
Abstract: Developing products based on machine learning algorithms require relevant and accurate datasets. In particular when it comes to supervised learning algorithms whose performance is directly related to the quality and amount of training data. Within the field of computer vision, classification is a task that require training data in the form of annotated images. Annotating images is a manual task and I propose that the annotation difficulty of an image should be interpreted as the likelihood of someone else annotating an image differently. Knowing in advance which images are hard to annotate would facilitate the distribution of work between annotators with varying experience. In this thesis, it is shown that the uncertainty derived from Monte Carlo dropout resembles the variance of a group of persons annotations of the same image. This finding indicates that the level of agreement between persons can be predicted, and thus enable for better distribution of work between annotators. Furthermore, the finding could also be used to order images during training by prioritizing harder images higher.
Keywords: Fysik;Physical Sciences
Issue Date: 2019
Publisher: Chalmers tekniska högskola / Institutionen för fysik (Chalmers)
Chalmers University of Technology / Department of Physics (Chalmers)
URI: https://hdl.handle.net/20.500.12380/256747
Collection:Examensarbeten för masterexamen // Master Theses



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.