Importance Sampling in Deep Learning Object Detection - An empirical investigation into accelerating deep learning object detection training by exploiting informative data points
Ladda ner
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Program
Data science and AI (MPDSC), MSc
Publicerad
2023
Författare
Huang, Alexander
Kvernes, Johannes
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
In recent years, the field of deep learning object detection has witnessed a notable surge in progress, largely fueled by the growth of available data. While the ever growing amount of data has enabled complex object detection models to improve generalization performance and to facilitate training phases that are less prone to over-fitting, the time to reach desired performance with this remarkable amount of data has been reported to be an emerging problem in practice. This is certainly the case when each sample consist of high resolution image data that could burden the capacity of loading data. This thesis explores the possibilities of leveraging importance sampling as a means to accelerate gradient-based training of an object detection model. This avenue of research aims at biasing the sampling of training examples during training, in the hope of exploiting informative samples and reduce the amount of computation on uninformative, noisy and out-of-distribution samples. While previous art shows compelling evidence of importance sampling in deep learning, it is consistently reported in a context of less complex tasks. In this work, we propose techniques that
can be applied to single-stage anchor-free object detectors, in order to adopt importance sampling to accelerate training. Our methods do not require modifications to the objective function, and allow for a biased sampling procedure that remains consistent across runs and samples. Our results suggest that an uncertainty-based heuristic outperforms loss-based heuristics, and that object detection training can be subject to a remarkable speed-up in terms of reaching the baseline’s performance in fewer iterations, where the baseline samples the training examples uniformly without replacement. Furthermore, the empiric observations reported in this work also indicate that an increased final generalization performance can be achieved given an equal amount of training time when compared to the baseline.
Beskrivning
Ämne/nyckelord
Importance sampling , deep learning , object detection , accelerate training , sampling heuristic , gradient descent , loss , uncertainty , threshold-closeness