Deep Learning with Ensembles of Neural Networks

Abstract

Deep neural networks are powerful machine-learning models that excel at a large array of machine-learning tasks. A major challenge in machine-learning is the problem of assessing the uncertainty of deep-learning predictors. Coupled with this problem is that deep neural networks tend to make overconfident predictions, which can lead to incorrect decision making where errors go unnoticed. A number of schemes for uncertainty detection have been proposed in recent years ranging from using Bayesian methodology and Monte Carlo simulations to reading neuron states in the upstream layers and letting the neural-networks learn to recognize its certainty. In this thesis, we analyze one of the proposed methods that estimates the predictive uncertainty of deep learning algorithms using an ensemble of deep neural networks. Using classification of handwritten digits as the reference problem, we demonstrate that this method is effective at assessing predictive uncertainty when faced with outof- distribution inputs and inputs that are distorted by deformation and noise. Our results demonstrated that the distribution of the estimated predictive uncertainty differs substantially between correctly and incorrectly classified inputs, indicating that this method can be used to predict incorrect decision making.