Implementation of Deep Feedforward Neural Network with CUDA backend for Efficient and Accurate Natural Image Classification

Abstract

Recent advancements in techniques for constructing and training deep feedforward neural networks for classification tasks has enabled efficient training procedures leading to impressive results. This involves reducing overfitting due to over parameterized models, using an adaptive learning rate for avoiding exploding and vanishing gradients and symmetry breaking parameter initialization for efficient model optimization. Utilizing these techniques, this thesis concerns with the implementation of deep feedforward neural networks capable of efficient and accurate natural image classification. Four feedforward neural network models were constructed with the aim to classify tiny natural images from the CIFAR10 dataset. Having 3.274.634 trainable parameters for gray scale input and 4.259.274, 29.853.002 and 30.955.290 trainable parameters for rgb input, the training procedure utilizes a CUDA backend for efficient parameter optimization. The handwritten digits were classified with 97.31% accuracy and the tiny natural images were classified, using the best model, with 72.88% accuracy.