Training Binary Deep Neural Networks Using Knowledge Distillation
Typ
Examensarbete för masterexamen
Program
Publicerad
2020
Författare
Lundborg, Sofia
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Binary networks can be used to speed up inference time and make image analysis
possible on less powerful devices. When binarizing a network the accuracy drops.
The thesis aimed to investigate how the accuracy of a binary network can be improved
by using knowledge distillation. Three different knowledge distillation methods
were tested for various network types. Additionally, different architectures of a
residual block in ResNet were suggested and tested. Test on CIFAR10 showed an
1.5% increase in accuracy when using knowledge distillation and an increase of 1.1%
when testing on ImageNet dataset. The results indicate that the suggested knowledge
distillation method can improve the accuracy of a binary network. Further
testing needs to be done to verify the results, especially longer training. However,
there is great potential that knowledge distillation can be used to boost the accuracy
of binary networks.
Beskrivning
Ämne/nyckelord
deep neural networks , knowledge distillation , binary neural networks