Understanding deep neural networks with clustering analysis and structural causal model

Abstract

The blackbox problem of machine learning algorithms has been limiting human trust in deep neural networks’ decisions. While some researchers use clustering analysis to enhance networks’ transparency, some use attribution method to improve the interpretability. This raises a question "how it will be if we apply both approaches on a deep neural network at once". The current study is an exploratory study that investigates the possibilities of relieving the blackbox problem by combining clustering analysis and a structural causal modelling-based attribution method. The study is developed on a conditional β variational autoencoder (β-VAE). It estimates the average causal effects (ACEs) of the decoder’s inputs on the hidden layers and reconstruction layer, then conducts ACE-based and activation-based clustering analysis. We apply lesion test experiments to identify the clusters that are important to image reconstruction and answer our question "if there is a non-empty intersection of the two important clusters that contains neurons which are critical to image reconstruction". The results show that 1) there is one input neuron carrying the rotation and scaling roles in image reconstruction, 2) this neuron has positive ACEs on the stroke of the targeted digit, 3) both of the activation-based and ACE-based clustering analysis give us at least 5 clusters, 4) there is always one cluster that is important to image reconstruction, 5) the intersection of the two important clusters is not empty and contains neurons that are critical to image reconstruction and 6) among those critical neurons, there are only 4 to 21 hidden neurons in contrast to our decoder which has 896 hidden neurons. The findings suggest that there may exist the measure of chance of reducing the cost of training deep neural networks and protecting networks from being hacked by focusing on the critical hidden neurons.