Informed regularization Aiding the identification of spurious correlations

Abstract

Today, end-to-end neural networks that feature deep and complex architectures, are common tools to use in natural language processing. By using these methods it has become harder to identify which inputs have contributed the most to a model’s classification. This issue leads to the problem of models overfitting on features that cannot directly be identified by a developer. To open up the black box of complex deep learning natural language processing systems, this study aims to investigate what information can be extracted from the data used to train a model and how the model’s inputs are weighted during pre diction. This thesis aims to present methods that can aid in the identification of differences between the population a developer intends to model with a data set and what correlations a model makes from the true content of the data. By presenting three novel methods that can aid a developer with the task of identify ing spurious correlations, it was possible to present information regarding a spurious correlation between two pre-selected keywords and a model’s classification. It was also shown that the identification and reduction of spurious correlations is a tricky subject. Results showed that, from the reduction of the spurious correlation asso ciated with the selected keyword, the model made another correlation which could be considered as spurious.