Using Neural Tangent Kernel metrics to measure Intrinsic Motivation in Reinforcement Learning

Abstract

We aim to investigate if the Neural Tangent Kernel (NTK) is a useful perspective to explain why intrinsic motivation works, in an effort to try to apply theories from supervised learning to the reinforcement learning domain. We find some inconclusive evidence that suggests that an intrinsic motivation, Intrinsic Curiosity Module (ICM), does in fact increase NTK trace as a mechanism to improve performance, and show that NTK trace and other metrics based on the NTK, can be used to artificially select better training sets that decreases test loss.