Discrete Geometry for Comparing and Transferring Neural Representations

Abstract

Neural networks transform data through a sequence of intermediate representations, but comparing and transferring the structure of these representations remains challenging. This thesis develops a geometric framework for neural representations based on manifold learning, representational similarity, and diffusion geometry, incorporating tools from multi-view learning into this field for the first time. A key contribution is an exact Markov reformulation of a broad class of centered, scale-invariant RSMbased similarity measures in terms of row-stochastic Markov matrices, which then opens the door to manipulations from diffusion geometry. Building on this, the thesis introduces multi-scale variants of CKA and DistCorr, which compare powers of the associated Markov operators, and alternating-diffusion variants, which fuse the Markov matrices of several layers into a single network-level operator. Empirically, these diffusion-based measures achieve state-of-the-art performance in accuracy and output correlation for both language and vision tasks across different models, on the Representational Similarity (ReSi) benchmark. They also obtain the best results on an additional out-of-distribution challenge benchmark. The thesis further applies the geometric viewpoint to knowledge distillation. A graph-Laplacian distillation objective is proposed, in which the student is trained to match the teacher’s sample geometry rather than activation coordinates. Together, these results show that operator-based discrete geometry provides a useful language for comparing, aggregating, and transferring neural representations. Keywords: