Home | Publications | NGD24

Challenges in Explaining Representational Similarity Through Identifiability

MCML Authors

Andrea Dittadi

Dr.

→ Group Stefan Bauer
Algorithmic Machine Learning & Explainable AI

Abstract

The phenomenon of different deep learning models producing similar data representations has garnered significant attention, raising the question of why such representational similarity occurs. Identifiability theory offers a partial explanation: for a broad class of discriminative models, including many popular in representation learning, those assigning equal likelihood to the observations yield representations that are equal up to a linear transformation, if a suitable diversity condition holds. In this work, we identify two key challenges in applying identifiability theory to explain representational similarity. First, the assumption of exact likelihood equality is rarely satisfied by practical models trained with different initializations. To address this, we describe how the representations of two models deviate from being linear transformations of each other, based on their difference in log-likelihoods. Second, we demonstrate that even models with similar and near-optimal loss values can produce highly dissimilar representations due to an underappreciated difference between loss and likelihood. Our findings highlight key open questions and point to future research directions for advancing the theoretical understanding of representational similarity.

inproceedings

UniReps @NeurIPS 2024

2nd Workshop on Unifying Representations in Neural Models at the 37th Conference on Neural Information Processing Systems. Vancouver, Canada, Dec 10-15, 2024.