ExPLAIND: Unifying Model, Data, and Training Attribution to Study Model Behavior
MCML Authors
Florian Eichin
Philipp Mondorf
Barbara Plank
Prof. Dr.
Core PI
Michael Hedderich
Dr.
JRG Leader Human-Centered NLP
Abstract
Florian Eichin
Philipp Mondorf
Barbara Plank
Prof. Dr.
Core PI
Michael Hedderich
Dr.
JRG Leader Human-Centered NLP
Abstract
Post-hoc interpretability methods typically attribute a model’s behavior to its components, data, or training trajectory in isolation, and are often tied to a particular level of granularity along the local-to-global spectrum. This leads to explanations that lack a unified view and may miss key interactions. We present ExPLAIND, a theoretically grounded, unified framework that integrates model components, data, and training trajectory while supporting explanations across granularities. We generalize recent work on gradient path kernels, reformulating models trained by AdamW as kernel machines. From the resulting kernel feature maps, we derive novel parameter-wise and step-wise influence scores. We empirically validate the resulting decomposition of model behavior in several settings and apply ExPLAIND to two case studies. Our findings on a Transformer exhibiting Grokking support previously proposed learning phases, while refining the final phase as one in which outer layers align around a representation pipeline learned after memorization. For EuroLLM pretraining, ExPLAIND reveals a two-phase dynamic, with the first characterized by outer-layer MLP learning and the second by increased relative influence of intermediate attention layers. These results establish ExPLAIND as a unified framework for interpreting model behavior and training dynamics.
inproceedings EDM+26
ICML 2026
43rd International Conference on Machine Learning. Seoul, South Korea, Jul 06-11, 2026. To be published. Preprint available.
Authors
F. Eichin • Y. Du • P. Mondorf • M. Matveev • B. Plank • M. A. HedderichLinks
URL GitHubResearch Areas
BibTeXKey: EDM+26