Home | Publications | HGF+26

Rethinking Relation-Specific Neurons in Large Language Models

MCML Authors

Link to Profile Hinrich Schütze

Hinrich Schütze

Prof. Dr.

Core PI

Computational Linguistics

Abstract

Previous work has identified relation-specific neurons that selectively activate on specific semantic relations in factual knowledge tasks. However, the conclusions we draw about these representations depend heavily on the methodological assumptions underlying this procedure. We systematically reflect on three such assumptions, showing that (i) the number of relevant neurons varies across relations; (ii) the choice of internal signal for neuron identification shapes the results; (iii) cross-relation entanglement is structural rather than an artifact of subject overlap. We additionally present a preliminary investigation into the mismatch between benchmark-defined relation categories and model-internal organization. For instance, we show that the absence of a strong expert set for the product_company relationship reflects conceptual heterogeneity within the category rather than localization failure, and that targeted ablation of the subrelation car_company yields substantially stronger results. Together, our findings show that the apparent structure of relational representations is jointly shaped by the model's internal organization and the methodological lens applied to study it.

inproceedings HGF+26

Mech Interp @ICML 2026

Workshop on Mechanistic Interpretability at the43rd International Conference on Machine Learning. Seoul, South Korea, Jul 06-11, 2026. To be published. Preprint available.

Authors

L. Hirlimann • S. Gerstner • F. Yvon • H. Schütze

Links

URL

Research Area

 B2 | Natural Language Processing

BibTeXKey: HGF+26

#p-schuetze
Back to Top