Through a Compressed Lens: Investigating the Impact of Quantization on LLM Explainability and Interpretability

MCML Authors

Mingyang Wang

→ Group Hinrich Schütze
 Computational Linguistics

Hinrich Schütze

Prof. Dr.

Core PI

Computational Linguistics

Abstract

Quantization methods are widely used to accelerate inference and streamline the deployment of large language models (LLMs). Although quantization’s effects on various LLM capabilities have been extensively studied, one critical area remains underexplored: factual knowledge recall (FKR), the process by which LLMs access stored knowledge. To this end, we conduct comprehensive experiments using three common quantization techniques at distinct bit widths, in conjunction with interpretability-driven analyses on two tasks, knowledge memorization and latent multi-hop reasoning. We show that quantization typically results in information loss within LLMs, consequently diminishing their capacity for FKR. This effect is particularly amplified in smaller models within the same architectural families. However, models quantized at reduced bit precision do not consistently exhibit inferior performance and occasionally quantization may even enhance model FKR. We find that BitSandBytes demonstrates highest preservation of the original full-precision model’s FKR. Despite variability across models and methods, quantization causes modest performance degradation and remains an effective compression strategy.

inproceedings WWF+26

---

TrustNLP @ACL 2026

6th Workshop on Trustworthy NLP at the 64th Annual Meeting of the Association for Computational Linguistics. San Diego, CA, USA, Jul 02-07, 2026.

---

Authors

Q. Wang • M. Wang • N. Feldhus • S. Ostermann • Y. Cao • H. Schütze • S. Möller • V. Schmitt

---

Links

DOI

---

In Collaboration

 Bosch

---

Research Area

 B2 | Natural Language Processing