Home | Publications | Kas25

Consistency and Completeness of Knowledge Acquired by Language Models

MCML Authors

Nora Kassner

* Former Member

→ Group Hinrich Schütze
Computational Linguistics

Abstract

This dissertation studies world and commonsense knowledge in pretrained language models, focusing on improving knowledge consistency and completeness. It reveals that LMs often produce self-contradictory answers and proposes a “symbolic executive” architecture that helps models maintain coherent beliefs over time. Additionally, it explores retrieval-based augmentation, reasoning during pretraining, and integration of new entities to enhance factual coverage, moving toward language models with more consistent and evolving world knowledge. (Shortened).

phdthesis Kas25