ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge

MCML Authors

Zeinab Sadat Taghavi

* Former Member

→ Group Hinrich Schütze
 Computational Linguistics

Ali Modarressi

→ Group Hinrich Schütze
 Computational Linguistics

Yunpu Ma

Dr.

→ Group Volker Tresp
 Database Systems, Data Mining and AI
→ Co-Group Hinrich Schütze

Hinrich Schütze

Prof. Dr.

Core PI

Computational Linguistics

Abstract

Retrieval systems are central to many NLP pipelines, but often rely on surface-level cues such as keyword overlap and lexical semantic similarity. To evaluate retrieval beyond these shallow signals, recent benchmarks introduce reasoning-heavy queries; however, they primarily shift the burden to query-side processing techniques – like prompting or multi-hop retrieval – that can help resolve complexity. In contrast, we present Impliret, a benchmark that shifts the reasoning challenge to document-side processing: The queries are simple, but relevance depends on facts stated implicitly in documents through temporal (e.g., resolving “two days ago”), arithmetic, and world knowledge relationships. We evaluate a range of sparse and dense retrievers, all of which struggle in this setting: the best nDCG@10 is only 14.91%. We also test whether long-context models can overcome this limitation. But even with a short context of only thirty documents, including the positive document, GPT-o4-mini scores only 55.54%, showing that document-side reasoning remains a challenge.

inproceedings TMM+25

---

EMNLP 2025

Conference on Empirical Methods in Natural Language Processing. Suzhou, China, Nov 04-09, 2025.

---

Authors

Z. S. Taghavi • A. Modarressi • Y. Ma • H. Schütze

---

Links

DOI GitHub

---

Research Areas

 A3 | Computational Models

 B2 | Natural Language Processing