Home | Publications | SSR+26b

Climatextract: A Systematic Prompt Evaluation for Climate Information Extraction

MCML Authors

Link to Profile Frauke Kreuter PI Matchmaking

Frauke Kreuter

Prof. Dr.

Core PI

Social Data Science and AI

Link to Profile Alexander Fraser

Alexander Fraser

Prof. Dr.

Core PI

Data Analytics & Statistics

Abstract

Automated extraction of corporate greenhouse gas (GHG) emissions from sustainability reports is critical for climate crisis governance, yet the choice of prompt design, context provision, and model in LLM-based pipelines is largely guided by informal best-practice recommendations rather than systematic evaluation. With Climatextract, we assess these choices in a RAG pipeline, comparing closed-source against open-weight models, a structured “best-practice” prompt against a simple question-style prompt, and text-only against text-plus-table contexts. We find that performance is non-monotonic across model generations, a structured prompt with roles, definitions, extraction rules, and JSON formatting is significantly outperformed by a question-style prompt, and tabular context helps only in a specific model-prompt combination. A cost-benefit analysis shows that performance gains are often disproportionate to cost increases.

inproceedings SSR+26b

ClimateNLP @EMNLP 2026

Tutorials at the Conference on Empirical Methods in Natural Language Processing. Budapest, Hungary, Oct 24-29, 2026.

Authors

A. Steinberg Schulten • M. Schierholz • L. Reichenbach • F. Kreuter • A. Fraser

Links

DOI

Research Areas

 B2 | Natural Language Processing

 C4 | Computational Social Sciences

BibTeXKey: SSR+26b

#p-fraser #p-kreuter
Back to Top