Large Language Models for Statistical Inference: Context Augmentation With Applications to the Two-Sample Problem and Regression

The colloquium introduces context expansion, an approach that uses large language models to generate additional contexts around strings to enable valid statistical inferences.

These contexts reduce uncertainty, provide additional information and improve the interpretability of the results. Using synthetic data and a two-sample test, it is shown that the method has correct null behavior, is powerful and can be easily replicated. In addition, a text-to-text regression is introduced in which generated contexts serve as mediating variables and semantic from syntactic effects can be separated.

Theoretical analyzes provide identification conditions, efficiency gains and error bounds and overall show that context extension meaningfully combines LLMs with classical statistics.

Marc Ratkovic is Professor and Chair of Social Data Science at the Department of Political Science, University of Mannheim. He is working to integrate machine learning and deep learning methods into practical social science methodology.