Causal Inference on Outcomes Learned From Text

This lecture introduces a machine learning method that investigates causal effects in randomized controlled trials using text data. The tool answers three key questions: whether an intervention influences the text, which outcome measures this influence affects, and how complete the effects are. Large language models are used to identify systematic differences between texts from different experimental groups. For robust results, the approach combines LLM analysis with statistical validation through sample splitting and human annotations for content review. The applicability of the method is demonstrated using a sample study with abstracts of scientific manuscripts.

Jann Spiess is an Associate Professor of Operations, Information & Technology at the Graduate School of Business, Stanford University. He holds a PhD in economics from Harvard University.