10.06.2026
How Should Researchers Report Their Use of LLMs?
MCML Research Insight – With Stefan Feuerriegel, Barbara Plank, Kerstin Forster, Dominique Geissler, Abdurahman Maarouf, Sebastian Maier
Large Language Models (LLMs) are increasingly becoming part of scientific research. They can generate text, analyze data, simulate participants, and support researchers in entirely new ways.
But there is a catch: even small changes in prompts, settings, or model versions can lead to substantially different results. Without clear reporting, it becomes difficult to understand what model was used, how it was configured, which prompts were given, or how outputs were validated.
MCML PI Stefan Feuerriegel, together with Junior Members Kerstin Forster, Dominique Geissler, Abdurahman Maarouf, Sebastian Maier, MCML PI Barbara Plank, and other international collaborators, developed GUIDE-LLM — a framework to help researchers use LLMs more transparently. Their work was published in Nature Human Behaviour, one of the leading journals for interdisciplinary research on human behavior and society.
Key Insight
Research using LLMs needs new reporting standards.
A Challenge for Research Rigour
Science depends on reproducibility. Researchers need to understand how a study was conducted in order to evaluate or replicate it. That becomes harder with LLMs. The same model may behave differently depending on its version, size, runtime environment, or system instructions, and even small changes in prompts can lead to substantially different outputs. At the same time, models may also reproduce biases present in their training data.
Without proper reporting, this can lead to:
- Replication failures
- Misinterpretation of results
- Difficulty comparing studies across fields
- Reduced trust in findings using LLMs
- Challenges for reviewers evaluating methodological quality
Core Idea
GUIDE-LLM establishes a common reporting standard that makes AI-assisted research easier to evaluate, compare, and reproduce.
A Shared Standard for Transparent AI Research
GUIDE-LLM introduces a reporting framework that helps researchers systematically document how LLMs are used in behavioural and social science research. It was created through a global panel of 80 experts across disciplines including psychology, economics, sociology, AI, and ethics.
The final checklist contains 14 core items that help researchers clearly document:
- Why and how LLMs were used
- Which models and versions were used
- Prompting and configuration choices
- Validation procedures for LLM-generated outputs
- Reproducible workflows and code sharing
Importantly, GUIDE-LLM does not tell researchers how to use AI. Instead, it establishes a minimum standard for transparency.
Takeaway
Without transparent reporting, AI-assisted research risks becoming impossible to evaluate or reproduce.
Why It Matters
As LLMs become standard tools in research, transparency becomes essential for interpreting results. Without it, studies risk becoming difficult to evaluate, compare, or reproduce.
GUIDE-LLM aims to make AI-assisted research visible, structured, and reliable — helping ensure that research remains trustworthy in the age of AI.
Further Reading & Reference
The links below lead to the full paper, published in Nature Human Behaviour - a leading journal at the intersection of the social, behavioral, and computational sciences - as well as the project page with additional information and the downloadable checklist.
A reporting checklist for large language models in behavioural science.
Nature Human Behaviour 10. Jun. 2026. PDF URL
Share Your Research!
Get in touch with us!
Are you an MCML Junior Member and interested in showcasing your research on our blog?
We’re happy to feature your work. Get in touch with us to present your paper.
Related
16.09.2026
Vincent Fortuin Receives ERC Starting Grant for Reliable Small-Data AI
MCML PI Vincent Fortuin receives an ERC Starting Grant for AutoBayes, advancing reliable AI methods for learning from small datasets.
