23.07.2025

Teaser image to How Reliable Are Machine Learning Methods? With Anne-Laure Boulesteix and Milena Wünsch

How Reliable Are Machine Learning Methods? With Anne-Laure Boulesteix and Milena Wünsch

Research Film

Often a new machine learning method claims to outperform the last. Whether it’s in bioinformatics, finance, or image recognition, the message is the same: this algorithm is faster, more accurate, more powerful. But can we trust those claims?

«It’s not just about the algorithms. It’s about how we compare them—and what we choose to report or ignore.»


Milena Wünsch

MCML Junior Member

Beneath the surface of many benchmarking studies lies a quiet problem: subtle biases that skew comparisons and inflate performance. These issues often go unnoticed — but they can have real consequences, especially when such models are used to inform research or high-stakes decisions.

«It doesn’t matter whether the bias is deliberate or not. It still shapes how methods are judged and used.»


Anne-Laure Boulesteix

MCML PI

Anne-Laure Boulesteix, Professor of Biometry at LMU and MCML PI, and Milena Wünsch, PhD student at LMU and MCML, study how seemingly harmless methodological choices can lead to misleading results.

One common issue: when a method fails on a dataset, researchers may simply drop it from the analysis. While convenient, this can introduce bias and overstate performance.

Bias can also arise from less obvious sources — like spending more time tuning one method, being more familiar with a tool, or unconsciously interpreting results in its favor.

With so many studies promoting the “next best” algorithm, it’s hard to know which results to trust. Researchers may end up using a method that only looked good due to biased comparisons. Still, the researchers are hopeful. In recent years, the methodological machine learning community has made real progress — pushing for better standards, more transparency, and more careful benchmarking.

Watch in Full Quality on youTube

The film was produced and edited by Nicole Huminski and Nikolai Huber.

 

23.07.2025


Subscribe to RSS News feed

Related

Link to Predicting Health with AI - with researcher Simon Schallmoser

22.09.2025

Predicting Health With AI - With Researcher Simon Schallmoser

Simon Schallmoser uses AI to predict health risks, detect low blood sugar in drivers, and advance personalized, safer healthcare.

Link to

19.09.2025

MCML Researchers With 24 Papers at MICCAI 2025

28th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2025). Daejeon, Republic of Korea, 23.09.2025 - 27.09.2025

Link to Robots Seeing in the Dark - with researcher Yannick Burkhardt

15.09.2025

Robots Seeing in the Dark - With Researcher Yannick Burkhardt

Yannick Burkhardt erforscht Event-Kameras, die Robotern ermöglichen, blitzschnell zu reagieren und auch im Dunkeln zu sehen.

Link to

12.09.2025

MCML Researchers With Eight Papers at ECML-PKDD 2025

European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Database (ECML-PKDD 2025). Porto, Portugal, 15.09.2025 - 19.09.2025

Link to Niki Kilbertus Receives Prestigious ERC Starting Grant

08.09.2025

Niki Kilbertus Receives Prestigious ERC Starting Grant

Niki Kilbertus wins ERC Starting Grant for his DYNAMICAUS project on causal AI and scientific modeling.

Back to Top