What Shapes Participant Data Quality? a Scoping Review and Case Study of Crowdsourced Webcam Eye Tracking in AI Interviews

MCML Authors

Ka Hei Carrie Lau

→ Group Enkelejda Kasneci
 Human-Centered Technologies for Learning

Enkelejda Kasneci

Prof. Dr.

Core PI

Human-Centered Technologies for Learning

Abstract

Webcam-based eye tracking is a cost-effective, scalable method for remote research that effectively reaches broader populations. However, uncontrolled environments and hardware diversity lead to inconsistent data quality in crowdsourcing. To assess current practices, we conducted a scoping review of crowdsourced eye-tracking from 2011–2025. The review confirms fragmented reporting and a lack of established quality benchmarks. To address this lack of predictive insight, we conducted a case study on AI fairness interviews (N = 205) using the RealEye platform. Applying Ordered Logistic Regression (OLR) to the platform’s quality metric, we found that behavioral and technical factors significantly predict data quality. Specifically, within the RealEye platform, higher fixation counts, shorter sessions, and operating system choice yield significantly higher quality grades. Based on this review and platform-specific predictive insights, we provide actionable recommendations to enhance the reliability, transparency, and replicability of future crowdsourced webcam eye tracking in HCI and behavioral science.

inproceedings LK26

---

CHI 2026

ACM CHI Conference on Human Factors in Computing Systems. Barcelona, Spain, Apr 13-17, 2026.

---

Authors

K. H. C. Lau • E. Kasneci

---

Links

DOI

---

Research Area

 B3 | Multimodal Perception