Home  | Publications | FRW+24

Predicting the Validity and Reliability of Survey Questions

MCML Authors

Abstract

The Survey Quality Predictor (SQP) is an open-access system to predict the quality, i.e., the reliability and validity, of survey questions based on the characteristics of the questions. The prediction is based on a meta-regression of many multitrait-multimethod (MTMM) experiments in which characteristics of the survey questions were systematically varied. The release of SQP 3.0 that is based on an expanded data base as compared to previous SQP versions raised the need for a new meta-regression. To find the best method for analyzing the complex data structure of SQP (e.g., the existence of various uncorrelated predictors), we compared four suitable machine learning methods in terms of their ability to predict both survey quality indicators: LASSO, elastic net, boosting and random forest. The article discusses the performance of the models and illustrates the importance of the individual item characteristics in the random forest model, which was chosen for SQP 3.0.

misc

Preprint

Jun. 2024

Authors

B. Felderer • L. Repke • W. Weber • J. Schweisthal • L. Bothmann

Links

DOI

Research Area

 A1 | Statistical Foundations & Explainability

BibTeXKey: FRW+24

Back to Top