When Small Decisions Have Big Impact: Fairness Implications of Algorithmic Profiling Schemes

MCML Authors

Christoph Kern

Prof. Dr.

Associate

Social Data Science and AI

Frauke Kreuter

Prof. Dr.

Principal Investigator

Social Data Science and AI

Abstract

Algorithmic profiling is increasingly used in the public sector with the hope of allocating limited public resources more effectively and objectively. One example is the prediction-based profiling of job seekers to guide the allocation of support measures by public employment services. However, empirical evaluations of potential side-effects such as unintended discrimination and fairness concerns are rare in this context. We systematically compare and evaluate statistical models for predicting job seekers’ risk of becoming long-term unemployed concerning subgroup prediction performance, fairness metrics, and vulnerabilities to data analysis decisions. Focusing on Germany as a use case, we evaluate profiling models under realistic conditions using large-scale administrative data. We show that despite achieving high prediction performance on average, profiling models can be considerably less accurate for vulnerable social subgroups. In this setting, different classification policies can have very different fairness implications. We therefore call for rigorous auditing processes before such models are put to practice.

article

---

ACM Journal on Responsible Computing

Nov. 2024.

---

Authors

C. Kern • R. Bach • H. Mautner • F. Kreuter

---

Links

DOI

---

Research Area

 C4 | Computational Social Sciences