How Fair Does AI Seem in Job Interviews?

Many companies rely on AI for recruitment. A new study shows that job applicants mostly trust the avatars conducting their interviews – but only until they receive a rejection. How fair this decision is perceived to be depends on the avatars’ identity characteristics. The test subjects who felt treated the most unfairly were not those who differed most from the avatar – but rather those who matched it in either gender or skin color.

Companies are increasingly using artificial intelligence in their hiring processes. It’s not just CVs that are evaluated automatically. AI tools can also conduct job interviews – usually in the form of avatars, which are animated characters – and make hiring decisions. An important reason for this, aside from saving time, is that AI is said to be less biased than humans. “What has been largely overlooked so far is that we all unconsciously react to the avatars’ appearance, even when we know we’re talking to a machine. That’s why a conversation with artificial intelligence becomes a social interaction as soon as it behaves like a human,” says MCML PI Enkelejda Kasneci, professor of Human-Centered Technologies for Learning at the Technical University of Munich (TUM).

A team of researchers from TUM and Lund University therefore investigated how applicants perceive AI decisions in job interviews depending on the avatar’s appearance. For the study, approximately 220 participants from Germany, the United Kingdom and the United States conducted a short simulated job interview for a fictional position in a company’s customer support department. They were interviewed by a photorealistic avatar capable of reacting to their answers and asking follow-up questions in a human-like manner. The research team had programmed four variants to embody the AI. The avatars were either female or male and had either dark or light skin. The participants’ eye movements were monitored using eye-tracking technology. After the interview, they completed a questionnaire.

Differences in Eye Tracking

The eye-tracking analysis showed that participants looked more closely at the avatar’s face when its skin color differed from their own (as self-reported). However, trust in artificial intelligence was consistently high at this point regardless of whether the gender and the skin color of the test subject and the avatar matched or not.

That changed after all applicants had received a rejection for the job position and were interviewed again. At that point, the test subjects were more likely to believe that they had not been judged impartially. Whether they perceived the decision as fair depended on the characteristics of the avatars.

If the avatar’s skin color differed from that of the test participants, they were more likely to attribute the rejection to bias. However, the participants who felt they were treated the most unfairly were those who matched the avatar in one characteristic, either gender or skin color. They judged the level of fairness of the decision more negatively than participants who matched the avatar in both characteristics – but also more negatively than participants who differed completely from the avatar.

“Taking Social Behavior Into Account in AI Design”

“The discussion about fairness in the use of artificial intelligence has so far revolved mainly around whether the models are programmed and trained without bias. But even if that is the case, AI may still be perceived as unfair. And this effect can arise for reasons other than what we might assume at first glance,” says study author Kasneci. “That’s why insights into our social behavior must be given greater consideration in the design of AI if these technologies are to fulfill their purpose – such as a recruitment process whose quality is recognized by all involved.”

@TUM