“See, Don’t Assume”: Revealing and Reducing Gender Bias in AI

Using AI and LLMs at work feels almost unavoidable today: they make things easier, but they can also go wrong in important ways. One of the trickiest problems? Gender bias. For example, if you ask about someone’s skills from a photo, it may confidently label them a “born leader” or “working well under pressure” with no real evidence.

A recent ICLR 2025 paper from MCML, Helmholtz Munich, and UC Berkeley puts 22 popular open-source vision-language assistants under the microscope and shows how to spot gender bias and dial it down without reducing overall performance. In their paper “Revealing and reducing Gender Biases in Vision and Language Assistants (VLAs)”, MCML Junior Members Leander Girrbach, Yiran Huang and former member Stephan Alaniz, with MCML PI Zeynep Akata and collaborator Trevor Darrell investigate what biases these models exhibit and evaluate ways to reduce them.

---

How it Works

The authors test three areas, i.e., personality traits, skills, and occupations, to see how the models assign these to men and women. Since the data used to train such systems can contain hidden stereotypes, those same biases may quietly carry over into the model’s behavior. Without checking or correcting for these biases, AI systems risk repeating and reinforcing real-world stereotypes in their answers.

To measure gender bias, they select a set of 5,000 pictures, balanced by gender and with no obvious job clues like uniforms or tools. Then, for the different personality traits and skills, they ask models whether the person shown in the image possesses them, or for occupations, whether they would be well-suited to do the job. Models are prompted to answer with Yes/No/Unsure, and prompts are phrased in multiple ways to avoid effects of specific formulations. As a bias metric, they track the model’s chance of saying “Yes”. The gap in those “Yes” scores for men versus women on the same question is considered a bias.

Once they find bias, they test different ways to fix it: retraining the model (fully or with parameter-efficient (low-rank) updates), changing prompts, pruning the model, while adding a small penalty that encourages neutral answers (like “Unsure”) when there’s no clear evidence in the image.

---

Results

The results showed a clear pattern of bias. Men were more often described with negative traits like “moody” or “arrogant”, while women were labeled with more positive ones. In terms of skills, men were linked to things like “leadership” and “working under pressure”, while women were more often said to be “good communicators” or “great at multitasking”.

Occupation biases closely matched real-world patterns, too: roles like “construction worker” and “mechanic” were mostly assigned to men, while jobs like “nurse” and “secretary” were more often linked to women, mirroring existing labor market inequalities. Many models also avoided saying “Unsure”, giving confident answers even when there wasn’t enough visual evidence.

When the researchers adjusted the models to be fairer, the results improved without a significant loss in performance on general tasks unrelated to gender bias. The most effective way to reduce bias is to train the model to be less biased. When training the entire model, bias is reduced the most. However, performance on unrelated tasks also takes a larger hit. Parameter-efficient fine-tuning reduces bias slightly less well, but preserves performance on other tasks better. These changes reduce gender bias by roughly half for personality traits and nearly eliminate it for skills and jobs. Other fixes, such as tweaking prompts or pruning model parameters, helped less and sometimes even hurt accuracy. After these adjustments, the models responded more reasonably, such as choosing “Unsure” when there was no clear visual proof, rather than guessing.

---

Why it Matters

These models show up in hiring tools, education, support, and everyday apps. Unchecked bias equals scaled stereotypes. Therefore, it is necessary to test these models for potential biases, especially gender bias, and develop methods to reduce them.

---

Challenges

Though the authors understand that gender is not only a binary concept, due to data constraints, they used a binary male/female distinction to understand bias. Future work could address this limitation, and also add an axis of ethnicity in the analysis to understand how biases regarding it could perhaps be reflected in the output of LLMs.

---

Further Reading & Reference

If you’d like to dig deeper into how the team tested, quantified, and reduced gender bias in vision–language models or try the benchmark yourself, check out the full paper, presented at ICLR 2025, one of the world’s leading machine learning conferences.

---

Subscribe to RSS News feed