13.11.2025

Explaining AI Decisions: Shapley Values Enable Smart Exosuits

MCML Research Insight - With Julia Herbinger, Giuseppe Casalicchio, Yusuf Sale, Bernd Bischl and Eyke Hüllermeier

Engineers are tuning the settings for each warehouse worker, like how strongly the suit assists when you lift something. But finding the perfect balance isn’t easy. You can’t simply write down a formula for “comfort”. Instead, you try, test, adjust, and try again.

---

From Trial and Error to Intelligent Optimization

This kind of trial-and-error process is exactly what Bayesian optimization (BO) does in AI. It’s a powerful way to search for the best solution when the problem is too complex to solve directly, from tuning exosuits to designing drugs or ML models. The catch: BO can feel like a black box. It proposes new settings, but rarely explains why. That’s where ShapleyBO comes in: a framework in collaboration with MCML researchers that turns BO’s next move into a short, readable rationale and invites experts into the loop.

In the paper “Explaining Bayesian Optimization by Shapley Values Facilitates Human-AI Collaboration for Exosuit Personalization”, MCML members Julia Herbinger, Yusuf Sale, and Giuseppe Casalicchio along with MCML Director Bernd Bischl and PI Eyke Hüllermeier as well as first author Julian Rodemann and collaborators Federico Croppi, Philipp Arens, Thomas Augustin, and Conor J. Walsh, outline ShapleyBO, their method that creates explainable BO and adds an expert to weigh in on the outputs. The result: better choices made sooner, and with reasons. The project was carried out in cooperation with the Harvard Biodesign Lab, which provided real data on a new generation of exosuits.

---

Interpreting Bayesian Optimization Using Shapley Values

The idea behind ShapleyBO is surprisingly intuitive. It borrows a concept from game theory, the world of mathematical reasoning about cooperation and fairness. In a cooperative game, you want to figure out how much each player contributed to the team’s success. The Shapley value is a method for fairly dividing that “credit.”

Now imagine each parameter of your optimization problem, like the lift assistance parameter in the exosuit problem, as a “player” in the game. ShapleyBO uses the same principle to ask: how much did each parameter contribute to the optimizer’s latest decision?

In other words, it translates the algorithm’s reasoning into a human-understandable explanation. Instead of saying “I picked this configuration because it’s optimal”, ShapleyBO can explain “I picked it because the lifting gain strongly reduced uncertainty, while the lowering gain fine-tuned the mean performance.”

---

Peeking Inside the AI’s Mind

Bayesian optimization constantly juggles two goals:

• Exploration – trying new things to learn more about the problem
• Exploitation – using what it already knows to improve results.

ShapleyBO helps untangle these two forces. Even more, it can separate different types of uncertainty:

• Aleatoric uncertainty, which comes from noise or randomness that can’t be reduced (like measurement errors)
• Epistemic uncertainty, which comes from missing knowledge and can, in principle, be reduced with more data.

By showing how much each parameter contributes to reducing each type of uncertainty, ShapleyBO provides a richer picture of what’s going on under the hood.

---

Humans and Algorithms Working Together

Perhaps the most exciting part of the research is how ShapleyBO also improves human-AI collaboration. In the exosuit example, engineers could watch ShapleyBO’s explanations in real time. When the optimizer proposed a new configuration, they could see why, and decide whether to accept or override it.

In simulated experiments, teams that had access to ShapleyBO’s explanations were able to reach good solutions faster than those without them. The explanations didn’t just make the AI more transparent; they made the collaboration more efficient.

---

Why It Matters

As the complexity of AI systems increases, so does their opacity. If we want to rely on them to make critical decisions, it is crucial to understand why those decisions are being made.

By extending interpretability to optimization algorithms, ShapleyBO opens a new frontier for explainable AI. It helps transform AI from a mysterious black box into a transparent partner that we can reason with, challenge, and ultimately trust.

---

Open Challenges

Some challenges that lie ahead in this area are the explainability as the functions become more complex, or as more parameters are added.

---

Interested in Exploring Further?

The full paper, presented at the ECML-PKDD 2025 conference, provides a more in-depth exploration of the potential of this new method and technology, laying the foundations for future advancements in 3-D modeling and deformation.

The Börsen Zeitung also covered the research in an article

---

#blog #research #bischl #huellermeier

---