German AI Competence Centers

This work introduces ShaplEIG, a Bayesian experimental design method for efficiently approximating Shapley values when value-function evaluations are expensive. By using a Gaussian process surrogate and selecting coalitions based on expected information gain, the method achieves more accurate Shapley value estimates than existing approaches under limited evaluation budgets.

The paper shows that task-relevant information in foundation models is spread across all layers, not just the final ones. It introduces an attentive probing method that dynamically combines representations from multiple layers, leading to consistently better performance than standard linear probes across diverse datasets.

This work formalizes calibration for probabilistic label ranking, introducing a hierarchy of calibration notions for full, partial, and top-k rankings. It shows that existing models are often poorly calibrated and demonstrates that calibration provides complementary insights beyond prediction accuracy, including for RLHF reward models.

The paper argues that activation steering should be viewed as a form of model adaptation alongside fine-tuning and prompting. It introduces functional criteria to compare steering with traditional methods and shows that steering enables targeted, reversible behavior changes without updating model parameters.

The paper evaluates multilingual counterfactual generation with LLMs and finds that translation-based counterfactuals outperform direct generation but remain inferior to English. Multilingual counterfactual data augmentation improves performance, especially for low-resource languages, though errors limit robustness.

The paper tackles mode collapse in text-to-image models by optimizing the input noise rather than modifying the model itself. This simple noise optimization approach improves diversity and generation quality while preserving fidelity, with frequency-aware noise initializations further enhancing results.

The paper introduces MO-CAPO, a multi-objective prompt optimization method that jointly optimizes LLM performance and inference cost. By efficiently exploring performance–cost trade-offs, it identifies diverse, high-quality prompts and consistently outperforms existing multi-objective baselines while remaining cost-efficient.

The paper shows that task-relevant information in Vision Transformers is distributed across multiple layers, not just the final one used in standard linear probing. It introduces Attentive Layer Fusion (ALF), which dynamically combines representations from all layers, improving performance across diverse tasks and revealing how foundation models organize information hierarchically.

This work introduces promptolution, a modular and extensible open-source framework that unifies contemporary prompt optimization techniques for large language models.

The paper introduces the first integer linear programming approach for partial–partial 3D shape matching, addressing the challenge of finding both correspondences and unknown overlap regions. By leveraging geometric consistency, the method achieves accurate, smooth, and scalable matching, outperforming existing approaches in realistic partial-observation scenarios.

This work introduces ControlEvents, a diffusion-based model that generates high-quality event data guided by text labels, 2D skeletons, and 3D poses. Leveraging priors from foundation models like Stable Diffusion, it enables efficient, low-cost labeled data synthesis that boosts performance in event-based vision tasks.

The paper introduces CASHomon sets, extending Rashomon sets to the combined algorithm selection and hyperparameter optimization (CASH) setting to capture multiple well-performing models across model classes. It proposes TruVaRImp, an active learning method with convergence guarantees, which efficiently identifies these models and highlights variability in predictions and feature importance, challenging single-model interpretations.

The paper redefines forgetting in LLM post-training as broader behavioral drift rather than just loss of factual knowledge. It introduces CapTrack, a framework to analyze capability changes, and shows that post-training can degrade robustness and behavior, with instruction tuning causing the strongest drift and no universal mitigation strategy.

The paper introduces Linear-LLM-SCM, a benchmarking framework to evaluate how well large language models perform quantitative causal reasoning by estimating parameters in linear Gaussian structural causal models. Experiments reveal strong variability, stochasticity, and sensitivity to structural errors, highlighting current limitations of LLMs as reliable causal parameter estimators.

Addressing the gap in domain-specific vision-language modeling, this work presents GAIA, a large-scale, multi-sensor, multi-modal remote sensing dataset with 205,150 carefully curated image-text pairs. Experiments show that GAIA enables significant improvements in RS image classification, cross-modal retrieval, and captioning, providing rich, scientifically grounded descriptions of environmental and dynamic phenomena.

Detoxification, the task of rewriting harmful language into non-toxic text, has become increasingly important amid the growing prevalence of toxic content online. However, high-quality parallel datasets for detoxification, especially for hate speech, remain scarce due to the cost and sensitivity of human annotation. In this paper, we propose a novel LLM-in-the-loop pipeline leveraging GPT-4o-mini for automated detoxification. We first replicate the ParaDetox pipeline by replacing human annotators with LLM and show that LLM performs comparably to the human annotation. Building on this, we construct ParaDeHate, a large-scale parallel dataset specifically for hate speech detoxification. We release ParaDeHate as a benchmark of over 8,000 hate/non-hate text pairs and evaluate a wide range of baseline methods. Experimental results show that models such as BART fine-tuned on ParaDeHate achieve better performance in style accuracy, content preservation, and fluency, demonstrating the effectiveness of LLM-generated detoxification text as a scalable alternative to human annotation.

The paper proposes IABO, a Bayesian optimization method that preserves disagreement between multiple GP hyperparameter settings by evaluating the acquisition function under each model instead of collapsing them into a single posterior. Using either aggregated or decision-level comparisons, IABO yields more robust selections and often outperforms standard BO baselines.

The paper introduces a Bayesian framework for contextual multi-task bandits that learns shared latent reward dependencies across tasks and arms from partially observed context. Using a particle-based Gaussian process, it captures key sources of uncertainty and achieves more efficient exploration, outperforming existing hierarchical bandit methods in complex settings.

The paper introduces Gradient Slingshots, a method for manipulating feature visualizations in deep neural networks without changing model performance or architecture. By steering optimization toward arbitrary target images, the work reveals a major vulnerability in feature visualization explanations and proposes a simple defense to improve their trustworthiness.

The paper shows that recourse explanations in algorithmic decision systems are performative, meaning that if many applicants follow the recommendations, the resulting data shift can invalidate those same recommendations after model retraining. It proves that recourse fails when it targets non-causal variables and argues that only causal-variable-based recourse can ensure long-term validity.

This work introduces Equivariance by Contrast (EbC), a method that learns equivariant embeddings from observation pairs affected by group actions without group-specific biases. EbC jointly learns a latent space and group representation, achieving faithful equivariance for both abelian and non-abelian groups and enabling general-purpose encoder-only equivariant learning.

This work presents METok, a training-free, multi-stage event-based token compression framework that accelerates Video Large Language Models while preserving accuracy. By progressively removing redundant visual tokens across encoding, prefilling, and decoding, METok achieves over 80% FLOPs reduction and 93% memory savings without performance loss.

This work presents a compression strategy for Multimodal Large Language Models (MLLMs) that integrates structural pruning with efficient recovery training. Results show that widthwise pruning combined with supervised finetuning and knowledge distillation preserves over 95% of model performance while requiring only 5% of the original training data.

The paper introduces a scalable method to train text-to-image models with DPO using fully synthetic preference data generated by a reward model, removing the need for human annotation. With RankDPO, which leverages ranked preferences, the approach improves prompt alignment and image quality on SDXL and SD3-Medium, offering an efficient alternative to human-labeled datasets.

The paper introduces TriDi, the first unified model for 3D human-object interaction. Unlike previous methods that only predict in one direction (human→object or object→human), TriDi can model both together using a three-way diffusion process. It links humans, objects, and their interactions through a shared transformer network, controllable by text or contact maps. Despite being a single model, TriDi outperforms specialized methods and generates more diverse, realistic results for 3D interaction tasks.

This work introduces VGGSounder, a re-annotated, multi-label test set designed to address critical flaws in the widely used VGGSound benchmark. With detailed modality annotations, VGGSounder enables more accurate evaluation of audio-visual foundation models and uncovers limitations previously overlooked.

The paper introduces PF-SDM, a smooth and interpretable shape representation for biomedical imaging that captures geometry, topology, and temporal dynamics. It enables robust shape comparison and learning, outperforming CNN baselines in predicting body-axis formation in mouse gastruloids while offering improved accuracy and efficiency.

The paper explores feature importance (FI) methods as a means to understand the data-generating process (DGP) in machine learning models, which are often opaque. It provides a comprehensive review of FI methods, new theoretical insights, and practical recommendations for selecting the right approach. The study also discusses uncertainty estimation in FI and future directions for statistical inference from black-box models.

This work introduces ECHO, a unified framework for egocentric modeling of human-object interactions using only head and wrist tracking. Leveraging a Diffusion Transformer with a three-variate diffusion process, ECHO jointly reconstructs human pose, object motion, and contact, achieving state-of-the-art performance in flexible HOI reconstruction.

This work introduces MAGBIG, a controlled benchmark designed to evaluate gender bias in multilingual text-to-image (T2I) generation models. Despite advancements in multilingual capabilities, the study reveals significant gender bias and language-specific inconsistencies, with prompt engineering proving largely ineffective at mitigating these issues.

This work presents WikiBigEdit, a large-scale benchmark of real-world Wikidata edits designed to advance and future-proof research in lifelong knowledge editing. By evaluating existing editing methods against over 500K question-answer pairs, the study reveals their practical strengths and limitations compared to broader approaches like retrieval augmentation and continual finetuning.

This work investigates whether modern vision models exhibit compositional understanding by systematically varying data scale, concept diversity, and combination coverage. The findings show that data diversity—not scale—drives compositional generalization, with effective learning emerging from linearly factored representational structures.

This study explores reducing the number of layers in Large Language Models (LLMs) to address size and efficiency challenges. Remarkably, even models with significantly fewer layers—sometimes just one—can match or outperform fully layered models in prompt-based text classification tasks.

This work introduces a procedural framework for generating virtually infinite, realistic partial 3D shape matching scenarios from complete geometry and establishes cross-dataset correspondences across seven shape datasets (2543 shapes total). It defines challenging partial-matching benchmarks and evaluates state-of-the-art methods as baselines.

This paper introduces TIME, a unified framework for temporal model merging—integrating expert models trained over time on emerging tasks. Through extensive experiments, TIME explores key design choices in initialization, merging, and deployment to improve model performance across dynamic learning scenarios.

This work proposes using minimum ratio cycles in conjugate product graphs to solve shape matching problems more effectively. This approach improves accuracy and significantly reduces runtimes by enabling higher-order costs and better geometric regularization.

This work extends multimodal pretraining to improve few-shot adaptation by enabling models to better use contextual information, achieving up to 4× sample efficiency and 5% average gains across 21 tasks—without sacrificing zero-shot performance.

This work introduces VGGSounder, a re-annotated, multi-label test set designed to address critical flaws in the widely used VGGSound benchmark. With detailed modality annotations, VGGSounder enables more accurate evaluation of audio-visual foundation models and uncovers limitations previously overlooked.

The paper gives an overview of reinforcement learning from human feedback (RLHF) — a method where AI systems learn from human judgments instead of fixed reward functions. It explains how RLHF helps align AI behavior with human values and has been key to the success of large language models. The article reviews RLHF across fields like robotics and control, describing its basic ideas, how human feedback guides learning, and current research trends.

This work analyzes why merging many expert models yields diminishing returns, showing that task vector spaces suffer from rank collapse during merging. To address this, the authors introduce Subspace Boosting, which preserves task vector ranks and boosts merging efficacy by over 10% across vision benchmarks, while offering new insights via higher-order task similarity analysis.

To address false refusals in large language models, this work introduces XSB and MS-XSB, two benchmarks for assessing and mitigating exaggerated safety behaviors. Combined with post-hoc explanations and lightweight inference-time methods, the approach improves safe prompt compliance while maintaining strong safety safeguards across LLMs.

This study investigates the influence of MBTI-based persona prompts on hate speech classification in Large Language Models (LLMs), a previously unexplored aspect of subjectivity in annotation. By demonstrating substantial persona-driven variation and bias, the work emphasizes the need for careful prompt design to support fair and value-aligned model behavior.

This tutorial provides an overview of 3D shape analysis and shape matching, covering both traditional optimization-based methods and modern learning-based approaches for finding correspondences between shapes. It discusses applications in areas such as autonomous driving, medical imaging, and augmented reality, while also highlighting recent advances in spectral methods, foundation models, and challenges like partial shape matching.

This work introduces a survey of common scoring rules for survival analysis, focusing on their theoretical and empirical properness, and proposes a new marginal definition of properness. While the Integrated Survival Brier Score (ISBS) and Right-Censored Log-Likelihood (RCLL) are theoretically improper under this definition, simulations show they behave properly in practice, supporting their continued use—particularly in automated model evaluation—despite practical estimation challenges.

This paper investigates how quantization impacts the explainability and interpretability of large language models, an area previously unexplored. Through experiments with multiple quantization methods, analysis techniques, and a user study, the results show that quantization can unpredictably degrade or even improve transparency, highlighting important implications for LLM deployment.

This paper introduces DeLoRA, a new parameter-efficient finetuning method that normalizes and scales learnable low-rank matrices to bound transformation strength. By doing so, it improves robustness to hyperparameters and training duration while maintaining strong performance, consistently outperforming popular PEFT approaches like LoRA across image generation and LLM instruction tuning tasks.

Disentangled representation learning is key to improving generalization and fairness, but aligning data with a prior while preserving geometric features is difficult. This work introduces a new method using quadratic optimal transport and a Gromov-Monge-Gap regularizer to minimize geometric distortion, achieving strong disentanglement performance across benchmarks.

This paper introduces TIME, a unified framework for temporal model merging—integrating expert models trained over time on emerging tasks. Through extensive experiments, TIME explores key design choices in initialization, merging, and deployment to improve model performance across dynamic learning scenarios.

Accurate lymph node segmentation in 3D CT scans is vital but challenging due to the limited availability of fully annotated datasets. The LNQ challenge at MICCAI 2023 demonstrated that weakly-supervised methods show promise, but combining them with fully annotated data significantly boosts performance, underscoring the continued need for high-quality annotations.

The study introduces a novel method for improving text-to-image (T2I) models by optimizing the initial noise using human preference reward models. This approach significantly enhances T2I model performance, outperforming existing open-source models and achieving efficiency and quality levels comparable to proprietary systems.

The paper introduces a new benchmark, FoMo-in-Flux, for continual multimodal pretraining, designed to tackle the challenges of updating multimodal foundation models. The guide provides practical advice for practitioners on how to update models effectively and efficiently in real-world applications.

The comprehensive review addresses the growing need for transparency in AI applications, especially in critical fields such as geospatial data analysis. The paper highlights methods, objectives, challenges, and findings, providing a much-needed summary of the state of XAI in this specialized area.

The paper introduces EgoCVR, a new benchmark for Composed Video retrieval, where a video and a text description modifying the video content are used to retrieve the relevant video. The study shows that existing methods struggle with this task, and proposes a training-free approach with a re-ranking framework.

The work explores how resource efficiency can be integrated into Automated Machine Learning (AutoML), which traditionally focuses on maximizing predictive quality without considering factors like running time or energy consumption.

The work explores parameter-efficient fine-tuning (PEFT) techniques in the context of continual learning and examines the strengths and limitations of rehearsal-free methods, providing valuable insights into how they can be improved for better performance in dynamic, real-world environments.

The work introduces GNNavi, a novel prompt-based parameter-efficient fine-tuning (PEFT) approach for Large Language Models (LLMs). The approach addresses the high resource demands of traditional fine-tuning by leveraging Graph Neural Networks (GNNs) to efficiently guide the flow of information during prompt processing.

The paper introduces ETHER, a new approach to parameter-efficient fine-tuning (PEFT), which aims to optimize the adaptation of foundation models to downstream tasks while maintaining generalization ability and minimizing the introduction of extra parameters and computational overhead.

The position paper critiques the current focus on high predictive accuracy in deep learning, particularly for supervised tasks involving large image and language datasets, and calls for greater attention to overlooked metrics and data types, such as uncertainty, active learning, continual learning, and scientific data.

The paper explores a new method for learning structured representations by leveraging quadratic optimal transport, enhancing the interpretability of learned features.

The paper explores counterfactual explanations, which help users understand algorithmic decisions by identifying changes that would lead to a desired outcome. These explanations enhance transparency, guide user actions, and provide grounds for contesting decisions.

The paper explores feature importance (FI) methods as a means to understand the data-generating process (DGP) in machine learning models, which are often opaque. It provides a comprehensive review of FI methods, new theoretical insights, and practical recommendations for selecting the right approach. The study also discusses uncertainty estimation in FI and future directions for statistical inference from black-box models.

The study introduces Divergent Token Metrics as a novel method for evaluating compressed large language models (LLMs), offering a deeper analysis of model degradation during compression, and providing a more effective way to optimize these models beyond traditional metrics like perplexity and accuracy.

The paper improves Optimal Transport (OT), a method for efficiently transforming one set of data into another. The new UOT-FM approach helps in areas like predicting biological changes and improving image processing by making the transformation more flexible and accurate. This makes OT more useful for real-world applications.

The work explores the ethical challenges in the algorithmization of concepts like fairness and diversity in AI. The authors advocate for caution when algorithmically implementing ethical principles and emphasize the importance of human oversight to ensure these systems do not mislead or oversimplify complex ethical dilemmas.

The work introduces a way to create new test functions for optimization problems, where specific characteristics of the problem landscape can be chosen in advance. By adjusting random data and training a simple neural network, the method can recreate known test functions and also generate new ones with properties not seen before.

The paper presents the mlr3fairness package which builds upon the ML framework mlr3. The extension contains fairness metrics, fairness visualizations, and model-agnostic pre- and post-processing operators that aim to reduce biases in ML models.

The paper investigates whether pre-trained multilingual language models (PMLMs) impose English moral norms on other languages or show random, potentially harmful biases. Experiments in five languages reveal that the models do encode different moral biases, but these do not consistently reflect real cultural differences. This could cause problems when using such models across languages.