Home | Events

08

Feb

Teaser image to The highs and lows of performance evaluation: Towards a measurement theory for machine learning

 

The highs and lows of performance evaluation: Towards a measurement theory for machine learning

Peter Flach, University of Bristol

   08.02.2023

   5:00 pm - 7:00 pm

   Livestream on YouTube

The understanding of performance evaluation measures for machine-learned classifiers has progressed, but gaps persist, leading to questionable evaluation practices. This raises concerns about the trustworthiness of systems utilizing these algorithms. To address this, a proper measurement theory of machine learning is proposed, akin to measurement theory in other domains. This theory would explore relevant concatenation operations in data science and AI and their implications for measurement scales. The need for latent-variable models is highlighted to capture key properties like classification ability and dataset difficulty, enabling causal inferences in machine learning experiments.

Related

Link to Multi-Modal and Multi-Robot Coordination in Challenging Environments

 

Munich AI Lectures  •  22.07.2024  •  TUM Garching Campus, FMI Building, Hörsaal 2 (00.04.011), Boltzmannstr. 3, 85748 Garching bei München or online via Livestream

Multi-Modal and Multi-Robot Coordination in Challenging Environments

Munich AI Lecture with Sebastian Scherer from CMU outlining some of their approaches progress, and results on multi-modal sensing.

Link to We are (still?) not giving data enough credit

 

Munich AI Lectures  •  17.07.2024  •  Bayerische Akademie der Wissenschaften, Plenarsaal, 1. Stock, Alfons-Goppel-Straße 11, 80539 München

We are (still?) not giving data enough credit

Munich AI Highlight Lecture with Alexei A. Efros from UC Berkeley, emphasizing the critical role of data in Computer Vision with historical examples and recent work.

Link to Physical AI: Promises and Challenges

 

Munich AI Lectures  •  15.07.2024  •  TU Munich, Institute for Advanced Study, Auditorium (Ground floor), Lichtenbergstraße 2a, 85748 Garching

Physical AI: Promises and Challenges

Munich AI Lecture with Daniela Rus from MIT discussing recent developments in ML and robotics.

Link to From Video Understanding to Embodied Agents

 

Munich AI Lectures  •  25.06.2024  •  TUM Campus Munich, Room 0790, Arcisstraße 21, 80333 München

From Video Understanding to Embodied Agents

The MCML invites you to the Munich AI Lectures with Ivan Laptev discussing AI models that make reliable predictions from explanatory videos.

Link to Learning complex robotic behaviors with optimal control

 

Munich AI Lectures  •  18.06.2024  •  TUM, Arcisstr. 21, 80333 Munich, Room 0790 (ground floor)

Learning complex robotic behaviors with optimal control

Munich AI Lecture with Ludovic Righetti from NYU presenting their recent work with a particular eye towards unifying learning and numerical optimal control.