Home  | Publications | KAC+24

Automatic Speech-Based Charisma Recognition and the Impact of Integrating Auxiliary Characteristics

MCML Authors

Link to Profile Björn Schuller

Björn Schuller

Prof. Dr.

Principal Investigator

Health Informatics

Abstract

Automatic recognition of speaker’s states and traits is crucial to facilitate a more naturalistic human-AI interaction – a key focus in human-computer interaction to enhance user experience. One particularly important trait in daily life is charisma. To date, its definition is still controversial. However, it seems that there are characteristics in speech that the majority perceives as charismatic. To this end, we address the novel speech-based task of charisma recognition in a three-fold approach. First, we predict charismatic speech using both interpretable acoustic features and embeddings of two audio Transformers. Afterwards, we make use of auxiliary labels that are highly correlated with charisma, including enthusiastic, likeable, attractive, warm, and leader-like, to check their impact on charisma recognition. Finally, we personalise the best model, taking individual speech characteristics into account. In our experiments, we demonstrate that the charisma prediction model benefits from integrating auxiliary characteristics as well as from the personalised approach, resulting in a best Pearson’s correlation coefficient of 0.4304.

inproceedings

TELEPRESENCE 2024

IEEE Conference on Telepresence. Pasadena, CA, USA, Nov 16-17, 2024.

Authors

A. Kathan • S. Amiriparian • L. Christ • S. Eulitz • B. W. Schuller

Links

DOI

Research Area

 B3 | Multimodal Perception

BibTeXKey: KAC+24

Back to Top