Home | Research | Groups | Alexander Fraser

Research Group Alexander Fraser

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing

Alexander Fraser

holds the Chair for Data Analytics & Statistics at TU Munich.

He is renowned for his work in machine learning approaches to machine translation, language modeling, and multilingual natural language processing. He focuses on addressing data sparsity and integrating linguistic and world knowledge in AI systems. Additionally, he collaborates with language communities to develop technology for their languages. His contributions to natural language processing and machine learning emphasize both theoretical advancements and practical applications.

Team members @MCML

Link to Lukas Edman

Lukas Edman

Dr.

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Faeze Ghorbanpour

Faeze Ghorbanpour

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Katharina Hämmerl

Katharina Hämmerl

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Wen Lai

Wen Lai

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Shu Okabe

Shu Okabe

Dr.

Data Analytics & Statistics

B2 | Natural Language Processing

Publications @MCML

[16]
L. Edman, H. Schmid and A. Fraser.
CUTE: Measuring LLMs' Understanding of Their Tokens.
Conference on Empirical Methods in Natural Language Processing (EMNLP 2024). Miami, FL, USA, Nov 12-16, 2024. To be published. Preprint at arXiv. arXiv.
Abstract

Large Language Models (LLMs) show remarkable performance on a wide variety of tasks. Most LLMs split text into multi-character tokens and process them as atomic units without direct access to individual characters. This raises the question: To what extent can LLMs learn orthographic information? To answer this, we propose a new benchmark, CUTE, which features a collection of tasks designed to test the orthographic knowledge of LLMs. We evaluate popular LLMs on CUTE, finding that most of them seem to know the spelling of their tokens, yet fail to use this information effectively to manipulate text, calling into question how much of this knowledge is generalizable.

MCML Authors
Link to Lukas Edman

Lukas Edman

Dr.

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[15]
W. Lai, V. Hangya and A. Fraser.
Style-Specific Neurons for Steering LLMs in Text Style Transfer.
Conference on Empirical Methods in Natural Language Processing (EMNLP 2024). Miami, FL, USA, Nov 12-16, 2024. To be published. Preprint at arXiv. arXiv.
Abstract

Text style transfer (TST) aims to modify the style of a text without altering its original meaning. Large language models (LLMs) demonstrate superior performance across multiple tasks, including TST. However, in zero-shot setups, they tend to directly copy a significant portion of the input text to the output without effectively changing its style. To enhance the stylistic variety and fluency of the text, we present sNeuron-TST, a novel approach for steering LLMs using style-specific neurons in TST. Specifically, we identify neurons associated with the source and target styles and deactivate source-style-only neurons to give target-style words a higher probability, aiming to enhance the stylistic diversity of the generated text. However, we find that this deactivation negatively impacts the fluency of the generated text, which we address by proposing an improved contrastive decoding method that accounts for rapid token probability shifts across layers caused by deactivated source-style neurons. Empirical experiments demonstrate the effectiveness of the proposed method on six benchmarks, encompassing formality, toxicity, politics, politeness, authorship, and sentiment.

MCML Authors
Link to Wen Lai

Wen Lai

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Viktor Hangya

Viktor Hangya

Dr.

* Former member

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[14]
W. Lai, M. Mesgar and A. Fraser.
LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback.
Findings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024). Bangkok, Thailand, Aug 11-16, 2024. URL.
MCML Authors
Link to Wen Lai

Wen Lai

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[13]
A. Dimmelmeier, H. Doll, M. Schierholz, E. Kormanyos, M. Fehr, B. Ma, J. Beck, A. Fraser and F. Kreuter.
Informing climate risk analysis using textual information - A research agenda.
Workshop Natural Language Processing meets Climate Change (ClimateNLP 2024) at the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024). Bangkok, Thailand, Aug 11-16, 2024. URL.
MCML Authors
Link to Malte Schierholz

Malte Schierholz

Dr.

Social Data Science and AI Lab

C4 | Computational Social Sciences

Link to Bolei Ma

Bolei Ma

Social Data Science and AI Lab

C4 | Computational Social Sciences

Link to Jacob Beck

Jacob Beck

Social Data Science and AI Lab

C4 | Computational Social Sciences

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Frauke Kreuter

Frauke Kreuter

Prof. Dr.

Social Data Science and AI Lab

C4 | Computational Social Sciences


[12]
M. Di Marco, K. Hämmerl and A. Fraser.
A Study on Accessing Linguistic Information in Pre-Trained Language Models by Using Prompts.
Conference on Empirical Methods in Natural Language Processing (EMNLP 2023). Singapore, Dec 06-10, 2023. DOI.
MCML Authors
Link to Katharina Hämmerl

Katharina Hämmerl

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[11]
V. Hangya, S. Severini, R. Ralev, A. Fraser and H. Schütze.
Multilingual Word Embeddings for Low-Resource Languages using Anchors and a Chain of Related Languages.
Conference on Empirical Methods in Natural Language Processing (EMNLP 2023). Singapore, Dec 06-10, 2023. DOI.
MCML Authors
Link to Viktor Hangya

Viktor Hangya

Dr.

* Former member

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Hinrich Schütze

Hinrich Schütze

Prof. Dr.

Statistical NLP and Deep Learning

B2 | Natural Language Processing


[10]
W. Lai, A. Chronopoulou and A. Fraser.
Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation.
Findings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2023). Singapore, Dec 06-10, 2023. DOI.
MCML Authors
Link to Wen Lai

Wen Lai

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[9]
W. Lai, V. Hangya and A. Fraser.
Extending Multilingual Machine Translation through Imitation Learning.
Preprint at arXiv (Nov. 2023). arXiv.
MCML Authors
Link to Wen Lai

Wen Lai

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Viktor Hangya

Viktor Hangya

Dr.

* Former member

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[8]
V. Hangya and A. Fraser.
LMU at HaSpeeDe3: Multi-Dataset Training for Cross-Domain Hate Speech Detection.
Final Workshop of the 8th evaluation campaign EVALITA 2023. Parma, Italy, Sep 07-08, 2023. PDF.
MCML Authors
Link to Viktor Hangya

Viktor Hangya

Dr.

* Former member

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[7]
Y. Liu, A. Chronopoulou, H. Schütze and A. Fraser.
On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss.
20th International Conference on Spoken Language Translation (IWSLT 2023). Toronto, Canada, Jul 09-14, 2023. DOI.
MCML Authors
Link to Yihong Liu

Yihong Liu

Statistical NLP and Deep Learning

B2 | Natural Language Processing

Link to Hinrich Schütze

Hinrich Schütze

Prof. Dr.

Statistical NLP and Deep Learning

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[6]
K. Hämmerl, B. Deiseroth, P. Schramowski, J. Libovický, C. A. Rothkopf, A. Fraser and K. Kersting.
Speaking Multiple Languages Affects the Moral Bias of Language Models.
Findings of the 61th Annual Meeting of the Association for Computational Linguistics (ACL 2023). Toronto, Canada, Jul 09-14, 2023. DOI.
MCML Authors
Link to Katharina Hämmerl

Katharina Hämmerl

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[5]
K. Hämmerl, A. Fastowski, J. Libovický and A. Fraser.
Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity.
Findings of the 61th Annual Meeting of the Association for Computational Linguistics (ACL 2023). Toronto, Canada, Jul 09-14, 2023. DOI.
MCML Authors
Link to Katharina Hämmerl

Katharina Hämmerl

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[4]
A. Chronopoulou, D. Stojanovski and A. Fraser.
Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation.
6th Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2023) at the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023). Dubrovnik, Croatia, May 02-06, 2023. DOI.
MCML Authors
Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[3]
H. S. Saadi, V. Hangya, T. Eder and A. Fraser.
Comparative Analysis of Cross-lingual Contextualized Word Embeddings.
2nd Workshop on Multi-lingual Representation Learning (MRL 2022) at the Conference on Empirical Methods in Natural Language Processing (EMNLP 2022). Abu Dhabi, United Arab Emirates, Nov 07-11, 2022. DOI.
MCML Authors
Link to Viktor Hangya

Viktor Hangya

Dr.

* Former member

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[2]
V. Hangya, H. S. Saadi and A. Fraser.
Improving Low-Resource Languages in Pre-Trained Multilingual Language Models.
Conference on Empirical Methods in Natural Language Processing (EMNLP 2022). Abu Dhabi, United Arab Emirates, Nov 07-11, 2022. DOI.
MCML Authors
Link to Viktor Hangya

Viktor Hangya

Dr.

* Former member

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing


[1]
W. Lai, A. Chronopoulou and A. Fraser.
m4 Adapter: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter.
Findings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2022). Abu Dhabi, United Arab Emirates, Nov 07-11, 2022. DOI.
MCML Authors
Link to Wen Lai

Wen Lai

Data Analytics & Statistics

B2 | Natural Language Processing

Link to Alexander Fraser

Alexander Fraser

Prof. Dr.

Data Analytics & Statistics

B2 | Natural Language Processing