Interpretable Additive Tabular Transformer Networks

MCML Authors

David Rügamer

Prof. Dr.

Principal Investigator

Statistics, Data Science and Machine Learning

Abstract

Attention based Transformer networks have not only revolutionized Natural Language Processing but have also achieved state-of-the-art results for tabular data modeling. The attention mechanism, in particular, has proven to be highly effective in accurately modeling categorical variables. Although deep learning models recently outperform tree-based models, they often lack a complete comprehension of the individual impact of features because of their opaque nature. In contrast, additive neural network structures have proven to be both predictive and interpretable. Within the context of explainable deep learning, we propose Neural Additive Tabular Transformer Networks (NATT), a modeling framework that combines the intelligibility of additive neural networks with the predictive power of Transformer models. NATT offers inherent intelligibility while achieving similar performance to complex deep learning models. To validate its efficacy, we conduct experiments on multiple datasets and find that NATT performs on par with state-of-the-art methods on tabular data and surpasses other interpretable approaches.

article

---

Transactions on Machine Learning Research

May. 2024.

---

Authors

A. F. Thielmann • A. Reuter • T. Kneib • D. Rügamer • B. Säfken

---

Links

URL

---

Research Area

 A1 | Statistical Foundations & Explainability