Home | Publications | JAH+26

Not All Nodes Are Created Equal: Homophily-Aware Stratification for Stable GNN Evaluation

MCML Authors

Abstract

Graph neural networks are widely used for transductive node classification, with accuracy<br>typically measured on randomly drawn train/validation/test splits. Reported accuracy has<br>been shown to shift substantially across different random splits of the same dataset, making<br>published comparisons between architectures unreliable. The classical remedy in non-<br>graph settings is stratified $k$-fold cross-validation, which ensures each test fold reflects<br>the full class distribution of the dataset. We argue that class stratification alone is<br>insufficient for graphs: nodes are not isolated but connected, and folds that differ in their<br>distribution of local neighbourhood homophily expose the model to systematically different<br>relational conditions that directly affect message-passing behaviour. The resulting cross-<br>fold variation reflects the homophily composition of each split, inflating reported variance<br>beyond what model behaviour alone would produce. To address this, we propose hp{}, a<br>topology-aware stratification procedure that treats node homophily as the primary<br>stratification axis, aligning folds with respect to local relational consistency alongside the<br>class marginal that standard stratification already controls. Stratifying on homophily alone<br>does not guarantee class balance, so hp{} incorporates class label as a secondary axis,<br>preserving class representativeness as a natural consequence of the procedure. We<br>evaluate hp{} on a broad benchmark suite comprising 15 node-classification datasets<br>spanning the full homophily spectrum and 8 GNN architectures. hp{} achieves the lowest<br>mean stability rank on 13 of 15 datasets, 1.48 overall against 2.33 for random $k$-fold,<br>while preserving class balance close to class-stratified splits and substantially better than<br>random. We argue that homophily-aware split construction merits broader adoption for<br>GNN evaluation.

inproceedings JAH+26

LoG 2026

5th Learning on Graphs Conference. Boston, MA, USA, Nov 20-22, 2026. To be published. Preprint available.

Authors

N. V. S. J. Jami • T. Altstidl • S. Müller • D. Zanca • B. Eskofier • H. Leutheuser

Links

arXiv GitHub

Research Area

 C1 | Medicine

BibTeXKey: JAH+26

#p-eskofier
Back to Top