From Growing to Looping: A Unified View of Iterative Computation in LLMs

MCML Authors

Emmanouil Angelis

→ Group Stefan Bauer
 Algorithmic Machine Learning & Explainable AI

Stefan Bauer

Prof. Dr.

Core PI

Algorithmic Machine Learning & Explainable AI

Abstract

Looping, reusing a block of layers across depth, and depth growing, training shallow-to-deep models by duplicating middle layers, have both been linked to stronger reasoning, but their relationship remains unclear. We provide a mechanistic unification: looped and depth-grown models exhibit convergent depth-wise signatures, including increased reliance on late layers and recurring patterns aligned with the looped or grown block. These shared signatures support the view that their gains stem from a common form of iterative computation. Building on this connection, we show that the two techniques are adaptable and composable: applying inference-time looping to the middle blocks of a depth-grown model improves accuracy on some reasoning primitives by up to , despite the model never being trained to loop. Both approaches also adapt better than the baseline when given more in-context examples or additional supervised fine-tuning data. Additionally, depth-grown models achieve the largest reasoning gains when using higher-quality, math-heavy cooldown mixtures, which can be further boosted by adapting a middle block to loop. Overall, our results position depth growth and looping as complementary, practical methods for inducing and scaling iterative computation to improve reasoning.

inproceedings KAM+26

---

ICML 2026

43rd International Conference on Machine Learning. Seoul, South Korea, Jul 06-11, 2026. To be published. Preprint available.

---

Authors

F. Kapl • E. Angelis • K. Maile • J. von Oswald • S. Bauer

---

Links

URL

---

Research Area

 A1 | Statistical Foundations & Explainability