Iterative Reasoning Transformers

How this thread developed

1. Oct 2026

   Loop-native residuals let Transformers reason across tens of thousands of iterations

   Proposes InfiLoop with loop-native residuals enabling Transformers to reason across tens of thousands of iterations.

2. Oct 2026 · NVIDIA

   A training recipe for looped language models that matches non-recurrent models at one loop and improves with more compute.

   Removes the one-loop performance tax so looped language models match non-recurrent models and improve with more compute.

3. 6 further papers

   Oct 2026 · Technical University of Munich, Helmholtz AI

   Reusing computations across model depth favors learning from the prompt

   Shows that models with shared or copied layers (reusing computations across depth) favor in-context learning, an architectural modification for iterative reasoning.

4. Oct 2026 · The Hong Kong University of Science and Technology, Huawei Noahah’s Ark Lab

   Reasoning models usually inherit “wait” from training data, not reinforcement learning

   Traces the origin of the 'wait' token in reasoning traces, showing it typically comes from training data rather than RL.

5. Oct 2026 · Apple

   Halting heads can overstate compute savings in looped language models

   Provides pretrained looped language model evidence that halting heads overstate compute savings, informing when to stop iterative computation.

6. 1 further paper

   Nov 2026 · Microsoft Applied Sciences Group

   Reusing layers in trained AI models pays off only for the smallest models

   Finds that layer looping only benefits the smallest models, clarifying when iterative depth helps versus unlooped compute.

7. Nov 2026 · Peking University, IQuest Research

   Training shifts a small theorem prover toward fewer, larger proof steps

   Uses Monte Carlo tree search in Lean 4 to train a theorem prover, analyzing how training shifts proof step size, directly exploring iterative reasoning.

7 of 14 papers shown