The authors formalize recent Muon variants as alternating compositions of LMOs, where each LMO maps a momentum matrix to a steepest descent direction under a norm ball.
Composed matrix normalizations unify Muon optimizers and cut pretraining token use
Kim, Ozkara and Park introduce chained LMOs, prove which compositions retain convergence, and propose TensorChain, a layer-wise normalization that outperforms Muon in Qwen3 pretraining.
Big Tech
Sungyoon Kim · Kaan Ozkara · Youngsuk Park
Stanford University · Amazon Annapurna Labs
Research Digest··3 min read
The authors unify recent Muon optimizer variants as chained linear minimization oracles (LMOs), compositions of norm-ball projection steps such as the matrix sign.
Why this paper
From Amazon Annapurna Labs and Stanford University
In one line
TensorChain, a chained LMO optimizer, achieves 9.6% average token savings over Muon at matched validation loss in LLM pretraining.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§