Optimizer Unification and Efficiency
Unifying optimizer variants (e.g., Muon) through composed matrix normalizations to reduce pretraining token use.
3 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
News without the noise
Unifying optimizer variants (e.g., Muon) through composed matrix normalizations to reduce pretraining token use.
3 papers
The written synthesis of this thread is for subscribers. Subscribe.