Optimizer Batch Scaling
Analyzes how optimizer performance and rankings change with batch size in large-scale pretraining, and implications for scaling rules.
3 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.