Compute-Efficient Scaling
Investigates scaling laws and training recipes for language models under fixed data and compute budgets, including overfitting from repeated data and compute-optimal stopping criteria.
4 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
How this thread developed
Nov 2026 · Johns Hopkins University, OpenAI
Language models can use compute better by stopping large models early
Finds that stopping large models early can be compute-optimal within a fixed training budget.
2 further papers
Nov 2026 · University of Michigan, KAIST
Language models can stretch limited training data by masking inputs
Shows that masking inputs improves LM training when data is fixed but compute is abundant.
2 of 4 papers shown