← All concepts

scaling law

A scaling law is a mathematical formula that predicts how a model's performance will improve as you increase resources like model size, training data, or compute. Think of it as a recipe: if you know the quantity of your ingredients (parameters, tokens, compute) and how they scale, the law tells you what quality of result to expect, allowing you to plan experiments efficiently.

The papers show that scaling laws are emerging across diverse areas, from standard language model pretraining to reinforcement learning and physiological waveform modeling. However, they also reveal important nuances: the optimal mix of synthetic data increases with compute, and adding more tools helps more than adding more tasks. Crucially, one paper warns that a single best-fit curve can be dangerously misleading, proposing certified prediction intervals instead, while others extend laws to account for quantization or latent prediction errors in robotics.

Papers
32
Released code
0
First seen
Oct 2026
Latest
Nov 2026

32 papers in the last two months, against 0 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Language models can use compute better by stopping large models early

    Language model performance follows power-law scaling with model size, dataset size, and compute, making larger models more sample-efficient.

    Nov 2026
  2. 2AI trained on simulated data can reconstruct unfamiliar medical scans

    Training on physics-simulated synthetic data unlocks scaling laws for zero-shot MRI reconstruction.

    Nov 2026
  3. 3Latent feedback cuts compute by up to 48% compared to training more epochs

    Latent feedback amplifies compute and data, reaching a target loss with up to 48% less compute or 47% less unique data than more epochs.

    Nov 2026

Who is working on it

Stanford University 3Google DeepMind 3Cornell University 2Princeton University 2Carnegie Mellon University 2Tsinghua University 2

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.