Language model pretraining loss is governed by the effective learning rate, defined as the ratio of learning rate to parameter norm. When runs with markedly different learning rate and norm trajectories are constructed to match ELR step by step, their loss trajectories collapse to nearly identical curves, with mean discrepancies around a few times 10^-3, below the seed-to-seed variation seen in a representative configuration. This holds across optimizers, architectures, datasets, and model scales.
Collapse precision depends on normalization design: removing QK-Norm weakens the collapse, and fixing RMSNorm gains weakens it further. Weight decay and Hyperball affect loss dynamics mainly through the ELR schedules they induce. Using ELR instead of LR also lets a fitted functional scaling law transfer across norm-control methods and explains delayed acceleration, making ELR a common coordinate for LR scheduling, norm control, and loss.