The paper introduces Loop Scaling Laws, the first scaling law that jointly models recurrence and sparsity alongside model size and data. The core contribution is a bounded recurrence mapping whose effective-parameter gain depends on the number of experts, capturing diminishing returns from additional recurrent passes and how sparsity raises and sustains that gain. The law predicts held-out loss more accurately than prior alternatives and recovers standard dense and MoE scaling laws as special cases.
Downstream evaluations show complementary benefits: sparsity delivers roughly 3x active-parameter efficiency, while recurrence delivers roughly 2x total-parameter efficiency on reasoning. At trillion-token scale, a looped MoE with law-derived recurrence matches a roughly 2x larger non-looped MoE at matched training compute, and recurrence also enables . This makes the result directly useful for choosing recurrence and sparsity under compute and memory constraints.