The work measures repeated training tokens against two baselines: one epoch over the same data and fresh data at equal compute. Across model sizes and budgets, the penalty relative to fresh data follows z=(R-1)N_total/U, combining extra epochs, parameter count, and unique-token count.
A second epoch delivers 0.87 to 1.02 times the value of fresh tokens. Repeats fall to half-value near a critical epoch count of 2.7(D/N)^0.24. With a fixed corpus, the loss-minimizing count drops from about 15 epochs for 127M parameters to 4 for 2B parameters.
Counts are not sufficient by themselves. Consecutive shard replay increased loss by up to 0.46 bits per byte, while concentrating repeats and repeating lower-entropy text also hurt. This helps reconcile conflicting scaling results by showing that model-size trends depend on what an experiment holds fixed.