← All threads

Pretraining Repetition Scaling

Investigates the scaling rules that govern the diminishing returns of repeating training tokens during language model pretraining.

2 papers

Where this stands

The written synthesis of this thread is for subscribers. Subscribe.