Foundation models are built in stages: pretraining on large text, fine-tuning on examples, and from feedback, each with its own algorithm and data. This separation complicates training and limits how choices interact across stages. The authors propose that all these stages can be seen as one statistical fitting task, where the model adjusts to match a mix of fixed data and its own generated responses.
They derive a single maximum-likelihood rule, which means selecting the explanation most consistent with the data, to cover pretraining, fine-tuning, and suitable reinforcement learning. Instead of switching algorithms, the trainer always uses the same likelihood objective, but varies which data it learns from: static texts, demonstrations, or generated responses that score well on a reward signal. This lets stages overlap and blend smoothly.
In tests, this unified approach improved the tradeoff between accuracy and compute, expanded task coverage, and did not sacrifice broader abilities. It also gave a stronger basis for later learning on new tasks. The work suggests that choosing data mixtures and timing, not algorithm changes, is the key lever in training foundation models.