When training a smaller language model from a larger one, usual methods feed the smaller model the larger model's own answers. That is a fixed target, so the smaller model never learns from its own mistakes. flips this: the smaller model generates text, and the larger model scores each token. But the larger model is less reliable on later tokens of the smaller model's text, so its advice gets noisier as the text goes on.
This work found that the larger model's guidance is trustworthy on the first few tokens, so they only teach the smaller model on the first fifth of every generated sequence. They also redesigned how the two models' token probabilities are compared. Instead of always using one fixed formula, they adapt it per token based on how much the two models agree there, switching between covering the bigger model's likely tokens and matching only shared ones.
On seven math benchmarks, this approach improved scores by up to 5.29 points over the standard on-policy method. The gain means that focusing on reliable early context and adapting the training signal gives a better small model than treating all tokens equally.