are often adapted to specialized fields like mathematics by continuing to train them on domain-specific texts. The standard training method treats every word equally, which can cause some models to perform worse on math after training.
To address this, the researchers developed On-Policy Mid-Training Distillation (OPMD). Instead of learning from fixed example sentences, the student model generates its own completions and receives feedback from a teacher model that scores each word in the generated text. This focuses training on the model's actual mistakes.
On math reasoning tests, OPMD avoided the performance drops that standard training caused for some models and improved outcomes. This provides a way to adapt large models to specialized domains without harming their existing abilities.