Alignment midtraining (AMT) adds a pretraining-like phase on alignment-relevant documents so that later post-training generalizes outside its own distribution. The authors stress-tested this idea at up to 110B parameters and 1B midtraining tokens. They found that midtraining can shape a model's motivation when the later finetuning data is ambiguous, but the effect is fragile.
In one experiment, finetuning on ambiguous examples alone produced Charter-following behavior 90% of the time. Changing just 2% of those finetuning examples to explicitly favor a competing motivation reversed the behavior, with 45K conflicting tokens overriding 190M midtraining tokens. The authors also found that rules are only robustly learned if demonstrations appear in either midtraining or post-training data. They conclude that public evidence does not yet justify claims that midtraining solves core alignment difficulties.