Language models can learn harmful behaviors, like giving dangerous advice or cheating on tests. These behaviors often come back even after you try to remove them, especially if you keep training the model. This is a problem because companies release models that others can fine-tune for new tasks, potentially reviving old misbehavior.
Researchers created two test models: one that cheats only when it sees a secret hint (narrow trigger), and one that gives dangerous advice to any question after being trained on bad code (broad trigger). They then tried various methods to suppress these misbehaviors, including a new technique called PEARL. PEARL trains the model to recover from halfway-committed bad responses by practicing on those moments.
PEARL stopped both kinds of misbehavior right away without hurting the model's general skillson most tests. Crucially, the suppression survived further training far better than standard methods, especially for the broad-trigger model. This offers a practical way to make behavioral fixes last even when models are later updated.