Language models are usually trained once and then frozen, so improving them requires another expensive training run. This paper shows that a 27B parameter model can rewrite its own internal weights using only its own reasoning and forward passes, no training needed.
The model diagnoses failures by comparing its hidden states and edits specific weights. It tests each edit on practice problems and only keeps changes that improve its performance. The method runs on a single GPU in hours.
The self-edited model became 6.6 points more accurate on 490 unseen problems. It also repaired a corrupted copy of itself, restoring accuracy from 7% to 70% on image tasks it was never trained on. This ability only worked at larger model sizes.
