When language models are updated during use, current evaluations only count updates or measure performance immediately after. This hides whether the model really learns, remembers later, or damages other abilities.
The authors propose a Control Map that records the order of updates before each answer and separately tests immediate learning (acquisition), later recall (retention), and preservation of unrelated behavior (locality).
Different schedules with the same update count gave accuracy ranging from 19.7% to 34.2%. Scheduling matters, and learning an edit does not guarantee it lasts or stays contained. Developers should evaluate updates more carefully.