← All concepts

catastrophic forgetting

When a model learns a new task, it can overwrite what it already knew. That loss of old skills is a central obstacle in continual learning, and it shows up in language models, robots, and even deepfake detectors. Researchers combat it by remembering old data, constraining parameter changes, or using softer training signals that let old knowledge coexist with new abilities.

Across these studies, a shared lesson emerges: forgetting is not merely a matter of accuracy. It can be tracked through entropy shifts, decomposed into distinct interference patterns, or reduced by restricting updates to less important directions and anchoring critical parameters. Soft, model-generated targets also help. Yet even successful methods can worsen demographic unfairness, so stability must be measured beyond average performance.

Papers
7
Released code
0
First seen
Oct 2026
Latest
Nov 2026

7 papers in the last two months, against 0 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Continual learning's core challenges stem from a single evolving update-behavior interaction

    A token-level update-behavior interaction unifies data selection, forgetting mechanisms, and plasticity loss in continually trained language models.

    Oct 2026
  2. 2Restricting weight updates to non-principal subspace preserves old skills while learning new ones

    Subspace Tuning prevents catastrophic forgetting by restricting LLM fine-tuning updates to non principal weight subspaces.

    Oct 2026
  3. 3Language models can learn new tasks while forgetting less

    Output entropy can replace reference-model KL as a real-time forgetting signal, and dynamic replay weighting reduces catastrophic forgetting in LLM fine-tuning.

    Oct 2026

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.

catastrophic forgetting · Research concepts | Zotpaper