This paper shows that attempts to erase a concept (e.g., horse) from a text-to-image while preserving related concepts (e.g., donkey) face a fundamental trade-off rooted in the model's internal representations. The authors prove mathematically that if two concepts overlap in cross-attention activation space, any robust erasure must damage the neighbor concept, with the damage scaling linearly with the overlap. Experiments across 13 unlearning methods confirm this trade-off: methods that achieve strong erasure cause over 75% reduction in neighbor generation, while others preserve neighbors but leak the target concept under indirect prompts. The finding reframes unlearning as a Pareto optimization problem rather than a search for perfect erasure.
Concept entanglement forces a fundamental trade-off in diffusion model unlearning
The authors prove a geometric lower bound showing that robust erasure of a target concept inevitably damages semantically related concepts, and validate it across thirteen methods on Stable Diffusion and SDXL.
Academic
Yian Wang · Ali Ebrahimpour-Boroojeny · Hari Sundaram · Varun Chandrasekaran
University of Illinois Urbana-Champaign
Research Digest··2 min read
In one line
Concept unlearning in diffusion models is fundamentally limited by the geometric overlap of target and neighbor concepts in activation space.
Before you read this
An idea this paper builds onRead it as
The paper shows that the failure of concept unlearning methods—either leaking the target under indirect prompts or degrading related concepts—is not a bug of individual algorithms but a structural consequence of overlapping concept representations in activation space.
The authors formalize concepts as regions in the cross-attention activation space of and define an entanglement coefficient (\hat{\kappa}) to quantify overlap between target and neighbor concepts.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.