Fast image and video generators made by distilling a slow, high-quality model can produce nearly identical outputs from different random seeds, a problem called diversity collapse. The authors discovered this happens because the fast model concentrates its internal signal into too few directions at one specific processing stage. They developed two fixes, one that fine-tunes a small adapter and another that works without any training, that restore variety without hurting quality.
Sped-up AI image generators lose diversity at a single layer
By probing internal activations, the authors locate the source of diversity collapse to a specific block and reintroduce variation with or without retraining.
Top University
Furkan Horoz · Ozgur Kara · Kaan Oktay · Adil Kaan Akan · Atakan Kara · Anand Bhattad · +1 more
University of Illinois at Urbana-Champaign · Johns Hopkins University
Research Digest··2 min read

In one line
Distilled generators collapse diversity because they concentrate activation variance into fewer directions at a single intermediate block.
Before you read this
Three ideas this paper builds ondiffusion modelA type of generative model that learns to reverse a gradual noising process to create data.197 papers →activation steeringActivation steering modifies a model's internal vector representations during inference to shift behavior, such as editing outputs, removing capabilities, or controlling trust.83 papers →mechanistic interpretabilityThe study of how neural networks compute through internal circuits, features, and causal interventions, rather than just input-output behavior.143 papers →
Read it as
The authors investigate why distilled diffusion models produce less varied outputs than their teachers.
The authors compared activation variances in teacher and student generators block by block across many seeds for identical prompts.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.