The authors studied on-policy distillation (OPD) for diffusion models in the large-to-small regime, where a larger teacher model is distilled into a smaller student.
Large-to-small diffusion model distillation fails due to CFG error amplification, new method resolves it
The authors identify that classifier-free guidance in on-policy distillation amplifies distributional mismatches between a small student and large teacher, and propose Guidance-Folding Distillation (GFD) to absorb the teacher's guided policy directly.
Chinese Tech
Zhenxing Zhang · Jiayan Teng · Wenxu Wu · Zhuoyi Yang · Jiazheng Xu · Wendi Zheng · +3 more
Hefei University of Technology · Tsinghua University · Zhipu AI
Research Digest··3 min read
Zhang et al.
Why this paper
From Zhipu AI and 2 others
In one line
Folding a large diffusion teacher’s guided policy into a small student’s conditional branch prevents guidance error amplification and improves cross-scale distillation.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§