In image generation, powerful diffusion models produce high-quality results by slowly removing noise over many steps, which is slow. To speed them up, researchers train simpler student models that mimic the teacher in only a few steps, a process called distillation.
A popular method, , uses an extra fake model to track what the student produces. But it trained this fake model on the student's intermediate steps, causing flawed optimization signals that can persist even after the student's final outputs match the teacher's. This new approach trains the fake model only on the student's final outputs, ensuring the signals vanish when they should.
The method, called endpoint distribution matching distillation, consistently produces better images in 4 steps across several large models. This makes fast generation more practical without sacrificing quality.