Tuning only the first denoising step restores lost diversity in distilled models

ReSeed adds a tiny bias adapter to the first student step, matching the teacher's velocity, and recovers most output diversity without retraining later steps.

Academic

Wenwu Tang · Dong Wang · Yun Cheng · Olga Saukh

Graz University of Technology · Complexity Science Hub Austria · Swiss Data Science Center

Research Digest·28 October 2026·2 min read

Illustration: Tuning only the first denoising step restores lost diversity in distilled models

In one line

Diversity loss in distilled diffusion models is concentrated in the first step and can be recovered by a lightweight first-step bias adapter.

Before you read this

Three ideas this paper builds on

distribution matching distillationA training method that aligns student and teacher output distributions to compress many-step generative models into efficient few-step versions.25 papers →diffusion modelA type of generative model that learns to reverse a gradual noising process to create data.152 papers →diffusion transformerA neural network architecture combining transformer layers with diffusion processes for high-quality image and video generation.59 papers →

Read it as

Distilled image generators run fast but often produce similar outputs from different random seeds, reducing variety for users.

The researchers traced the loss to the first step. They added learnable biases only to that step, trained to match the original model's average direction over the same interval.

Their method, ReSeed, improved measured diversity by 48.3% using just 8,000 training examples, while leaving the rest of the model unchanged. This shows that a small first-step fix can restore diversity without costly retraining.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

distribution matching distillation diffusion model flow matching diffusion transformer classifier-free guidance activation steering