Fast generative models improve with RL when noise split is optimized geometrically

The method selects the admissible policy that minimizes conditional KL divergence induced by velocity updates, yielding higher reward in 4-step image and video generation.

Chinese Tech

Zhuguanyu Wu · Congjia Chen · Ruihao Gong · Yang Yong · Shiqiao Gu · Yi Xu · +1 more

Beihang University · Sensetime Research

Research Digest·2 November 2026·3 min read

Illustration: Fast generative models improve with RL when noise split is optimized geometrically

In one line

Geo-FSRL selects stochastic policies minimizing conditional KL divergence, achieving higher reward scores than prior methods in few-step flow models.

Before you read this

Two ideas this paper builds on

group relative policy optimizationA reinforcement learning method that optimizes policy by computing advantages based on group-level reward statistics.160 papers →diffusion modelA type of generative model that learns to reverse a gradual noising process to create data.162 papers →

Read it as

Flow models generate images or videos by gradually removing noise over many steps. Recent methods use only a few steps for speed, but training them with reinforcement learning requires adding randomness to each step. The way this randomness is set strongly affects performance.

That's the problem Geo-FSRL solves. It chooses the amount of randomness in each step to minimize how much the model's predictions change the random distribution. This choice can be computed exactly with a simple formula.

In tests on image and video generation using four-step models, Geo-FSRL achieved higher reward scores than previous approaches while keeping the fast inference. This shows that carefully designing the random transitions in few-step flow models can significantly improve reinforcement learning outcomes.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

group relative policy optimization flow matching conditional KL policy selection few-step flow models gradient-scale matching diffusion model