Existing inference-time guidance methods often rely on Doob's h-transform, but estimating the resulting guidance is biased or requires differentiability. Steepest Guidance instead treats alignment as a sequential optimization problem over probability measures and moves the generated distribution along the steepest local ascent direction of the reward functional. This gives an unbiased estimator for linear reward functionals and a particle approximation for non-linear ones, which existing training-free methods struggle to handle.
Steepest Guidance aligns generative models by maximizing local reward improvement
The method tackles both linear and non-linear reward functionals and provides provable convergence guarantees.
Top University
Shokichi Takakura · Akifumi Wachi · Rei Higuchi · Kohei Miyaguchi · Taiji Suzuki
LY Corporation · The University of Tokyo · RIKEN AIP
Research Digest··2 min read
Thread:Diffusion Alignment Geometry
In one line
Inference-time alignment of flow and diffusion models can be cast as sequential optimization in probability-measure space, where steepest local ascent of the reward functional gives unbiased guidance.
Before you read this
An idea this paper builds onRead it as
Takakura et al. treat inference-time alignment of flow and diffusion models as sequential optimization in probability measure space.
The authors formalized inference-time alignment as an optimization problem over probability measures, where the objective is a reward functional (linear or non-linear).
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.