Preference alignment for text-to-image often uses Diffusion-DPO, which optimizes in the full latent space without considering the data manifold. This causes off-manifold drift, degrading image quality and diversity. The authors propose APO, which introduces a geometry-aware anisotropic metric derived from the reference model's score field. APO strengthens regularization in manifold-normal directions (prone to drift) and relaxes it in tangent directions (safe semantic adjustments). Experiments on SD1.5 and SDXL show APO achieves over 60% win rate against existing methods, requires fewer training steps, and preserves generation diversity throughout training.
Geometry-aware alignment keeps diffusion models near natural image structure
APO uses a reference model’s local geometry to constrain risky diffusion updates while permitting safer semantic changes.
Chinese Tech
Lei Wang · Zhen Wang · Yuexiang Xie · Yaliang Li
Sun Yat-Sen University · Alibaba Group
Research Digest··3 min read
Thread:Diffusion Alignment Geometry
In one line
Anisotropic geometry-aware optimization prevents off-manifold drift in diffusion model preference alignment.
Before you read this
Two ideas this paper builds onRead it as
Wang et al. argue that standard preference optimization can gradually push diffusion models away from the low-dimensional structure associated with natural images, reducing quality and diversity.
The authors reinterpret Diffusion-DPO, a method that learns from pairs of preferred and dispreferred images, through the manifold hypothesis.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.