The paper introduces Variational Stackelberg Discrete Diffusion (VSDD), a method that learns the forward corruption process in discrete . Instead of using fixed or uniform noise, VSDD selects corruptions based on how much they improve the denoiser after a training step, formulated as a leader-follower game where the leader (corruption process) anticipates the follower's (denoiser's) response.
A Stackelberg game framework lets discrete diffusion learn which token corruptions most improve the denoiser
The method, VSDD, trains a forward corruption process to maximize denoiser improvement after learning, not immediate reconstruction.
Industry
Dmitrii Moor · Federico Tomasi · Paul N. Bennett · Alice Wang · Mounia Lalmas
Spotify
Research Digest··2 min read
In one line
Learning which token corruptions best train a denoiser improves discrete diffusion generation across molecules, text, and playlists.
Before you read this
An idea this paper builds onRead it as
Discrete diffusion models can revise earlier predictions during generation, but their performance depends on the fixed forward noise process.
Moor et al. address the challenge of learning the forward corruption process in discrete .
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.