Diffusion-based large language models used as agents in embodied environments suffer from retry loops: they keep re-issuing a failed action because the masked decoding process over-weights actions that are already salient in the context, regardless of the task. The authors model this as a task-blind corruption that inflates probabilities by a factor depending only on the state and action, not the goal. They propose a training-free correction: instead of asking the model which action fits the task (forward scoring), they ask whether the state plus a candidate action still explains the task (reverse scoring). Under their assumptions, the task-blind factor cancels from this reverse conditional. They implement Reflect Reverse on four embodied benchmarks and show it improves task success and progression rates over forward-scoring baselines.
Reverse scoring helps diffusion agents escape repeated failed actions
Reflect Reverse reranks candidate actions by how well they explain the task, reducing retry loops without additional training.
Chinese Tech
Jiacheng Qiu · Christopher E. Mower · Jan Peters · Haitham Bou-Ammar · Matthieu Zimmer
Huawei Noah’s Ark Lab · Technical University of Darmstadt · UCL Centre for AI
Research Digest··3 min read
In one line
Diffusion language model agents retry failed actions due to task-blind bias; reverse scoring corrects this without retraining.
Before you read this
Three ideas this paper builds ondiffusion modelA type of generative model that learns to reverse a gradual noising process to create data.197 papers →test-time adaptationAdapting a deployed model during inference using new data or interactions, without a separate training phase.84 papers →verificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.341 papers →
Read it as
Qiu et al. identify a decoding pathology in diffusion language model agents: after an action fails, adaptive masked decoding can favor repeating that same action instead of responding to the failure.
The authors model action selection using a task description, interaction history and candidate next action.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.