The authors propose CFG-Policy, which maintains two distinct generative policies: one for exploration (frozen) and one for exploitation (trainable).
Separating exploration and exploitation policies with classifier-free guidance improves diffusion RL fine-tuning
By composing two generative policies during sampling, CFG-Policy preserves behavioral diversity while optimizing for rewards across online and offline-to-online settings.
Academic
Shutong Ding · Zhaoyang Liu · Yimiao Zhou · Jingya Wang · Jiayuan Gu · Ye Shi
ShanghaiTech University · InstAdapt
Research Digest··3 min read
Thread:Diffusion RL Fine-tuning
The authors introduce CFG-Policy, a plug-and-play framework that decouples exploration and exploitation into two separate diffusion or flow policies combined via classifier-free guidance during action generation.
Why this paper
From ShanghaiTech University and InstAdapt
In one line
CFG-Policy combines separate exploration and exploitation diffusion policies via classifier-free guidance to improve RL performance while preserving behavioral diversity.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§