Research · Language models
Diffusion models learn better short-budget reasoning from their longer-window trajectories
HiLo-RD: Cross-Budget Self-Distillation for Masked Diffusion Language Models
HiLo-RD transfers a model’s own large-window representations into small-window decoding, improving GSM8K accuracy while reducing readable output length.
GSM8K: 78.24% accuracy (as reported)
The Hong Kong University of Science and Technology (Guangzhou), Zhejiang University, Nanyang Technological University