Models that fill in text by rewriting can finalize any part. With a small response window, they sometimes lock in an answer too early, before supporting reasoning is done, and that sends the rest of generation off track.
A method called HiLo-RD uses the same model with a larger window as teacher for the small-window version, aligning their internal states at positions where words get fixed, so the small-window model learns to postpone decisions like the large-window one.
On the GSM8K math benchmark, accuracy rose 9.4 percentage points and average readable length fell about 25%. Gains also appeared for code generation without code-specific training: constrained decoders became more accurate and more concise.