The authors run RL fine-tuning on four Qwen-family LLMs using separate inference and training engines, both operating on BF16 tensors with identical weights.
Aligning rounding locations between inference and training stabilizes BF16 LLM reinforcement learning
Selectively retaining intermediates in FP32 for RoPE, SwiGLU, and residual RMSNorm prevents training collapse without changing kernels or the learning objective.
Independent
Yijing Ke · Tianle Cai · Ruosong Wang
Research Digest··2 min read
The authors investigate why BF16 reinforcement learning for LLMs can collapse despite identical weights, and show that selectively aligning rounding locations between inference and training is enough to stabilize training.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.