Language models can learn faster when tiny updates survive rounding

The authors report that randomized rounding closes a numerical-format convergence gap in reinforcement learning without the extra model drift caused by increasing the learning rate.

Big Tech

Mahdi Nazemi · Oleg Rybakov · Tijmen Blankevoort · Asit Mishra · Carlo del Mundo

NVIDIA

Research Digest·11 November 2026·2 min read

Illustration: Language models can learn faster when tiny updates survive rounding

In one line

Stochastic rounding closes the BF16-FP16 convergence speed gap in RL post-training without increasing KL divergence.

Before you read this

An idea this paper builds on

reinforcement learning with verifiable rewardsReinforcement learning that uses rewards from objectively checkable signals like correct answers or verification feedback.156 papers →

Read it as

When fine-tuning large language models with reinforcement learning, BF16 saves memory but is less precise than FP16. At low learning rates, small weight updates get lost in rounding, slowing convergence.

The researchers solved this by using stochastic rounding for BF16 weights: each weight is rounded up or down based on a probability, so even small updates affect the expected weight.

This matches FP16's convergence speed without increasing model drift from its original behavior, improving training efficiency while preserving learned capabilities.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

stochastic rounding reinforcement learning with verifiable rewards mixed precision training weight sticking