Language models train faster when generated samples arrive more evenly

The authors report up to 3.22 times higher throughput for an asynchronous training pipeline, with scheduling alone reducing mean iteration time by 10.9%.

Chinese Tech

Salah Chikhi · Luis Zuin · Entong Li · Zeng Li

École polytechnique · Huawei Technologies · Shenzhen Research Institute of Big Data

Research Digest·9 November 2026·2 min read

Thread:Compute-Efficient Scaling

Illustration: Language models train faster when generated samples arrive more evenly

In one line

An offline scheduler that balances generation batching efficiency and trainer queue stability triples async LLM reinforcement learning throughput.

Before you read this

Two ideas this paper builds on

reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.281 papers →large language modelA neural network trained on massive text corpora to generate coherent text and perform diverse language tasks.290 papers →

Read it as

Training a (LLM) with (RL) uses generated responses as learning data. The generation step is inefficient because it wastes time waiting for long outputs and suffers from irregular delivery of samples to the training step.

The authors built a scheduler that predicts each response's length, groups similar ones for efficient generation, and sequences them to supply the trainer smoothly without bursts or idle waiting.

In tests, the scheduler tripled system throughput and cut iteration time by 76.8% compared to a leading baseline. The scheduling alone accounted for a 10.9% improvement without hurting model quality, and the system was used to compress a real-world training job from over three months to two.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

asynchronous reinforcement learning rollout scheduling reinforcement learning large language model output length prediction bounded policy lag