New system halves compute waste during AI response generation training
Researchers developed FlexFloor, a system that cuts wasted compute time in reinforcement-learning post-training for mixture-of-experts language models. By releasing workers that have finished generating their responses while keeping their experts available for ongoing responses, FlexFloor boosts response tokens per occupied compute time by 65% and overall throughput by 53% with no loss in output quality.
