MoE Load Balancing
Methods for balancing expert utilization and routing in mixture-of-experts models under shifting data distributions.
5 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
How this thread developed
Nov 2026 · Umeå University
Feedback control theory yields simpler MoE load balancer that adapts to shifting data
Frames MoE load balancing as a feedback control problem and derives a simpler PID controller that adapts to shifting data.
3 further papers
Nov 2026
New training stack keeps mixture-of-experts AI models efficient at large scale
Uncovers token garbage collection issues in large-scale MoE training, providing evidence for load-balancing challenges
2 of 5 papers shown