load balancing
- Papers
- 6
- Released code
- 0
- First seen
- Oct 2026
- Latest
- Nov 2026
6 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Feedback control stabilizes expert loads in extremely sparse language models
ID Balancing reduces expert load imbalance in extremely sparse MoE training using an integral-derivative controller.
Oct 2026 - 2Feedback control theory yields simpler MoE load balancer that adapts to shifting data
A pure integral feedback controller achieves the best load balance for MoE training under shifting data distributions.
Nov 2026 - 3AI models run faster when busy experts share chip resources
Fuse-All MoE fuses expert computation and communication to reduce MoE inference latency by up to 21.3% on TPUs.
Nov 2026
How fast it is moving
Often studied with
mixture of expertsA neural architecture with many specialized sub-networks, only a few activated per input via a router, balancing capacity and efficiency.4 togethercontinual learningA learning paradigm where a model sequentially acquires knowledge from multiple tasks without significantly forgetting earlier ones.1 togethermodel routingDirecting each query or subtask to the most suitable model to optimize cost, accuracy, or quality.1 togetherThe papers
Most central to this idea first, not most recent.
- Academiccs.LG
Feedback control theory yields simpler MoE load balancer that adapts to shifting data
A pure integral feedback controller achieves the best load balance for MoE training under shifting data distributions.
Umeå University · Nov 2026
- Industrycs.LG
AI models run faster when busy experts share chip resources
Fuse-All MoE fuses expert computation and communication to reduce MoE inference latency by up to 21.3% on TPUs.
Meta SuperIntelligence Lab · Nov 2026
- Chinese Techcs.LG
Feedback control stabilizes expert loads in extremely sparse language models
ID Balancing reduces expert load imbalance in extremely sparse MoE training using an integral-derivative controller.
Qwen Team, Alibaba Token Hub · Oct 2026
- Top Universitycs.DC
Co-scheduling CPU preprocessing and GPU execution slashes training stalls for multimodal models
DataHelix minimizes GPU idle time in multimodal training by scheduling data preprocessing to meet per-rank deadlines.
Shanghai Jiao Tong University, Xiamen University · Nov 2026
- Chinese Techcs.LG
Mask-efficient distributed attention speeds up long-context training by up to 3.7x
FlashMask-3 improves throughput for long-context training with structured attention masks on Hopper GPUs.
Baidu Inc. · Oct 2026
- Big Techcs.LG
Tree-based routing balances expert use without auxiliary loss in MoE models
BRANCH-MoE uses a binary decision tree with EMA anchors to balance expert utilization without auxiliary losses.
Google Research, University of Southern California · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.