The authors formulate batch-level expert selection as minimizing the squared change in a mixture-of-experts layer’s output when some experts are not loaded.
Predicting expert removal error makes batched MoE decoding more efficient
BASE coordinates expert loading across concurrent requests by predicting which omissions would most alter each mixture-of-experts layer’s output.
Academic
Ali Abbasi · Justin Shi · Soheil Kolouri
Vanderbilt University
Research Digest··2 min read
Abbasi, Shi and Kolouri address a mismatch in mixture-of-experts serving: each token activates few experts, but a batch of tokens can collectively require weights from much of the expert pool.
Why this paper
From Vanderbilt University
In one line
BASE selects experts by predicting removal error, improving MoE decoding accuracy by 29.5 points under tight batch budgets.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§