Large language models with mixture-of-experts (MoE) reduce per-token computation, but during batched decoding the set of experts needed across multiple requests grows large, causing memory bandwidth bottlenecks. Existing batch-aware methods select experts based on router weights or calibration statistics, which do not directly capture the impact on output quality.
BASE introduces a new criterion: the squared change in the MoE layer output if an expert's contribution is removed. To compute this before the expert is fetched, they train a lightweight linear predictor per expert during calibration that estimates its output energy from the token representation. Combined with router weights, this gives a removal cost per token, and experts with the highest aggregated cost across the batch are selected within a fixed budget. Custom GPU kernels implement the prediction and selection.
Across three MoE architectures, BASE improves average accuracy by 3 to 29.5 points over existing methods at comparable speed under tight expert budgets. At relaxed budgets, it achieves 22-60% higher throughput than dense inference while staying within 0.4 accuracy points. The method requires additional offline calibration but no retraining.