The authors designed a differentiable optimization method that jointly learns sparse support patterns and quantized weights for each expert in an MoE model.
Joint sparsity and quantization enable efficient trillion-parameter MoE serving
A hardware-software co-design framework compresses expert weights into low-precision sparse representations, achieving up to 4.35% accuracy improvement and 1.65x kernel speedup on NVIDIA B200 GPUs.
Top University
Kwanhee Lee · Namhoon Lee · Dan Alistarh
POSTECH · ISTA
Research Digest··3 min read
The authors propose moesq, an end-to-end framework that jointly optimizes semi-structured sparsity and low-precision quantization for Mixture-of-Experts models, leveraging hardware Sparse Tensor Cores.
Why this paper
From ISTA and POSTECH · Part of Mixture-of-Experts Inference Efficiency, now 3 papers
In one line
A hardware-software co-design framework jointly sparsifies and quantizes MoE weights for acceleration on Sparse Tensor Cores.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§