RAZOR is a training-free pruning method for mixture-of-experts (MoE) models. It scores experts by their replaceability using consensus residuals, which measure the output deviation from the original weighted mixture under survivor renormalization and router refill.
Across two backbones at 25% and 50% expert removal, RAZOR achieved the highest macro average over nine reasoning tasks among all evaluated methods, outperforming REAP by 2.12 to 5.59 points. It also lowered reverse KL relative to REAP in all settings. On two additional backbones, it achieved the highest macro average among residual criteria.
The method requires only forward computation and no gradients or recovery training. However, local exactness does not guarantee better joint pruning, and generation analysis showed changes in diversity, formatting, and termination despite higher task scores, indicating that task performance and generation behavior need separate evaluation.