Low-precision training of AI expert models needs matching fixes on both sides

A 4-bit recovery recipe that works for ordinary language models recovers no accuracy on mixture-of-experts models, but applying the same correction to both training and inference restores it.

Chinese Tech

Hei Yi Mak · Hoang Le · Mehran Taghian Jazi · Yunke Peng · Huan Lin · Zhengzhi Tang · +5 more

Huawei

Research Digest·4 November 2026·3 min read

Illustration: Low-precision training of AI expert models needs matching fixes on both sides

In one line

Symmetric quantization recovery closes the accuracy gap in FP4 reinforcement learning for mixture-of-experts LLMs.

Before you read this

Three ideas this paper builds on

mixture of expertsA neural architecture with many specialized sub-networks, only a few activated per input via a router, balancing capacity and efficiency.71 papers →reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.256 papers →post-training quantizationQuantizing pre-trained neural network weights and activations to44 papers →

Read it as

are often fine-tuned with , but this requires huge memory. One way to reduce memory is to use 4-bit floating point numbers instead of the usual 16-bit ones. This compression, however, can break how mixture-of-experts models work because the router that selects which expert to use is very sensitive to errors.

The researchers found that previous methods only fixed the quantization error on one side of the process, causing the router to behave differently during training and when the model generates outputs. They created a new recipe called MoeResQ that applies the same correction on both sides, keeping the router's decisions consistent.

MoeResQ restored the model's accuracy to nearly the same level as using full 16-bit precision. This matters because it makes low-precision reinforcement learning practical for mixture-of-experts models, saving memory and computation without sacrificing performance.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

mixture of experts reinforcement learning post-training quantization training-inference alignment model routing large language model