QATFactory is an open-source framework for (QAT) and distillation (QAD) of large language models. It simulates deployment-time quantization during training using engine-aligned fake quantizers, supporting NVFP4, MXFP4, and llama.cpp's Q4_K formats. This allows models to adapt to numerical noise without requiring hardware that natively supports the target format—for example, NVFP4 training on H100 GPUs, which lack FP4 Tensor Cores.
On Qwen3.5-9B, QAD achieves average benchmark accuracies of 68.9% under NVFP4 and 66.0% under MXFP4, outperforming the best (PTQ) results of 65.4% and 56.4%, respectively. The experiments also reveal format-dependent training strategies: NVFP4 performs better when only weights are quantized during training, while MXFP4 benefits from quantizing both weights and activations. Training on fewer 32K sequences improves average accuracy by 1.9 points over training on more 4K sequences under a fixed token budget.
The framework is significant because it bridges the gap between training numerics and production inference engines. It exports checkpoints directly to vLLM, SGLang, and llama.cpp without additional lossy conversion, and it works for dense and mixture-of-experts models from 8B to 230B parameters. This provides a practical, accessible path to exploit low-precision hardware throughput without sacrificing model quality.