QATFactory inserts a simulation of the target inference format into the training forward pass while retaining BF16 matrix multiplication.
Training models for deployment preserves quality under four-bit quantization
QATFactory reproduces production quantization behavior during training and exports adapted models directly to common inference engines.
AI Startup
Weili Xu · Jisen Li · Yuqing Jian · Chenxi Li · Zhizhou Sha · Yifan Yu · +5 more
Together AI · University of Illinois Urbana-Champaign · The University of Texas at Austin
Research Digest··2 min read
Xu and colleagues present QATFactory, an open-source framework that trains large language models to tolerate the numerical errors introduced by deployment-grade quantization.
Why this paper
From Together AI and 2 others
In one line
QATFactory enables quantization-aware training that improves deployed LLM quality over post-training quantization for NVFP4, MXFP4, and Q4_K formats.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (3 noted)
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§