The authors trained 180 transformer text classifiers from scratch: four classifier formulations (discriminative, pseudo-generative, fully generative) × three model sizes × five datasets × three random seeds.
Text classifier predictions shift with serving context, even with fixed model and input
A systematic study of 180 classifiers shows that changing batch size, precision, or padding length alters probabilities and labels, with the largest effect from position-encoding shifts due to variable padding.
Big Tech
Santhosh Kumar Kasa · Siva Rajesh Kasa · Sumit Negi
Amazon
Research Digest··3 min read
Kasa et al.
Why this paper
From Amazon
In one line
Text classifier predictions change with padding length, batch shape, precision, backend, and other serving settings even when inputs and model weights are fixed.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§