Using activation patching across twenty-five models from eight LLM families, the work identifies a shared early-layer (L0) attention routing circuit in VQ-tokenized . The circuit is present in five natural VQ models across three families and five induced variants, and absent in continuous-projector models. A single-variable swap from CLIP+MLP to VQ+Linear installs it, while a matched-compute MLP control does not, isolating as the cause.
A three-gate diagnostic, measuring residual-stream divergence, attention-routing concentration at L0, and codebook off-manifold rate, separates the ten positive models from fifteen negative ones. On interventions, tuned DoLA improves binary yes/no calibration, but only L0 ablation reduces object hallucination in open-ended generation, cutting CHAIR_i from 10.3% to 7.1% (31% relative), while VCD and DoLA leave it unchanged or make it worse.
This recasts object hallucination in VQ-tokenized models as an architectural and pretraining property rather than a generic calibration problem, and points to a targeted mechanism that decoding-time fixes cannot replicate.