The work compares unwatermarked and watermarked LLM outputs using identical retrieval context, queries, and decoding configurations. Across six watermarking methods, watermarked responses lost factual accuracy despite remaining fluent and having the necessary evidence in context.
The degradation comes from both immediate token perturbations and accumulated changes to the generated prefix that draw attention away from factual evidence. Two plug-in interventions targeting these mechanisms reduced factual errors by approximately 90% relative to watermark-only decoding at a matched TPR of 0.90 at 1% FPR, while preserving fluency and comparable decoding efficiency.
This matters for fact-critical applications because conventional evaluations based on detectability, robustness, perplexity, or readability can miss factual changes to numbers, dates, identifiers, and citations.