TTMark extends distortion-free watermarking from single tokens to adjacent token pairs, enlarging the effective watermarking alphabet from vocabulary size V to V^2. This allows the detector to exploit both token entropy and conditional entropy, overcoming a key limitation of single-token methods in low-entropy regimes. The authors develop an efficient branch-isolating generation algorithm to construct the joint distribution in a single forward pass.
Theoretically, they show that pairwise watermarking yields up to a sqrt(2) improvement in aggregate z-score in low-entropy settings. Empirically, on the C4 dataset with Llama-3.2-3B-Instruct at a 0.001% false positive rate, TTMark raises the true positive rate of SynthID-Text from 50.3% to 73.2% and ENS-MCMark from 54.0% to 81.8%. These gains generalize across models, datasets, and sampling conditions, while preserving generation quality and improving robustness to edits and localized detection.
This is interesting because it addresses a practical bottleneck: watermark detectability degrades when sampling is conservative or text is short. By leveraging pairs instead of individual tokens, TTMark extracts stronger detection signals without introducing distributional bias.