Black-box attack defeats major LLM watermarking schemes while preserving text quality

Omidi et al. show that detection scores in seven schemes are monotonically linked to token probability shifts, enabling greedy score-minimizing token selection.

Academic
Romina Omidi · Nima Naderloui · Yuan Hong · Binghui Wang

Illinois Institute of Technology · University of Connecticut

Research Digest··3 min read
Romina Omidi and colleagues prove that statistical LLM watermarks such as KGW, GumbelSoft, DipMark, ITS and SynthID share a structural property: per-token detection scores are monotone functions of the ratio of watermarked to base token probabilities.

The authors formalized score-ratio alignment as a theorem for five major watermarking families, including KGW and its variants, GumbelSoft, DipMark, ITS and SynthID.

§