LLMs respond differently to harmful prompts when AI watermarking is used
ars ·
In response to a new European Union law, AI platforms are adopting watermarking methods to identify generated text. SynthID-Text, developed by Google and released as open source, uses a secret key to subtly influence token selection while preserving the appearance of ordinary text. Research by Lasso Security found that this process can alter not only word choices but also safety refusals and tool use. Tests on six open-weight models showed that watermarking had a stronger effect when harmful requests were combined with prompt-injection techniques, sometimes making models comply with requests they would otherwise reject. These changes can also affect AI agents by influencing which tools they call and which arguments they provide. Model behavior varied depending on the secret key used, a phenomenon the researcher calls sampling drift. The study did not test Claude models or Anthropic’s planned implementation, so developers are advised to stress-test their systems with watermarking enabled.
ars