On the Robustness of Watermarking for Autoregressive Image Generation

Researchers demonstrate critical vulnerabilities in watermarking techniques designed for autoregressive image generators, showing that watermarks can be removed or forged with access to only a single watermarked image and no knowledge of model secrets. These findings undermine the reliability of watermarking as a defense against synthetic content in training datasets and enable attackers to manipulate authentic images to falsely appear as AI-generated content.

The emergence of autoregressive image generation models has created an urgent need for reliable watermarking systems to combat misinformation and prevent dataset contamination. This research exposes fundamental weaknesses in existing watermarking schemes by introducing three novel attack vectors: vector-quantized regeneration removal, adversarial optimization-based attacks, and frequency injection techniques. The researchers demonstrate that sophisticated adversaries require minimal resources—merely a single watermarked reference image—to circumvent protections without accessing proprietary model parameters or watermarking secrets.

The security landscape for synthetic content detection has matured significantly as AI-generated media becomes increasingly sophisticated and prevalent. Watermarking emerged as the primary mechanism to provide attribution and detection capabilities at scale. However, this research reveals that current implementations fail to provide the robustness necessary for critical applications like dataset filtering, which prevents model collapse when synthetic training data contaminates future model development.

The discovery of Watermark Mimicry presents a particularly concerning threat vector where legitimate content creators can be harmed through false detection triggered by manipulated authentic images. This creates perverse incentives and undermines trust in watermarking as a detection mechanism. For developers and platform operators, this indicates that watermarking alone cannot serve as a primary defense against synthetic content proliferation. The findings suggest that multi-layered detection approaches combining cryptographic verification, metadata analysis, and behavioral model characteristics may be necessary to achieve reliable synthetic content identification in production environments.

• →Existing watermarking schemes for autoregressive image generators can be defeated with access to only a single watermarked image and no model secrets
• →Three new attack methods (regeneration, adversarial optimization, and frequency injection) effectively remove or forge watermarks in AR-generated images
• →Watermark Mimicry attacks enable authentic images to be manipulated to trigger false AI-detection, creating potential harm to legitimate content creators
• →Current watermarking techniques cannot reliably filter synthetic images from training datasets, limiting their utility in preventing model collapse
• →Single-layer watermarking defense proves insufficient; multi-layered detection approaches combining cryptography and metadata analysis appear necessary