Seed Hijacking of LLM Sampling and Quantum Random Number Defense

Researchers demonstrate SeedHijack, a supply-chain attack exploiting pseudorandom number generators in LLM sampling to inject arbitrary tokens without modifying model weights, achieving 99.6% success rates across multiple models. A quantum random number generator-based defense is proposed that neutralizes the attack with minimal performance overhead.

This research exposes a fundamental vulnerability in how large language models generate text that has been largely invisible to the AI safety community. Rather than attacking model parameters or weights, SeedHijack targets the randomness layer used during token sampling—the mechanism that selects which word comes next during generation. By controlling the seed or output of the pseudorandom number generator, an attacker can deterministically force specific tokens, effectively hijacking model outputs without leaving traces in the model itself. This represents a critical supply-chain vulnerability because the attack surface extends to infrastructure components (servers, hardware, software libraries) rather than just the model weights themselves.

The attack's near-perfect success rates across multiple model sizes and alignment techniques suggest that current LLM defenses—including RLHF, supervised fine-tuning, and reasoning distillation—provide no protection against sampling-layer manipulation. This highlights how alignment efforts focus almost exclusively on model behavior rather than the operational environment where models run. The threat model becomes particularly concerning in cloud deployment scenarios where infrastructure components may be outside direct user control.

The proposed quantum random number generator defense offers a practical mitigation with minimal computational cost. However, deployment challenges remain: QRNG hardware integration, costs, and the need for standardized implementation across diverse inference platforms. For developers and infrastructure providers, this research underscores the importance of hardening randomness sources and considering hardware-based solutions for sensitive applications. The work demonstrates that AI security requires thinking beyond model parameters to the full computational stack, potentially shifting how organizations architect inference systems and manage supply-chain risks.

• →SeedHijack hijacks LLM outputs by manipulating pseudorandom number generators rather than model weights, achieving 99.6% success on GPT-2 and 100% on aligned models.
• →The attack bypasses all tested alignment methods (RLHF, SFT, reasoning distillation), revealing a critical gap in current AI safety approaches.
• →Quantum random number generator-based defense successfully neutralizes the attack with only 0.6% median latency overhead and 7.7 MB memory increase.
• →The vulnerability represents a supply-chain attack surface in LLM infrastructure that extends beyond model parameters to hardware and software randomness sources.
• →QRNG deployment requires standardization across cloud platforms and inference infrastructure to provide comprehensive protection at scale.