The Defense Trilemma: Why Prompt Injection Defense Wrappers Fail?

Researchers prove mathematically that no continuous input-preprocessing defense can simultaneously maintain utility, preserve model functionality, and guarantee safety against prompt injection attacks in language models with connected prompt spaces. The findings establish a fundamental trilemma showing that defenses must inevitably fail at some threshold inputs, with results verified in Lean 4 and validated empirically across three LLMs.

This research addresses a critical vulnerability in large language model deployment: prompt injection attacks, where adversaries manipulate inputs to bypass safety guidelines. The authors prove a mathematical impossibility theorem demonstrating that wrapper-based defenses—functions that preprocess inputs before the model processes them—cannot achieve three desirable properties simultaneously: continuity, utility preservation, and complete safety coverage.

The theoretical framework establishes three progressive impossibility results. First, any continuous defense must leave some threshold-level inputs unchanged (boundary fixation). Second, under Lipschitz regularity assumptions, unsafe regions persist around these boundary points. Third, under transversality conditions, positive-measure subsets of inputs remain strictly unsafe. The mechanical verification in Lean 4 provides unprecedented rigor for AI safety research, moving beyond empirical demonstrations to formal proof.

These findings carry significant implications for AI safety architectures. Organizations investing in prompt injection defenses face a sobering reality: wrapper-based approaches fundamentally cannot guarantee comprehensive safety without sacrificing either model utility or computational continuity. The research does not preclude alternative approaches—training-time alignment, architectural redesigns, or utility-sacrificing defenses remain viable—but eliminates an entire class of seemingly elegant solutions.

For practitioners, this suggests defending against prompt injection requires fundamentally different strategies than applying preprocessing layers. Multi-turn interactions and stochastic defenses face parallel constraints. The theoretical characterization of where defenses must fail enables researchers to focus resources on architectural innovations rather than iterating on provably limited approaches.

• →No continuous wrapper defense can simultaneously achieve utility preservation, functionality preservation, and complete safety against prompt injection
• →Mathematical proof establishes boundary fixation—defenses must leave some threshold inputs unchanged—making complete coverage impossible
• →Results verified mechanically in Lean 4 provide formal proof rather than empirical evidence of the defense trilemma
• →Alternative approaches like training-time alignment and architectural changes remain viable despite wrapper defense limitations
• →Organizations must redesign safety strategies rather than iterating on preprocessing-based defenses with inherent theoretical limits