When Compression Becomes an Attack Surface: Black-Box Attacks on Prompt-Compressed LLM Agents

Researchers demonstrate that prompt compression—a technique used to reduce LLM latency and costs—creates a new security vulnerability when processing mixed trusted and untrusted inputs. By strategically perturbing untrusted data before compression, attackers can force compressors to discard critical task information or safety guardrails, achieving 71% attack success rates through a black-box method called COMA.

This research identifies a fundamental tension in LLM agent architecture: the compression layer designed to improve efficiency inadvertently becomes an attack vector. Unlike traditional prompt injection or jailbreaks that manipulate the language model directly, adversarial information loss (AIL) targets the compressor itself, exploiting its lossy transformation to strategically remove critical information. This represents a paradigm shift in LLM security thinking, as the vulnerability exists before the backend model even processes the input.

Prompt compression has emerged as essential infrastructure for cost-conscious LLM deployments, particularly in agentic systems handling real-time queries. As these systems become more prevalent across enterprise applications, the attack surface expands. The researchers demonstrate that attackers need neither sophisticated encoding nor knowledge of the backend model—perturbations can be crude, as their only purpose is steering compression decisions. COMA's transfer-based approach proves effective across multiple compressors and real-world case studies, suggesting broad applicability.

For the AI industry, this finding raises immediate architectural questions. Teams deploying compression must now consider isolation between trusted and untrusted inputs, potentially requiring separate compression pipelines or enhanced compression robustness. The 71% attack success rate indicates this isn't a theoretical edge case but a practical threat. Organizations relying on prompt compression for cost optimization face a difficult tradeoff: maintain efficiency but accept compression-based vulnerabilities, or redesign compression strategies with adversarial robustness as a first-class concern.

Future work likely focuses on compression algorithms designed to resist adversarial perturbations and detection mechanisms that identify suspicious information loss patterns before inference occurs.

• →Prompt compression creates a new attack surface independent of backend LLM vulnerabilities, exploitable through perturbations that need not survive compression
• →COMA achieves 71% average attack success rate across six compressors using black-box transfer-based optimization
• →Adversarial information loss (AIL) differs fundamentally from prompt injection by targeting lossy compression rather than direct LLM manipulation
• →This vulnerability affects production LLM agents already deployed for cost and latency optimization
• →Defense strategies require architectural redesign, potentially including input isolation or compression-specific robustness mechanisms