AGORA: Adapter-Grounded Observation-Action Retention for Inference-Free Prompt Compression in LLM Agents

Researchers introduce AGORA, a new compression method for LLM agents that addresses critical failures in existing token-level compressors. Unlike general-purpose compression techniques that destroy action semantics by removing low-entropy tokens, AGORA operates at step-granularity with structural awareness, achieving 1.0-11.5x compression while retaining 75%+ performance across most test scenarios.

Token compression has become essential for reducing computational costs in large language model applications, but existing techniques designed for general text processing fail catastrophically when applied to LLM agents. The core problem identified is that general-purpose compressors rank action-critical tokens—including identifiers, brackets, and action verbs—as low-priority for retention because they carry predictable information patterns. When these tokens are removed, the resulting prompts become syntactically or semantically invalid, causing the agent's environment to reject actions entirely.

AGORA addresses this gap by shifting from token-level to step-level compression, introducing structural awareness that preserves format-critical content and recent observations. The system combines three components: a structural prompt parser that understands agent syntax, a keep-floor mechanism protecting essential formatting and recency-dependent information, and a lightweight 125M-parameter relevance scorer trained on counterfactual labels indicating whether action changes would occur. This approach incurs minimal computational overhead—approximately 2 milliseconds per step with zero additional LLM inference calls.

The significance lies in enabling practical deployment of LLM agents at scale. Previous compression methods achieved high compression ratios but rendered agents non-functional, presenting a false efficiency gain. AGORA demonstrates the first method maintaining 75%+ performance across diverse agent configurations while achieving adaptive compression ratios up to 11.5x. For developers building cost-sensitive agent systems, this provides a viable path to reduce token consumption without sacrificing reliability.

• →Existing token-level compression methods destroy action semantics in LLM agents by removing syntactically essential but low-entropy tokens
• →AGORA achieves 1.0-11.5x adaptive compression while retaining 75%+ performance through step-granularity and structural awareness
• →The method combines structural parsing, format-critical keep-floors, and a 125M-parameter scorer with minimal computational overhead (~2ms per step)
• →Component ablation shows structural preservation is the dominant quality factor, with learned scoring enabling adaptive compression ratios
• →This addresses a critical deployment bottleneck for LLM agents by enabling cost reduction without functional degradation