4 articles tagged with #prompt-compression. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce CORE, a lightweight prompt compression method that optimizes large language models for edge devices without requiring auxiliary smaller models. The approach achieves 30% accuracy improvements while reducing memory usage by 50% and cutting energy consumption by 95% on smartphones compared to existing methods.
Researchers demonstrate that prompt compression—a technique used to reduce LLM latency and costs—creates a new security vulnerability when processing mixed trusted and untrusted inputs. By strategically perturbing untrusted data before compression, attackers can force compressors to discard critical task information or safety guardrails, achieving 71% attack success rates through a black-box method called COMA.
Researchers introduce AGORA, a new compression method for LLM agents that addresses critical failures in existing token-level compressors. Unlike general-purpose compression techniques that destroy action semantics by removing low-entropy tokens, AGORA operates at step-granularity with structural awareness, achieving 1.0-11.5x compression while retaining 75%+ performance across most test scenarios.
A large-scale study of prompt compression techniques for LLMs found that LLMLingua can achieve up to 18% speed improvements when properly configured, while maintaining response quality across tasks. However, compression benefits only materialize under specific conditions of prompt length, compression ratio, and hardware capacity.
