47 articles tagged with #token-efficiency. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers demonstrate that training vision-language models (VLMs) on curated, concise data significantly reduces inference costs without sacrificing accuracy. By focusing on output brevity rather than traditional model compression techniques, the approach achieves 35x efficiency gains over verbose models while maintaining competitive performance.
Researchers introduce EquivPruner, a method that reduces token consumption in LLM reasoning searches by identifying and pruning semantically equivalent steps. Combined with MathEquiv, a new dataset for mathematical equivalence detection, the approach achieves 48.1% token reduction on GSM8K while maintaining or improving accuracy.
Researchers introduce HORMA, a hierarchical memory system for LLM agents that organizes experience into structured hierarchies with linked summaries and raw trajectories. The system achieves 22% token efficiency on long tasks while maintaining performance, addressing critical limitations in how language model agents manage working memory for multi-step reasoning.
Researchers introduce Latent Memory, a novel memory paradigm that compresses multimodal evidence (text and images) into single high-dimensional tokens for retrieval-augmented generation systems. The approach achieves competitive QA performance while reducing token consumption by 3-10x, addressing critical efficiency constraints in resource-limited deployments.
Researchers propose optical reasoning, a novel approach that uses images as the primary medium for AI reasoning tasks rather than text. The method demonstrates 28.57% token reduction on language tasks and 16% on multimodal tasks while matching or exceeding traditional text-based reasoning performance across mathematical, scientific, and multimodal benchmarks.
Researchers propose PACT, a new protocol for multi-agent AI systems that compresses inter-agent communication into compact action-state records, reducing token usage by up to 50% while maintaining or improving task performance. The approach addresses a critical efficiency bottleneck in large language model-based multi-agent systems, with demonstrated improvements in production coding applications.
Researchers introduce LatentSkill, a framework that converts textual skills into efficient LoRA adapters for LLM agents, storing knowledge in model weights rather than context prompts. The approach reduces token overhead by 64-72% while improving task performance, enabling more scalable and modular AI agent systems.
MIRAGE is a new AI framework that enables mobile agents to reason internally using compressed latent representations instead of generating verbose reasoning chains. By aligning hidden states with future interface screenshots, the system achieves comparable performance to explicit chain-of-thought approaches while reducing token generation by 3-5x, offering significant efficiency gains for AI-powered mobile automation.
Microsoft launched MAI-Code-1, an AI model that generates source code from written prompts, available through GitHub Copilot and Visual Studio Code. The company also introduced MAI-Thinking-1, a reasoning model optimized for lower token costs in private preview, as Microsoft continues building proprietary AI models alongside its OpenAI partnership.
AdaCodec introduces a predictive visual coding approach for video multimodal large language models that adaptively allocates visual tokens based on scene complexity. Rather than encoding each frame independently as RGB images, the system sends full reference frames only when scenes are unpredictable and uses compact tokens for inter-frame changes, achieving superior performance at 1/7th the token budget while reducing latency significantly.
Researchers introduce stochastic backtracking, a novel test-time scaling method for language models that revisits previously generated solution paths rather than committing irreversibly to frontier candidates. The approach uses subpool selection and power backtrack sequential Monte Carlo to improve reasoning accuracy while reducing token generation, outperforming existing PRM-guided methods across mathematical benchmarks.
AutoTTS has achieved a 69.5% reduction in token usage for large language model reasoning tasks, potentially lowering operational costs for AI systems. This efficiency gain has significant implications for crypto infrastructure and AI-driven sectors that rely on LLM inference, making computational resources more economical.
Researchers introduce ZipRL, an adaptive context compression framework that uses reinforcement learning to efficiently reduce token usage in multi-turn LLM agent tasks while preserving task-critical information. The method incorporates Hindsight Response Replay to address sparse reward problems and demonstrates 27-35% performance improvements over existing approaches on benchmark tasks.
Researchers demonstrate that tool-schema compression reduces token consumption by 44-50%, enabling large language model agents to function under tight context constraints. Testing across 14 models shows compressed schemas restore RAG functionality with +20.5 percentage point exact-match improvements at 8K tokens, while frontier models can now handle 800+ tools instead of ~494.
Researchers introduce Slipstream, a system that validates LLM agent trajectory compression by running compaction asynchronously alongside continued agent execution, enabling independent validation of summarized context. The approach improves task accuracy by up to 8.8 percentage points while reducing latency by 39.7% on long-horizon coding and web-browsing tasks.
Researchers introduce the Context Gathering Decision Process (CGDP), a POMDP framework that formalizes how LLM agents should search and gather information from environments exceeding their context windows. The approach yields measurable improvements in multi-hop reasoning (up to 11.4%) and token efficiency (up to 39% savings) through explicit belief state management and programmatic exhaustion detection.
Researchers introduce ObjectGraph (.og), a new file format designed specifically for how AI agents consume documents through retrieval rather than linear reading. The format reduces token consumption by up to 95.3% while maintaining task accuracy, addressing a fundamental architectural mismatch between traditional documents and LLM agent workflows.
Researchers introduce ContextCurator, a reinforcement learning-based framework that decouples context management from task execution in LLM agents, addressing the context bottleneck problem. The approach pairs a lightweight specialized policy model with a frozen foundation model, achieving significant improvements in success rates and token efficiency across benchmark tasks.
Researchers demonstrate that large speech language models contain significant redundancy in their token representations, particularly in deeper layers. By introducing Affinity Pooling, a training-free token merging technique, they achieve 27.48% reduction in prefilling FLOPs and up to 1.7× memory savings while maintaining semantic accuracy, challenging the necessity of fully distinct tokens for acoustic processing.
JoyAI-LLM Flash is a new efficient Mixture-of-Experts language model with 48B parameters that activates only 2.7B per forward pass, trained on 20 trillion tokens. The model introduces FiberPO, a novel reinforcement learning algorithm, and achieves higher sparsity ratios than comparable industry models while being released open-source on Hugging Face.
Researchers propose an Adaptive Social Learning (ASL) framework with Adaptive Mode Policy Optimization (AMPO) algorithm to improve language agents' reasoning abilities in social interactions. The system dynamically adjusts reasoning depth based on context, achieving 15.6% higher performance than GPT-4o while using 32.8% shorter reasoning chains.
Researchers present a comprehensive framework comparing RAG (Retrieval-Augmented Generation) variants—including GraphRAG, Modular RAG, and Agentic RAG—across 9 standardized scenarios. They introduce a novel context optimization method that reduces token usage by 19-53% while identifying a retrieval-generation gap suggesting advanced retrieval methods may not proportionally improve output quality.
Researchers compare retrieval-augmented generation (RAG) versus long-context prompting for document-grounded AI applications, finding that while long-context achieves higher accuracy (73.1% vs 65.4%), it incurs a 26x higher token cost. The study frames this trade-off as an 'epistemic accuracy' versus computational expense frontier, with significant implications for resource-constrained organizations.
Researchers introduce DART, a training-free routing framework that dynamically allocates computational thinking budgets in hybrid reasoning models by sampling cheap draft responses and using agreement patterns to decide between direct answers and extended reasoning. The approach achieves significant accuracy improvements on math and code tasks while reducing token consumption by 15-69%, without requiring labeled data or model fine-tuning.
Researchers introduce SEVRA, a serving-layer system that selectively decides whether to verify AI reasoning outputs, reducing computational waste while maintaining accuracy. The approach achieves comparable or better results than always-verifying strategies while cutting token usage significantly, though longer initial reasoning sometimes proves more efficient overall.
