#prompt-caching News & Analysis

5 articles tagged with #prompt-caching. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

5 articles

AIBullisharXiv – CS AI · May 277/10

🧠

PANDO: Efficient Multimodal AI Agents via Online Skill Distillation

PANDO introduces an efficient multimodal AI agent framework that improves performance while reducing computational costs through online skill distillation, achieving 58.3% success on VisualWebArena tasks with 58-61% fewer tokens than competing approaches. The system addresses inefficiencies in web agent design by maintaining a skill library and employing hierarchical routing, visual compression, and cache-aware prompting without requiring expensive pre-evaluation.

AIBullisharXiv – CS AI · May 127/10

🧠

Shepherd: A Runtime Substrate Empowering Meta-Agents with a Formalized Execution Trace

Shepherd is a new runtime substrate that enables meta-agents to supervise and optimize other agents through formalized execution traces, achieving 5x faster forking than Docker and demonstrating measurable improvements in coding assistance, optimization, and reinforcement learning tasks. The open-source system mechanizes core operations in Lean and enables replay, branching, and counterfactual exploration of agent behaviors.

AIBullishOpenAI News · Nov 137/107

🧠

Introducing GPT-5.1 for developers

OpenAI has released GPT-5.1 through its API, featuring enhanced adaptive reasoning capabilities, extended prompt caching, and improved coding performance. The update includes new developer tools like apply_patch and shell functionality for better development workflows.

AIBullisharXiv – CS AI · May 126/10

🧠

Agent-X: Full Pipeline Acceleration of On-device AI Agents

Researchers introduce Agent-X, a software framework that accelerates LLM-based agents running on edge devices by optimizing both prefill and decode stages through prompt rewriting and LLM-free speculative decoding. The framework achieves 1.61x end-to-end speedup with no accuracy loss, addressing a critical performance bottleneck in on-device AI deployments.

AIBullishOpenAI News · Oct 15/107

🧠

Prompt Caching in the API

An API service is introducing prompt caching functionality that automatically provides cost discounts when the model processes inputs it has recently encountered. This optimization technique reduces computational overhead and costs for repeated or similar queries.