139 articles tagged with #llm-optimization. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
ScaleDoc is a new system that enables efficient semantic analysis of large document collections using LLMs by combining offline document representation with lightweight online filtering. The system achieves 2x speedup and reduces expensive LLM calls by up to 85% through contrastive learning and adaptive cascade mechanisms.
Researchers propose Many-Shot In-Context Fine-tuning (ManyICL), a novel approach that significantly improves large language model performance by treating multiple in-context examples as supervised training targets rather than just prompts. The method narrows the performance gap between in-context learning and dedicated fine-tuning while reducing catastrophic forgetting issues.
Researchers introduce Group Tree Optimization (GTO), a new training method that improves speculative decoding for large language models by aligning draft model training with actual decoding policies. GTO achieves 7.4% better acceptance length and 7.7% additional speedup over existing state-of-the-art methods across multiple benchmarks and LLMs.
Researchers introduce FreeKV, a training-free optimization framework that dramatically improves KV cache retrieval efficiency for large language models with long context windows. The system achieves up to 13x speedup compared to existing methods while maintaining near-lossless accuracy through speculative retrieval and hybrid memory layouts.
Researchers introduce Arbor, a framework that decomposes large language model decision-making into specialized node-level tasks for critical applications like healthcare triage. The system improves accuracy by 29.4 percentage points while reducing latency by 57.1% and costs by 14.4x compared to single-prompt approaches.
Researchers introduce U-Mem, an autonomous memory agent system that actively acquires and validates knowledge for large language models. The system uses cost-aware knowledge extraction and semantic Thompson sampling to improve performance, showing significant gains on benchmarks like HotpotQA and AIME25.
Researchers present a systematic analysis of hybrid multi-agent systems combining cloud-based large language models with on-device small language models, revealing that optimal architecture design is highly task-dependent and that increased frontier compute does not guarantee better performance across the power-cost-accuracy Pareto frontier.
Researchers propose a novel method for optimizing multi-agent LLM systems by decomposing credit assignment into temporal and structural components, enabling more efficient prompt optimization through targeted refinement rather than global updates. The approach uses state-space bottleneck analysis and role-based policy isolation to identify and fix weak components in collaborative AI systems, reducing computational queries while improving reasoning performance across benchmarks.
Researchers introduce NaRA (Noise-aware Low-Rank Adaptation), a parameter-efficient fine-tuning method designed specifically for diffusion large language models that adapts to noise levels during the denoising process. Unlike existing methods like LoRA that use static parameters, NaRA employs a hypernetwork to dynamically adjust low-rank matrices based on noise, achieving better performance on reasoning and code generation tasks.
Researchers introduce Loong, an AI agent designed to improve long document translation by selectively retrieving relevant context from a 3E memory module rather than processing all available information. The system uses reinforcement learning to optimize context selection and demonstrates significant translation quality improvements across multiple language pairs, achieving gains up to 13 points on standard evaluation metrics.
Researchers introduce RACE-Sched, an asynchronous AI framework that combines real-time symbolic heuristics with LLM-powered reasoning to solve dynamic job shop scheduling problems in industrial systems. The approach decouples fast reactive execution from slower deliberative optimization, enabling superior performance over deep reinforcement learning baselines while maintaining interpretability and millisecond-level response times.
CORE-T introduces a training-free framework for improving table retrieval in text-to-SQL systems by combining dense retrieval with LLM-generated metadata and compatibility caching. The approach achieves significant performance gains—up to 22.7 points in table-selection F1 and 24.4 points in multi-table execution accuracy—while reducing inference tokens by 64-76% compared to LLM-intensive alternatives.
Researchers introduce Agent-Radar, a training-free context management method that improves multi-agent LLM systems by dynamically filtering irrelevant information from long conversation histories. The technique uses temporal and spatial decay mechanisms to maintain focus on relevant context, achieving up to 7.64% performance improvements across five benchmarks.
TIMEGATE is a new policy framework that optimizes machine learning system adaptation by intelligently managing computational budgets across training, labeling, and evaluation cycles. The research demonstrates 2.3x efficiency gains in labeling versus training and achieves 66% evaluation-compute savings without compromising model accuracy, with validated results across tabular data and large language models like LLaMA-3.1-8B.
Researchers propose BaSE, a multi-armed bandit algorithm that optimizes how large language models allocate computational resources during evolutionary search tasks. By dynamically distributing LLM calls across parallel trajectories, BaSE improves mean fitness by 12.3% over existing baselines while addressing the reliability gap between reported best-case and typical run performance.
Researchers introduce CoHyDE, an iterative co-training method that jointly optimizes a dense encoder and LLM rewriter to improve tool retrieval for AI agents. The approach outperforms single-component baselines by 2.5-8 percentage points on standard and vague queries, addressing the fundamental challenge of bridging colloquial user language with technical API vocabularies.
Researchers demonstrate an autoresearch framework where an AI agent autonomously optimizes LLM-based policy synthesis for multi-agent cooperation problems. The system discovers objective-dependent pipeline designs that outperform hand-crafted baselines, with fairness mechanisms emerging only when optimizing for equitable outcomes rather than efficiency.
Researchers identify a critical failure mode in test-time reinforcement learning (TTRL) where majority voting locks onto incorrect answers, permanently suppressing correct signals in low-ability problems. They introduce TTRL-Guard, a framework using flip-rate monitoring and selective updating to prevent this 'Correct-Answer Extinction Window,' achieving 54% relative improvement on AIME 2025 benchmarks.
Researchers introduce MGRetrieval, a novel retrieval strategy for long-term dialogue agents that uses semantic memory structures to guide multi-step retrieval rather than one-shot approaches. The method improves performance on dialogue benchmarks by 8-11% while maintaining computational efficiency, addressing a key limitation in LLM-based conversational systems.
Researchers present twelve token optimization strategies for using LLMs to migrate Oracle databases to PostgreSQL, addressing cost and quality degradation challenges. Adaptive routing emerges as the optimal approach, reducing token consumption by 8.72% while maintaining 88.40% semantic match accuracy, demonstrating that token optimization requires balancing multiple objectives rather than simple prompt shortening.
Researchers present an LLM-powered framework that enables non-expert end users to re-optimize deployed decision-support systems through natural language interaction, eliminating dependency on operations research specialists. The system combines language models with an optimization toolbox to dynamically adapt models to changing business conditions while maintaining solution quality and interpretability.
Researchers demonstrate that cross-lingual contrastive preference tuning (CroCo) enables large language models to improve performance across 14 languages without language-specific annotations by leveraging English-trained reward models. The method shows consistent gains in both structured and open-ended generation tasks across multiple languages while avoiding catastrophic forgetting.
Researchers introduce Dense2MoE, a framework that converts dense language models into efficient Mixture of Experts (MoE) architectures through unified pruning and upcycling, enabling viable on-device LLM deployment with improved latency-accuracy tradeoffs.
Researchers introduce Persona Generators, AI functions that create diverse synthetic populations for evaluating AI systems across varied user demographics without needing extensive real-world data collection. Using iterative optimization with large language models, the approach generates lightweight code that produces synthetic personas spanning rare trait combinations and long-tail behaviors, outperforming existing baselines on diversity metrics.
Researchers introduce BOSQ, a framework that optimizes the use of large language models for graph neural network tasks by selectively querying LLMs only when necessary. This approach reduces computational costs by orders of magnitude while maintaining or improving performance on text-attributed graph datasets, addressing a critical bottleneck in practical LLM-enhanced graph learning.
