y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#inference-optimization News & Analysis

297 articles tagged with #inference-optimization. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

297 articles
AIBullisharXiv – CS AI · May 276/10
🧠

AGORA: Adapter-Grounded Observation-Action Retention for Inference-Free Prompt Compression in LLM Agents

Researchers introduce AGORA, a new compression method for LLM agents that addresses critical failures in existing token-level compressors. Unlike general-purpose compression techniques that destroy action semantics by removing low-entropy tokens, AGORA operates at step-granularity with structural awareness, achieving 1.0-11.5x compression while retaining 75%+ performance across most test scenarios.

AINeutralarXiv – CS AI · May 276/10
🧠

Tail-Aware HiFloat4: W4A4 Post-Training Quantization for Wan2.2

Researchers have developed Tail-Aware HiFloat4, a post-training quantization method that compresses text-to-video generation models using W4A4 (4-bit weights and activations) while maintaining output quality. The technique introduces activation-tail-aware calibration to handle statistical outliers, enabling efficient model deployment without retraining.

AINeutralarXiv – CS AI · May 276/10
🧠

Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models

Researchers propose Token-to-Mask (T2M) remasking as an improved alternative to Token-to-Token editing in discrete diffusion language models, addressing fundamental limitations in error detection and context corruption. The method resets suspected erroneous tokens to mask state for re-prediction, demonstrating 5.92% improvement on mathematical benchmarks and fixing 59.4% of final-answer corruption cases.

AINeutralarXiv – CS AI · May 276/10
🧠

DynFrame: Adaptive Reasoning-Driven Multimodal Framework with Dynamic Frame Augmentation for Complex Video Understanding

Researchers introduce DynFrame, an advanced video understanding framework that enables multimodal language models to dynamically select both temporal windows and frame sampling rates during inference. The approach achieves competitive performance with smaller 4B models against larger 7B-8B baselines and sets new state-of-the-art results with its 8B variant across six video understanding benchmarks.

AINeutralarXiv – CS AI · May 276/10
🧠

Genre Controlled Music Generation via Activation Steering

Researchers present a novel method for controlling music generation in the MusicGen transformer by using activation steering techniques applied at inference time. The approach enables precise genre control through linear probes that manipulate the model's residual stream, demonstrating how interpretable AI behaviors can enhance collaborative music creation.

AINeutralarXiv – CS AI · May 276/10
🧠

Inference-Time Search Using Side Information for Diffusion-Based Image Reconstruction

Researchers propose DISS, a training-free framework that enhances diffusion-based image reconstruction by incorporating side information through inference-time search. The method demonstrates consistent quality improvements across multiple inverse problems (inpainting, super-resolution, deblurring) and diffusion solvers while supporting diverse side information types including reference images, text, and medical scans.

AINeutralarXiv – CS AI · May 126/10
🧠

WindINR: Latent-State INR for Fast Local Wind Query and Correction in Complex Terrain

WindINR is a machine learning framework that enables fast, localized wind forecasting in complex terrain by using implicit neural representations to query wind conditions at specific user-defined locations rather than generating dense grid-based forecasts. The system achieves 2.6x speedup in corrections by updating only a compact latent state instead of retraining full networks, making it practical for real-time wind estimation applications.

AINeutralarXiv – CS AI · May 126/10
🧠

Primal-Dual Guided Decoding for Constrained Discrete Diffusion

Researchers introduce primal-dual guided decoding, an inference-time method for discrete diffusion models that enforces global constraints during token generation through adaptive Lagrangian multipliers and KL-regularized optimization. The approach requires no model retraining, supports multiple simultaneous constraints, and demonstrates effectiveness across text generation, molecular design, and music applications.

AINeutralarXiv – CS AI · May 126/10
🧠

MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphs

MAGE introduces a novel framework for self-evolving language model agents that uses co-evolutionary knowledge graphs to preserve learned knowledge across iterations without modifying the base model. The system externalizes learning into structured memory subgraphs, enabling frozen backbone models to improve through retrieved guidance while maintaining inference stability across nine diverse benchmarks.

AIBullisharXiv – CS AI · May 126/10
🧠

TMAS: Scaling Test-Time Compute via Multi-Agent Synergy

Researchers introduce TMAS, a multi-agent framework that improves test-time compute scaling for large language models by enabling specialized agents to collaborate through hierarchical memory systems. The approach balances exploration and exploitation more effectively than existing methods, achieving stronger iterative scaling on challenging reasoning benchmarks.

AINeutralarXiv – CS AI · May 126/10
🧠

DARE: Diffusion Language Model Activation Reuse for Efficient Inference

Researchers introduce DARE, a technique that reduces computational redundancy in Diffusion Language Models by reusing cached attention activations across tokens. The method achieves up to 1.20x per-layer latency improvements while maintaining generation quality, addressing efficiency gaps between diffusion-based and auto-regressive language models.

AINeutralarXiv – CS AI · May 126/10
🧠

NoisyCoconut: Counterfactual Consensus via Latent Space Reasoning

NoisyCoconut is an inference-time method that improves LLM reliability by injecting controlled noise into internal representations to generate diverse reasoning paths, enabling models to abstain when uncertain without requiring retraining. The technique reduces error rates from 40-70% to below 15% on mathematical reasoning tasks through unanimous agreement among noise-perturbed paths, offering practical reliability improvements compatible with existing models.

AIBullisharXiv – CS AI · May 126/10
🧠

When Few Steps Are Enough: Training-Free Acceleration of Identity-Preserved Generation

Researchers demonstrate that identity-preserved image generation using FLUX can be accelerated 5.9x by replacing the standard diffusion backbone with a distilled version, without retraining the identity adapter. Analysis reveals identity fidelity stabilizes within 4-8 steps while later steps primarily refine visual details, enabling efficient personalized generation at deployment.

AIBullisharXiv – CS AI · May 126/10
🧠

TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLM

Researchers introduce TAD, a temporal-aware self-distillation framework that improves diffusion large language models' accuracy-parallelism trade-off by using adaptive loss functions based on token decoding timelines. The method increases accuracy from 46.2% to 51.6% while enabling aggressive acceleration modes, addressing a fundamental limitation in parallel text generation.

AIBullisharXiv – CS AI · May 126/10
🧠

The Silent Vote: Improving Zero-Shot LLM Reliability by Aggregating Semantic Neighborhoods

Researchers propose Semantic Softmax, a novel inference-time method that improves zero-shot LLM classification by recovering probability mass lost during constrained decoding. The approach aggregates scores from semantic synonyms, reducing calibration errors and boosting accuracy on emotion and toxicity detection tasks.

AINeutralarXiv – CS AI · May 116/10
🧠

The Single-File Test: A Longitudinal Public-Interface Evaluation of First-Output LLM Web Generation with Social Reach Tracking

A comprehensive eight-week study evaluated 68 HTML generations from four major LLM families (GPT, Gemini, Grok, Claude) in standardized web generation tasks, finding Claude delivered the most consistent performance while questioning assumptions about reasoning time and social media predictability. The research reveals significant evaluation bias in LLM-as-judge systems and that code verbosity correlates more with model architecture than prompt specificity.

🧠 Claude🧠 Gemini🧠 Grok
AINeutralarXiv – CS AI · May 116/10
🧠

LensVLM: Selective Context Expansion for Compressed Visual Representation of Text

LensVLM is a new inference framework that enables Vision Language Models to process highly compressed images of text by selectively expanding relevant sections, achieving 4.3x compression while maintaining accuracy comparable to full-resolution processing. The approach combines learned tool selection with post-training techniques to overcome the fundamental limitation that compressed text becomes illegible to standard vision encoders.

AINeutralarXiv – CS AI · May 116/10
🧠

Amortized-Precision Quantization for Early-Exit Vision Transformers

Researchers introduce Amortized-Precision Quantization (APQ) and MAQEE, a framework that optimizes Vision Transformers for low-precision deployment with early-exit mechanisms. By jointly optimizing exit thresholds and bit-widths while accounting for quantization noise across layers, the approach achieves up to 95% reduction in computational operations while maintaining accuracy across vision tasks.

AIBullisharXiv – CS AI · May 116/10
🧠

Fast Byte Latent Transformer

Researchers introduce the Byte Latent Transformer (BLT), a new approach to byte-level language models that dramatically accelerates generation speed through diffusion-based and speculative decoding techniques. The methods reduce memory-bandwidth costs by over 50% compared to standard byte-level models, potentially making byte-level LMs practical for real-world deployment.

AINeutralarXiv – CS AI · May 116/10
🧠

Skip-It? Theoretical Conditions for Layer Skipping in Vision-Language Models

Researchers propose a theoretical framework for identifying when layer skipping in vision-language models reduces computational costs without sacrificing performance. The work establishes experimentally verifiable redundancy conditions that unify and improve upon existing pruning heuristics, confirming that early and late vision tokens contain significant redundancies across models.

AIBullisharXiv – CS AI · May 116/10
🧠

AdaCorrection: Adaptive Offset Cache Correction for Accurate Diffusion Transformers

Researchers introduce AdaCorrection, a framework that improves the efficiency of Diffusion Transformers (DiTs) used in image and video generation by adaptively correcting cached features during inference. The method maintains generation quality while reducing computational costs through intelligent cache reuse without requiring retraining or additional supervision.

AINeutralarXiv – CS AI · May 116/10
🧠

KV Cache Offloading for Context-Intensive Tasks

Researchers demonstrate that KV-cache offloading techniques, designed to reduce memory usage in large language models, significantly degrade performance on context-intensive tasks requiring extensive information extraction. The study introduces the Text2JSON benchmark and identifies low-rank projection and unreliable landmarks as key failure points, proposing improved alternatives.

🧠 Llama
AINeutralarXiv – CS AI · May 96/10
🧠

Inference-Time Budget Control for LLM Search Agents

Researchers propose a two-stage inference-time budget control system for LLM search agents that optimizes how language models allocate computational resources between tool calls and token generation during multi-hop question answering. The method uses Value-of-Information scoring to decide when to retrieve information, decompose questions, or commit to final answers, demonstrating consistent performance gains across multiple benchmarks and model sizes.

AINeutralarXiv – CS AI · May 96/10
🧠

Open-SAT: LLM-Guided Query Embedding Refinement for Open-Vocabulary Object Retrieval in Satellite Imagery

Researchers introduce Open-SAT, a training-free algorithm that uses Large Language Models to refine query embeddings for satellite image retrieval tasks. The method improves upon existing vision-language models by leveraging LLM-guided contextual refinement at inference time, achieving up to 16% F1 score improvement on open-vocabulary satellite imagery tasks without requiring additional training.

← PrevPage 10 of 12Next →