#token-routing News & Analysis

5 articles tagged with #token-routing. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

5 articles

AIBullisharXiv – CS AI · Jun 97/10

🧠

WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routing

WhiFlash introduces a novel speculative decoding method that combines autoregressive and diffusion-based drafting models through token-level routing, achieving up to 69.6% throughput improvements over existing approaches. The system uses lightweight controllers to dynamically switch between drafting paradigms based on per-token conditions, addressing a key bottleneck in LLM inference efficiency.

AIBullisharXiv – CS AI · Jun 57/10

🧠

You Only Index Once: Cross-Layer Sparse Attention with Shared Routing

Researchers propose Cross-Layer Sparse Attention (CLSA), a novel architecture that optimizes long-context LLM inference by sharing both key-value caches and routing indices across decoder layers. The method achieves up to 7.6x decoding speedup and 17.1x throughput improvement at 128K context while maintaining accuracy, addressing the efficiency-quality tradeoff that has constrained existing sparse attention approaches.

AINeutralarXiv – CS AI · Jun 116/10

🧠

Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models

Researchers propose Reroute, a training-free method that improves vision-language model efficiency by recoverable token routing instead of permanent token removal. The approach dynamically reroutes less important visual tokens through decoder layers rather than discarding them, improving performance on grounding tasks while maintaining computational efficiency.

AINeutralarXiv – CS AI · Jun 26/10

🧠

Optimizing Diversity and Quality through Base-Aligned Model Collaboration

Researchers propose Base-Aligned Model Collaboration (BACo), an inference-time framework that dynamically combines base and aligned language models to improve both output diversity and quality simultaneously. The method uses token-level routing strategies based on uncertainty signals, achieving a 21.3% joint improvement in diversity-quality metrics without requiring expensive retraining or multi-pass decoding.

AINeutralarXiv – CS AI · May 126/10

🧠

TRACE: Distilling Where It Matters via Token-Routed Self On-Policy Alignment

Researchers introduce TRACE, a novel training method that improves AI model performance by selectively applying different optimization techniques to critical versus routine tokens in reasoning tasks. The approach addresses inefficiencies in standard self-distillation by concentrating training effort on important decision points, achieving 2.76 percentage point improvements over baseline methods while better preserving out-of-distribution generalization.