VIA-SD: Verification via Intra-Model Routing for Speculative Decoding
Researchers propose VIA-SD, a multi-tier verification framework for speculative decoding that uses a lightweight slim-verifier to handle medium-confidence tokens instead of always invoking full model verification. The approach reduces rejection rates by 10-22% and achieves 10-20% speedup improvements over existing speculative decoding methods while maintaining compatibility with current frameworks.