CLP: Collocation-Length Prediction for Zero-Loss Adaptive Multi-Token Inference

Researchers propose CLP (Collocation-Length Predictor), a lightweight neural architecture that improves multi-token prediction inference for large language models by eliminating competition between prediction heads and backbone models. The method achieves 1.20x-1.29x speedup on smaller models with zero quality degradation, significantly outperforming existing approaches that suffer from repetitive outputs.

CLP addresses a critical inefficiency in large language model inference by identifying and solving a fundamental architectural problem in multi-token prediction systems. Previous approaches attempted to accelerate autoregressive decoding—the sequential process where each token requires a complete forward pass—by predicting multiple tokens simultaneously. However, these methods failed because competing prediction heads created conflicting objectives, resulting in degraded output quality marked by repetition and incoherence.

The Backbone-as-Architect principle represents a conceptual shift in how acceleration systems should be designed. By restricting the backbone's language model head to first-token generation and delegating subsequent predictions to specialized MTP heads, the approach eliminates architectural conflicts. CLP implements this principle through a remarkably efficient span-level decision layer requiring only 4.6K-7.7K parameters, reducing complexity by over 99% compared to previous gate-based networks that used 1M parameters.

The performance metrics indicate substantial practical value across model scales. Testing on Qwen2.5 variants (0.5B through 7B parameters) demonstrates consistent speedups without quality loss, a critical requirement for production deployment. The finding that shorter prediction horizons (k=2) recover 24% higher accuracy on large models establishes important scaling principles for future development.

This work has implications for AI infrastructure efficiency and deployment costs. Faster inference reduces computational requirements, lowering operational expenses for AI service providers and enabling broader adoption on resource-constrained devices. The research also establishes MTP head prediction accuracy as the primary bottleneck, providing a clear research direction for future improvements in model acceleration techniques.

• →CLP uses a minimal 4.6K-7.7K parameter layer versus 1M-parameter competitors, achieving superior speedups through architectural simplification
• →Zero quality degradation with repetition ratios below 0.02 addresses the critical failure point of prior multi-token prediction methods
• →Backbone-as-Architect principle eliminates head-backbone competition by dedicating the backbone LM head exclusively to first-token generation
• →Shorter prediction horizons improve accuracy on large models, suggesting scaling-aware design is essential for effective acceleration
• →MTP head prediction accuracy identified as the binding constraint, establishing a research roadmap for future inference optimization