PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding

PARD-2 introduces a dual-mode speculative decoding framework that accelerates large language model inference by up to 6.94× through improved draft model training aligned with token acceptance rather than prediction accuracy. The advancement uses Confidence-Adaptive Token optimization to enable single draft models to operate in both target-dependent and target-independent modes, significantly outperforming existing methods like EAGLE-3.

PARD-2 addresses a fundamental misalignment in speculative decoding—the gap between how draft models are trained and how they perform during inference. Traditional approaches optimize for token prediction accuracy, but the actual goal during inference is maximizing consecutive token acceptance rates. This paper's key innovation shifts the optimization objective entirely, focusing draft model training on acceptance length rather than raw prediction performance.

Speculative decoding has emerged as a critical efficiency technique for LLM deployment, particularly as model sizes continue growing. The technique uses lightweight draft models to generate candidate tokens that are verified in parallel by the target model, reducing latency without sacrificing output quality. Prior implementations like PARD and EAGLE struggled with this train-inference mismatch, limiting acceleration gains and requiring separate models for different deployment scenarios.

The dual-mode capability is particularly significant for production environments. Organizations can deploy a single draft model across both target-dependent scenarios (where the draft model aligns with specific target models) and target-independent scenarios (where flexibility matters more). The reported 1.9× improvement over EAGLE-3 and 1.3× over PARD on Llama3.1-8B demonstrates material performance gains that directly translate to reduced computational costs and faster response times in production systems.

This advancement matters for the broader LLM economics. As inference costs dominate operational expenses for deployed language models, efficiency improvements compound across millions of requests. The open-source release suggests rapid adoption potential within the AI community, likely accelerating similar optimization techniques across competing implementations.

• →PARD-2 achieves up to 6.94× lossless acceleration through target-aligned draft model training focused on token acceptance rather than prediction accuracy.
• →Dual-mode framework enables single draft models to support both target-dependent and target-independent inference scenarios, improving deployment flexibility.
• →Confidence-Adaptive Token optimization adaptively reweights tokens to better align with the verification process during inference.
• →Outperforms EAGLE-3 by 1.9× and PARD by 1.3× on Llama3.1-8B, demonstrating significant practical efficiency improvements.
• →Open-source availability at AMD-AGI GitHub repository positions PARD-2 for rapid community adoption and further optimization.