DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention

Researchers introduce DyLLM, a training-free inference framework that accelerates diffusion language model decoding by up to 9.6x by selectively computing only salient tokens rather than processing entire sequences at each step. The approach identifies important tokens through attention context similarity and reuses cached activations for stable tokens, maintaining baseline accuracy across benchmarks.

DyLLM addresses a fundamental computational bottleneck in diffusion language models, which have emerged as a promising alternative to autoregressive generation due to their parallel decoding capability. While masked diffusion LLMs offer architectural advantages, their iterative denoising process demands repeated full-sequence processing, making them computationally expensive at inference time. The research identifies that most token representations remain statistically stable across denoising steps, creating an opportunity for dynamic computation optimization.

This work builds on broader trends in efficient LLM inference, where researchers increasingly recognize that not all computational operations contribute equally to model outputs. Similar sparsity-based approaches have gained traction in attention mechanisms and token pruning, but applying this principle to diffusion model iteration represents a meaningful contribution. The cosine similarity metric for identifying salient tokens provides an interpretable mechanism grounded in how transformer architectures function.

For the AI infrastructure ecosystem, this research has practical implications for deployment costs. A 9.6x throughput improvement directly translates to reduced latency and energy consumption in production environments, particularly valuable for resource-constrained applications. The training-free nature of DyLLM means existing diffusion LLM deployments could adopt the optimization without retraining, lowering adoption barriers.

The technique's effectiveness across diverse benchmarks—reasoning tasks and code generation—suggests broad applicability rather than task-specific optimization. However, questions remain about performance degradation in edge cases and whether accuracy preservation holds across all model scales. Future work examining how saliency patterns vary with model size and task complexity could reveal whether dynamic computation strategies become increasingly valuable as systems scale.

• →DyLLM achieves 9.6x throughput improvement in diffusion LLM inference through selective computation of salient tokens.
• →The framework identifies important tokens via cosine similarity of attention contexts between adjacent denoising steps, enabling computational reuse.
• →Training-free implementation allows existing diffusion language models to adopt the optimization without model retraining or fine-tuning.
• →Baseline accuracy largely preserved across reasoning and code-generation benchmarks, indicating practical viability for production deployment.
• →Exploits temporal sparsity in token representations, a property that could extend to other iterative decoding architectures.