Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models

Researchers demonstrate that diffusion language models exhibit superior jailbreak robustness compared to autoregressive models due to their sampling mechanisms' ability to recover from harmful intermediate generations. They introduce a Step-Wise Refusal Internal Dynamics (SRI) signal that enables effective jailbreak detection without modifying inference, generalizing to unseen attacks.

This research addresses a critical gap in understanding how different language model architectures respond to adversarial attacks. The study reveals that diffusion language models' parallel decoding mechanism inherently provides better protection against jailbreak attempts than traditional autoregressive approaches, a finding with significant implications for AI safety.

The research builds on growing recognition that model architecture fundamentally shapes safety properties. While diffusion models have gained traction for their computational efficiency and generation quality, this work establishes that their sampling characteristics provide an additional security benefit. The ability to recover from harmful intermediate outputs represents a qualitative difference in how these models process adversarial prompts, suggesting the architectural choice itself serves as a form of robustness rather than relying solely on training-based safety measures.

The introduction of SRI signals provides practical value for AI developers and security teams. By detecting anomalous generation patterns without modifying the inference pipeline, organizations can implement detection layers with minimal computational overhead. This non-invasive approach is particularly valuable for production systems where altering model behavior carries risks. The detector's ability to generalize to unseen attacks suggests it captures fundamental patterns rather than memorizing specific threats.

For the AI safety community, these findings suggest architectural choices warrant greater consideration in safety frameworks. Future development may increasingly evaluate models not just on generation quality or inference speed, but on inherent robustness properties. The work opens questions about whether other architectural innovations—whether in transformer variants, mixture-of-experts, or emerging paradigms—similarly provide security benefits without explicit safety training.

• →Diffusion language models demonstrate superior jailbreak robustness compared to autoregressive models due to their sampling mechanism's recovery capabilities
• →Step-Wise Refusal Internal Dynamics (SRI) signals enable effective jailbreak detection without modifying model inference or requiring knowledge of specific attacks
• →Recovery failures appear anomalous in SRI signal space for autoregressive models, enabling simple detection through benign signal training
• →Architectural design choices fundamentally shape language model safety properties, suggesting architectural evaluation should complement training-based safety measures
• →The non-invasive detection approach adds negligible computational overhead while matching or exceeding existing jailbreak detection baselines