29 articles tagged with #jailbreak-attacks. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers demonstrate that conventional detect-and-block defenses against AI jailbreak attacks fail as automated attackers scale their efforts, but a new misdirection strategy called CMPE significantly reduces attack success rates by feeding false positives to attacker judges instead of predictable refusals.
JailbreakOPT is a new framework that optimizes adversarial prompts to exploit safety vulnerabilities in large language models through iterative refinement and tool composition. The approach combines atomic jailbreak techniques with contextual bandits to achieve higher attack success rates while reducing the number of queries needed, demonstrating meaningful progress in LLM security testing.
Researchers developed AutoInject, a reinforcement learning framework that automatically generates adversarial prompts to exploit LLM agents through prompt injection attacks. The method outperforms existing attack techniques on production models and successfully breaks defenses specifically designed to resist prompt injection, highlighting a significant vulnerability gap in AI system security.
Researchers introduce BadRobot, an attack paradigm that exploits vulnerabilities in embodied LLM agents to make them perform harmful physical actions through voice commands. The study demonstrates successful attacks against prominent frameworks like Voxposer and Code as Policies, revealing critical safety gaps in AI systems integrated into physical robotics.
Researchers demonstrate Context-Fractured Decomposition (CFD), a new class of jailbreak attacks against tool-using LLM agents that exploit gaps in artifact provenance tracking across multiple steps and system boundaries. By decomposing harmful requests across time and contexts while maintaining benign-looking intermediate artifacts, CFD achieves up to 28.3% higher success rates than existing attack methods, revealing fundamental vulnerabilities in how AI agents enforce safety guardrails in fragmented deployment environments.
Researchers introduce SlotGCG, a novel jailbreak attack method that exploits positional vulnerabilities in large language models by strategically inserting adversarial tokens at optimal positions within prompts rather than just at the end. The approach achieves 14% higher success rates than existing GCG-based attacks while identifying that LLM vulnerability is significantly dependent on token insertion location.
Researchers discovered the 'Alignment Curse,' revealing that stronger text-audio alignment in multimodal AI models inadvertently enables more effective transfer of text-based jailbreak attacks to audio channels. The finding exposes a critical safety vulnerability in recent omni-models like Qwen, suggesting current audio safety evaluations significantly underestimate risks originating from text modalities.
Researchers have developed a comprehensive taxonomy of jailbreak attacks and defenses for Large Audio Language Models (LALMs), identifying vulnerabilities across semantic, acoustic, signal, and embedding layers. The study reveals that current defenses create tradeoffs between robustness and usability, highlighting the need for cost-aware safety evaluation beyond simple success-rate metrics.
Researchers propose CRaFT, a circuit-guided framework that identifies critical refusal features in large language models by analyzing inter-feature relationships rather than isolated activation signals. The method improves jailbreak attack success rates from 6.7% to 57.4% across benchmarks, advancing understanding of LLM safety mechanisms and highlighting vulnerabilities in model alignment.
Researchers propose TRACE, a credit assignment framework that improves multi-turn jailbreak attacks on large language models by identifying which dialogue turns actually contribute to harmful outcomes. The method achieves 25% higher attack success rates than existing approaches and can be repurposed to strengthen AI safety defenses.
A research paper argues that jailbreak attack evaluations should report distributional success rates across parameter configurations rather than single best-case scenarios. The authors propose two new metrics—Variant Sensitivity Measure (VSM) and Union Coverage (UC)—and demonstrate that attacks covering 81% in optimal configuration reach 100% coverage when all variants are tested, fundamentally changing threat assessments.
Researchers present a comprehensive framework for systematically generating, categorizing, and evaluating jailbreak attacks against large language models, introducing a dataset of 114,000 adversarial prompts, automated generation methods, and a novel continuous evaluation metric (OPTIMUS) that surpasses binary success rate measurements.
Researchers demonstrate that audio language models can be jailbroken using sparse token optimization rather than dense waveform updates, with Token-Aware Gradient Optimization (TAGO) achieving comparable attack success rates while modifying only 25% of audio tokens. The findings reveal that gradient energy concentrates in specific audio regions, suggesting future AI safety research should account for this heterogeneous token-level structure.
Researchers introduce Security Cube, a comprehensive evaluation framework for assessing Large Language Model robustness against jailbreak attacks. The study systematically catalogs existing attack and defense methods while establishing benchmarks across 13 attack vectors and 5 defense mechanisms, revealing critical gaps in current LLM safety practices.
Researchers have discovered a critical vulnerability in Large Reasoning Models (LRMs) like DeepSeek R1 and OpenAI o4-mini that allows attackers to inject harmful content into the reasoning process while keeping final answers unchanged. The Psychology-based Reasoning-targeted Jailbreak Attack (PRJA) framework achieves an 83.6% success rate by exploiting semantic triggers and psychological principles, revealing a previously understudied safety gap in AI systems deployed in high-stakes domains.
Researchers introduce MemJack, a multi-agent framework that exploits semantic vulnerabilities in Vision-Language Models through coordinated jailbreak attacks, achieving 71.48% attack success rates against Qwen3-VL-Plus. The study reveals that current VLM safety measures fail against sophisticated visual-semantic attacks and introduces MemJack-Bench, a dataset of 113,000+ attack trajectories to advance defensive research.
Researchers introduce TEMPLATEFUZZ, a fuzzing framework that systematically exploits vulnerabilities in LLM chat templates—a previously overlooked attack surface. The method achieves 98.2% jailbreak success rates on open-source models and 90% on commercial LLMs, significantly outperforming existing prompt injection techniques while revealing critical security gaps in production AI systems.
Researchers have developed Head-Masked Nullspace Steering (HMNS), a novel jailbreak technique that exploits circuit-level vulnerabilities in large language models by identifying and suppressing specific attention heads responsible for safety mechanisms. The method achieves state-of-the-art attack success rates with fewer queries than previous approaches, demonstrating that current AI safety defenses remain fundamentally vulnerable to geometry-aware adversarial interventions.
Researchers have discovered a new attack vulnerability in mobile vision-language agents where malicious prompts remain invisible to human users but are triggered during autonomous agent interactions. Using an optimization method called HG-IDA*, attackers can achieve 82.5% planning and 75.0% execution hijack rates on GPT-4o by exploiting the lack of touch signals during agent operations, exposing a critical security gap in deployed mobile AI systems.
Researchers identified critical security vulnerabilities in Diffusion Large Language Models (dLLMs) that differ from traditional autoregressive LLMs, stemming from their iterative generation process. They developed DiffuGuard, a training-free defense framework that reduces jailbreak attack success rates from 47.9% to 14.7% while maintaining model performance.
Researchers developed a genetic algorithm-based method using persona prompts to exploit large language models, reducing refusal rates by 50-70% across multiple LLMs. The study reveals significant vulnerabilities in AI safety mechanisms and demonstrates how these attacks can be enhanced when combined with existing methods.
Researchers developed Prefix-Shared KV Cache (PSKV), a new technique that accelerates jailbreak attacks on Large Language Models by 40% while reducing memory usage by 50%. The method optimizes the red-teaming process by sharing cached prefixes across multiple attack attempts, enabling more efficient parallel inference without compromising attack success rates.
Researchers have developed SAHA (Safety Attention Head Attack), a new jailbreak framework that exploits vulnerabilities in deeper attention layers of open-source large language models. The method improves attack success rates by 14% over existing techniques by targeting insufficiently aligned attention heads rather than surface-level prompts.
Researchers have developed a new 'untargeted jailbreak attack' (UJA) that can compromise AI safety systems in large language models with over 80% success rate using only 100 optimization iterations. This gradient-based attack method expands the search space by maximizing unsafety probability without fixed target responses, outperforming existing attacks by over 30%.
Researchers developed CC-BOS, a framework that uses classical Chinese text to conduct more effective jailbreak attacks on Large Language Models. The method exploits the conciseness and obscurity of classical Chinese to bypass safety constraints, using bio-inspired optimization techniques to automatically generate adversarial prompts.
