AIBearisharXiv – CS AI · May 97/10
🧠LeakDojo is a new research framework that systematically evaluates security vulnerabilities in Retrieval-Augmented Generation (RAG) systems, revealing that stronger LLM instruction-following capabilities correlate with higher data leakage risks. The study benchmarks six attack methods across multiple LLMs and datasets, providing critical insights into how RAG databases can be exploited and suggesting that improvements in RAG faithfulness may paradoxically increase security vulnerabilities.
AIBearisharXiv – CS AI · May 97/10
🧠Researchers have identified a critical vulnerability in LLM agents called Termination Poisoning, where adversaries inject malicious prompts to trick agents into believing tasks are incomplete, causing unbounded computation. The LoopTrap framework demonstrates this attack across 8 mainstream LLM agents with up to 25x step amplification, revealing systematic behavioral patterns that enable scalable red-teaming.
AIBearisharXiv – CS AI · May 97/10
🧠Researchers have identified a fundamental vulnerability in multimodal large language models where safety mechanisms can be bypassed by exploiting the tension between hiding harmful intent and maintaining reconstructability. The study demonstrates that character-removed text variants combined with keyword-related distractor images achieve effective jailbreaks, revealing that models' own reconstruction capabilities become a security liability.
AIBearisharXiv – CS AI · May 77/10
🧠Researchers demonstrate that LLM-based vulnerability detectors, increasingly used in software security pipelines, can be evaded through syntax-preserving code transformations. The study reveals that models with 70%+ accuracy on clean code can fail to detect 87%+ of vulnerabilities when subjected to minor edits, with adversarial attacks achieving up to 92.5% evasion rates—raising serious questions about the reliability of AI-driven security tools in production environments.
🧠 GPT-4
AIBearisharXiv – CS AI · May 77/10
🧠Researchers introduce DecodingTrust-Agent Platform (DTap), a red-teaming framework designed to systematically test AI agent vulnerabilities across 14 real-world domains. The platform includes an autonomous red-teaming agent (DTap-Red) that discovers attack strategies and a benchmarking dataset, revealing critical security gaps in popular AI agents that could enable API key theft, unauthorized transactions, and data deletion.
AIBearisharXiv – CS AI · May 47/10
🧠Researchers have demonstrated a novel white-box adversarial attack called Attention Redistribution Attack (ARA) that bypasses safety mechanisms in major large language models by redirecting attention away from safety-critical components using just 5 adversarial tokens. The attack reveals that AI safety emerges from attention routing patterns rather than localized, removable components, challenging current assumptions about how safety alignment works.
AIBearisharXiv – CS AI · May 47/10
🧠Researchers found that advanced jailbreaks against large language models impose minimal performance degradation on the most capable models, with frontier models like Claude Opus 4.6 losing only 7.7% of benchmark performance when compromised. This challenges the assumption that safety mechanisms inherently trade off capability, raising concerns that safety strategies relying on performance degradation are insufficient for protecting frontier AI systems.
🧠 Claude🧠 Haiku🧠 Opus
AIBearisharXiv – CS AI · May 47/10
🧠Researchers demonstrate four novel jailbreak techniques that exploit the visual modality of vision-language models to bypass safety alignment, revealing a significant gap between text-based and vision-based safety training. Testing across six frontier VLMs shows visual attacks achieve substantially higher success rates than equivalent textual attacks, with implications for the robustness of AI safety measures.
🧠 Claude
AIBearisharXiv – CS AI · May 17/10
🧠Researchers present the first comprehensive threat modeling of LLM-enabled robotic systems, mapping three categories of attacks (cyber, adversarial, and conversational) across the perception-planning-actuation pipeline. The analysis reveals critical architectural vulnerabilities where compromised inputs or unsafe model outputs can propagate to unsafe physical actions without proper validation boundaries.
AIBearisharXiv – CS AI · Apr 207/10
🧠Researchers introduce CREST-Search, a red-teaming framework that exposes vulnerabilities in web-augmented LLMs by crafting benign-seeming queries designed to trigger unsafe citations from the internet. The study reveals that integrating web search into language models creates new safety risks beyond traditional LLM harms, requiring specialized defensive strategies.
AIBearisharXiv – CS AI · Apr 207/10
🧠Researchers identify a critical vulnerability in federated learning systems where malicious 'dictator clients' can erase other participants' contributions while preserving their own, compromising the collaborative training process. The study provides theoretical and empirical analysis of single and multiple dictator scenarios, revealing fundamental security weaknesses in decentralized machine learning architectures.
AINeutralarXiv – CS AI · Apr 157/10
🧠Researchers have identified a critical vulnerability in large language models where safety guardrails fail across low-resource languages despite strong performance in high-resource ones. The team proposes LASA (Language-Agnostic Semantic Alignment), a new method that anchors safety protocols at the semantic bottleneck layer, dramatically reducing attack success rates from 24.7% to 2.8% on tested models.
AIBearisharXiv – CS AI · Apr 147/10
🧠Researchers discovered that large reasoning models (LRMs) like DeepSeek R1 and Llama become significantly more vulnerable to adversarial attacks when presented with conflicting objectives or ethical dilemmas. Testing across 1,300+ prompts revealed that safety mechanisms break down when internal alignment values compete, with neural representations of safety and functionality overlapping under conflict.
🧠 Llama
AIBearisharXiv – CS AI · Apr 147/10
🧠Researchers demonstrate critical vulnerabilities in watermarking techniques designed for autoregressive image generators, showing that watermarks can be removed or forged with access to only a single watermarked image and no knowledge of model secrets. These findings undermine the reliability of watermarking as a defense against synthetic content in training datasets and enable attackers to manipulate authentic images to falsely appear as AI-generated content.
AIBearisharXiv – CS AI · Apr 147/10
🧠Researchers have developed Adaptive Stealing (AS), a novel watermark stealing algorithm that exploits vulnerabilities in LLM watermarking systems by dynamically selecting optimal attack strategies based on contextual token states. This advancement demonstrates that existing fixed-strategy watermark defenses are insufficient, highlighting critical security gaps in protecting proprietary LLM services and raising urgent questions about watermark robustness.
AIBearisharXiv – CS AI · Apr 137/10
🧠Researchers demonstrate a critical vulnerability in diffusion-based language models where safety mechanisms can be bypassed by re-masking committed refusal tokens and injecting affirmative prefixes, achieving 76-82% attack success rates without gradient optimization. The findings reveal that dLLM safety relies on a fragile architectural assumption rather than robust adversarial defenses.
AINeutralarXiv – CS AI · Apr 107/10
🧠Researchers prove mathematically that no continuous input-preprocessing defense can simultaneously maintain utility, preserve model functionality, and guarantee safety against prompt injection attacks in language models with connected prompt spaces. The findings establish a fundamental trilemma showing that defenses must inevitably fail at some threshold inputs, with results verified in Lean 4 and validated empirically across three LLMs.
AIBearisharXiv – CS AI · Apr 107/10
🧠This research paper examines physical adversarial attacks on AI surveillance systems through a surveillance-oriented lens, emphasizing that robustness cannot be assessed from isolated image benchmarks alone. The study highlights critical gaps in current evaluation practices, including temporal persistence across frames, multi-modal sensing (visible and infrared), realistic attack carriers, and system-level objectives that must be tested under actual deployment constraints.
AIBullisharXiv – CS AI · Apr 77/10
🧠Researchers have developed CoopGuard, a new defense framework that uses cooperative AI agents to protect Large Language Models from sophisticated multi-round adversarial attacks. The system employs three specialized agents coordinated by a central system that maintains defense state across interactions, achieving a 78.9% reduction in attack success rates compared to existing defenses.
AIBearisharXiv – CS AI · Apr 67/10
🧠Research reveals that two methods for removing safety guardrails from large language models - jailbreak-tuning and weight orthogonalization - have significantly different impacts on AI capabilities. Weight orthogonalization produces models that are far more capable of assisting with malicious activities while retaining better performance, though supervised fine-tuning can help mitigate these risks.
AINeutralarXiv – CS AI · Mar 277/10
🧠Researchers have identified a new category of AI safety called 'reasoning safety' that focuses on protecting the logical consistency and integrity of LLM reasoning processes. They developed a real-time monitoring system that can detect unsafe reasoning behaviors with over 84% accuracy, addressing vulnerabilities beyond traditional content safety measures.
AIBearisharXiv – CS AI · Mar 277/10
🧠Research reveals that LLM system prompt configuration creates massive security vulnerabilities, with the same model's phishing detection rates ranging from 1% to 97% based solely on prompt design. The study PhishNChips demonstrates that more specific prompts can paradoxically weaken AI security by replacing robust multi-signal reasoning with exploitable single-signal dependencies.
AIBearisharXiv – CS AI · Mar 277/10
🧠Researchers have developed PIDP-Attack, a new cybersecurity threat that combines prompt injection with database poisoning to manipulate AI responses in Retrieval-Augmented Generation (RAG) systems. The attack method demonstrated 4-16% higher success rates than existing techniques across multiple benchmark datasets and eight different large language models.
AINeutralarXiv – CS AI · Mar 267/10
🧠Researchers have developed techniques to mitigate many-shot jailbreaking (MSJ) attacks on large language models, where attackers use numerous examples to override safety training. Combined fine-tuning and input sanitization approaches significantly reduce MSJ effectiveness while maintaining normal model performance.
AINeutralarXiv – CS AI · Mar 267/10
🧠Researchers developed Anti-I2V, a new defense system that protects personal photos from being used to create malicious deepfake videos through image-to-video AI models. The system works across different AI architectures by operating in multiple domains and targeting specific network layers to degrade video generation quality.
