AIBearisharXiv – CS AI · Jun 57/10
🧠Researchers demonstrate a reinforcement learning approach that enables AI agents to learn and execute adversarial attacks on machine learning models more efficiently than traditional methods. The RL-based system achieves 13.2% higher attack success rates and reduces queries needed per attack by 16.9%, while outperforming state-of-the-art adversarial methods by 17% on unseen inputs, revealing a significant new security vulnerability in deployed ML systems.
AIBearisharXiv – CS AI · Jun 57/10
🧠Researchers have developed a new adversarial attack method against automatic speech recognition systems that operates in feature space rather than directly on audio waveforms, achieving significantly higher transfer rates to black-box ASR models and bypassing existing defenses. The attack uses self-supervised learning representations and vocoders to reconstruct adversarial signals, revealing critical vulnerabilities in current ASR robustness evaluation protocols.
AIBearisharXiv – CS AI · Jun 47/10
🧠Researchers introduce MaskForge, a black-box attack method that exploits structural vulnerabilities in diffusion-based large language models (dLLMs) by leveraging their native masking capabilities. The technique achieves 79.3% average success rates across five models and transfers effectively to other benchmarks, demonstrating a significant security gap in an emerging class of language models distinct from standard autoregressive architectures.
AIBearisharXiv – CS AI · Jun 47/10
🧠Researchers demonstrate that LLM agents are vulnerable to credential exfiltration attacks when sensitive data shares context windows with untrusted content, enabling indirect prompt injection. The study proposes three defense mechanisms: activation probes for pre-output detection, honeytokens with calibrated thresholds, and multi-turn leakage accounting to prevent cumulative credential theft across conversations.
AIBearisharXiv – CS AI · Jun 47/10
🧠Researchers have identified systematic vulnerabilities in LLM-based AI agents that enable memory poisoning attacks, where adversaries inject malicious data into persistent memory to manipulate long-term agent behavior. The study reveals four memory write channels and nine structural vulnerabilities across system design, with existing security defenses proving ineffective against this threat vector.
AIBearisharXiv – CS AI · Jun 47/10
🧠Researchers have identified a critical security vulnerability in agentic AI systems called cross-session stored prompt injection, where malicious instructions can persist within system state and compromise future interactions long after the attacker disconnects. This threat fundamentally differs from traditional prompt injection by leveraging long-lived system artifacts like memories and filesystems, transforming ephemeral model-level attacks into durable system-level vulnerabilities that accumulate over time.
AIBearisharXiv – CS AI · Jun 47/10
🧠Researchers demonstrate that offline bandit algorithms—used to evaluate machine learning models like image generators and LLMs—are vulnerable to adversarial attacks on their reward models. The study reveals that in high-dimensional settings, attackers can achieve near-perfect success rates with imperceptibly small perturbations to publicly available reward model weights, creating a critical security gap in AI evaluation systems.
🏢 Hugging Face
AIBearisharXiv – CS AI · Jun 27/10
🧠Researchers have developed an A*-inspired framework that generates obfuscated prompts capable of triggering factual errors in large language models while preserving semantic intent. The method uses a hierarchical rewrite strategy with dynamic semantic dispersion to efficiently create adversarial prompts, demonstrating higher attack success rates than existing approaches and raising urgent concerns about LLM reliability in safety-critical applications.
AIBearisharXiv – CS AI · Jun 27/10
🧠Researchers demonstrate that adversarial patches—printable patterns designed to fool AI object detectors—can be physically deployed against aerial vehicle detection systems with significant effectiveness. The study reveals that patches placed directly on vehicles outperform digitally-optimized designs in real-world conditions, exposing critical vulnerabilities in deep neural network-based detection systems used for surveillance and monitoring applications.
AIBearisharXiv – CS AI · Jun 27/10
🧠Researchers present DEPO, a reinforcement learning algorithm that enables large language models to evade AI-text detectors through paraphrasing while maintaining semantic fidelity. The constrained optimization approach treats detector evasion as the primary objective with semantic preservation as an explicit constraint, demonstrating robust performance across multiple detectors and datasets.
AIBearisharXiv – CS AI · Jun 17/10
🧠Researchers demonstrate that LLM agent vulnerabilities to prompt injection attacks vary dramatically depending on the injection surface used, with the same attack payload showing 96% success on one model via tool outputs but only 4% via tool descriptions. The study reveals that vulnerability is determined by model-surface interaction rather than the injection channel alone, exposing critical blindspots in current AI security evaluation methodology.
🧠 GPT-4
AIBearisharXiv – CS AI · Jun 17/10
🧠Researchers demonstrate a novel adversarial attack using genetic algorithm-based prompt injection that can deceive LLM-powered reverse engineering tools like GhidraMCP into misinterpreting binary executables. This vulnerability exploits how large language models process decompiled code through surreptitious string variable assignments, potentially allowing malware to bypass automated detection systems that rely on AI-driven analysis.
AIBearisharXiv – CS AI · Jun 17/10
🧠Researchers identified that indirect prompt injection attacks against ReAct AI agents succeed at dramatically different rates depending on where malicious payloads appear in tool sequences, with success rates dropping from 60% at the first tool observation to 0% at deeper positions. The study reveals that payload framing and conversation turn limits have minimal impact on attack success, making injection depth the critical vulnerability factor for AI agent systems handling real-world tasks.
🧠 GPT-4🧠 Claude
AINeutralarXiv – CS AI · Jun 17/10
🧠Researchers identify two critical vulnerabilities in machine unlearning techniques: over-unlearning that damages nearby data and prototypical relearning attacks that can restore forgotten information. They propose Spotter, a new method combining masked knowledge-distillation and intra-class dispersion losses to address both security gaps in class-level unlearning.
AIBearisharXiv – CS AI · May 297/10
🧠Researchers conducted the first systematic study of prompt injection attacks in real-world LLM-based resume screening, analyzing approximately 200,000 resumes from hireEZ. They found that ~1% of resumes contain hidden prompt injections, with prevalence increasing significantly over the past 1-2 years, and discovered that over 90% of injected prompts use subtle methods rather than explicit instructions.
AIBearisharXiv – CS AI · May 297/10
🧠Researchers introduce GEO-Bench, a standardized benchmark for evaluating ranking manipulation attacks against large language models used in generative search. The study compares black-box and white-box adversarial attacks, revealing that simpler content-rewriting methods can match gradient-based approaches while remaining more difficult to detect.
🏢 Perplexity🧠 Llama
AIBearisharXiv – CS AI · May 297/10
🧠A comprehensive arXiv research review examines vulnerabilities in Large Language Models, particularly prompt injection and jailbreaking attacks, while analyzing existing defense mechanisms. The study identifies critical security gaps and proposes future research directions for safer LLM deployment across applications.
AIBearisharXiv – CS AI · May 287/10
🧠Researchers have identified a new vulnerability in LLM-based agents called 'Sleeper Attacks,' where adversarial content persists dormant in agent state across multiple interactions before being activated by benign queries. The attack threatens real-world LLM deployments by evading single-interaction detection mechanisms, with testing showing vulnerabilities across seven major language models.
AIBearisharXiv – CS AI · May 287/10
🧠Researchers demonstrate that large language model refusal behavior can be detected and exploited through intermediate layer activations before final output generation. A new attack method called Mechanistic AutoDAN leverages this discovery to achieve competitive jailbreak success rates while reducing computational time by up to 72%, raising concerns about LLM safety mechanisms.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers propose SPARD, a defense framework that protects large language models from harmful fine-tuning attacks by combining safety-constrained optimization with intelligent data selection. The method maintains task performance while significantly reducing adversarial attacks that attempt to remove safety guardrails from AI systems.
AIBearisharXiv – CS AI · May 287/10
🧠Researchers demonstrate MIRAGE, a technique that exploits vision-language model vulnerabilities in mobile GUI agents by injecting adversarial text into user-generated content regions. The attack achieves 23-30% success rates across five VLM agents without modifying apps or operating systems, revealing a critical security gap in AI-powered mobile automation that existing visual-quality defenses cannot reliably prevent.
AIBearisharXiv – CS AI · May 287/10
🧠Researchers present MM-PoisonRAG, a framework demonstrating critical vulnerabilities in multimodal RAG systems where adversaries can inject poisoned content into knowledge bases to manipulate AI outputs. Two attack strategies—localized poisoning targeting specific queries and globalized poisoning affecting all queries—achieve high success rates and bypass existing defenses, exposing fundamental security gaps in RAG-augmented language models.
AIBearisharXiv – CS AI · May 287/10
🧠Researchers demonstrate a practical attack called Bias-Inversion Rewriting Attack (BIRA) that defeats LLM watermarking schemes with over 99% success rate while maintaining semantic quality. The findings expose fundamental vulnerabilities in current watermarking detection methods, which are widely considered essential for identifying AI-generated content.
AINeutralarXiv – CS AI · May 277/10
🧠Researchers propose SALO, a jailbreak detection method that identifies persistent 'refusal trajectories' across model layers, rather than relying on static terminal representations. The detector demonstrates improved detection rates against adversarial attacks on multiple LLM architectures, though with acknowledged limitations against adaptive attacks.
🧠 Llama
AIBearisharXiv – CS AI · May 277/10
🧠Researchers introduce MemMorph, a novel attack method that compromises LLM-driven agents by poisoning their long-term memory modules rather than manipulating tool metadata. The attack achieves up to 85.9% success rates by injecting crafted records disguised as technical facts, exposing a critical security vulnerability in memory-augmented AI systems that existing defenses fail to address.
