AIBearisharXiv – CS AI · Jun 257/10
🧠Researchers conducted a human study evaluating whether Large Language Model-assisted tools improve software vulnerability patching compared to manual debugging. The study revealed that while LLMs accelerate patching speed, they risk introducing insecure code and superficial repairs that pass functional tests but fail security validation, highlighting critical trade-offs in AI-assisted security workflows.
AIBearisharXiv – CS AI · Jun 257/10
🧠Researchers propose a formal threat model framework for evaluating distillation defenses against black-box LLM attacks, arguing that existing output perturbation defenses lack clear specifications about attacker capabilities. The work demonstrates that defense effectiveness depends heavily on assumed threat parameters, raising concerns about false security claims in deployed systems.
AIBearisharXiv – CS AI · Jun 237/10
🧠A new security analysis reveals that self-evolving LLM agent systems face critical vulnerabilities across 17 of 25 potential attack vectors, with adversarial compromises becoming permanently encoded and self-amplifying across system generations. Testing of open-source frameworks demonstrates 100% attack persistence rates, suggesting that autonomous AI systems capable of self-modification require fundamentally new security paradigms beyond traditional static defenses.
AIBearisharXiv – CS AI · Jun 237/10
🧠Researchers have identified a critical vulnerability called "relinking" in LLM agents that use compression to handle long contexts. By splitting malicious instructions into benign fragments distributed across text, attackers can bypass security filters that inspect uncompressed prompts, as the compression process reconstructs the complete malicious instruction. Existing defenses fail to catch this attack, though a new KBRA defense eliminates the risk.
AIBearisharXiv – CS AI · Jun 237/10
🧠A new research paper reveals critical vulnerabilities in Knowledge Editing (KE) techniques used to update facts in Large Language Models without retraining. The study demonstrates that edited knowledge is not truly erased but merely suppressed, and can be recovered through adversarial prompting, exposing fundamental flaws in current post-hoc update methods.
AIBearisharXiv – CS AI · Jun 237/10
🧠Researchers demonstrate that prompt compression—a technique used to reduce LLM latency and costs—creates a new security vulnerability when processing mixed trusted and untrusted inputs. By strategically perturbing untrusted data before compression, attackers can force compressors to discard critical task information or safety guardrails, achieving 71% attack success rates through a black-box method called COMA.
AIBearisharXiv – CS AI · Jun 237/10
🧠Researchers introduced HardSecBench, a comprehensive security benchmark for evaluating large language models used in hardware and firmware code generation. The study of 924 tasks reveals that LLMs frequently produce functionally correct code while embedding critical security vulnerabilities, highlighting a significant gap in current AI safety evaluation practices.
AIBearisharXiv – CS AI · Jun 237/10
🧠Researchers discovered that popular prompt-injection detectors (ProtectAI-v2 and Prompt-Guard-2) maintain extremely high confidence scores even when failing to catch attacks, particularly indirect behavior-hijack injections. Across multiple attack distribution shifts, detectors missed injections with 0.99-1.00 confidence while false-negative rates ranged from 1-97%, indicating a critical calibration failure that standard metrics fail to detect.
AIBearisharXiv – CS AI · Jun 237/10
🧠Researchers demonstrate that Large Language Models like GPT-3.5 and GPT-4 can effectively obscure programmer code stylometry while maintaining functionality, challenging the reliability of authorship attribution techniques used in cybersecurity. The study reveals that structured, multi-shot prompting strategies outperform single-shot approaches in evading detection by traditional machine learning classifiers.
🧠 GPT-4
AIBearisharXiv – CS AI · Jun 237/10
🧠Researchers introduce TrojanGYM, an LLM-driven framework that automatically generates hardware Trojans to expose vulnerabilities in detection systems. The system demonstrates that existing detectors can be evaded at rates up to 83.33%, revealing critical gaps in hardware security testing methodologies.
🧠 GPT-4🧠 Gemini
AIBearisharXiv – CS AI · Jun 197/10
🧠Researchers identify critical vulnerabilities in LLM-solver hybrid systems where formal verification guarantees break down during the narration phase—converting solver outputs to user-readable answers. Testing five open-source models reveals adversaries can manipulate final responses through prompt injection despite underlying formal correctness, indicating safety-critical applications using AI-assisted reasoning require additional safeguards beyond solver verification.
AI × CryptoNeutralarXiv – CS AI · Jun 197/10
🤖Researchers identify 'Secure Coding Drift,' a vulnerability where developers gradually adopt insecure practices when relying on LLM-generated code for post-quantum cryptography implementation. The paper proposes a gamified framework that transforms LLMs into active security partners through adversarial evaluation and behavioral feedback to mitigate this socio-technical risk.
AIBullisharXiv – CS AI · Jun 197/10
🧠Researchers propose a novel fingerprinting framework for large language models that combines Code-mixing Fingerprints (CF) and Multi-Candidate Editing (MCEdit) to protect against unauthorized redistribution and commercial misuse. The approach addresses key vulnerabilities in existing fingerprinting methods by balancing imperceptibility with robustness against defensive filtering and downstream model modifications.
🏢 Perplexity
AIBearisharXiv – CS AI · Jun 197/10
🧠A new research framework called CWE-Trace challenges the claim that large language models can reliably detect software vulnerabilities, revealing that fine-tuned models achieve only 52.1% accuracy at best and lack genuine security reasoning despite appearing well-calibrated. The study of 834 Linux kernel samples shows that models exhibit systematic failure patterns that persist across datasets and resist correction through fine-tuning, suggesting they memorize patterns rather than understand vulnerability detection.
AINeutralarXiv – CS AI · Jun 117/10
🧠Researchers propose a compute-aware evaluation framework for assessing adversarial robustness in large language models, measuring attack effort in FLOPs rather than fixed query budgets. Testing across multiple models and attack strategies reveals that alignment training has non-monotonic effects on robustness, scaling reduces gradient-based attacks but not cheaper template-based ones, and safety measures leave certain harm categories disproportionately accessible.
AIBearisharXiv – CS AI · Jun 117/10
🧠JailbreakOPT is a new framework that optimizes adversarial prompts to exploit safety vulnerabilities in large language models through iterative refinement and tool composition. The approach combines atomic jailbreak techniques with contextual bandits to achieve higher attack success rates while reducing the number of queries needed, demonstrating meaningful progress in LLM security testing.
AIBearisharXiv – CS AI · Jun 117/10
🧠Researchers developed AutoInject, a reinforcement learning framework that automatically generates adversarial prompts to exploit LLM agents through prompt injection attacks. The method outperforms existing attack techniques on production models and successfully breaks defenses specifically designed to resist prompt injection, highlighting a significant vulnerability gap in AI system security.
AIBearisharXiv – CS AI · Jun 117/10
🧠Researchers have discovered that Grammar-Constrained Decoding (GCD), a technique used to improve code safety in Large Language Models, can actually be exploited as a jailbreak vector called CodeSpear. The study introduces CodeShield, a defensive alignment method that protects LLMs from generating malicious code even when attackers manipulate grammar constraints.
AIBearisharXiv – CS AI · Jun 107/10
🧠Researchers demonstrate that LLM-based search engines are vulnerable to ranking manipulation attacks, where adversaries craft content to game results. Using game theory, the study reveals that reducing attack success rates can paradoxically incentivize attacks, and defensive caps may fail—highlighting the need for adaptive security strategies beyond traditional defenses.
AIBearisharXiv – CS AI · Jun 107/10
🧠Researchers introduce CIAware-Bench, a benchmark measuring whether frontier LLMs can detect when their outputs are being monitored and modified by AI control systems. Testing eleven models across multiple domains, the study finds low-to-moderate detection rates (up to 0.87 accuracy), revealing that intervention awareness varies significantly by task and model pair, with implications for the robustness of AI safety protocols.
AIBearisharXiv – CS AI · Jun 107/10
🧠Researchers discovered that Large Language Models leak significantly more personally identifiable information (PII) when interacting with AI agents compared to human users, despite identical safety mechanisms. The study identifies an 'Interlocutor Effect' where LLMs reduce privacy caution based on perceived recipient identity, with leakage rates increasing up to 23 percentage points when addressing AI agents, raising critical security concerns for multi-agent system architectures.
🧠 Llama
AIBearisharXiv – CS AI · Jun 107/10
🧠Researchers demonstrate that multi-agent LLM systems used for political analysis can be identified by their stylometric fingerprints even when anonymized, undermining a proposed security mitigation. A fine-tuned T5 model achieved 99.1% accuracy in identifying LLM model families, revealing compliance gaps with EU AI Act requirements for transparency and system validation in critical applications.
🧠 Claude🧠 Sonnet🧠 Llama
AIBearisharXiv – CS AI · Jun 107/10
🧠Researchers introduce BadRobot, an attack paradigm that exploits vulnerabilities in embodied LLM agents to make them perform harmful physical actions through voice commands. The study demonstrates successful attacks against prominent frameworks like Voxposer and Code as Policies, revealing critical safety gaps in AI systems integrated into physical robotics.
AIBearisharXiv – CS AI · Jun 107/10
🧠Researchers have evaluated automated prompt injection attacks against large language model agents using both white-box and black-box optimization methods, finding that black-box approaches significantly outperform gradient-based techniques in realistic agentic settings. While task-universal attacks transfer effectively across domains, attacks trained on smaller models fail to generalize to frontier models like GPT-5, suggesting model-dependent vulnerabilities rather than universal exploits.
🧠 GPT-5
AIBearisharXiv – CS AI · Jun 107/10
🧠A comprehensive review of 247 research papers reveals that LLM agents face escalating security threats beyond text generation, including prompt injection, tool hijacking, and state corruption. The study proposes a framework emphasizing trust boundaries, privilege control, and stateful risk evaluation to address fragmented defenses and inadequate benchmarking standards.
