31 articles tagged with #ai-defense. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers propose a formal threat model framework for evaluating distillation defenses against black-box LLM attacks, arguing that existing output perturbation defenses lack clear specifications about attacker capabilities. The work demonstrates that defense effectiveness depends heavily on assumed threat parameters, raising concerns about false security claims in deployed systems.
France is developing Arcadia, an AI system designed as a European alternative to Palantir's intelligence platform, aiming to strengthen digital sovereignty and reduce dependence on US technology. The initiative aligns with broader European efforts to build independent technological capabilities for defense and governance sectors.
Anthropic has embedded engineers at the NSA to deploy Mythos AI for cyber operations, signaling institutional adoption of advanced AI systems for government security purposes. This collaboration underscores the strategic importance of AI in national defense infrastructure and the growing convergence between private AI companies and government agencies.
Merlin (MRLN) stock jumped 32% after-hours following a successful Critical Design Review for its C-130J autonomy program under a USSOCOM contract valued at $100M+. This milestone represents significant progress in defense-sector autonomous systems development.
Researchers introduce DiscourseFlip, a novel attack method against Retrieval-Augmented Generation (RAG) systems that manipulates opinions across multiple related queries by poisoning retrieval content at the discourse level. Unlike previous attacks targeting individual queries, this coordinated approach induces broader opinion shifts while evading detection, and existing defenses prove ineffective against it.
Researchers reveal a critical vulnerability in LLM agents operating in local workspaces, where attackers can plant hidden prompt injections across multiple steps to gain persistent control. The new ClawTrojan benchmark demonstrates 95.5% attack success rates against GPT-5.4, while a proposed defense mechanism called DASGuard offers runtime protection by tracing and sanitizing potentially malicious control text in sensitive files.
The US Navy has deployed AI-enhanced mine detection technology to counter Iranian naval threats in the Strait of Hormuz, potentially reducing geopolitical tensions in one of the world's most critical shipping chokepoints. This development could stabilize global oil trade routes and lower conflict risks in a region where maritime disruptions have significant economic implications.
The US Navy is implementing AI-powered mine detection technology in the Strait of Hormuz to accelerate maritime clearance operations. This initiative aims to enhance security along one of the world's most critical oil trade routes and stabilize global energy markets by 2026.
Researchers propose HyPE and HyPS, a two-part defense framework using hyperbolic geometry to detect and neutralize harmful prompts in Vision-Language Models. The approach offers a lightweight, interpretable alternative to blacklist filters and classifier-based systems that are vulnerable to adversarial attacks.
Researchers have developed CoopGuard, a new defense framework that uses cooperative AI agents to protect Large Language Models from sophisticated multi-round adversarial attacks. The system employs three specialized agents coordinated by a central system that maintains defense state across interactions, achieving a 78.9% reduction in attack success rates compared to existing defenses.
Researchers identified critical security vulnerabilities in Diffusion Large Language Models (dLLMs) that differ from traditional autoregressive LLMs, stemming from their iterative generation process. They developed DiffuGuard, a training-free defense framework that reduces jailbreak attack success rates from 47.9% to 14.7% while maintaining model performance.
Researchers have identified a new attack vector called Epistemic Bias Injection (EBI) that manipulates AI language models by injecting factually correct but biased content into retrieval-augmented generation databases. The attack steers model outputs toward specific viewpoints while evading traditional detection methods, though a new defense mechanism called BiasDef shows promise in mitigating these threats.
Researchers developed a genetic algorithm-based method using persona prompts to exploit large language models, reducing refusal rates by 50-70% across multiple LLMs. The study reveals significant vulnerabilities in AI safety mechanisms and demonstrates how these attacks can be enhanced when combined with existing methods.
Researchers introduce GroupGuard, a defense framework to combat coordinated attacks by multiple AI agents in collaborative systems. The study shows group collusive attacks increase success rates by up to 15% compared to individual attacks, while GroupGuard achieves 88% detection accuracy in identifying and isolating malicious agents.
Researchers introduced VideoSafetyEval, a benchmark revealing that video-based large language models have 34.2% worse safety performance than image-based models. They developed VideoSafety-R1, a dual-stage framework that achieves 71.1% improvement in safety through alarm token-guided fine-tuning and safety-guided reinforcement learning.
Researchers developed Sysformer, a novel approach to safeguard large language models by adapting system prompts rather than fine-tuning model parameters. The method achieved up to 80% improvement in refusing harmful prompts while maintaining 90% compliance with safe prompts across 5 different LLMs.
Researchers propose a game-theoretic framework using Stackelberg equilibrium and Rapidly exploring Random Trees to model interactions between attackers trying to jailbreak LLMs and defensive AI systems. The framework provides a mathematical foundation for understanding and improving AI safety guardrails against prompt-based attacks.
Researchers introduce WARP, a new defense mechanism for machine unlearning protocols that protects against privacy attacks where adversaries can exploit differences between pre- and post-unlearning AI models. The technique reduces attack success rates by up to 92% while maintaining model accuracy on retained data.
Researchers propose T2S, a rehearsal-based watermarking framework that protects AI models against extraction attacks by simulating the theft process during training. The method embeds watermarks that remain detectable even when adversaries steal and replicate models, addressing a critical vulnerability in AI intellectual property protection.
Researchers present a game-theoretic framework analyzing the tension between model utility and distillation vulnerability, introducing Product-of-Experts (PoE) as an efficient defense mechanism. Their adaptive evaluation methodology reveals that existing defenses are significantly weaker against adaptive attacks than passive evaluation suggests, challenging current benchmarking practices in AI security.
Researchers demonstrate that Large Language Model-based multi-agent systems are vulnerable to coordinated attacks where malicious agents collaborate to spread misinformation more effectively than independent attackers. They propose STAR, a defense mechanism using sentence-level analysis that recovers 36.76% of lost performance by identifying and correcting misleading information in agent communications.
OpenAI has expanded its Trusted Access for Cyber program by introducing GPT-5.5 and a specialized GPT-5.5-Cyber model to help verified cybersecurity defenders accelerate vulnerability research and strengthen critical infrastructure protection. This initiative enables authorized security professionals to leverage advanced AI capabilities for defensive purposes while maintaining controlled access.
Artemis has secured $70 million in funding to develop AI-powered defense systems against increasingly sophisticated AI-driven cyberattacks. The funding reflects growing market demand for advanced security solutions as AI-enabled threats become faster and more cost-effective to deploy.
Researchers introduce Critical-CoT, a defense framework that protects large language models against reasoning-level backdoor attacks by fine-tuning models to develop critical thinking behaviors. Unlike token-level backdoors, these attacks inject malicious reasoning steps into chain-of-thought processes, making them harder to detect; the proposed defense demonstrates strong robustness across multiple LLMs and datasets.
Researchers propose CanaryRAG, a runtime defense mechanism that protects Retrieval-Augmented Generation systems from adversarial attacks that extract proprietary data from knowledge bases. The solution uses embedded canary tokens to detect leakage in real-time while maintaining normal system performance, offering a practical safeguard for organizations deploying RAG-based AI systems.
