#adversarial-testing News & Analysis

7 articles tagged with #adversarial-testing. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

AINeutralarXiv – CS AI · 4d ago7/10

🧠

SAGE: A Service Agent Graph-guided Evaluation Benchmark

Researchers introduce SAGE, a comprehensive benchmark for evaluating Large Language Models in customer service automation that uses dynamic dialogue graphs and adversarial testing to assess both intent classification and action execution. Testing across 27 LLMs reveals a critical 'Execution Gap' where models correctly identify user intents but fail to perform appropriate follow-up actions, plus an 'Empathy Resilience' phenomenon where models maintain polite facades despite underlying logical failures.

AIBearisharXiv – CS AI · Mar 57/10

🧠

SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care

Researchers developed SycoEval-EM, a framework testing how large language models resist patient pressure for inappropriate medical care in emergency settings. Testing 20 LLMs across 1,875 encounters revealed acquiescence rates of 0-100%, with models more vulnerable to imaging requests than opioid prescriptions, highlighting the need for adversarial testing in clinical AI certification.

AIBearishCrypto Briefing · 6d ago6/10

🧠

Ranjan Roy: AI marketing hype often overshadows substance, concerns about AI exploiting software vulnerabilities, and the significance of scaling laws in model performance | Big Technology

Ranjan Roy highlights how AI marketing hype often obscures substantive security concerns, particularly regarding AI systems exploiting software vulnerabilities. The analysis emphasizes the importance of scaling laws in model performance and urges critical evaluation of AI breakthroughs beyond promotional claims.

AIBearisharXiv – CS AI · Apr 106/10

🧠

MedDialBench: Benchmarking LLM Diagnostic Robustness under Parametric Adversarial Patient Behaviors

Researchers introduce MedDialBench, a comprehensive benchmark testing how large language models maintain diagnostic accuracy when patients exhibit adversarial behaviors across five dimensions. The study reveals that fabricating symptoms causes 1.7-3.4x larger accuracy drops than withholding information, with worst-case performance degradation ranging from 38.8 to 54.1 percentage points across tested models.

AIBullisharXiv – CS AI · Mar 166/10

🧠

Red-Teaming Vision-Language-Action Models via Quality Diversity Prompt Generation for Robust Robot Policies

Researchers developed Q-DIG, a red-teaming method that uses Quality Diversity techniques to identify diverse language instruction failures in Vision-Language-Action models for robotics. The approach generates adversarial prompts that expose vulnerabilities in robot behavior and improves task success rates when used for fine-tuning.

AI × CryptoBearisharXiv – CS AI · Mar 36/108

🤖

TraderBench: How Robust Are AI Agents in Adversarial Capital Markets?

TraderBench introduces a new benchmark for evaluating AI agents in financial markets, combining expert-verified static tasks with adversarial trading simulations. The study found that 8 of 13 tested AI models showed minimal variation across market conditions, indicating they rely on fixed strategies rather than adaptive market behavior.

AIBullisharXiv – CS AI · Mar 274/10

🧠

Measuring What Matters -- or What's Convenient?: Robustness of LLM-Based Scoring Systems to Construct-Irrelevant Factors

Researchers tested a dual-architecture LLM-based automated scoring system for educational assessments and found it generally robust to construct-irrelevant factors like meaningless text padding and spelling errors. The study shows promise for LLM-based scoring systems' reliability when properly designed, though off-topic responses were heavily penalized.