AINeutralarXiv – CS AI · Apr 147/10
🧠
Evaluating Reliability Gaps in Large Language Model Safety via Repeated Prompt Sampling
Researchers introduce Accelerated Prompt Stress Testing (APST), a new evaluation framework that reveals safety vulnerabilities in large language models through repeated prompt sampling rather than traditional broad benchmarks. The study finds that models appearing equally safe in conventional testing show significant reliability differences when repeatedly queried, indicating current safety benchmarks may mask operational risks in deployed systems.