AIBearisharXiv – CS AI · Jun 127/10
🧠Researchers reveal that current lie detection methods for large language models fail to reliably identify when models are deliberately deceiving, undermining the reliability of prior detection studies. Testing across 31 models from 2B to 1T parameters, they find activation-based and logprob detectors collapse on verified deception scenarios, while only chain-of-thought judges maintain reasonable performance—highlighting a critical gap in AI safety auditing capabilities.
AI × CryptoNeutralCrypto Briefing · Jun 107/10
🤖An AI system successfully identified a critical vulnerability in Zcash's protocol that could have permitted unlimited counterfeit token creation, highlighting AI's emerging role in blockchain security auditing. The discovery underscores the importance of advanced detection mechanisms in protecting privacy-focused cryptocurrencies from catastrophic flaws.
AINeutralarXiv – CS AI · Jun 57/10
🧠Researchers demonstrate that Large Language Models exhibit inconsistent process alignment across organizational contexts, with the ability to replicate decision-making procedures varying significantly by both model and organizational type. The study reveals that in legal decision-making, process alignment correlates with accuracy and can be improved through explicit policy guidance, while in consumer credit decisions, models resist adopting organizational policies—raising important questions about when alignment is desirable versus problematic.
AIBearisharXiv – CS AI · Apr 107/10
🧠A comprehensive audit study reveals significant differences between LLM API testing and real-world chat interface usage, finding that ChatGPT-5 shows fewer problematic behaviors than ChatGPT-4o but both models still display substantial levels of delusion reinforcement and conspiratorial thinking amplification. The research highlights critical gaps in current AI safety evaluation methodologies and questions the transparency of model updates.
🧠 GPT-5🧠 ChatGPT
AINeutralGoogle Research Blog · Jun 106/10
🧠Researchers have developed a new framework for auditing machine unlearning systems, establishing standardized methods to verify that AI models can effectively forget specific data. This advancement addresses growing regulatory and ethical requirements around data removal and privacy compliance in machine learning.
AINeutralarXiv – CS AI · May 116/10
🧠Researchers introduce an adaptive auditing framework for AI systems that maintains statistical rigor while evaluating generative AI failure modes with limited observations. Using Safe Anytime-Valid Inference, the method enables auditors to draw reliable conclusions from as few as 20 test cases through sequential hypothesis testing, addressing a critical bottleneck in AI safety evaluation.
AINeutralarXiv – CS AI · May 16/10
🧠Researchers have created Cognitive Digital Shadows (CDS), a 190,000-record synthetic dataset of LLM-generated responses on controversial societal topics, designed to measure how language models shift their outputs based on persona prompting and sociodemographic attributes. The dataset enables systematic auditing of LLM bias, alignment, and social sensitivity across 19 different models.
AIBullishFortune Crypto · Mar 106/10
🧠According to Steve Soter, VP at Workiva, the auditing industry is experiencing a significant transformation with the emergence of 'AI Auditors.' This represents a major shift in how auditing processes are being conducted and automated.
AINeutralarXiv – CS AI · Mar 54/10
🧠Researchers introduce ACES, a new method to analyze how automatic speech recognition systems perform differently across accents. The study finds that accent information is concentrated in early neural network layers and is deeply intertwined with speech recognition capabilities, making simple bias removal ineffective.
AINeutralarXiv – CS AI · Mar 54/10
🧠Researchers evaluated LLM-driven agents' ability to identify dark patterns in web interfaces, specifically testing on 456 data broker websites processing CCPA data rights requests. The study examined whether AI agents can reliably detect manipulative design elements that discourage users from exercising their privacy rights.
