AIBearisharXiv – CS AI · 4d ago7/10
🧠Researchers demonstrate that LLM agents are vulnerable to credential exfiltration attacks when sensitive data shares context windows with untrusted content, enabling indirect prompt injection. The study proposes three defense mechanisms: activation probes for pre-output detection, honeytokens with calibrated thresholds, and multi-turn leakage accounting to prevent cumulative credential theft across conversations.
AIBearisharXiv – CS AI · May 297/10
🧠Researchers introduce GEO-Bench, a standardized benchmark for evaluating ranking manipulation attacks against large language models used in generative search. The study compares black-box and white-box adversarial attacks, revealing that simpler content-rewriting methods can match gradient-based approaches while remaining more difficult to detect.
🏢 Perplexity🧠 Llama
AIBearisharXiv – CS AI · May 297/10
🧠Researchers demonstrate that LoRA adapters, widely used for fine-tuning large language models, can be backdoored through training data poisoning while maintaining clean performance. The backdoor generalizes at the token level rather than structural patterns, making it harder for defenders to detect generically. Two complementary detection methods—behavioral probing and weight-level analysis—successfully identify poisoned adapters without false positives.
AINeutralarXiv – CS AI · May 117/10
🧠Researchers propose a geometric framework for detecting hallucinations in large language models by analyzing embedding space structure, categorizing three types of errors with different detectability profiles. The approach outperforms standard NLI baselines on expert-annotated datasets, providing interpretable diagnostics for production systems operating under black-box constraints.
AIBearisharXiv – CS AI · Mar 37/103
🧠New research reveals that benchmark contamination in language reasoning models (LRMs) is extremely difficult to detect, allowing developers to easily inflate performance scores on public leaderboards. The study shows that reinforcement learning methods like GRPO and PPO can effectively conceal contamination signals, undermining the integrity of AI model evaluations.
$NEAR
AINeutralarXiv – CS AI · Mar 37/104
🧠IARPA's TrojAI program investigated AI Trojans - malicious backdoors hidden in AI models that can cause system failures or allow unauthorized control. The multi-year initiative developed detection methods through weight analysis and trigger inversion, while identifying ongoing challenges in AI security that require continued research.
AINeutralarXiv – CS AI · Feb 277/105
🧠Researchers have developed a new decision-theoretic framework to detect steganographic capabilities in large language models, which could help identify when AI systems are hiding information to evade oversight. The method introduces 'generalized V-information' and a 'steganographic gap' measure to quantify hidden communication without requiring reference distributions.
AINeutralarXiv – CS AI · 4d ago6/10
🧠Researchers have released AITDNA, a new benchmark dataset for detecting AI-generated text that includes detailed edit histories and human-machine co-creation information. The study reveals that existing AI text detectors perform inconsistently across different types of AI-generated content, highlighting the need for standardized definitions of what constitutes problematic AI-generated text and more robust detection methods.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers have developed Synthesis Data Reversion (SDR), a technique to detect unauthorized LLM training data even when that data has been deliberately obfuscated through stylistic transformation. The method works by inferring laundering patterns and generating synthetic queries that mimic the transformed data, effectively countering data laundering practices that previously evaded detection.
🧠 Llama
AINeutralarXiv – CS AI · May 116/10
🧠Researchers have developed Token Probability Deviation (TPD), a method to detect whether questions were included in a reasoning model's distillation training data. The technique addresses data contamination risks in reasoning distillation, where benchmark data may inadvertently inflate model performance metrics, achieving up to 31% improvement in detection accuracy.
AINeutralarXiv – CS AI · Mar 36/103
🧠Researchers introduce FaithCoT-Bench, the first comprehensive benchmark for detecting unfaithful Chain-of-Thought reasoning in large language models. The benchmark includes over 1,000 expert-annotated trajectories across four domains and evaluates eleven detection methods, revealing significant challenges in identifying unreliable AI reasoning processes.
