#machine-learning-reliability News & Analysis

3 articles tagged with #machine-learning-reliability. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBearisharXiv – CS AI · Jun 197/10

🧠

Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

A new research framework called CWE-Trace challenges the claim that large language models can reliably detect software vulnerabilities, revealing that fine-tuned models achieve only 52.1% accuracy at best and lack genuine security reasoning despite appearing well-calibrated. The study of 834 Linux kernel samples shows that models exhibit systematic failure patterns that persist across datasets and resist correction through fine-tuning, suggesting they memorize patterns rather than understand vulnerability detection.

AINeutralarXiv – CS AI · Jun 16/10

🧠

Conditional Coverage Diagnostics for Conformal Prediction

Researchers introduce Excess Risk of Target Coverage (ERT), a new metric framework for evaluating conditional coverage in conformal prediction systems. The approach reformulates coverage assessment as a classification problem, providing more statistically powerful diagnostics than existing methods while offering conservative estimates of miscoverage and enabling distinction between over- and under-coverage effects.

AINeutralarXiv – CS AI · May 116/10

🧠

Divide and Conquer: Object Co-occurrence Helps Mitigate Simplicity Bias in OOD Detection

Researchers propose OCO (Object Co-occurrence), a new out-of-distribution detection framework that leverages object co-occurrence patterns within images to improve the reliability of deep learning models. The method addresses simplicity bias by learning disentangled representations and using divide-and-conquer logic to distinguish near-OOD samples, achieving competitive results across multiple OOD detection benchmarks.