Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

A new research framework called CWE-Trace challenges the claim that large language models can reliably detect software vulnerabilities, revealing that fine-tuned models achieve only 52.1% accuracy at best and lack genuine security reasoning despite appearing well-calibrated. The study of 834 Linux kernel samples shows that models exhibit systematic failure patterns that persist across datasets and resist correction through fine-tuning, suggesting they memorize patterns rather than understand vulnerability detection.

Researchers at y0.exchange have published findings that fundamentally question whether current LLMs possess meaningful security reasoning capabilities or merely perform sophisticated pattern-matching on training data. Using the CWE-Trace framework—a meticulously designed benchmark with 834 manually curated Linux kernel samples spanning 74 vulnerability classes—the team conducted one of the most rigorous evaluations of LLM security detection to date. Their methodology accounts for data contamination through strict temporal splits and preserves vulnerable-patched context pairs, addressing critical weaknesses in prior benchmarking efforts.

The research exposes a critical distinction: fine-tuning creates the illusion of improvement without fundamental capability changes. Models exhibit stable directional biases (measured by the Directional Failure Index) that persist from historical to post-cutoff datasets, suggesting the underlying decision-making process remains unchanged while only output calibration shifts. Notably, data contamination—long suspected as inflating vulnerability detection scores—provided no measurable advantage, with 84% of nominally contaminated samples carrying no usable memorization signal. This contradicts common assumptions in the field and refocuses attention on genuine reasoning gaps.

The practical implications are substantial for enterprises deploying LLMs in security workflows. Current models cannot reliably identify or classify vulnerabilities in systems software, with exact CWE classification accuracy below 1.3%. Organizations relying on LLM-powered security tools face hidden risks: models may appear confident while fundamentally lacking security reasoning. The decoupling of detection capabilities from classification accuracy reveals these are distinct problems requiring separate solutions. As LLMs become integrated into development pipelines, this research underscores the need for rigorous validation and human oversight rather than automated reliance on model outputs.

• →LLMs achieve only 52.1% accuracy on vulnerability detection—barely above 50% chance—despite years of fine-tuning optimization efforts.
• →Data contamination provides no measurable performance advantage; 84% of supposedly contaminated samples lack usable memorization signals.
• →Fine-tuning adjusts output calibration without changing underlying security reasoning, creating a false impression of capability improvement.
• →Systematic failure patterns persist across datasets and resist correction, indicating models lack generalizable vulnerability understanding.
• →Exact CWE classification remains below 1.3% accuracy, confirming LLMs cannot reliably categorize vulnerability types in systems software.