IDEA: An Interpretable and Editable Decision-Making Framework for LLMs via Verbal-to-Numeric Calibration

Researchers introduce IDEA, a framework that converts Large Language Model decision-making into interpretable, editable parametric models with calibrated probabilities. The approach outperforms major LLMs like GPT-5.2 and DeepSeek R1 on benchmarks while enabling direct expert knowledge integration and precise human-AI collaboration.

IDEA addresses a critical limitation in LLM deployment: the gap between impressive performance and unreliable decision-making in high-stakes environments. Current large language models suffer from miscalibrated confidence scores, opaque reasoning that doesn't accurately reflect their logic, and inflexibility when incorporating domain expertise. This framework bridges that gap by extracting LLM knowledge into a structured parametric model that maps verbal explanations to numerical probabilities, making both interpretable and editable.

The research emerges from growing recognition that raw LLM capabilities alone are insufficient for regulated domains like healthcare, finance, and legal services. Organizations deploying these models face regulatory pressure to explain decisions and demonstrate reliability. Prior approaches relied on prompt engineering or fine-tuning, offering limited control and transparency. IDEA's mathematical framework with EM-based joint learning and correlated sampling provides formal guarantees about calibration and factor exclusion—precision previously unattainable.

The benchmark results demonstrate substantial practical value: Qwen-3-32B with IDEA achieves 78.6% accuracy versus 77.9% for GPT-5.2 and 68.1% for DeepSeek R1, while enabling quantitative editing of decision factors. This matters for enterprises seeking to deploy LLMs responsibly. Financial institutions could recalibrate risk assessments; healthcare systems could exclude irrelevant factors from diagnostic recommendations; compliance teams gain verifiable audit trails.

The open-source release signals the technique's maturity and invites broader adoption. Watch whether major enterprise AI platforms integrate IDEA-like calibration methods, regulatory bodies accept this approach for compliance, and whether performance-interpretability tradeoffs improve further with scale.

• →IDEA framework extracts LLM decisions into interpretable, editable parametric models with mathematical guarantees on calibration.
• →Outperforms GPT-5.2 and DeepSeek R1 on benchmarks while achieving perfect factor exclusion and exact probability calibration.
• →Enables quantitative human-AI collaboration by allowing direct parameter editing with verifiable mathematical properties.
• →Addresses critical gaps in LLM deployment for high-stakes domains by providing explainability and reliability beyond prompt engineering.
• →Open-source implementation accelerates adoption in enterprise and regulated industry applications requiring trustworthy decision-making.