ClinDet-Bench: Beyond Abstention, Evaluating Judgment Determinability of LLMs in Clinical Decision-Making

Researchers developed ClinDet-Bench, a new benchmark that reveals large language models fail to properly identify when they have sufficient information to make clinical decisions. The study shows LLMs make both premature judgments and excessive abstentions in medical scenarios, highlighting safety concerns for AI deployment in healthcare settings.

• →ClinDet-Bench is a new benchmark for evaluating LLMs' ability to determine when clinical information is sufficient for decision-making.
• →Current LLMs fail to properly identify determinability under incomplete information in clinical scenarios.
• →LLMs demonstrate both premature clinical judgments and excessive abstention, both of which can compromise patient safety.
• →Existing benchmarks are insufficient for evaluating the safety of LLMs in clinical and high-stakes environments.
• →The benchmark framework has potential applications beyond medicine to other high-stakes decision-making domains.