#clinical-llm News & Analysis

3 articles tagged with #clinical-llm. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBullisharXiv – CS AI · Jun 127/10

🧠

Deployment-Centered Evaluation: Predicting Query-Level Rejection Risk in a Clinical LLM System

Researchers developed a pre-response classifier for clinical LLMs that predicts user rejection risk with 71.9% accuracy by leveraging deployment-specific context like provider type and department. This deployment-centered evaluation approach addresses a critical gap in clinical AI assessment, moving beyond static benchmarks to measure real-world user acceptance in a healthcare system.

AINeutralarXiv – CS AI · Jun 17/10

🧠

Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs

Researchers propose a semantic verification framework to evaluate robustness of clinical LLMs against prompt variations that preserve meaning. Testing 16 models reveals that domain-specific medical models show mixed results compared to general-purpose counterparts, with sensitivity to rephrasing posing safety risks in healthcare applications.

AINeutralarXiv – CS AI · Jun 96/10

🧠

Beyond English benchmarks: clinical llm evaluation in Brazilian Portuguese

Researchers introduce ClinicalBr, the first bilingual clinical benchmark using 2,892 real Brazilian Portuguese-English case reports to evaluate large language models. The study reveals that English-language advantages in clinical AI are task-dependent, with Portuguese performing comparably in differential diagnosis, exam recommendations, and treatment planning.