Learning Preference-Based Objectives from Clinical Narratives for Sequential Treatment Decision-Making

Researchers propose Clinical Narrative-informed Preference Rewards (CN-PR), a machine learning framework that extracts reward signals from patient discharge summaries to train reinforcement learning models for treatment decisions. The approach achieves strong alignment with clinical outcomes, including improved organ support-free days and faster shock resolution, offering a scalable alternative to traditional reward design in healthcare AI.

This research addresses a fundamental problem in healthcare AI: designing meaningful reward functions for reinforcement learning systems that guide treatment decisions. Traditional approaches rely on sparse clinical outcomes or structured data that fail to capture the full complexity of patient recovery, treatment burden, and clinical stability. The CN-PR framework leverages clinical narratives as implicit supervision, using large language models to derive trajectory quality scores from discharge summaries and construct pairwise preference signals between patient pathways.

The work builds on growing recognition that unstructured clinical text contains rich information absent from structured databases. By treating narrative summaries as scalable supervision rather than manual expert annotation, the researchers bypass the bottleneck of handcrafted reward design. The incorporation of confidence weighting acknowledges that not all narratives equally inform decision-making, addressing practical variability in documentation quality.

The validation results demonstrate meaningful clinical impact: learned policies correlate with increased organ support-free days and faster shock resolution while maintaining mortality parity. External validation persistence strengthens claims of generalizability. This approach has significant implications for healthcare AI deployment, where reward misspecification can lead to suboptimal or harmful policies. For AI developers and healthcare institutions, the framework offers a pathway to scale RL applications in clinical settings without relying on expensive expert supervision or potentially misleading outcome proxies.

Future applications could extend this methodology to other clinical domains and explore how narrative-derived rewards integrate with real-time decision systems. The research also raises questions about documentation bias and whether narrative-based supervision reflects evidence-based medicine or local institutional practices.

• →CN-PR extracts reward functions from clinical narratives, enabling reinforcement learning for treatment decisions without manual reward engineering
• →The framework achieved Spearman correlation of 0.63 with trajectory quality and improved organ support-free days in validation studies
• →Confidence weighting mechanisms address variability in narrative informativeness, improving supervision relevance
• →Results persisted under external validation, suggesting potential scalability across healthcare institutions
• →This approach offers an alternative to outcome-based reward design that can better capture treatment burden and recovery dynamics