ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models

Researchers propose ClinCoT, a new framework for medical AI that improves Visual Language Models by grounding reasoning in specific visual regions rather than just text. The approach reduces factual hallucinations in medical AI systems by using visual chain-of-thought reasoning with clinically relevant image regions.

• →ClinCoT addresses factual hallucinations in medical AI by connecting reasoning to specific visual regions in medical images.
• →The framework shifts from response-level correction to visual-driven reasoning through hypothesis-driven region proposals.
• →Multiple medical AI evaluators rank responses to create training supervision for improved clinical accuracy.
• →An iterative learning scheme dynamically regenerates preference data as the model evolves during training.
• →Testing on medical VQA and report generation benchmarks shows superior performance compared to existing alignment methods.