Retrieval-Augmented Anatomical Guidance for Text-to-CT Generation

Researchers propose a retrieval-augmented approach for generating CT scans from radiology reports that combines semantic control with anatomical consistency by retrieving structurally similar clinical cases and using their annotations as guidance. The method improves image fidelity and clinical consistency compared to text-only baselines while enabling spatial controllability without requiring ground-truth annotations at inference time.

This research addresses a fundamental challenge in medical image synthesis: balancing semantic flexibility with anatomical accuracy. Text-conditioned generative models offer intuitive control through natural language but often produce spatially incoherent outputs, while structure-driven methods ensure anatomical plausibility but require unavailable ground-truth data during synthesis. The proposed solution elegantly bridges this gap through retrieval-augmentation, leveraging a 3D vision-language encoder to find semantically similar clinical cases whose anatomical annotations serve as structural templates for generation.

The approach represents a meaningful advancement in medical AI infrastructure. By injecting retrieved anatomical proxies through a ControlNet branch into a latent diffusion model, the method maintains semantic flexibility while providing coarse guidance that grounds outputs in plausible anatomy. This dual-pathway conditioning is particularly valuable in clinical contexts where both interpretability and accuracy matter. The experiments on CT-RATE demonstrate consistent improvements across evaluation metrics, with retrieval quality directly correlating to output quality—a finding that validates the underlying methodology.

For the medical imaging and AI development communities, this work opens practical pathways for deploying generative models in clinical workflows. The approach's scalability depends on retrieval quality, suggesting that investments in robust medical vision-language models will yield multiplicative returns. The code release enables broader adoption and iteration, potentially accelerating development of similar retrieval-augmented systems in other medical imaging modalities. This represents incremental but solid progress toward more clinically viable generative medical imaging systems.

• →Retrieval-augmented generation improves anatomical consistency in text-to-CT synthesis by using similar cases as structural templates
• →The method maintains semantic flexibility while providing anatomical guidance without requiring ground-truth annotations at inference
• →Experiments show consistent gains in image fidelity and clinical consistency over text-only baselines
• →Retrieval quality directly impacts generation quality, emphasizing the importance of robust medical vision-language encoders
• →Open-source code release enables broader adoption and accelerates development of similar approaches