Any2Any 3D Diffusion Models with Knowledge Transfer: A Radiotherapy Planning Study

Researchers introduced DiffKT3D, a 3D diffusion model framework that applies knowledge transfer from video diffusion models to radiotherapy dose prediction. The approach achieves state-of-the-art results by reducing prediction error by 7% compared to previous benchmarks while maintaining clinical alignment through reinforcement learning post-training.

DiffKT3D represents a significant advancement in medical imaging AI by demonstrating how generative models trained at scale in vision domains can transfer effectively to specialized clinical applications. The framework addresses a fundamental challenge in radiotherapy planning: developing models that generalize across diverse clinical settings and institutional preferences without requiring complete retraining for each new environment. This matters because dose prediction directly impacts treatment safety and efficacy.

The technical innovation combines several elements that overcome traditional limitations. Rather than building specialized models from scratch, the authors leverage pretrained diffusion models and introduce modality-specific embeddings that enable flexible conditioning across multiple clinical inputs—CT scans, anatomical structures, beam parameters—without the computational overhead of cross-attention mechanisms. The reinforcement learning post-training step that follows integrates institutional clinical preferences through a Scorecard mechanism, effectively aligning model outputs with real-world treatment protocols.

For the medical AI industry, this approach demonstrates practical pathways for deploying foundation models in clinical settings where generalization and institutional customization are prerequisites. The 7% error reduction from 2.07 to 1.93 in voxel-level MAE and superior preference matching suggest that diffusion-based approaches may become standard in radiotherapy planning. The methodology extends beyond radiotherapy—the Any2Any conditional paradigm could apply to other medical imaging tasks requiring multi-modal conditioning.

Looking forward, adoption depends on regulatory clearance, clinical validation across diverse institutions, and integration with existing radiotherapy planning systems. The ability to rapidly adapt models to institutional preferences through RL post-training could accelerate deployment cycles compared to traditional approaches requiring extensive retraining.

• →DiffKT3D achieves 7% error reduction in voxel-level dose prediction compared to previous state-of-the-art benchmarks
• →Knowledge transfer from pretrained video diffusion models eliminates need to train specialized models from scratch
• →Any2Any conditioning framework enables flexible multi-modal inputs without computational cross-attention overhead
• →Reinforcement learning post-training with clinical scorecards aligns predictions with institutional treatment preferences
• →Generalizable approach demonstrates foundation models can effectively transfer to specialized medical imaging applications