Self-Supervised Vision Transformers for CBCT-Based Detection of Temporomandibular Joint Osteoarthritis

Researchers demonstrate that self-supervised Vision Transformers, particularly the DINO family, can effectively detect temporomandibular joint osteoarthritis from cone-beam CT scans with 90.2% AUC when partially adapted. The study shows that strategic backbone unfreezing of final transformer blocks outperforms fully frozen models and supervised baselines, providing practical guidance for deploying foundation models in medical imaging with limited training data.

This research addresses a critical challenge in medical imaging: detecting subtle bone degradation in cone-beam CT scans using modern deep learning approaches. Temporomandibular joint osteoarthritis presents genuine clinical difficulty because degenerative changes appear as fine osseous alterations, making manual detection unreliable and automated systems necessary. The study's contribution lies not in achieving state-of-the-art performance alone, but in systematically demonstrating how to adapt foundation models for medical imaging tasks with scarce annotated data.

The emergence of self-supervised vision transformers like DINO has shifted the AI landscape by enabling models trained on unlabeled data to capture generalizable visual features. However, deploying these models to specialized domains like medical imaging requires careful adaptation strategy. Previous work assumed either full fine-tuning or complete freezing, but this research reveals the sweet spot: unfreezing only the final two transformer blocks yields dramatically better results (0.902 AUC) than frozen variants (0.671 AUC) while avoiding catastrophic forgetting.

For the medical AI sector, this work establishes practical protocol for practitioners working with limited labeled datasets. The comparison between DINOv2 variants and a radiology-pretrained model (RAD-DINO) shows that adaptation strategy matters more than pre-training choice, democratizing access to high-performance medical AI. This finding reduces computational and data collection burdens for healthcare institutions deploying diagnostic tools.

Looking forward, the methodology could extend to other musculoskeletal conditions and imaging modalities. As foundation models become larger and more capable, understanding efficient adaptation mechanisms becomes increasingly valuable for domain-specific applications beyond radiology.

• →Partial unfreezing of transformer blocks increases CBCT osteoarthritis detection from 67.1% to 90.2% AUC, outperforming fully frozen and supervised baselines.
• →Self-supervised DINO models transfer effectively to medical imaging when adapted strategically, reducing dependence on large labeled datasets.
• →Adaptation strategy proves more impactful than backbone selection or pre-training approach alone in low-data medical imaging scenarios.
• →Attention-based multiple instance learning aggregates slice-level predictions effectively for patient-level binary classification tasks.
• →The findings provide practical deployment guidance for foundation model adaptation in specialized medical domains with resource constraints.