SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transport

Researchers introduce SOTAlign, a new framework for aligning vision and language AI models using minimal supervised data. The method uses optimal transport theory to achieve better alignment with significantly less paired training data than traditional approaches.

• →SOTAlign enables alignment of vision and language models with substantially less supervised training data than existing methods.
• →The framework uses a two-stage approach combining linear teachers and optimal transport theory for efficient alignment.
• →The method effectively leverages unpaired images and text data, unlike existing semi-supervised approaches.
• →SOTAlign significantly outperforms both supervised and semi-supervised baseline methods across multiple datasets.
• →The research addresses the computational efficiency challenge in multimodal AI model training.