Data Selection Through Iterative Self-Filtering for Vision-Language Settings

Researchers propose a Self-Filtering method that trains CLIP vision-language models on dynamically evolving datasets by iteratively balancing clean samples with diverse data. This bootstrapped approach improves model performance without requiring additional data or pre-trained models, addressing the challenge of training on large-scale noisy datasets.

The paper addresses a fundamental challenge in machine learning: scaling neural network training without proportional increases in manual data curation. As datasets grow larger, maintaining quality becomes computationally expensive and impractical, yet noisy data degrades model performance. The Self-Filtering approach offers a practical solution by creating a feedback loop where the model itself guides data selection, reducing dependency on external validation frameworks or pre-trained models.

This work builds on established concepts in active learning and data selection, but applies them specifically to vision-language models like CLIP. The iterative refinement process—training, evaluating, and reselecting data—creates a bootstrapped system where each cycle produces cleaner training data. By maintaining both high-confidence clean samples and diverse edge cases, the method preserves representational breadth while improving dataset quality, avoiding the common pitfall of over-filtering that reduces model robustness.

For the AI development community, this approach has substantial implications. Organizations training large vision-language models can reduce annotation costs and infrastructure requirements while achieving better performance. The method's independence from pre-trained models makes it particularly valuable for specialized domains where transfer learning may be limited. This efficiency gain accelerates model development cycles and democratizes training of high-quality models across organizations with varying resources.

Future work likely explores applying this methodology to other modalities and model architectures. The reproducibility and scalability of Self-Filtering across different datasets and domains will determine its practical adoption in production environments.

• →Self-Filtering creates a bootstrapped feedback loop where models iteratively select and train on improving data distributions without external supervision.
• →The method balances data quality and diversity by retaining both high-confidence clean samples and diverse edge-case examples from the full distribution.
• →No requirement for additional datasets or pre-trained models reduces computational overhead and makes the approach accessible to resource-constrained teams.
• →Vision-language model performance improves through dataset refinement alone, suggesting data quality impacts downstream performance more than previously quantified.
• →The approach addresses scalability challenges in machine learning by automating data curation, reducing manual annotation burden at large scales.