MIRCaps: A Large-Scale Mixed-Domain Dataset with Image-Level and Region-Level Captions for Fine-Grained Vision-Language Learning

Researchers introduce MIRCaps, a large-scale multimodal dataset containing 141,364 images with 981,947 image-level and 1,742,264 region-level captions designed to improve Vision-Language Models (VLMs) for general imagery and CCTV surveillance applications. The dataset demonstrates effective fine-tuning of lightweight VLMs across image captioning and object detection tasks, with code and data publicly available.

The introduction of MIRCaps addresses a significant gap in the machine learning landscape where mixed-domain vision-language datasets remain scarce despite rapid progress in VLM development. This resource is particularly valuable because it bridges two distinct use cases—general-purpose image understanding and specialized CCTV surveillance systems—within a single comprehensive framework. The dataset's dual-caption structure, providing both image-level and region-level descriptions, enables models to learn fine-grained visual attributes across multiple semantic levels, a capability essential for complex real-world applications.

The research reflects broader industry trends toward democratizing AI development by releasing high-quality, annotated datasets publicly. The successful fine-tuning of lightweight VLMs like SmolVLM-256M and BLIP2 demonstrates that effective vision-language learning doesn't require massive model architectures, opening deployment possibilities on resource-constrained devices and edge systems. This aligns with industry momentum toward efficient AI inference.

For developers and organizations, MIRCaps provides a foundation for building specialized vision-language applications without requiring expensive annotation campaigns. The dataset's applicability to surveillance systems carries particular significance for security infrastructure modernization, where AI-driven scene understanding can enhance monitoring capabilities. The public availability accelerates research velocity and enables rapid prototyping of downstream applications across retail, security, and smart city domains.

Future impact depends on community adoption and how effectively the dataset drives improvements in real-world surveillance and general vision tasks. Researchers should monitor emerging applications, particularly in automated scene understanding and event detection systems that leverage the rich regional annotations.

• →MIRCaps contains 141,364 images with nearly 1 million image-level captions and 1.7 million region-level captions for fine-grained vision-language learning
• →The dataset enables effective fine-tuning of lightweight VLMs including SmolVLM-256M, BLIP, and Qwen2.5-VL with demonstrated results on image captioning and object detection
• →Dual-caption architecture supports learning visual attributes including object categories, colors, actions, states, and environmental context at both global and regional levels
• →Dataset addresses mixed-domain use cases including general-purpose imagery and CCTV surveillance systems, broadening applicability across security and monitoring sectors
• →Public release on Zenodo democratizes access to high-quality annotated data, reducing barriers for researchers and developers building vision-language applications