Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation

Researchers demonstrate that training vision-language models (VLMs) on curated, concise data significantly reduces inference costs without sacrificing accuracy. By focusing on output brevity rather than traditional model compression techniques, the approach achieves 35x efficiency gains over verbose models while maintaining competitive performance.

The research addresses a critical gap in AI efficiency optimization. While the industry has focused heavily on model compression through distillation, pruning, and quantization, output token proliferation has remained largely unchecked—a counterintuitive oversight given that token generation directly drives computational costs and latency. The study's core insight is elegant: training on naturally concise, high-quality data teaches models to answer efficiently without sacrificing correctness.

This work emerges as VLMs increasingly power real-world applications where inference costs directly impact deployment economics. The MAmmoTH-VL curation experiment provides concrete evidence that data quality matters as much as model architecture. By holding output length constant through regression analysis, the researchers isolate brevity's contribution from reasoning capability, revealing that verbose outputs rarely improve accuracy—challenging assumptions underlying current training practices.

For the AI infrastructure industry, this has immediate implications. The 35x Cost-of-Pass improvement at 4B parameters demonstrates that efficiency gains scale across the model size spectrum most enterprises actually deploy. The finding that reasoning-structured verbosity provides diminishing returns—shrinking from 4 of 8 capability groups at 2B to just 1 of 8 at 4B—suggests industry consensus about lengthy reasoning chains may be economically misguided.

Looking forward, this work validates data curation as a primary efficiency lever, likely shifting investment priorities toward curated datasets over hardware acceleration. The approach applies across VLM architectures and scales, making it a generalizable technique for cost-conscious deployment. Whether this becomes standard practice depends on how broadly the findings generalize beyond the MAmmoTH-VL domain.

• →Data curation enabling output brevity delivers 35x inference efficiency gains without accuracy loss compared to verbose models
• →Verbose reasoning outputs provide minimal accuracy benefits at 4B parameters, contradicting assumptions underlying current training approaches
• →Holding accuracy constant, concise models reach correct answers that verbose reasoning models miss, positioning brevity as a distinct optimization target
• →The efficiency-through-brevity approach generalizes across 1B-4B parameter scales, with gains growing from +16.7 pp to +21.2 pp accuracy advantage
• →This research reframes inference efficiency from a model-size problem to a tokens-per-correct-answer problem with direct practical cost implications