LensVLM: Selective Context Expansion for Compressed Visual Representation of Text

LensVLM is a new inference framework that enables Vision Language Models to process highly compressed images of text by selectively expanding relevant sections, achieving 4.3x compression while maintaining accuracy comparable to full-resolution processing. The approach combines learned tool selection with post-training techniques to overcome the fundamental limitation that compressed text becomes illegible to standard vision encoders.

LensVLM addresses a critical efficiency challenge in vision language models: processing text rendered as images typically requires prohibitively high resolution to maintain character legibility, resulting in long token sequences that slow inference. The researchers identified that fixed-size visual token outputs create an inherent trade-off between compression and accuracy, but this limitation can be mitigated through selective, learned expansion rather than uniform decompression.

The technical innovation lies in training VLMs to recognize when visual information becomes too degraded and automatically invoke expansion tools only for relevant regions. This selective approach differs fundamentally from prior compression strategies by operating at inference time with learned decision-making, rather than applying uniform compression globally. Testing on seven text QA benchmarks demonstrates the framework maintains performance at 4.3x compression and extends to 10.1x against weaker baselines, while generalizing to document and code understanding tasks.

For AI infrastructure developers and researchers, LensVLM presents a practical pathway to reduce inference costs and latency when processing text-heavy documents. The framework's ability to generalize across task types suggests the selective expansion principle addresses a universal constraint in vision-language architectures. The finding that text expansion performs better than image expansion for rendered content provides actionable guidance for practitioners choosing rendering strategies.

Future implications depend on whether this technique becomes standard in production VLM systems and whether similar selective mechanisms emerge for other modalities. The work signals growing sophistication in compression-aware AI design, where models learn adaptive efficiency rather than sacrificing quality for speed.

• →LensVLM achieves 4.3x effective compression while maintaining full-resolution accuracy by selectively expanding compressed image regions during inference.
• →The framework trains vision-language models to recognize illegible content and invoke learned expansion tools only when necessary, reducing unnecessary computational overhead.
• →Testing across seven text QA benchmarks and multimodal tasks shows consistent accuracy improvements over retrieval-based and traditional compression baselines up to 10.1x compression.
• →Analysis reveals the model increasingly relies on expanded content as compression increases, validating that learned tool selection addresses fundamental vision encoder resolution limitations.
• →Practical guidance for practitioners indicates text expansion suits rendered documents while high-resolution image expansion benefits native documents where layout contains task-relevant information.