DRIFT: A Residual Flow Adapter for Decoding Continuous Outputs in Vision-Language Models
Researchers introduce DRIFT, a framework that adapts pretrained vision-language models to handle continuous numerical outputs rather than discrete tokens. By combining a base predictor with a flow-matching refinement module, DRIFT improves performance on tasks like temporal localization and robotic control across multiple model architectures.
DRIFT addresses a fundamental limitation in how modern vision-language models operate. While autoregressive token-based decoding enables scalable pretraining and strong generalization, this approach struggles with tasks requiring precise continuous outputs—a critical constraint for robotics, temporal grounding, and spatial reasoning applications. The framework's innovation lies in its residual formulation, which reframes the generative modeling challenge from learning a global output distribution to refining localized predictions around a strong prior.
This advancement reflects the broader evolution in AI systems architecture. As VLMs mature beyond pure language generation, the field increasingly recognizes that different output modalities demand specialized decoding strategies. DRIFT's approach of layering a flow-matching refinement module on top of existing pretrained models demonstrates how architectural innovations can extend model capabilities without requiring complete retraining, reducing computational costs and implementation barriers.
The framework's applicability across multiple model types—multimodal language models (MLLMs), vision-language action models (VLAs), and world action models (WAMs)—suggests it could become a standard adapter pattern in the industry. For developers building robotics applications or spatial reasoning systems, DRIFT offers a practical pathway to leverage existing pretrained models. The consistent performance improvements over regression and alternative generative baselines indicate this approach meaningfully advances the state-of-the-art in continuous output prediction.
Looking forward, the integration of such refinement modules into vision-language models could accelerate development of embodied AI systems and precise perception tasks where token-based interfaces previously imposed fundamental accuracy constraints.
- →DRIFT enables vision-language models to generate precise continuous outputs by combining coarse predictions with iterative flow-matching refinement
- →The residual formulation simplifies optimization by modeling localized distributions rather than global output spaces
- →Framework demonstrates consistent improvements across multiple architectures including MLLMs, VLAs, and WAMs
- →Approach reduces computational requirements by adapting pretrained models rather than requiring full retraining
- →Innovation addresses critical gaps in robotics and spatial reasoning tasks previously constrained by token-based decoding