From Clouds to Hallucinations: Atmospheric Retrieval Hijacking in Remote Sensing Vision-Language RAG

Researchers introduce CloudWeb, an adversarial attack that manipulates remote sensing images with realistic cloud and haze patterns to hijack vision-language retrieval systems in multimodal RAG pipelines. The attack achieves significant success rates—increasing weather-related evidence injection from 0.71% to 43.29% on benchmark tests—demonstrating that input-space threats to retrieval stages remain largely undefended in production systems.

CloudWeb represents a critical vulnerability in the emerging ecosystem of multimodal AI systems that combine vision-language models with retrieval-augmented generation. The attack exploits a fundamental assumption: that deploying frozen retrievers and generators creates a secure pipeline when only inputs can be modified. Researchers demonstrate this assumption is dangerously flawed by overlaying parameterized atmospheric patterns on satellite imagery that reliably redirect retrieval systems toward target evidence while suppressing legitimate scene information.

This work builds on growing recognition that RAG systems introduce new attack surfaces beyond traditional adversarial ML. While prior research focused on corrupting training data or knowledge bases, CloudWeb targets the evidence retrieval stage—arguably the most critical phase where factual grounding occurs. The attack's effectiveness across five different CLIP-style retrievers, including domain-specific models like GeoRSCLIP and RemoteCLIP, suggests the vulnerability is architectural rather than implementation-specific.

For practitioners deploying vision-language systems in high-stakes domains like environmental monitoring, disaster response, or agricultural assessment, CloudWeb exposes practical risks. The attack produces visually plausible weather patterns that could evade human review while poisoning downstream outputs. Downstream generators demonstrably hallucinate false weather information based on hijacked retrieval results, meaning the failure mode compounds through the pipeline.

The security implications extend beyond remote sensing. Any multimodal RAG system relying on frozen vision-language retrievers faces similar input-space vulnerabilities. Organizations should prioritize adversarial robustness testing for retrieval components and consider dynamic defenses that validate retrieved evidence consistency with actual image content.

• →CloudWeb achieves 60x improvement in injecting false atmospheric evidence through realistic cloud pattern overlays, revealing critical vulnerabilities in frozen vision-language retrievers.
• →The attack succeeds across multiple CLIP variants and domains, indicating the vulnerability is systemic to current retrieval-augmented generation architectures rather than isolated implementation issues.
• →Hijacked retrieval directly propagates to downstream hallucinations in vision-language generators, demonstrating that retrieval-stage attacks compromise end-to-end system integrity.
• →Input-space adversarial threats to multimodal RAG retrieval remain largely undefended in production deployments despite growing adoption in remote sensing and critical applications.
• →Natural-looking atmospheric perturbations can evade human inspection while reliably manipulating evidence rankings, requiring automated validation mechanisms for retrieval consistency.