Beyond Templates: Revisiting Zero-Shot Remote Sensing through Meta-Prompting

Researchers analyze how vision-language models perform zero-shot remote sensing tasks across multiple datasets and find that textual design choices critically impact performance. The study reveals that semantically rich LLM-generated descriptions don't consistently outperform simpler template-based descriptions due to noise in text embeddings, but lightweight query embedding calibration effectively improves results.

This research addresses a fundamental challenge in applying large vision-language models to specialized domains like Earth Observation. While VLMs have demonstrated impressive zero-shot capabilities across general vision tasks, their adaptation to remote sensing requires careful engineering of textual inputs—a finding with significant implications for practitioners deploying these systems.

The core tension the researchers identify is compelling: intuitively, semantically richer descriptions from language models should provide better classification signals, yet empirically they often underperform. This paradox stems from how text embeddings cluster in CLIP feature space. When LLMs generate verbose descriptions, they introduce semantic nuance that increases noise rather than signal robustness. The whitened feature space analysis provides a principled explanation for why simpler, domain-adapted prompts often work better despite being less expressive.

For the AI and machine learning community, this reveals that prompt engineering for vision-language models requires domain-specific calibration rather than just semantic optimization. The proposed query embedding calibration solution is particularly valuable because it's lightweight and generalizable—practitioners can apply it across different VLM variants and remote sensing datasets without retraining. This democratizes better performance for organizations using off-the-shelf models.

The broader impact extends to Earth Observation applications spanning agriculture, urban planning, and environmental monitoring, where zero-shot capabilities reduce labeling costs. Future work should investigate whether this semantic richness-robustness trade-off manifests in other specialized domains and whether adaptive calibration strategies can further optimize the balance between expressiveness and stability.

• →Zero-shot remote sensing performance is highly sensitive to textual design choices, not just model selection.
• →Semantically richer LLM-generated descriptions can introduce embedding noise that reduces robustness compared to simpler templates.
• →Lightweight query embedding calibration consistently improves zero-shot classification and retrieval across multiple datasets.
• →Text log-likelihood analysis in whitened CLIP feature space reveals why template-based descriptions often outperform verbose LLM descriptions.
• →Domain-adapted prompting strategies are more critical than semantic richness for specialized vision-language applications.