Gram-Anchored Prompt Learning for Vision-Language Models via Second-Order Statistics

Researchers propose Gram-Anchored Prompt Learning (GAPL), a new framework that improves Vision-Language Model adaptation by incorporating second-order statistical features via Gram matrices. This approach enhances robustness against domain shifts and local noise compared to existing methods that rely solely on first-order spatial features.

• →GAPL introduces second-order statistical streams via Gram matrices to augment standard first-order spatial interactions in Vision-Language Models.
• →The framework addresses limitations of existing prompt learning methods that are susceptible to domain shifts and local noise.
• →The approach enables language representations to dynamically adapt to statistical distribution shifts across diverse domains.
• →Extensive experiments demonstrate compelling performance improvements across various benchmarks.
• →The method synergizes local semantic alignment with global structural consistency for more robust VLM adaptation.