Do Foundation Models Know Geometry? Probing Frozen Features for Continuous Physical Measurement

Research reveals that vision-language models internally encode geometric information that cannot be effectively expressed through their text pathways. A lightweight linear probe can extract hand joint angles with 6.1 degrees accuracy from frozen features, while text output only achieves 20.0 degrees accuracy, indicating a significant bottleneck in geometric understanding translation.

• →Vision-language models have a 3.3x accuracy bottleneck between internal geometric understanding and text expression capabilities.
• →Different AI architectures achieve similar geometric accuracy despite low representational similarity, showing functional convergence.
• →Autoregressive text generation damages geometric fidelity, but the issue stems from generation process rather than language alignment.
• →Mid-network layers (18-22) carry the strongest geometric signals across all tested architectures.
• →Lightweight probes can enable frozen AI models to function as multi-task geometric sensors without fine-tuning.