What Spatial Memory Must Store: Occlusion as the Test for Language-Agent Memory

Researchers demonstrate that spatial memory systems for language agents must fundamentally separate memory recall from visibility computation, using occlusion testing as a validation method. The study shows that geometry-based weighting outperforms traditional blending approaches, and introduces a ray-casting technique to properly handle occluded spatial information.

This research addresses a critical architectural problem in embodied AI systems: how language agents should store and retrieve spatial information about their environments. The work challenges conventional approaches that treat spatial proximity as just another weighted factor alongside recency and importance, revealing that geometry must take precedence when queries are spatial in nature.

The core innovation lies in distinguishing between two separate operations: memory recall (what the agent knows happened in a location) and visibility prediction (what the agent can currently perceive). Traditional systems conflate these, leading to incorrect inferences about occluded spaces. By implementing a ray-casting differential analyzer—essentially a one-line geometric test—the researchers achieve near-perfect discrimination between visible and hidden objects, reaching 0.982 accuracy compared to 0.000 for baseline approaches.

The research demonstrates measurable improvements through pre-registered experiments across multiple scripted worlds with automated validation. These controlled conditions provide strong empirical evidence that coordinate-based recall can resolve spatial ambiguities that cosine similarity metrics cannot, achieving perfect discrimination (1.000 vs 0.533) on near-duplicate locations.

This work has implications for any system requiring spatial reasoning: robotics, embodied question-answering, multi-agent navigation, and virtual world interactions. The findings suggest that naive integration of spatial information into text-based systems produces suboptimal results without proper geometric foundations. The separation of concerns—storing coordinate data separately from retrieval mechanisms—represents a design principle that could improve reliability and interpretability in embodied AI systems.

• →Geometry-weighted recall outperforms traditional blended approaches by 8.5x on spatial queries
• →Occlusion-aware visibility computation requires explicit ray-casting, not text-based inference
• →Memory recall and visibility are distinct operations that should be architecturally separated
• →Pre-registered experiments validated improvements across 96 behind-wall targets with McNemar p<10^-6
• →Coordinate-based spatial storage enables perfect disambiguation where embedding metrics fail