Scalable Object Relation Encoding for Better 3D Spatial Reasoning in Large Language Models

Researchers introduce QuatRoPE, a novel positional embedding method that improves 3D spatial reasoning in Large Language Models by encoding object relations more efficiently. The method maintains linear scalability with the number of objects and preserves LLMs' original capabilities through the Isolated Gated RoPE Extension.

• →QuatRoPE addresses scalability issues in 3D spatial reasoning for LLMs by using linear rather than quadratic input length relative to object count.
• →The method explicitly calculates pairwise spatial relations through dot products in attention layers rather than premature feature fusion.
• →Isolated Gated RoPE Extension (IGRE) limits QuatRoPE's influence to object-related tokens, preserving LLM capabilities.
• →The approach maintains spatial consistency and geometric integrity through holistic vector encoding of 3D coordinates.
• →Code and experimental data are publicly available, enabling reproducible research in embodied AI applications.