The Geometry of Reasoning: Flowing Logics in Representation Space

Researchers propose a geometric framework showing how large language models 'think' through representation space as flows, with logical statements acting as controllers of these flows' velocities. The study provides evidence that LLMs can internalize logical invariants through next-token prediction training, challenging the 'stochastic parrot' criticism and suggesting universal representational laws underlying machine understanding.

• →LLM reasoning corresponds to smooth flows in representation space that can be analyzed using geometric quantities like position, velocity, and curvature.
• →Logical statements act as local controllers of reasoning flows' velocities, allowing LLMs to internalize logic beyond surface form.
• →Training via next-token prediction can lead LLMs to develop logical invariants as higher-order geometry in representation space.
• →Experiments across Qwen and LLaMA model families suggest universal representational laws underlying machine understanding.
• →The framework provides new tools for interpretability and formal analysis of LLM reasoning behavior.