Rotate2Think: Geometric Priming via Orthogonal Rotation to Improve Language Model Reasoning

Researchers introduce Rotate2Think, a training-free method that improves language model reasoning by applying geometric transformations to embedding space. The technique identifies that input and reasoning embeddings occupy distinct directional regions and uses orthogonal rotation to geometrically prime the model before generating reasoning traces, showing consistent accuracy improvements across 30 of 32 tested model-benchmark configurations.

Rotate2Think addresses a fundamental gap in understanding how language models structure their internal representations during reasoning tasks. The research reveals that embeddings exhibit extreme conicity—clustering tightly around single mean directions—with a crucial distinction: input embeddings and thinking embeddings point in measurably different directions. This geometric insight transforms reasoning improvement from a nebulous training problem into a concrete mathematical optimization challenge solvable through Procrustes analysis.

The work builds on the growing recognition that model reasoning requires more sophisticated prompting than simple instruction-following tasks. Prior approaches relied on chain-of-thought prompting or fine-tuning, which either required extensive computation or yielded inconsistent improvements. Rotate2Think circumvents these limitations by requiring only a small set of correct examples to estimate the optimal rotation matrix, making it computationally lightweight and widely applicable.

For the AI development community, this approach democratizes reasoning enhancement—practitioners can improve existing models without retraining or access to proprietary optimization techniques. The zero-shot generalization to multimodal reasoning on MATH-Vision suggests the geometric principle generalizes beyond single-modality tasks. However, the method's reliance on correctly solved examples creates a bootstrapping problem: models that already struggle with a task may lack sufficient positive examples to calibrate the rotation effectively.

Future research should investigate whether different task domains require different rotation angles and whether this framework extends to other model architectures beyond standard transformers. The 30-of-32 improvement rate warrants deeper analysis of the two failing configurations to identify fundamental boundaries. This technique potentially opens new directions in prompt engineering by treating inference-time optimization as a geometric rather than purely linguistic problem.

• →Rotate2Think improves reasoning accuracy without training by applying orthogonal rotations to embedding space based on the geometric distinction between input and thinking directions.
• →The method demonstrates consistent improvements across 30 of 32 model-benchmark configurations using only a small set of correctly solved examples for calibration.
• →Reasoning embeddings and input embeddings occupy geometrically distinct regions despite high conicity, enabling closed-form optimization via Procrustes analysis.
• →The training-free approach generalizes zero-shot to multimodal reasoning tasks, suggesting broad applicability across different model architectures and domains.
• →This geometric interpretation of reasoning shifts optimization from language-based prompting toward mathematical embedding space manipulation.