Unlocking the Working Memory of Large Language Models for Latent Reasoning

Researchers introduce Reasoning in Memory (RiM), a novel method that enables large language models to perform internal reasoning using fixed memory blocks instead of generating intermediate tokens. The approach matches or exceeds existing reasoning methods while being more compute-efficient, as memory blocks process in a single forward pass rather than through autoregressive generation.

Reasoning in Memory represents a meaningful shift in how AI systems approach complex problem-solving. Rather than forcing language models to externalize every reasoning step through token generation—a computationally expensive process—RiM leverages the concept of working memory, allowing models to hold and manipulate information internally before producing a final answer. This mirrors human cognition more closely and addresses a fundamental inefficiency in current large language model architectures.

The advancement emerges from growing recognition that scaling test-time compute through autoregressive generation conflates two separate concerns: internal reasoning and external communication. Previous approaches like chain-of-thought prompting generate intermediate reasoning steps as tokens, consuming computational resources and latency. RiM decouples these processes by using fixed sequences of special tokens that function as memory placeholders, enabling models to reason without the overhead of generating human-readable intermediate steps.

The practical implications extend across AI development and deployment. For organizations running large language models, compute efficiency directly impacts operational costs and inference speed. By processing memory blocks in a single forward pass, RiM reduces the computational burden while maintaining or improving reasoning accuracy across multiple model families and sizes. The two-stage curriculum approach—first grounding memory blocks with explicit reasoning supervision, then discarding it—demonstrates a pragmatic training methodology that scales.

This work positions latent reasoning as a competitive alternative to autoregressive reasoning methods, suggesting the next generation of AI systems may prioritize internal reasoning capacity over interpretability through explicit thought externalization. The implications for AI safety, model interpretability, and real-world deployment warrant close attention from the research community.

• →RiM enables latent reasoning through fixed memory blocks that process in a single forward pass, improving compute efficiency over autoregressive reasoning methods.
• →The approach decouples internal reasoning from external communication, allowing models to manipulate information without generating intermediate tokens.
• →Two-stage curriculum training first grounds memory blocks with explicit reasoning supervision, then refines answers iteratively without step-level guidance.
• →Results show RiM matches or exceeds existing latent reasoning methods across different language model families and sizes.
• →The method addresses a fundamental inefficiency in current LLM architectures by enabling working memory-like computation.