Micro-Macro Retrieval: Reducing Long-Form Hallucination in Large Language Models

Researchers propose Micro-Macro Retrieval (M2R), a framework that reduces hallucination in large language models during long-form text generation by keeping key information closer to model outputs. The method combines coarse-grained external retrieval with fine-grained extraction from an internal knowledge repository, addressing a critical bottleneck where proximity of evidence to final answers directly correlates with factual accuracy.

The hallucination problem in large language models represents a fundamental challenge limiting their deployment in high-stakes applications. LLMs generate plausible-sounding but factually incorrect content, particularly in extended reasoning tasks where accumulated errors compound through lengthy output sequences. The M2R framework addresses this by implementing a dual-level retrieval strategy that targets a specific phenomenon: evidence positioned closer to model outputs yields significantly higher factual accuracy than evidence injected earlier in the reasoning chain.

This research builds on the established retrieval-augmented generation (RAG) paradigm, which augments LLMs with external knowledge sources. However, conventional RAG systems suffer from degradation in long-form contexts where multiple retrieval cycles introduce noise and dilute critical information. M2R innovates by maintaining a dynamic "key information repository" that extracts and preserves essential facts during reasoning, ensuring these facts remain accessible for final answer generation. The curriculum learning approach with rule-based rewards provides a structured training methodology that enables models to learn both when and how to retrieve information effectively.

For the AI development community, M2R suggests that hallucination reduction may not require architectural revolution but rather strategic information management during inference. This has practical implications for deploying LLMs in domains like healthcare, legal analysis, and financial services where factual accuracy directly impacts outcomes. The framework's effectiveness in lengthy-context settings addresses a critical gap as applications increasingly demand processing of multi-document inputs and extended narratives. Future research should examine whether M2R's principles scale to very large models and whether the rule-based reward system generalizes across diverse domains and task types.

• →M2R reduces LLM hallucination by maintaining key information proximity to model outputs rather than early-stage retrieval.
• →The framework employs curriculum learning with reinforcement learning to train stable retrieval and grounding capabilities.
• →Proximity of evidence to final answers directly correlates with factual accuracy in long-form generation tasks.
• →The approach builds a dynamic key-information repository during reasoning to prevent noise accumulation in multi-turn retrieval.
• →Effectiveness is particularly pronounced in lengthy-context settings where conventional RAG systems typically degrade.