Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)

Researchers present a graph-based retrieval-augmented generation (RAG) system that reduces AI hallucinations by integrating lightweight graph structures with vector search tools. Testing on Wikipedia QA benchmarks shows the approach halves hallucinated answers while improving factual precision and recall with minimal token overhead.

Large language models have revolutionized NLP but introduce a critical vulnerability: hallucination—confidently generating false information. This research addresses a core deployment challenge by enhancing RAG systems, which retrieve external knowledge to ground LLM responses. The key innovation involves layering graph-based tools alongside vector search, enabling the system to traverse structured relationships in knowledge bases rather than relying solely on semantic similarity matching. This dual-retrieval approach mirrors how human reasoning combines keyword lookup with relational understanding.

The work builds on growing recognition that RAG alone proves insufficient for complex reasoning. While RAG prevents models from inventing facts outright, unstructured retrieval can still fail to disambiguate entities or establish causal relationships required for accurate QA. Graph structures impose semantic constraints that naturally reduce confabulation by forcing the system to justify answers through verifiable paths in knowledge graphs.

For practitioners deploying LLMs in compliance-sensitive domains—legal, financial, medical—this technique offers tangible risk reduction. The findings suggest meaningful improvements in precision and truthfulness without proportional computational cost, making the approach economically viable for production systems. The modest token increase matters significantly given LLM inference costs.

The open question remains scalability. Testing on Wikipedia QA represents a controlled environment; real-world proprietary knowledge graphs introduce complexity around schema design and maintenance. Future research should explore whether simple graph schemas generalize across domains, and whether the approach handles sparse or inconsistent knowledge representations common in enterprise settings.

• →Graph-based RAG tools reduce hallucinations by 50% and improve factual correctness precision compared to vector-only retrieval
• →Lightweight graph structures enable agentic systems to reason through structured relationships rather than relying solely on semantic similarity
• →The approach achieves highest truthfulness scores with minimal token usage increase, making it economically viable for production deployment
• →Combining vector search with graph query tools creates complementary retrieval mechanisms that address different failure modes of standalone RAG
• →Results on MoNaCo benchmark validate effectiveness for complex multi-hop question answering tasks requiring entity disambiguation