FD-RAG: Federated Dual-System Retrieval-Augmented Generation

FD-RAG introduces a federated framework for retrieval-augmented generation that enables decentralized LLM deployment across edge devices without centralizing sensitive data. The system achieves 7.8% accuracy improvements and 8.4x latency reductions by splitting lightweight memory access from expensive LLM reasoning, while aggregating anonymized knowledge across fragmented device networks.

FD-RAG addresses a fundamental infrastructure challenge in deploying large language models across distributed edge environments where traditional centralized RAG systems prove impractical. Current RAG architectures assume centralized knowledge repositories and sufficient computational resources, assumptions that collapse in scenarios where devices cannot share raw data due to privacy constraints and repeated LLM inference becomes economically prohibitive.

The framework's innovation lies in its dual-system approach: semantic-aware adaptive hypergraphs encode local knowledge into compact question-answer memories, enabling the system to resolve well-covered queries through direct memory matching without invoking expensive LLM reasoning. This architectural decoupling reduces computational overhead while the federated aggregation layer combines anonymized memories across devices, preserving privacy while broadening the knowledge base each node can access.

The technical contribution includes theoretical guarantees establishing O(1/ε²) convergence rates for hypergraph learning, providing formal assurance that the approach scales tractably in edge settings. Experimental validation demonstrates simultaneous improvements in both accuracy (7.8%) and latency (8.4x reduction), addressing the typical accuracy-efficiency tradeoff that constrains edge deployments.

This work carries implications for distributed AI infrastructure, particularly relevant as organizations increasingly deploy language models in privacy-sensitive domains like healthcare and finance. The federated memory aggregation pattern could influence how edge AI systems manage knowledge sharing without exposing raw data, establishing architectural patterns for privacy-preserving distributed intelligence. The convergence guarantees also validate federated learning approaches for LLM-adjacent systems, potentially informing broader federated AI development practices.

• →FD-RAG achieves 7.8% accuracy improvement and 8.4x latency reduction through decoupled memory matching and on-demand LLM reasoning
• →Semantic-aware adaptive hypergraphs compress local knowledge into efficient question-answer memories for edge deployment
• →Federated aggregation enables privacy-preserving knowledge sharing across devices without exposing raw documents
• →Theoretical O(1/ε²) convergence guarantees ensure tractable scaling in computationally constrained edge environments
• →Framework addresses privacy and computational constraints preventing centralized RAG deployment in regulated industries