On the impact of retrieved content representations in RAG Pipelines

Researchers conducted a controlled study examining how retrieved documents should be formatted when fed into language models within RAG pipelines, rather than for human readers. Testing 14 different document representations across summarization, selection, and reformulation techniques, they found that answer retention—whether documents preserve answer-bearing content after transformation—is the primary driver of generation accuracy, while other factors like wording and length have minimal impact.

This research addresses a fundamental gap in how Retrieval-Augmented Generation systems are designed. While RAG has become critical for enhancing LLM capabilities with external knowledge, most pipelines repurpose retrieval and document formatting methods originally engineered for human consumption. The study's controlled methodology isolates document representation from retrieval quality, enabling precise measurement of what actually matters when machines process retrieved content.

The finding that answer retention dominates other representation characteristics has significant implications for RAG optimization. Previous work claiming improvements from specific transformations—shorter summaries, query-dependent reformulations, or structural reorganizations—may have succeeded primarily by preserving answer-bearing passages rather than through their stated mechanisms. This suggests researchers have been attributing performance gains to the wrong causal factors. The research reveals that when documents successfully maintain their answer content after transformation, downstream accuracy remains robust regardless of format changes.

For practitioners building RAG systems, this indicates that optimization efforts should prioritize content preservation over sophisticated representation engineering. Complex document restructuring and query-dependent formatting approaches provide minimal benefit if simpler methods retain the same critical information. This could reduce computational overhead and system complexity while maintaining performance. The work also establishes a new evaluation metric—answer retention—that should become standard for assessing document transformation approaches.

Future research should explore why answer retention is so dominant and whether this finding holds across different domains, document types, and generator architectures. Understanding the mechanisms behind retention-driven accuracy could lead to more efficient RAG designs.

• →Answer retention is the primary determinant of LLM accuracy in RAG systems, far outweighing representation format, length, or query-dependence.
• →Complex document transformations provide minimal performance gains when retention is high, suggesting prior optimization claims may be misattributed.
• →Simpler representation approaches could achieve equivalent results while reducing computational overhead in RAG pipelines.
• →Answer retention should become a standard evaluation metric when assessing document transformation techniques.
• →RAG optimization should focus on preserving answer-bearing content rather than sophisticated reformulation strategies.