SmartChunk Retrieval: Query-Aware Chunk Compression with Planning for Efficient Document RAG

Researchers have developed SmartChunk retrieval, a query-adaptive framework that improves retrieval-augmented generation (RAG) systems by dynamically adjusting chunk sizes and compression for document question answering. The system uses a planner to predict optimal chunk abstraction levels and a compression module to create efficient embeddings, outperforming existing RAG baselines while reducing costs.

• →SmartChunk addresses limitations of current RAG systems that use static, fixed-size document chunks for retrieval.
• →The framework includes a planner that predicts optimal chunk abstraction levels and a lightweight compression module for embeddings.
• →A novel reinforcement learning scheme called STITCH improves accuracy and generalization capabilities.
• →Testing across five QA benchmarks plus out-of-domain datasets shows superior performance compared to state-of-the-art RAG baselines.
• →The system demonstrates strong scalability with larger corpora while maintaining cost efficiency.