Multi-Faceted Self-Consistent Preference Alignment for Query Rewriting in Conversational Search

Researchers introduce MSPA-CQR, a machine learning approach that improves conversational query rewriting by aligning preferences across three dimensions: query rewriting, passage retrieval, and response generation. The method uses self-consistent preference data and direct preference optimization to generate more diverse and effective rewritten queries in conversational search systems.

This academic research addresses a fundamental challenge in conversational search systems: the tendency of earlier approaches to optimize query rewriting in isolation without considering downstream effects on retrieval and response quality. MSPA-CQR represents a shift toward holistic system optimization, recognizing that an isolated rewrite may perform well on its own metric while degrading overall search performance.

The approach builds on recent advances in preference learning and direct preference optimization (DPO), techniques that have gained prominence in large language model fine-tuning. By constructing training data that reflects preferences across multiple dimensions simultaneously, the researchers create a framework where query rewrites are evaluated not just for grammatical correctness or semantic clarity, but for their actual utility in retrieving relevant passages and enabling coherent response generation.

For the conversational AI and search industry, this research demonstrates practical value in improving user experience across information-seeking tasks. The method's effectiveness in both in-distribution and out-of-distribution scenarios suggests robustness that could translate to real-world deployment, where systems encounter diverse query types and contexts beyond training data.

The multi-faceted preference alignment approach has implications beyond conversational search. Similar methodologies could enhance other sequential generation tasks where downstream components depend on upstream model decisions. As conversational interfaces become more prevalent in enterprise and consumer applications, optimizing for end-to-end performance rather than component-level metrics becomes increasingly important for competitive differentiation.

• →MSPA-CQR optimizes query rewriting by considering feedback from retrieval and response generation simultaneously rather than in isolation.
• →The method constructs self-consistent preference alignment data across three distinct dimensions to improve training quality.
• →Prefix-guided multi-faceted direct preference optimization enables learning from multiple performance objectives at once.
• →The approach shows effectiveness in both in-distribution and out-of-distribution test scenarios, suggesting practical robustness.
• →This research represents a broader trend toward end-to-end system optimization in conversational AI rather than component-level optimization.