←Back to feed
🧠 AI🟢 BullishImportance 6/10
Retrieval-Feedback-Driven Distillation and Preference Alignment for Efficient LLM-based Query Expansion
🤖AI Summary
Researchers developed a framework to make large language model-based query expansion more efficient by distilling knowledge from powerful teacher models into compact student models. The approach uses retrieval feedback and preference alignment to maintain 97% of the original performance while dramatically reducing inference costs.
Key Takeaways
- →A new distillation framework transfers query expansion capabilities from large teacher models to smaller, more efficient student models.
- →The method uses retrieval-metric-driven strategy to automatically create training pairs based on nDCG@10 performance differences.
- →The distilled Qwen3-4B model achieves 97% of DeepSeek-685B's performance on TREC DL19 benchmark with much lower inference cost.
- →Direct Preference Optimization is applied to align model generation with retrieval objectives rather than relying on few-shot examples.
- →The approach demonstrates effectiveness across both English and Chinese retrieval tasks, showing cross-language applicability.
#llm#query-expansion#model-distillation#retrieval#preference-alignment#efficiency#nlp#information-retrieval#ai-optimization
Read Original →via arXiv – CS AI
Act on this with AI
Stay ahead of the market.
Connect your wallet to an AI agent. It reads balances, proposes swaps and bridges across 15 chains — you keep full control of your keys.
Related Articles