AIBullisharXiv – CS AI · 6h ago7/10
🧠
AutoRelAnnotator: Calibrated Model Cascades for Cost-Efficient Relevance Evaluation in Sponsored Search
Researchers introduced AutoRelAnnotator, a calibrated model cascade system that generates high-quality relevance annotations for search ranking systems at significantly lower cost than human labeling. The approach combines domain-specific fine-tuning, progressive model cascading, and isotonic calibration to achieve production-grade accuracy while reducing compute costs by approximately 50%, with validation across 150M+ annotations in real-world search and advertising systems.