Rethinking Sales Lead Scoring with LLM-based Hierarchical Preference Ranking

Researchers introduce HPRO, an LLM-based framework for sales lead scoring that combines structured CRM data with unstructured customer interactions using hierarchical preference ranking. A 132-day A/B test with a major NEV manufacturer showed 9.5% sales volume uplift and 39.7% precision improvement, demonstrating practical commercial viability beyond traditional machine learning approaches.

This research addresses a genuine operational challenge in high-stakes B2B sales environments where traditional lead scoring methods fall short. Unlike e-commerce recommendations with immediate feedback loops, automotive and real estate sales involve extended decision cycles and multi-stage evaluation processes that existing pointwise models struggle to capture. The semantic gap between unstructured CRM notes and quantifiable scores has historically required manual intervention or rule-based systems with limited scalability.

The innovation lies in leveraging LLMs' semantic understanding while constraining their output to comparative rankings rather than free-text generation. The margin-aware Bradley-Terry formulation transforms sparse binary labels into dense preference pairs, enabling the model to learn relative lead priorities aligned with sales funnel stages. This hierarchical approach mirrors how sales teams actually think about prospect quality—not as absolute scores but as contextual rankings within specific funnel segments.

The empirical results carry meaningful weight. An AUC of 0.8161 on classification and 39.7% precision gains among top-ranked leads suggest the model effectively identifies conversion-ready prospects. Critically, the 132-day online A/B test demonstrating 9.5% sales volume uplift validates that the approach translates from academic metrics to measurable business outcomes at scale with a leading NEV (New Energy Vehicle) manufacturer.

This work expands the practical applications of LLMs beyond content generation into structured decision systems. For enterprise SaaS vendors, CRM platforms, and sales automation tools, this methodology offers a blueprint for incorporating LLM capabilities into existing workflows without requiring architectural overhauls. The approach likely generalizes across other long-cycle B2B domains including enterprise software, industrial equipment, and commercial real estate.

• →HPRO framework combines LLM semantic understanding with hierarchical preference ranking to improve lead scoring in high-stakes B2B sales.
• →Achieved 9.5% sales volume uplift in 132-day production A/B test with major NEV manufacturer, validating real-world commercial impact.
• →LLM-based approach outperforms traditional rule-based and pointwise machine learning methods by capturing relative lead priority across sales funnel stages.
• →Margin-aware Bradley-Terry formulation transforms sparse binary labels into dense preference pairs, enabling both pointwise and pairwise supervision.
• →Model achieved 0.8161 AUC classification performance with 39.7% precision improvement among top-ranked leads compared to baselines.