DSIRM: Learning Query-Bridged Discrete Semantic Identifiers for E-commerce Relevance Modeling

Researchers have developed DSIRM, a machine learning model that improves e-commerce search relevance by combining discrete semantic identifiers with query-dependent ranking. The system achieved a 1.54% offline AUC improvement and significant online gains (+0.13% UCTR, +0.25% UCTCVR) when deployed on Tmall's platform, demonstrating practical value for large-scale recommendation systems.

E-commerce search relevance remains a critical technical challenge as platforms struggle to balance semantic understanding with computational efficiency. Traditional continuous embeddings excel at capturing general similarity but often fail to distinguish fine-grained product attributes that users care about during search. DSIRM addresses this gap by leveraging discrete semantic identifiers—categorical tags that group items based on relevance to specific queries—rather than relying solely on continuous vector representations.

The innovation centers on supervised quantization: instead of using unsupervised clustering methods that lack query context, the researchers inject explicit query-item interaction data into the learning process. This allows the model to learn which products should share identifiers based on actual relevance signals. The approach combines two strategies—query-bridged contrastive quantization for item partitioning and generative LLM integration for query-side SID prediction—creating a hybrid system that handles both well-defined and ambiguous search intents.

For the e-commerce and AI industries, this work demonstrates how incorporating explicit supervision into discrete representation learning yields tangible business results. The 0.13% UCTR (user click-through rate) improvement translates directly to increased engagement and revenue at scale. The deployment architecture proves that complex models can remain efficient in production environments, encouraging further research into hybrid dense-discrete systems.

Future developments may focus on extending this approach to cross-platform recommendation scenarios and applying similar query-bridged quantization techniques to other domains like content ranking and ad matching. The emphasis on LLM integration suggests continued convergence between traditional recommendation systems and large language models.

• →DSIRM combines supervised discrete semantic identifiers with query context to improve e-commerce relevance ranking over unsupervised baselines.
• →Query-bridged contrastive quantization injects explicit supervision signals into item partitioning, enabling relevance-aware semantic clustering.
• →The system achieved 1.54% offline AUC improvement and 0.13% UCTR online lift on Tmall's production data.
• →Generative LLMs predict item SIDs from text queries, resolving tail query ambiguity and intent disambiguation.
• →Hybrid dense-discrete architecture proves computationally efficient for large-scale e-commerce deployment.