LoopFM: Learning frOm HistOrical RePresentations of Foundation Model for Recommendation

LoopFM introduces a novel knowledge distillation framework that transfers rich intermediate representations from large foundation models to compact vertical models, achieving significant conversion improvements (0.5-1.22%) in industrial-scale systems by structuring FM embeddings as input features rather than relying on single scalar predictions.

LoopFM addresses a fundamental limitation in knowledge distillation where large foundation models fail to efficiently transfer their capabilities to smaller, production-ready vertical models. Traditional KD methods compress complex learned representations into single scalar outputs, creating an information bottleneck that limits how much improvement smaller models can capture. This research proposes structuring intermediate embeddings from foundation models as sequential input features, creating a high-bandwidth information channel without requiring real-time FM inference during serving.

The framework emerges from the growing tension between the capabilities of large foundation models and the operational constraints of production systems. As FMs scale to trillion parameters, deploying them directly becomes economically and technically infeasible. LoopFM decouples this dependency through offline embedding generation, allowing engineers to leverage FM knowledge without architectural coupling or serving overhead.

The empirical results carry substantial weight for industry applications. On public benchmarks, the framework achieves 6%+ AUC improvements on real e-commerce data, while deployment evidence from trillion-parameter systems shows it approximately doubles the knowledge transfer ratio compared to traditional KD alone. The conversion improvements (0.5-1.22%) directly impact revenue metrics that drive business decisions.

This advancement signals a maturing approach to model efficiency in machine learning systems. Rather than viewing foundation models as separate entities, LoopFM demonstrates how to systematically extract and repurpose their learned representations. For companies operating at scale with resource constraints, this framework offers a practical method to harness FM capabilities without proportional infrastructure investment.

• →LoopFM approximately doubles knowledge transfer ratio on trillion-parameter foundation models compared to traditional knowledge distillation alone
• →Framework achieves 0.5-1.22% conversion improvements in industrial deployments with billions of examples
• →Offline embedding approach eliminates real-time foundation model inference requirements during serving
• →6%+ AUC improvements demonstrated on public recommendation benchmarks including TaobaoAd dataset
• →Method enables decoupling of architectural dependencies between large foundation models and compact production models