Learning Chain Of Thoughts Prompts for Predicting Entities, Relations, and even Literals on Knowledge Graphs

Researchers introduce RALP, a novel method that uses chain-of-thought prompts with large language models to improve knowledge graph predictions, outperforming traditional embedding models by over 5% on standard benchmarks while better handling unseen entities, relations, and numerical data.

Knowledge graphs power recommendation systems, semantic search, and AI reasoning across industry applications, but current embedding-based models hit a fundamental wall: they cannot generalize to unseen entities or relations encountered after training. RALP addresses this limitation by reframing link prediction as a prompt learning problem, leveraging the generalization capabilities of pretrained language models rather than training fixed vector embeddings. The approach uses Bayesian optimization to discover effective chain-of-thought prompts from minimal training data—fewer than 30 examples—without requiring gradient-based learning.

This work represents a significant shift in how the AI community approaches structured data prediction. Traditional knowledge graph embedding methods like TransE or DistMult excel at benchmark performance but fail on dynamic, heterogeneous real-world graphs where new entities and relations continually emerge. The reliance on gradient-free optimization (MIPRO algorithm) is particularly notable, as it enables prompt discovery without access to model parameters, making the approach applicable even to closed-source commercial LLMs.

For practitioners building production systems, RALP offers tangible advantages: improved performance on numerical and literal predictions—historically weak points for embeddings—and the flexibility to reason over complex logical expressions from OWL ontologies with 88% accuracy. The open-source release democratizes access to these techniques, likely accelerating adoption in enterprise knowledge management and semantic AI applications. The implicit message is clear: prompt-based reasoning may become the default approach for dynamic knowledge graphs, potentially reshaping investment in embedding-specific research and infrastructure.

• →RALP achieves 5%+ improvement in link prediction accuracy over state-of-the-art embedding models across multiple benchmarks.
• →The method learns effective prompts from fewer than 30 training examples using gradient-free Bayesian optimization, enabling rapid adaptation.
• →RALP successfully predicts unseen entities, relations, and numerical literals—persistent weaknesses of traditional embedding-based approaches.
• →Complex logical reasoning on OWL class expressions reaches 88% Jaccard similarity, demonstrating capability beyond simple triple prediction.
• →Open-source release of implementation and evaluation pipeline lowers barriers to adoption in production knowledge graph systems.