A unified foundational framework for knowledge injection and evaluation of Large Language Models in Combustion Science

Researchers developed the first comprehensive framework for creating domain-specialized Large Language Models for combustion science, using 3.5 billion tokens from scientific literature and code. The study found that standard RAG approaches hit a performance ceiling at 60% accuracy, highlighting the need for more advanced knowledge injection methods including knowledge graphs and continued pretraining.

• →First end-to-end framework created for developing combustion science-specialized LLMs using 3.5 billion tokens of domain data.
• →Standard retrieval-augmented generation (RAG) accuracy peaks at 60%, well below the theoretical upper bound of 87%.
• →Context contamination severely constrains RAG performance, creating a hard ceiling for knowledge injection.
• →The framework includes CombustionQA benchmark with 436 questions across eight combustion science subfields.
• →Advanced approaches using knowledge graphs and continued pretraining are necessary to overcome RAG limitations.