Agentic LLM Planning via Step-Wise PDDL Simulation: An Empirical Characterisation

Researchers developed PyPDDLEngine, an open-source tool that allows large language models to perform task planning through interactive PDDL simulation. Testing on 102 planning problems showed agentic LLM planning achieved 66.7% success versus 63.7% for direct LLM planning, but at 5.7x higher token cost, while classical planning methods reached 85.3% success.

• →PyPDDLEngine enables LLMs to perform interactive task planning through step-by-step PDDL simulation rather than generating complete action sequences upfront.
• →Agentic LLM planning showed only modest 3 percentage point improvement over direct LLM planning but at significantly higher computational cost.
• →Classical planning methods still outperform LLMs substantially, achieving 85.3% success compared to 66.7% for the best LLM approach.
• →LLM-generated plans were shorter than classical methods, suggesting reliance on training data recall rather than genuine planning capabilities.
• →The research indicates agentic gains depend heavily on quality of environmental feedback, with PDDL providing less effective signals than coding environments.