LLMTM: Benchmarking and Optimizing LLMs for Temporal Motif Analysis in Dynamic Graphs

Researchers introduced LLMTM, a comprehensive benchmark to evaluate Large Language Models' performance on temporal motif analysis in dynamic graphs. The study tested nine different LLMs and developed a structure-aware dispatcher that balances accuracy with cost-effectiveness for graph analysis tasks.

• →LLMTM benchmark evaluates LLM performance across six temporal motif tasks and nine motif types in dynamic graphs.
• →Nine LLMs were tested including GPT-4o-mini, DeepSeek-R1, and Qwen2.5-32B-Instruct models.
• →A tool-augmented LLM agent achieved high accuracy but at substantial computational cost.
• →The structure-aware dispatcher successfully maintains accuracy while reducing operational costs.
• →This research addresses the relatively unexplored area of LLMs processing dynamic graph structures.