TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents

Researchers introduce TowerMind, a lightweight tower defense game environment designed to evaluate Large Language Models as autonomous agents. The benchmark tests LLMs' capabilities in strategic planning and real-time decision-making while revealing significant performance gaps compared to human experts and highlighting key limitations in model reasoning.

TowerMind addresses a critical gap in AI evaluation infrastructure by providing a computationally efficient alternative to existing real-time strategy game environments for testing LLM agent capabilities. The environment's design reflects growing recognition that evaluating advanced language models requires complex decision-making scenarios that demand both long-term planning and tactical adaptation—capabilities increasingly central to deploying LLMs as autonomous agents in practical applications.

The research emerges from a broader trend in AI development toward creating standardized benchmarks that can rigorously assess model performance beyond traditional language tasks. Tower defense games offer an ideal testing ground because they require agents to balance resource allocation, threat assessment, and dynamic strategy adjustment. Previous RTS environments either demanded substantial computational resources or lacked multimodal observation spaces compatible with LLM inputs, limiting their utility for language model evaluation.

The findings carry significant implications for the AI development community. By demonstrating clear performance gaps between state-of-the-art LLMs and human experts across multiple dimensions—including hallucination propensity, planning validation, and decision-making efficiency—the research provides quantifiable evidence of current model limitations in agentic scenarios. The evaluation of both LLMs and classical reinforcement learning algorithms enables direct comparison of different AI paradigms for agent development.

Looking forward, TowerMind's open-source availability positions it as a potential standard benchmark in the AI agent evaluation ecosystem. Future work will likely explore how architectural improvements in LLMs translate to enhanced agent performance, and whether insights from this environment transfer to real-world agentic applications in robotics, autonomous systems, and complex decision-making domains.

• →TowerMind provides a computationally efficient benchmark for evaluating LLMs as agents, addressing limitations of existing RTS game environments.
• →Performance gaps between LLMs and human experts reveal critical limitations in planning validation, decision-making diversity, and action execution efficiency.
• →The environment's multimodal observation space (pixel, text, and structured data) enables comprehensive evaluation across different input modalities.
• →Open-source availability suggests TowerMind could become a standard benchmark for evaluating autonomous agent capabilities across the AI research community.
• →Comparative analysis of LLMs versus reinforcement learning algorithms provides insights into different approaches for developing autonomous agents.