GLEE: A Unified Framework and Benchmark for Language-based Economic Environments

Researchers introduce GLEE, a new framework for studying how Large Language Models behave in economic games and strategic interactions. The study reveals that LLM performance in economic scenarios depends heavily on market parameters and model selection, with complex interdependent effects on outcomes.

• →GLEE provides a standardized benchmark for testing LLM behavior in two-player economic games with natural language communication.
• →The framework evaluates LLM performance on individual gains, efficiency, and fairness metrics across various economic environments.
• →Market parameters and LLM choice have complex, interdependent effects on economic outcomes in strategic interactions.
• →The research addresses critical questions about LLM integration into real-world economic systems like retail platforms.
• →Results suggest careful design and analysis is needed when deploying language-based AI agents in economic ecosystems.