Payoff scaling shapes cooperation in LLM agents across languages

Researchers analyzed how Large Language Models behave in repeated game scenarios, finding that LLMs become more cooperative as financial stakes increase—contrary to evolutionary game theory predictions. The study reveals that alignment training and human reasoning patterns embedded in LLM training data override expected selfish behavior, with implications for designing multi-agent AI systems in high-stakes environments.

This research addresses a critical gap in understanding LLM behavior when deployed as autonomous negotiating agents. By using supervised classifiers to identify canonical game-theoretic strategies rather than raw action counts, the researchers created a more sophisticated analytical framework than previous studies. Their key finding—that LLMs increase cooperation with higher payoffs while evolutionary theory predicts the opposite—suggests these models have internalized human prosocial values through training data rather than behaving as pure economic actors.

The divergence between LLM and evolutionary baselines matters because it reveals the hidden influence of alignment training. As organizations increasingly deploy LLM agents in negotiation, trading, and coordination tasks, understanding these behavioral patterns becomes essential for risk management. The research extends beyond frontier models like GPT-4 to demonstrate the phenomenon occurs in smaller, open-weight LLMs, indicating this is a structural property rather than isolated to specific architectures.

For AI governance and system design, the findings suggest that payoff structures and linguistic framing are underutilized control mechanisms. Organizations deploying multi-agent LLM systems could optimize cooperation or competition by adjusting how stakes are presented and in which language interactions occur. This has direct implications for autonomous trading systems, resource allocation algorithms, and international diplomatic AI applications where linguistic and cultural nuances could shift behavioral outcomes significantly.

• →LLMs become more cooperative as financial stakes increase, contradicting evolutionary game theory predictions of defection-dominance at higher payoffs
• →Alignment training and human-like reasoning inherited from training data override expected selfish economic behavior in LLM agents
• →Payoff design and linguistic framing are powerful but underexplored levers for steering multi-agent AI behavior
• →The cooperation pattern holds across frontier and smaller open-weight LLMs, indicating it's a structural behavioral property
• →Findings have critical implications for governing and evaluating LLM deployment in high-stakes multilingual negotiation and coordination scenarios