Discovering Cooperative Pipelines: Autoresearch for Sequential Social Dilemmas

Researchers demonstrate an autoresearch framework where an AI agent autonomously optimizes LLM-based policy synthesis for multi-agent cooperation problems. The system discovers objective-dependent pipeline designs that outperform hand-crafted baselines, with fairness mechanisms emerging only when optimizing for equitable outcomes rather than efficiency.

This research advances multi-agent AI coordination by automating the discovery of effective policy-synthesis pipelines through meta-level optimization. Rather than relying on human-designed prompts and feedback mechanisms, a researcher agent iteratively modifies system components—prompts, evaluation functions, and helper libraries—to improve cooperation outcomes in sequential social dilemmas. The framework's key strength lies in its objective-sensitivity: when optimizing for Rawlsian maximin (fairness), the system independently incorporates explicit fairness mechanisms absent from its base configuration, while efficiency-focused runs omit these features. This reveals how bounded-rational AI systems can be guided toward different welfare goals through information design rather than direct instruction.

The work extends beyond traditional prompt engineering by treating the entire synthesis pipeline as a learnable object. Across two game environments (Cleanup and Gathering) and multiple LLM backends, discovered pipelines consistently outperform baselines with reduced variance, suggesting the approach generalizes. The emergence of fairness mechanisms without explicit fairness-objective specification demonstrates that the researcher agent infers instrumental strategies aligned with underlying goals—a finding relevant to AI alignment research.

For AI systems deployed in multi-agent coordination contexts, this approach offers a principled path toward objective-aligned behavior discovery. The framework's ability to tailor mechanisms to welfare criteria suggests practical applications in resource allocation, negotiation systems, and cooperative game-playing. Future work should explore scaling to more complex environments and understanding why specific mechanisms emerge under particular objectives, potentially yielding insights into mechanism design and AI interpretability.

• →Autoresearch framework autonomously discovers superior LLM policy-synthesis pipelines by iteratively modifying system prompts, feedback mechanisms, and helper libraries.
• →Discovered pipelines exhibit objective-dependency, automatically incorporating fairness mechanisms only when optimizing for equitable outcomes rather than efficiency.
• →Approach outperforms hand-designed baselines and prompt-only optimization across multiple games and LLM variants with reduced run-to-run variance.
• →Framework demonstrates information-design principle where AI systems infer welfare-aligned strategies through meta-level optimization without explicit instruction.
• →Results suggest practical applications in multi-agent coordination, resource allocation, and mechanism design where objective-aware policy synthesis is needed.