Solipsistic Superintelligence is Unlikely to be Cooperative

A new research paper argues that AI systems designed with a solipsistic approach—treating the world as a static source of feedback—will unlikely produce cooperative superintelligence. The authors propose that deploying such systems creates self-undermining optimization effects, and advocate for a fundamentally different research paradigm centered on cooperation and human agency as core design principles rather than secondary objectives.

This arXiv paper presents a theoretical critique of current AI development methodologies, arguing that the field faces a fundamental misalignment between how systems are trained and how they must function in dynamic, multi-agent environments. The core claim centers on the 'self-undermining property of unilateral optimization'—the notion that powerful agents optimized in isolation will create distributional shifts upon deployment, breaking their own effectiveness assumptions.

The research emerges from growing concerns within AI safety circles about scalable alignment. Traditional reinforcement learning treats environments as static; this paper identifies that production deployment violates this assumption, creating what researchers call the train-test-deploy gap. This observation gains relevance as AI systems increasingly interact with other AI agents, markets, and human institutions simultaneously.

For the broader AI industry, this challenges the dominant paradigm of capability scaling. The implications suggest that simply building more powerful task solvers without addressing cooperative equilibrium selection will produce systems prone to unexpected failures. Organizations deploying large language models, autonomous agents, or trading algorithms face potential friction if these systems lack cooperative design principles.

The paper's prescription—treating institutions as design primitives, building dynamic testbeds with adaptive counterparties, and preserving human agency as structural features—represents a significant departure from current practice. This framework matters particularly for decentralized systems, multi-agent markets, and human-AI teaming scenarios. Future AI development cycles will likely incorporate these cooperative design principles, potentially slowing pure capability gains but improving deployment robustness and alignment.

• →Solipsistic AI design (treating environments as static) produces systems unlikely to cooperate effectively in dynamic deployment contexts
• →Unilateral optimization creates self-undermining properties where training-deployment gaps emerge due to endogenous non-stationarity
• →Current AI research paradigm misses interdependence as a core design principle, treating cooperation as a secondary task rather than foundational requirement
• →Future AI systems must incorporate dynamic evaluation with adaptive counterparties and institutions as explicit design primitives
• →Human agency preservation becomes a structural requirement for trustworthy AI deployment rather than an optional safeguard