DEI: Diversity in Evolutionary Inference for Quality-Diversity Search

Researchers present DEI, a distributed Quality-Diversity search framework that uses heterogeneous large language models as mutation operators to solve competitive programming tasks. A four-model ensemble achieved 124% higher performance than single-model baselines, demonstrating that model diversity—not just computational parallelism—drives superior outcomes in evolutionary AI search.

DEI represents a significant shift in how distributed AI systems can be designed and optimized. Rather than replicating identical models across multiple nodes, the framework leverages the distinct creative biases of different LLMs working collaboratively. This approach extends the Digital Red Queen framework by sharing optimal solutions between nodes, creating cross-model adversarial pressure that builds robustness beyond what self-play alone can achieve. The empirical results are compelling: testing on Core War, a competitive programming domain where AI-generated Redcode warrior programs battle in a simulated environment, shows a four-node heterogeneous ensemble substantially outperforms both single-node baselines and homogeneous multi-node setups at equal computational budgets. The 124% improvement in merged-archive QD-Score and 28% coverage gain suggest fundamental insights about how diverse reasoning patterns contribute to solution quality and generalization. This research holds implications for distributed AI systems beyond competitive programming. Organizations building large-scale AI infrastructure have traditionally prioritized homogeneous deployment for consistency and simplification. DEI suggests this approach may be suboptimal, and that deliberate diversity in model selection could yield better problem-solving capabilities. The finding that heterogeneous ensembles outperform homogeneous ones even at equal budget challenges conventional wisdom about scaling. As LLM architectures continue evolving with different training approaches and design philosophies, practitioners developing distributed systems may need to reconsider their deployment strategies to incorporate complementary models rather than identical copies.

• →Heterogeneous LLM ensembles achieve 124% higher Quality-Diversity scores than single-model systems with equivalent computational budgets
• →Model diversity drives performance gains in distributed search more effectively than homogeneous parallelism alone
• →Cross-model adversarial pressure from shared optimal solutions creates robustness improvements beyond self-play mechanisms
• →The framework demonstrates superior generalization across held-out test cases when models with different creative priors collaborate
• →DEI challenges conventional distributed AI deployment practices favoring homogeneous system replication