Beyond Consensus: Trace-Level Synthesis in Mixture of Agents

Researchers demonstrate that aggregating complete reasoning traces from multiple LLM agents recovers correct solutions more effectively than majority voting, even when agents unanimously agree. A new approach called Self-Consistent Mixture of Agents uses semantic-preserving perturbations to generate trace diversity while maintaining safety guarantees, outperforming heterogeneous model ensembles across mathematical and scientific reasoning tasks.

This research challenges a fundamental assumption in multi-agent AI systems: that consensus represents the optimal synthesis point. The aggregation paradox reveals that majority voting discards valuable minority reasoning chains containing correct intermediate steps, creating an artificial performance ceiling regardless of perturbation diversity. The key insight is architectural—the unit of analysis should shift from final answers to complete reasoning traces, enabling an aggregator to reconstruct superior solutions by cherry-picking valid intermediate logic across diverse agent attempts.

The Self-Consistent Mixture of Agents framework addresses a critical tension in ensemble systems: maximizing diversity while maintaining reliability. By using semantic-preserving input perturbations rather than different models, the approach maintains computational efficiency while generating trace variation. The anchored refinement mechanism with non-degradation guarantees prevents the synthesis process from corrupting the majority's correct reasoning, addressing the legitimate concern that aggressive aggregation could compound errors.

For the AI development community, this work suggests single-model systems with trace-level synthesis may replace expensive multi-model deployments without sacrificing performance. The consistent outperformance across structured reasoning, science, mathematics, and competitive programming indicates broad applicability. The research validates that reasoning diversity within a single model can match or exceed heterogeneous model pools, with significant implications for deployment costs and latency in production systems.

The practical challenge ahead involves scaling trace aggregation to longer reasoning chains and real-time constraints, particularly for applications requiring sub-second response times. Future work should explore whether trace-level synthesis extends to open-ended tasks beyond competition-style problems with verifiable correctness criteria.

• →Majority voting in multi-agent systems discards valuable minority reasoning chains, creating an artificial performance ceiling.
• →Aggregating complete reasoning traces outperforms consensus-based approaches even when all agents unanimously agree.
• →Single models with perturbation-induced trace diversity match or exceed heterogeneous multi-model ensemble performance.
• →Anchored refinement guarantees prevent aggregation from degrading the majority's correct reasoning.
• →Reasoning traces rather than final answers represent the optimal unit of aggregation in ensemble systems.