Representational Collapse in Multi-Agent LLM Committees: Measurement and Diversity-Aware Consensus

Research reveals that multi-agent LLM committees suffer from 'representational collapse' where agents produce highly similar outputs despite different role prompts, with mean cosine similarity of 0.888. A new diversity-aware consensus protocol (DALC) improves accuracy to 87% while reducing token costs by 26% compared to traditional self-consistency methods.

• →Multi-agent LLM committees show representational collapse with agents producing similar outputs despite different prompts, achieving only 2.17 effective rank out of 3 agents.
• →DALC consensus protocol outperforms traditional self-consistency methods, reaching 87% accuracy versus 84% on GSM8K benchmark.
• →The new approach reduces computational costs by 26% in token usage while improving performance.
• →Embedding encoder choice significantly impacts collapse severity, with different encoders showing varying cosine similarity scores.
• →Representational collapse worsens on harder tasks, making diversity-aware protocols increasingly important for complex AI applications.