Improving Role Consistency in Multi-Agent Collaboration via Quantitative Role Clarity

Researchers developed a quantitative method to improve role consistency in multi-agent AI systems by introducing a role clarity matrix that measures alignment between agents' assigned roles and their actual behavior. The approach significantly reduced role overstepping rates from 46.4% to 8.4% in Qwen models and from 43.4% to 0.2% in Llama models during ChatDev system experiments.

• →Multi-agent AI systems suffer from role disobedience where agents fail to adhere to their assigned responsibilities and behave like other agents.
• →A new quantitative role clarity matrix measures semantic similarity between agent behavior and role descriptions to improve consistency.
• →The method uses lightweight fine-tuning with the role clarity matrix as a regularizer to enhance role adherence.
• →Testing on ChatDev showed dramatic reductions in role overstepping rates across both Qwen and Llama language models.
• →Task success rates improved alongside role consistency, demonstrating practical benefits for multi-agent system performance.