Benchmarking Overton Pluralism in LLMs

Researchers introduced OVERTONBENCH, a framework for measuring viewpoint diversity in large language models through the OVERTONSCORE metric. In a study of 8 LLMs with 1,208 participants, models scored 0.35-0.41 out of 1.0, with DeepSeek V3 performing best, showing significant room for improvement in pluralistic representation.

• →OVERTONBENCH provides the first standardized framework for measuring viewpoint diversity in LLMs using the OVERTONSCORE metric.
• →All tested models scored poorly (0.35-0.41 out of 1.0) on pluralism, indicating substantial bias limitations.
• →DeepSeek V3 achieved the highest pluralism score among the 8 LLMs evaluated.
• →The automated benchmark shows high correlation with human judgments (ρ = 0.88), enabling scalable evaluation.
• →The framework transforms pluralistic AI alignment from abstract concept to measurable benchmark for systematic improvement.