LLM Probability Concentration: How Alignment Shrinks the Generative Horizon

Researchers introduce the Branching Factor (BF) metric to measure how alignment tuning reduces output diversity in large language models by concentrating probability distributions. The study reveals that aligned models generate 2-5x less diverse outputs and become more predictable during generation, explaining why alignment reduces sensitivity to decoding strategies and enables more stable Chain-of-Thought reasoning.

• →Alignment tuning reduces LLM output diversity by a factor of 2-5 overall and up to 10x at beginning positions through probability concentration.
• →The Branching Factor (BF) metric quantifies the effective number of plausible next tokens, typically decreasing as generation progresses.
• →Aligned Chain-of-Thought models achieve more stable outputs by generating longer reasoning chains that push generation into more deterministic stages.
• →Alignment appears to steer models toward stylistic tokens that unlock low-entropy trajectories already present in base models rather than fundamentally changing behavior.
• →Base models can be nudged toward similar low-diversity behavior by prompting with alignment-style tokens like 'Sure'.