The Well-Tempered Classifier: Some Elementary Properties of Temperature Scaling

Researchers provide the first rigorous theoretical analysis of temperature scaling, a widely-used technique for controlling uncertainty in machine learning models. The study reveals that while temperature scaling reliably increases entropy in classifiers, it does not necessarily increase diversity in large language models as commonly claimed, and establishes temperature scaling as the unique linear calibration method that preserves hard predictions.

Temperature scaling has become ubiquitous in machine learning as a simple knob for adjusting model confidence and output diversity, yet its theoretical foundations remain poorly understood. This research fills that gap by rigorously characterizing what temperature scaling actually does to probabilistic models. The findings challenge conventional wisdom in the LLM community, which has long assumed that higher temperatures automatically produce more diverse outputs. The reality appears more nuanced: while entropy increases predictably in classification settings, the relationship between temperature and diversity in language generation is less straightforward than practitioners assume.

The paper's geometric characterization—framing temperature scaling as an information projection onto an entropy-constrained manifold—provides valuable intuition for why the method works. This connects temperature scaling to broader calibration literature and information theory, grounding an otherwise empirical technique in principled mathematics. The second characterization, showing that temperature scaling is the sole linear scaler preserving hard predictions, explains its dominance in practice and distinguishes it from more complex calibration methods like matrix scaling or Dirichlet calibration.

For practitioners, these findings suggest temperature scaling's versatility comes with tradeoffs. In classification, it reliably manages uncertainty and calibration. In LLM applications, users should recalibrate expectations about what temperature actually controls—it modulates entropy and probability distributions, but may not achieve the output diversity they intuitively expect. Developers relying on temperature tuning for production systems should validate actual diversity metrics rather than assuming the theoretical properties transfer directly to their applications. This work establishes a more rigorous foundation for temperature scaling's continued evolution and broader adoption in emerging AI systems.

• →Temperature scaling provably increases model entropy in classifiers, providing theoretical justification for its calibration applications
• →The common assumption that higher temperature increases LLM output diversity lacks theoretical support and may mislead practitioners
• →Temperature scaling functions as an information projection onto entropy-constrained model families, offering geometric intuition for its behavior
• →Temperature scaling uniquely preserves hard predictions among linear calibration methods, explaining its practical preference over alternatives
• →Rigorous characterization of temperature scaling enables more informed hyperparameter tuning and model deployment decisions across ML applications