P-Cast Precision in FP8 Attention: Sink-Induced Collapse and the Optimality of S=2^8

Researchers analyze precision loss in FP8 (8-bit floating-point) attention computations, identifying how the Attention Sink phenomenon causes numerical underflow when probability matrices are cast to FP8. The study validates engineering choices in FlashAttention-3/4, proving that reverse KV iteration combined with a scaling factor of S=256 eliminates precision collapse and provides a closed-form threshold for predicting kernel-level accuracy loss.

This paper addresses a critical numerical stability problem in low-precision deep learning inference. As AI systems scale, reducing computational precision from FP32 to FP8 delivers substantial throughput gains, but the Attention Sink phenomenon—where softmax attention concentrates probability mass on specific tokens—creates conditions where the 3-bit mantissa in FP8 cannot represent the full range of values, causing non-sink probabilities to underflow to zero. This precision loss directly degrades model outputs and inference quality.

The research contextualizes a broader industry trend toward mixed-precision inference. As AI models grow larger and deployment costs rise, practitioners increasingly adopt lower-bit formats to reduce memory bandwidth and accelerate computation. However, naive quantization often introduces silent failures where numerical errors accumulate without triggering exceptions. The authors' contribution—proving why S=256 serves as an optimal static scaling factor—provides theoretical grounding for empirical design decisions already embedded in production systems. The threshold formula Delta_c enables practitioners to predict when precision loss will occur given specific attention characteristics.

For AI infrastructure developers and inference system designers, this work validates optimization techniques already deployed while providing mathematical guarantees. Organizations running large language models benefit from understanding when these techniques preserve accuracy versus when additional safeguards become necessary. The 3-10x MSE improvement at moderate sink strengths demonstrates meaningful practical gains. However, the analysis reveals precision degradation remains unavoidable at extreme sink strengths, suggesting limits to how aggressively FP8 quantization can proceed without accuracy compromise.

Future work likely focuses on adaptive scaling strategies that respond to measured attention patterns rather than static factors, or hybrid approaches using higher precision for sink-dominated layers.

• →Forward KV iteration in FP8 attention causes 'P-collapse' where non-sink probabilities underflow, while reverse iteration eliminates this effect
• →S=256 (2^8) serves as the optimal static scaling factor, simultaneously satisfying bit-exact IEEE 754 compliance and maximum normal-range coverage
• →A closed-form threshold formula Delta_c=6.93+ln(S)-delta_k predicts kernel-level precision loss without runtime measurement
• →The analysis validates engineering choices already deployed in FlashAttention-3/4, providing theoretical foundation for production systems
• →Precision improvements plateau when combining both optimizations, indicating a fundamental accuracy floor exists for aggressive FP8 quantization