dMX: Differentiable Mixed-Precision Assignment for Low-Precision Floating-Point Formats

Researchers introduce dMX, a differentiable mixed-precision quantization framework that enables dynamic floating-point bit-width assignment across different layers of large language models. The method uses continuous optimization with temperature-based annealing to efficiently compress models while maintaining accuracy, demonstrating improvements over existing quantization heuristics across multiple LLM families.

dMX addresses a fundamental challenge in LLM deployment: uniform quantization across all layers wastes computational resources by applying unnecessary precision to certain layers while potentially under-specifying others. The framework's innovation lies in treating bit-width assignment as a learnable optimization problem rather than a fixed hyperparameter, enabling dynamic allocation of precision based on each layer's importance to model performance.

The technical approach reflects broader trends in machine learning toward automated efficiency optimization. By parameterizing floating-point formats with a single scalar offset that gradually discretizes during training, dMX elegantly bridges the gap between continuous optimization and hardware constraints. This differentiable pipeline allows gradient-based learning of quantization strategies, eliminating the need for manual heuristic selection. The integration with Open Compute Project's MXFP standard ensures practical hardware compatibility, addressing a critical gap between research quantization schemes and real-world deployment.

For developers and inference service providers, this research has tangible implications for reducing computational costs and latency. The ability to target specific bit-width budgets while maintaining accuracy directly translates to lower infrastructure expenses and faster model serving. Evaluation across diverse LLM families—Llama, Qwen3, SmolLM2—demonstrates generalizability, suggesting the approach is robust across architectural variations.

The focus on hardware-native formats indicates the field is maturing beyond theoretical quantization research toward practical deployment. Organizations optimizing inference costs should monitor whether these techniques get integrated into popular quantization libraries and frameworks, as adoption would meaningfully impact the economics of LLM deployment at scale.

• →dMX enables learnable per-layer floating-point bit-width assignment for LLMs using continuous optimization with temperature annealing
• →The framework achieves Pareto-dominating efficiency-accuracy tradeoffs compared to existing KL divergence-based quantization heuristics
• →Integration with OCP's MXFP standard ensures hardware compatibility for practical deployment scenarios
• →Target-aware regularization allows users to specify inference cost budgets while optimizing model quality automatically
• →Consistent improvements demonstrated across multiple LLM families including Llama, Qwen3, and SmolLM2