Training Time Prediction for Mixed Precision-based Distributed Training

Researchers have developed a precision-aware training time predictor for distributed deep learning that accounts for floating-point precision settings, achieving 9.8% prediction accuracy compared to 147.85% error in existing models that ignore precision variations. The work addresses a critical gap in resource allocation and cost estimation for AI training workloads, where precision choices can create 2.4x variations in training time.

Distributed deep learning training has become increasingly complex as organizations scale AI workloads across multiple devices and data centers. The precision of numerical computations—whether using full 32-bit floating-point, 16-bit, or mixed approaches—significantly impacts both training speed and memory consumption. This research identifies and quantifies a previously underestimated variable in training time prediction models, demonstrating that ignoring precision settings leads to prediction errors exceeding 147% in some scenarios. The gap exists because traditional static computation graphs fail to capture how different precision modes alter computational bottlenecks and memory bandwidth requirements.

The implications extend across the AI infrastructure ecosystem. Accurate training time prediction directly influences infrastructure investment decisions, cloud resource billing models, and job scheduling efficiency. When prediction errors exceed 100%, organizations cannot reliably estimate training costs, leading to budget overruns or underutilized resources. This uncertainty cascades through data center operations, affecting everything from electricity consumption planning to GPU allocation strategies.

The proposed precision-aware predictor achieving 9.8% MAPE represents a substantial improvement that enables more reliable resource planning. For enterprises training large models like LLMs, reducing prediction uncertainty translates to better cost control and faster time-to-deployment. The work gains additional significance as mixed precision training becomes standard practice, making precision-agnostic models increasingly obsolete. Organizations and infrastructure providers will likely need to incorporate such precision-aware predictions into their scheduling systems to maintain competitive efficiency in AI workload management.

• →Floating-point precision settings create 2.4x variations in training time, a factor largely ignored by existing prediction models.
• →Current training time predictors suffer up to 147.85% error when precision variations are not considered.
• →The new precision-aware predictor reduces prediction error to 9.8% MAPE across diverse precision configurations.
• →Accurate training time prediction directly impacts resource allocation costs and infrastructure planning for AI workloads.
• →Mixed precision training requires dynamic prediction models rather than static computation graphs to achieve reliability.