AINeutralarXiv – CS AI · 10h ago6/10
🧠
The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling Model
Researchers present a roofline-inspired framework for accurately predicting energy consumption during Transformer model training across multiple GPUs. The study uses BERT architectural sweeps to correlate energy usage with computational proxies, hardware efficiency factors, and parallelism strategies, enabling more sustainable and cost-aware AI system design.