Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures

Researchers introduced Watt Counts, an open-access dataset containing over 5,000 energy consumption experiments across 50 LLMs and 10 NVIDIA GPUs, revealing that optimal hardware choices for energy-efficient inference vary significantly by model and deployment scenario. The study demonstrates practitioners can reduce energy consumption by up to 70% in server deployments with minimal performance impact, addressing a critical gap in energy-aware LLM deployment guidance.

The energy consumption of Large Language Models represents a growing operational and environmental challenge for organizations deploying AI infrastructure at scale. Watt Counts directly addresses this challenge by providing the first comprehensive, reproducible dataset quantifying energy trade-offs across heterogeneous GPU architectures—filling a meaningful void where system operators previously lacked empirical guidance for hardware selection decisions.

The research context reflects broader industry concerns about AI sustainability. As LLM adoption accelerates, energy costs and carbon footprints have become material considerations for enterprises and cloud providers. Previous benchmarking efforts focused primarily on inference speed or accuracy, largely ignoring the energy dimension despite its direct impact on operating expenses and environmental responsibility. This work rebalances that equation by making energy-efficiency data comparable and transparent.

For practitioners and infrastructure teams, the findings carry immediate practical value. The discovery that optimal GPU choices vary substantially across models and scenarios—rather than following a one-size-fits-all approach—suggests significant cost optimization opportunities through thoughtful hardware-workload matching. A 70% energy reduction in server scenarios and 20% in batch scenarios represents substantial financial savings for organizations running inference at scale, particularly in cost-sensitive regions or carbon-constrained environments.

The open-source, reproducible benchmark framework invites community expansion and validation, potentially establishing Watt Counts as an industry standard reference. Future impact depends on adoption by cloud providers, framework developers, and enterprise teams in deployment decision-making. The work signals growing maturity in AI operations, where energy efficiency becomes a primary optimization lever alongside traditional performance metrics.

• →Watt Counts provides the largest open-access dataset of LLM energy consumption with over 5,000 experiments across 50 models and 10 NVIDIA GPUs
• →Optimal GPU selection for energy efficiency varies significantly across different LLM models and deployment scenarios, rejecting one-size-fits-all approaches
• →Server deployments can achieve up to 70% energy consumption reduction with negligible user experience impact through proper hardware selection
• →The reproducible benchmark framework enables community submissions and establishes a foundation for standardized energy-aware LLM benchmarking
• →Energy-efficient hardware-workload matching represents a direct cost optimization opportunity for organizations operating LLM inference infrastructure