Distributed LLM Pretraining During Renewable Curtailment Windows: A Feasibility Study

Researchers developed a system that trains large language models using renewable energy during curtailment periods when excess clean electricity would otherwise be wasted. The distributed training approach across multiple GPU clusters reduced operational emissions to 5-12% of traditional single-site training while maintaining model quality.

• →A new system enables LLM training during renewable energy curtailment windows when excess clean electricity is available.
• →The approach uses geo-distributed GPU clusters that switch between local and federated training as sites become available.
• →Preliminary results show 88-95% reduction in operational emissions compared to traditional single-site training.
• →The prototype successfully trained a 561M-parameter transformer model across three clusters using real-world curtailment data.
• →This method could make AI training both more environmentally sustainable and cost-effective by utilizing otherwise wasted renewable energy.