Transformer autoencoder with local attention for sparse and irregular time series with application on risk estimation

Researchers present a Transformer Autoencoder framework with local attention mechanisms designed to detect non-technical losses (electricity theft) in power grids using sparse, irregular time series data. The model demonstrates superior performance in risk estimation for Greek electrical systems compared to existing methods, achieving high recall and precision while effectively handling data collection irregularities.

This research addresses a critical infrastructure challenge where traditional machine learning approaches fail: detecting anomalies in real-world power grid data characterized by sparse collection intervals and irregular timestamps. Electricity theft represents a significant economic burden on utilities, yet conventional risk estimation methods struggle with the temporal inconsistencies inherent in field measurements. The proposed Transformer Autoencoder leverages self-attention mechanisms to identify long-range dependencies that conventional time series models miss, while local attention prevents computational overhead and focuses learning on relevant temporal neighborhoods.

The framework bridges academic machine learning advances and practical utility operations. Transformers have revolutionized NLP and are increasingly applied to sequential data beyond text, yet their adaptation to irregular time series remains underexplored. This work demonstrates that combining architectural innovations (local attention) with robust preprocessing creates discriminative latent representations that generalize better than baseline methods.

For power utilities and grid operators, this approach offers measurable operational impact: improved theft detection reduces non-technical losses that currently plague many regional grids. The real-world Greek case study validates applicability beyond controlled environments, suggesting deployment potential across other regions facing similar challenges. Financial implications extend to utility profitability and grid stability—undetected losses propagate as higher costs for legitimate consumers and operational strain.

Future developments should explore whether this framework scales to continental-level grids, handles adversarial theft patterns, and integrates with existing SCADA systems. Cross-sector applications in IoT sensor networks and financial anomaly detection remain promising but untested.

• →Transformer Autoencoders with local attention effectively process sparse, irregular time series data that defeat traditional methods
• →Framework demonstrates high recall and precision for detecting electricity theft in Greek power systems—a major non-technical loss source
• →Local attention mechanisms reduce computational complexity while maintaining pattern discrimination in irregular sequences
• →Real-world validation on infrastructure data proves machine learning advances translate to operational utility improvements
• →Approach has potential cross-sector applications beyond power grids to IoT anomaly detection and financial risk systems