CITRAS: Covariate-Informed Transformer for Time Series Forecasting

Researchers introduce CITRAS, a Transformer-based model that improves time series forecasting by effectively integrating multiple data types: target variables, observed covariates (past-only data), and known covariates (advance-known data like calendar events). The model addresses a critical limitation in existing deep learning forecasting systems through two novel mechanisms that align future covariate information with predictions and refine cross-variable dependencies.

CITRAS represents a meaningful advancement in time series forecasting architecture, tackling a persistent challenge that limits current deep learning models. Traditional forecasting systems struggle when handling covariates with misaligned temporal windows—known covariates extend into the future while observed covariates remain historical. This fundamental asymmetry has forced practitioners to either discard valuable advance-known information or rely on suboptimal integration methods.

The model's innovation lies in two mechanisms embedded within its attention framework. The Key-Value Shift allows future covariate information to meaningfully influence predictions by establishing temporal alignment based on concurrent dependencies rather than forcing chronological consistency. Attention Score Smoothing then elevates patch-level attention patterns into coherent cross-variable relationships, preventing the model from over-fitting to local patterns while missing global covariate-target interactions.

For the forecasting industry, CITRAS addresses practical use cases where advance knowledge exists: retail demand forecasting with known promotional schedules, energy consumption with known calendar events, or financial markets with announced economic indicators. By demonstrating versatility across covariate-informed and multivariate settings, the model potentially reduces the engineering overhead practitioners face when preparing data for competing systems.

The research signals growing sophistication in Transformer-based time series modeling. As organizations increasingly recognize that external factors drive outcomes, models that elegantly incorporate multiple information sources without architectural compromises gain competitive value. Future developments likely involve deploying such approaches at scale across domains where forward-looking data improves forecast reliability.

• →CITRAS introduces KV Shift mechanism to seamlessly incorporate future-known covariates into time series forecasting models.
• →Attention Score Smoothing refines local patch-level attention into global cross-variable dependencies for improved accuracy.
• →The decoder-only Transformer preserves autoregressive capabilities while flexibly handling observed and known covariates.
• →Model demonstrates strong empirical performance across real-world datasets in both multivariate and covariate-informed forecasting scenarios.
• →Architecture addresses fundamental length-discrepancy problem that limits existing deep learning forecasting approaches.