Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation Models

Researchers applied sparse autoencoders to analyze Chronos-T5-Large, a 710M parameter time series foundation model, revealing how different layers process temporal data. The study found that mid-encoder layers contain the most causally important features for change detection, while early layers handle frequency patterns and final layers compress semantic concepts.

• →First successful application of sparse autoencoders to interpret time series foundation models using Chronos-T5-Large.
• →All 392 tested features showed causal relevance to model performance when ablated.
• →Mid-encoder layers contain the most critical features for forecasting accuracy, not the final semantically rich layers.
• →The model prioritizes abrupt-dynamics detection over periodic pattern recognition for time series forecasting.
• →A clear hierarchical feature organization emerges across model depth from low-level to high-level temporal concepts.