TIDES: Implicit Time-Awareness in Selective State Space Models

Researchers introduce TIDES, a new selective state space model architecture that combines the expressivity of input-dependent models like Mamba with the native irregular time-series handling of continuous-time models like S5. By moving input-dependence to the state matrix rather than the discretization step, TIDES maintains the physical meaning of time intervals while preserving per-token expressivity, achieving state-of-the-art results on time-series benchmarks.

TIDES addresses a fundamental architectural trade-off in state space models that has limited their effectiveness for temporal data. Selective SSMs like Mamba excel at per-token expressivity by making the discretization step input-dependent, but this approach sacrifices the physical interpretation of time intervals needed for irregular sampling. Continuous-time models like S5 preserve temporal semantics but remain constrained by linear time-invariant dynamics, reducing their ability to capture input-dependent patterns.

The paper's core innovation relocates input-dependence from the discretization step size to the diagonal state matrix itself. This architectural shift allows TIDES to maintain a meaningful relationship between discrete and continuous time representations while enabling the model to handle variable timestamp intervals natively. The researchers validate their approach through a novel diagnostic benchmark called Fading Flash, which jointly evaluates input-dependence and extrapolation capabilities while isolating failure modes in competing architectures.

For the broader machine learning community, TIDES represents progress toward more versatile sequence models capable of handling real-world temporal data where irregular sampling is common. The state-of-the-art results on UEA time-series classification and Physiome-ODE benchmarks demonstrate practical improvements beyond theoretical considerations. This work matters for applications spanning healthcare, finance, and sensor networks where irregular timestamps are prevalent.

The research signals growing sophistication in model design beyond simply scaling architectures. Future developments may focus on extending these principles to multimodal temporal data or exploring whether similar trade-off reconciliations apply to other model families. The open-sourced code enables rapid community adoption and extension.

• →TIDES reconciles selective and continuous state space models by moving input-dependence to the state matrix rather than discretization step
• →The architecture maintains physical meaning of time intervals while preserving per-token expressivity needed for sequence modeling
• →Fading Flash benchmark reveals distinct failure modes in competing architectures that TIDES avoids by design
• →Achieves state-of-the-art average rank on UEA time-series classification and Physiome-ODE regression benchmarks
• →Native handling of irregular timestamps without sacrificing model expressivity addresses a key limitation in current sequence models