TRACE: A Temporal Conditional Estimation for Multimodal Time Series Foundation Models

TRACE is a new conditional estimation framework for multimodal time series foundation models that handles temporal misalignment and missing data across different modalities. By inferring incomplete modalities from available data sources, TRACE outperforms existing approaches on healthcare and sentiment analysis benchmarks, demonstrating robust cross-modal representation learning.

TRACE addresses a critical limitation in current time series foundation models: their inability to handle real-world scenarios where different data streams arrive at irregular intervals or are partially absent. Traditional approaches rely on simple imputation or masking, which ignores the dependencies between modalities and produces degraded representations. This research is significant because time series foundation models represent a frontier in AI, aiming to create generalizable representations that can transfer across diverse downstream tasks—similar to how large language models revolutionized NLP.

The research emerges from growing recognition that multimodal learning requires sophisticated handling of incomplete data. In clinical settings like those represented in MIMIC-IV, patient monitoring systems generate heterogeneous data streams: vital signs, lab results, and imaging studies arrive on different schedules with varying completeness. Sentiment analysis benchmarks like CMU-MOSI face similar challenges when combining speech, text, and visual information. Prior work treated missingness as a nuisance to suppress rather than information to leverage.

TRACE's conditional estimation paradigm shifts this perspective by using available modalities to systematically infer missing ones, preserving cross-modal relationships. The framework's consistent outperformance across benchmarks suggests it could accelerate adoption of foundation models in domains requiring robust handling of incomplete multimodal data. For the AI research community, this work bridges theory and practical deployment by addressing real-world data characteristics that academic models often ignore.

Future development will likely focus on extending TRACE to longer sequences, higher-dimensional modalities, and online learning scenarios where data arrives continuously with variable completeness patterns.

• →TRACE enables multimodal time series foundation models to infer missing modalities from available data while preserving cross-modal dependencies.
• →The framework demonstrates superior performance on healthcare and sentiment analysis benchmarks compared to naive imputation approaches.
• →Temporal misalignment and partial missingness are common real-world challenges that existing foundation models inadequately address.
• →Conditional estimation paradigms may become standard practice for robust multimodal representation learning in production systems.
• →Clinical and affective computing applications stand to benefit most from improved handling of heterogeneous temporal data streams.