What changes after deployment? A survey on On-device Learning in TinyML

This survey examines on-device learning (ODL) in TinyML systems, analyzing how 70 existing solutions address the challenge of distribution shift in deployed machine learning models on microcontrollers. The research identifies a critical gap between academic benchmarks and real-world deployment scenarios, emphasizing that different types of distribution change require tailored technical approaches.

Machine learning models deployed on resource-constrained microcontroller devices face a fundamental problem: their static parameters degrade when real-world data distributions diverge from training conditions. This survey provides the first comprehensive taxonomy of on-device learning solutions by categorizing them under distribution change regimes, offering researchers and practitioners a structured framework for understanding how models can adapt after deployment without requiring cloud connectivity.

The TinyML ecosystem has grown substantially as edge AI applications expand across IoT, wearables, and embedded systems. However, most deployed models remain static, creating brittleness in dynamic environments. On-device learning addresses this by enabling microcontrollers to continuously improve themselves, but implementation requires careful consideration of hardware constraints, power budgets, and memory limitations. The survey's analysis of 70 works reveals how different distribution change patterns—concept drift, covariate shift, and label shift—demand different algorithmic strategies, affecting everything from model architecture choices to hardware selection.

For developers and organizations deploying TinyML systems, this research highlights a critical vulnerability: production models may fail silently as operational data diverges from training distributions. The identified gap between benchmark evaluations and real deployments suggests that academic ODL research may not adequately address actual field conditions, where devices face unpredictable environmental changes, sensor degradation, and heterogeneous user populations. This methodological gap could explain why adoption of adaptive edge learning remains limited despite clear theoretical benefits.

The survey's classification scheme enables practitioners to match distribution change characteristics to appropriate ODL solutions, potentially accelerating the transition from static to adaptive edge models. Future research should prioritize closing the benchmark-to-deployment gap through more realistic testbeds and longer-term field studies.

• →On-device learning enables TinyML models to adapt post-deployment by running learning directly on microcontrollers, addressing distribution shift limitations of static models.
• →Distribution change regime classification reveals that different shift types (concept drift, covariate shift, label shift) require fundamentally different technical solutions.
• →A significant gap exists between academic benchmarking methodologies and real-world deployment scenarios in TinyML research.
• →Hardware selection and model architecture choices are heavily influenced by the specific type of distribution change a deployed system must handle.
• →Bridging the research-to-production gap requires realistic testbeds and extended field studies beyond traditional benchmark evaluations.