RestoreML: Practical Unsupervised Tuning of Deployed Intelligent IoT Systems

As today's edge AI systems age, the need for incremental updates of intelligent sensor nodes (that fail or reach the end of their useful lifetime) becomes a growing concern. Replacement nodes may happen to use new versions of sensors, may have different physical properties (e.g., inertia or stiffness), or may run updated signal processing firmware, thereby changing the characteristics of sensor waveforms (such as acoustic, seismic, or acceleration measurements) made available to the downstream AI model. New downloadable AI models might be less than a perfect fit because they might be trained in an environment that differs from the conditions of the old deployment. As a result of such sensor and/or AI differences, new replacement nodes may not perform optimally when deployed. They will need to be fine-tuned after deployment. Recognizing this problem, this paper introduces RestoreML, a novel algorithm designed to fine-tune AI models in an unsupervised manner (i.e., without the need for labeling or human intervention). The algorithm leverages advances in test-time adaptation (TTA) to refine machine-learning (ML) models with unlabeled data collected after deployment. Innovations are introduced in the way deployment data are sampled for model fine-tuning, and the way less reliable nodes are automatically identified. RestoreML is implemented as a middleware library and a broker that can be easily integrated into existing applications. Evaluation results on a speciallycurated dataset, M3N-VC (that we make publicly available¹¹Out dataset M3N-VC is publicly available at: https://github.com/restoreml/m3n-vc and https://doi.org/10.5281/zenodo.15215210 demonstrate that RestoreML can significantly enhance model performance after deployment, especially in node replacement scenarios, outperforming state-of-the-art baselines.