ANNOE: Adaptive Neural Networks Optimization Engine

Abstract

As the demand for deploying machine learning models on high-end mobile devices and IoT devices increases, the need for efficient machine learning model optimization becomes critical to execute on-device AI tasks. One of the primary challenges in this context is the retraining process of these models on IoT and mobile devices with new data, which is constrained due to limited processing power. This limitation hinders the generation of accurate domain-specific inference results, as models may not be as well-trained with real-world data, potentially reducing their predictive accuracy and generalizability when applied to practical scenarios.