Predictive Autoscaler for Kubernetes with Kubernetes Event-Driven Autoscaler (KEDA)

Abstract

"Efficient resource management in cloud computing and microservices is crucial as organizations increasingly adopt cloud-native paradigms. Kubernetes, a leading container orchestration platform, offers autoscaling capabilities, but its current reactive mechanism may not respond quickly enough to changes in workload, potentially leading to service downtime and degraded Quality of Service (QoS). Proactive autoscaling methods, particularly those incorporating machine learning, offer a promising alternative. However, these methods are not widely adopted, and selecting the appropriate machine learning model for a given context remains challenging.

To address this problem, this research develops an event-driven predictive autoscaler by integrating machine learning models with Kubernetes Event-Driven Autoscaling (KEDA). The approach prioritizes predictive decisions over reactive ones, ensuring the system is prepared ahead of events. Various machine learning models, including XGBoost, LSTM, Bi-directional LSTM, GRU and Bi-directional GRU, are evaluated and integrated to support diverse input sources. The autoscaler framework is designed to dynamically adapt to workload patterns and optimize resource utilization while maintaining stringent QoS requirements.

The suggested approach involves integrating machine learning models with Kubernetes Event-Driven Autoscaling (KEDA) to enhance meeting quality of service (QoS) requirements, reducing downtime, and improving system reliability. The predictive autoscaler, particularly the Bi-directional Gated Recurrent Unit (Bi-GRU) model, demonstrated promising results in terms of prediction accuracy, scaling efficiency, and resource utilization across various datasets. Overall, Bi-GRU and GRU proved to be robust performers, with Bi-GRU delivering slightly superior results in a broader range of scenarios. "