Intuit Utilises Generative AI in Enhancing Kubernetes Cluster Management
Financial technology giant Intuit has recently detailed its innovative use of Generative AI (GenAI) to manage the intricate tasks of monitoring and debugging its extensive Kubernetes clusters. Intuit's efforts were geared towards improving the detection, debugging, and remediation processes to ensure the efficient operation of its services.
Experiment Overview and Background
Lili Wan, Senior Staff Software Engineer, and Anusha Ragunathan, Principal Software Engineer at Intuit provided insights into their pioneering experiment. The company's Kubernetes infrastructure is robust, supporting over 325 clusters that facilitate more than 7,000 applications and services. This scale of operations introduces substantial complexities, particularly in maintaining cluster health and managing the flood of alerts that on-call engineers encounter.
The engineers face significant challenges due to the vast and complex nature of the Intuit Kubernetes Service platform. The rapid expansion of applications and frequent updates to clusters add layers of complexity, often resulting in alert fatigue from the sheer volume of data sources and alerts, complicating the identification and resolution of issues.
Areas of Focus: Detection, Debugging, and Remediation
To tackle these challenges, Intuit identified three pivotal areas for improvement: detection, debugging, and remediation.
Detection: Intuit implemented a system referred to as “Cluster Golden Signals," designed to provide a holistic snapshot of a cluster's health by filtering out non-essential data and concentrating on critical signals for alerting. Using Prometheus expressions, core components of clusters are monitored via dashboards that aggregate various metrics into a consolidated health indicator—labelled as Healthy, Degraded, or Critical. This system enables engineers to quickly pinpoint and isolate problematic clusters, facilitating a faster mean time to detect issues (MTTD).
Debugging: For in-depth debugging, Intuit integrated an open-source tool known as K8sGPT. This tool, which has garnered significant attention within the Cloud Native Computing Foundation (CNCF) community, scans Kubernetes clusters to diagnose and triage issues using codified knowledge from Site Reliability Engineers. Through resource-specific analyzers, K8sGPT extracts pertinent error messages from clusters and enriches them with AI insights. When combined with Prometheus metrics and Cluster Golden Signals, K8sGPT enhances the understanding of potential root causes of alerts.
K8sGPT is noted for its versatility, supporting multiple models such as those from OpenAI, Azure, Cohere, Amazon Bedrock, Google Gemini, and local models. The project, which commenced in March 2023, has received widespread contributions and now boasts 5.6K stars and 88 contributors. This tool runs on various operating systems, including Windows, Mac, and Linux, and can be installed via multiple package managers.
Remediation: Post-debugging, the next step involves remediation. K8sGPT integrates with public Large Language Models (LLMs) from tech giants like OpenAI, Google, and Microsoft, suggesting remediation steps for Kubernetes-specific errors. However, the public LLMs often lack context specific to Intuit’s bespoke platform configurations. To address this, Intuit has developed a proprietary GenAI operating system (GenOS). This system hosts local models, which are augmented with Intuit-specific data through a process known as retrieval-augmented generation (RAG).
Future Plans
Moving forward, Intuit plans to closely monitor the progress in reducing both the mean time to detect (MTTD) and the mean time to resolve (MTTR) issues. The company also aims to explore other potential applications of GenAI within its infrastructure, including traffic management and Java virtual machine debugging.
By leveraging GenAI's capabilities, Intuit demonstrates a forward-thinking approach to enhancing the efficiency of its Kubernetes cluster management, ultimately striving to improve the operational stability of its wide range of applications and services.
Source: Noah Wire Services