In my previous post, we discussed how to take advantage of VPA to implement automatic vertical scaling for our Konnect Data Planes. In this post, we'll focus on HPA for horizontal Kubernetes Pods autoscaling.
VPA docs recommend not using VPA along with HPA on CPU or memory metrics at this moment. HPA should be configured with custom and external metrics instead.
Since we'll be running fundamental HPA use cases, let's delete the VPA policy with:
Horizontal Pod Autoscaler (HPA) is a standard API resource in Kubernetes that automatically updates a workload resource, for example a Deployment, to match a throughput demand.
The following diagram was taken from the official Kubernetes HPA documentation. Please check it out to learn more.
HPA requires metrics provided by the Kubernetes Metrics Server, we have already installed. The Metrics Server collects resource metrics from the kubelets in your cluster, and exposes those metrics through the Kubernetes API.
To have better control of the HPA environment, let's set our Konnect Data Plane deployment requesting more CPU and memory resources:
Also, to see HPA in action, we are going to replace the existing NodeGroup with a smaller one, this time based on the t3.large Instance Type with 2 vCPUs and 8GiB of memory:
The Kubernetes Autoscale command tells HPA how to proceed to instantiate new Pod replicas. The command tells that HPA should create new replicas of the Data Plane when the CPU usage reaches the 75% threshold. HPA should create up to 10 replicas of the Data Plane.
You can use the HorizontalPodAutoscaler CRD instead. Bear in mind this is a basic use case, there are many other scenarios addressed by HPA. Please, check the HPA documentation to learn more about it.
Check the HPA with. Since we haven't consumed the Data Plane yet, the HPA current usage is unknown.
Now, you are going to submit the Data Plane to a higher throughput in order to see HPA in action, spinning up new replicas of the Pod. Additionally, this is a long 20-minute run, so we should see how HPA performs in a scenario like this. First of all, make sure you delete any Fortio Pods you might have running.
After some minutes, you should see a new status:
If you check the running Pods you should see new replicas were started. However, there are two of them in the Pending status.
Let's check one of them a bit closer. The condition message says there are no more CPUs available to be allocated, hence the Pod has not been scheduled.
For more evidence, you can check the Nodes consumption themselves and see it has run out of resources:
The straightforward solution would be to create new Nodes for the Cluster. That's the main reason why we should use a cluster autoscaler mechanism alongside HPA.
Amazon EKS supports two autoscaling products:
Check out the third part of this series to see Kubernetes Cluster Autoscaler in action.
We're very excited to announce Kong Mesh 2.12 to the world! Kong Mesh 2.12 delivers two very important features: SPIFFE / SPIRE support, which provides enterprise-class workload identity and trust models for your mesh, as well as a consistent Kuma R
Meet Emily. She’s an API product manager at ACME, Inc., an ecommerce company that runs on dozens of APIs. One morning, her team lead asks a simple question: “Who’s our top API consumer, and which of your APIs are causing the most issues right now?”
In the last two parts of this series, we discussed How to Strengthen a ReAct AI Agent with Kong AI Gateway and How to Build a Single-LLM AI Agent with Kong AI Gateway and LangGraph . In this third and final part, we're going to evolve the AI Agen
In my previous post, we discussed how we can implement a basic AI Agent with Kong AI Gateway. In part two of this series, we're going to review LangGraph fundamentals, rewrite the AI Agent and explore how Kong AI Gateway can be used to protect an LLM
This is part one of a series exploring how Kong AI Gateway can be used in an AI Agent development with LangGraph. The series comprises three parts: Basic ReAct AI Agent with Kong AI Gateway Single LLM ReAct AI Agent with Kong AI Gateway and LangGr
What Is RAG, and Why Should You Use It? RAG (Retrieval-Augmented Generation) is not a new concept in AI, and unsurprisingly, when talking to companies, everyone seems to have their own interpretation of how to implement it. So, let’s start with a r
In February 2024, Kong became the first API platform to launch a dedicated AI gateway, designed to bring production-grade performance, observability, and policy enforcement to GenAI workloads. At its core, Kong’s AI Gateway provides a universal API