After getting our Konnect Data Planes vertically and horizontally scaled, with VPA and HPA, it's time to explore the Kubernete Node Autoscaler options. In this post, we start with the Cluster Autoscaler mechanism. (Part 4 in this series is dedicated to Karpenter.)
The Kubernetes Cluster Autoscaler documentation describes it as a tool that automatically adjusts the size of the Kubernetes Cluster when one of the following conditions is true:
The following Cluster Autoscaler diagram was taken from the AWS Best Practices Guides portal.
The Amazon EKS Cluster Autoscaler implementation relies on the EC2's Autoscaling Group capability to manage NodeGroups. That is the reason why we used the eksctl --asg-access parameter when we created our EKS Cluster. Check the eksctl autoscaling page to know more.
In fact, you can check all ASGs your EKS Cluster has in place with the following command, which asks for the regular MinSize, MaxSize, and DesiredCapacity values.
aws autoscaling \
describe-auto-scaling-groups \
--region us-west-1 \
--query "AutoScalingGroups[? Tags[? (Key=='eks:cluster-name') && Value=='kong35-eks129-autoscaling']].[AutoScalingGroupName, MinSize, MaxSize, DesiredCapacity]" \
--output tableThe expected output should look like this. As shown, each NodeGroup has its own ASG defined. Of course, we are interested in the kong NodeGroup.
--------------------------------------------------------------------------------
| DescribeAutoScalingGroups |
+--------------------------------------------------------------+----+-----+----+
| eks-nodegroup-fortio-f6c6a3b5-d248-111b-e890-cb5114697cae | 1 | 10 | 1 |
| eks-nodegroup-httpbin-72c6a3b5-c047-c19d-fde0-2646caa270df | 1 | 10 | 1 |
| eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb | 1 | 10 | 1 |
+--------------------------------------------------------------+----+-----+----+EKS Cluster Autoscaler is deployed and it runs as a Kubernetes Deployment. As such, it's recommended to use IRSA (IAM Roles for Service Accounts) to associate the Service Account that the Cluster Autoscaler Deployment runs as with an IAM role that is able to perform these functions.
IRSA requires the EKS Cluster to be associated with an IAM OpenID Connect Provider. This can be done with a specific eksctl command:
eksctl utils associate-iam-oidc-provider --cluster kong35-eks129-autoscaling --region=us-west-1 --approveYou can check the IAM OIDC provider with the following command. You should see your EKS Cluster OIDC Issuer endpoint listed as an ARN.
aws iam list-open-id-connect-providersIf you will, you can check the OIDC Issuer endpoint with:
aws eks describe-cluster --name kong35-eks129-autoscaling --region us-west-1 | jq -r ".cluster.identity"IRSA is based on a Kubernetes Service Account and IAM Role pair. In turn, the Kubernetes Deployment should refer to the Service Account, which has the IAM Role set as an Annotation. Finally, the IAM Role allows the Deployment to access AWS Services, including, in our case, Auto Scaling Group (ASG) services.
Here is an IAM Policy example that allows Roles to access the ASG Services:
aws iam create-policy \
--policy-name AmazonEKSClusterAutoscalerPolicy \
--policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeScalingActivities",
"autoscaling:DescribeTags",
"ec2:DescribeInstanceTypes",
"ec2:DescribeLaunchTemplateVersions"
],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": [
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"ec2:DescribeImages",
"ec2:GetInstanceTypesFromInstanceRequirements",
"eks:DescribeNodegroup"
],
"Resource": ["*"]
}
]
}'With the IAM Policy created we can run another eksctl command to create both Kubernetes Service Account and IAM Role:
eksctl create iamserviceaccount \
--name cluster-autoscaler \
--namespace kube-system \
--cluster kong35-eks129-autoscaling \
--region us-west-1 \
--approve \
--role-name kong35-eks129-autoscaling-role \
--override-existing-serviceaccounts \
--attach-policy-arn $(aws iam list-policies --query 'Policies[?PolicyName==`AmazonEKSClusterAutoscalerPolicy`].Arn' --output text)If you check the Service Account, you will see it has the required annotation referring to the Role created by the iamserviceaccount command:
kubectl describe sa cluster-autoscaler -n kube-systemYou can check the Role also.
aws iam get-role --role kong35-eks129-autoscaling-roleAs expected the Role refers to our Policy:
aws iam list-attached-role-policies --role kong35-eks129-autoscaling-roleNow we are ready to install Cluster Autoscaler in our EKS Cluster. This can be done using its Helm Charts. Note the command refers to the Service Account and IAM Role created previously.
helm repo add autoscaler https://kubernetes.github.io/autoscaler
helm repo update
helm install cluster-autoscaler autoscaler/cluster-autoscaler \
-n kube-system \
--set autoDiscovery.clusterName=kong35-eks129-autoscaling \
--set cloudProvider=aws \
--set awsRegion=us-west-1 \
--set image.tag=v1.29.0 \
--set rbac.serviceAccount.create=false \
--set rbac.serviceAccount.name=cluster-autoscaler \
--set rbac.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::<your_aws_account>:role/kong35-eks129-autoscaling-roleYou can check the Cluster Autoscaler log files to see how the installation went:
kubectl logs -f -l app.kubernetes.io/name=aws-cluster-autoscaler -n kube-systemWith the existing Load Generator still running (if you don't have it, please start one), you should see a new status for both ASG and the Konnect Data Plane Kubernetes Deployment:
aws autoscaling \
describe-auto-scaling-groups \
--region us-west-1 \
--query "AutoScalingGroups[? Tags[? (Key=='eks:cluster-name') && Value=='kong35-eks129-autoscaling']].[AutoScalingGroupName, MinSize, MaxSize, DesiredCapacity]" \
--output table
--------------------------------------------------------------------------------
| DescribeAutoScalingGroups |
+--------------------------------------------------------------+----+-----+----+
| eks-nodegroup-fortio-f6c6a3b5-d248-111b-e890-cb5114697cae | 1 | 10 | 1 |
| eks-nodegroup-httpbin-72c6a3b5-c047-c19d-fde0-2646caa270df | 1 | 10 | 1 |
| eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb | 1 | 10 | 2 |
+--------------------------------------------------------------+----+-----+----+ASG has requested a new instance of the Node:
% kubectl top node --selector='eks.amazonaws.com/nodegroup=nodegroup-kong'
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
ip-192-168-11-188.us-west-1.compute.internal 2000m 103% 6641Mi 93%
ip-192-168-53-177.us-west-1.compute.internal 54m 2% 1256Mi 17%And then, the Pending Pods, after getting scheduled, are finally running:
% kubectl get pod -n kong
NAME READY STATUS RESTARTS AGE
kong-kong-789ffd6f86-bmgh8 1/1 Running 0 15m
kong-kong-789ffd6f86-fcnts 1/1 Running 0 15m
kong-kong-789ffd6f86-k7cnm 1/1 Running 0 16m
kong-kong-789ffd6f86-nsqnn 1/1 Running 0 5m14s
kong-kong-789ffd6f86-sf65t 1/1 Running 0 7m10s
kong-kong-789ffd6f86-tv96x 1/1 Running 0 22m
kong-kong-789ffd6f86-twzd8 1/1 Running 0 16mSo, what happens if we submit our Konnect Data Plane Deployment to a longer and higher throughput? For instance, we change the HPA policy like this, allowing up to 20 replicas to run:
cat <<EOF | kubectl apply -f -
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: kong-hpa
namespace: kong
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: kong-kong
minReplicas: 1
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 75
EOFAlso, we can submit a heavier load test with:
kubectl delete pod fortio
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: fortio
labels:
app: fortio
spec:
containers:
- name: fortio
image: fortio/fortio
args: ["load", "-c", "800", "-qps", "5000", "-t", "60m", "-allow-initial-errors", "http://kong-kong-proxy.kong.svc.cluster.local:80/route1/get"]
nodeSelector:
nodegroupname: fortioThe expected result is to have extra Nodes getting created along the way. Here are the HPA, ASG and Kubernetes Nodes outputs:
% kubectl get hpa -n kong
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
kong-hpa Deployment/kong-kong 73%/75% 1 20 14 35m
aws autoscaling \
describe-auto-scaling-groups \
--region us-west-1 \
--query "AutoScalingGroups[? Tags[? (Key=='eks:cluster-name') && Value=='kong35-eks129-autoscaling']].[AutoScalingGroupName, MinSize, MaxSize, DesiredCapacity]" \
--output table
--------------------------------------------------------------------------------
| DescribeAutoScalingGroups |
+--------------------------------------------------------------+----+-----+----+
| eks-nodegroup-fortio-f6c6a3b5-d248-111b-e890-cb5114697cae | 1 | 10 | 1 |
| eks-nodegroup-httpbin-72c6a3b5-c047-c19d-fde0-2646caa270df | 1 | 10 | 1 |
| eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb | 1 | 10 | 4 |
+--------------------------------------------------------------+----+-----+----+
% kubectl top node --selector='eks.amazonaws.com/nodegroup=nodegroup-kong'
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
ip-192-168-11-188.us-west-1.compute.internal 1250m 64% 5870Mi 82%
ip-192-168-28-192.us-west-1.compute.internal 1254m 64% 6366Mi 89%
ip-192-168-48-1.us-west-1.compute.internal 839m 43% 670Mi 9%
ip-192-168-53-177.us-west-1.compute.internal 1212m 62% 6268Mi 88% The problem is that all Nodes are based on the same original t3.large Instance Type. You can check them out with:
% aws autoscaling describe-auto-scaling-groups --region us-west-1 | jq '.AutoScalingGroups[] | select (.Tags[0].Value == "kong35-eks129-autoscaling") | { AutoScalingGroupName }' | jq -r '.AutoScalingGroupName'
eks-nodegroup-fortio-f6c6a3b5-d248-111b-e890-cb5114697cae
eks-nodegroup-httpbin-72c6a3b5-c047-c19d-fde0-2646caa270df
eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb
% aws autoscaling describe-auto-scaling-groups --auto-scaling-group-name eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb --region us-west-1 | jq ".AutoScalingGroups[].Instances[].InstanceType"
"t3.large"
"t3.large"
"t3.large"
"t3.large"We could address that by deploying multiple instances of Cluster Autoscaler, each configured to work with a specific set of Node Groups. However, that's not recommended, as Cluster Autoscaler was not designed for this kind of configuration and it could lead to a situation where multiple Autoscaler would attempt to schedule a Pod. Please, check the EKS Best Practices Guide for Cluster Autoscaler provided by AWS.
That's one of the main reasons to switch to a more flexible and smarter cluster autoscaler as Karpenter implements. Let's check it out in the next (and final) part of this series, Kong Konnect Data Plane Node Autoscaling with Karpenter on Amazon EKS 1.29.
In the rapidly evolving landscape of API management, understanding the raw performance and reliability of your API gateway is not just an expectation — it's a necessity. At Kong, we're dedicated to ensuring our users have access to concrete, action
In this post, we're going to explore Karpenter, the ultimate solution for Node Autoscaling. Karpenter provides a cost-effective capability to implement your Kong Konnect Data Plane layer using the best EC2 Instances Types options available for your
In this series of posts, we will look closely at how Kong Konnect Data Planes can take advantage of Autoscalers running on Amazon Elastic Kubernetes Services (EKS) 1.29 to support the throughput the demands API consumers impose on it at the lowest c
In one of our posts, Kong AI/MCP Gateway and Kong MCP Server technical breakdown, we described the new capabilities added to Kong AI Gateway to support MCP (Model Context Protocol). The post focused exclusively on consuming MCP server and MCP tool
Managed Redis cache is a turnkey "Shared State" add-on for Kong Dedicated Cloud Gateways. It is designed to combine the performance of an in-memory data store with the simplicity of a SaaS product. When you spin up a Dedicated Cloud Gateway in Kong
Key Takeaways API testing is crucial for ensuring the reliability, security, and performance of modern applications. Different types of testing, such as functional, security, performance, and integration testing, should be employed to cover all aspe
So, what exactly is Kong Insomnia? Kong Insomnia is your all-in-one platform for designing, testing, debugging, and shipping APIs at speed. Built for developers who need power without bloat, Insomnia helps you move fast whether you’re working solo,