After getting our Konnect Data Planes vertically and horizontally scaled, with [VPA](https://konghq.com/blog/engineering/data-plane-elasticity-eks-1-29-pod-autoscaling-with-vpa)VPA and [HPA](https://konghq.com/blog/engineering/pod-autoscaling-with-hpa-on-eks-1-29)HPA, it's time to explore the Kubernete Node Autoscaler options. In this post, we start with the Cluster Autoscaler mechanism. (Part 4 in this series is dedicated to Karpenter.)
The [Kubernetes Cluster Autoscaler documentation](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/README.md)Kubernetes Cluster Autoscaler documentation describes it as a tool that automatically adjusts the size of the Kubernetes Cluster when one of the following conditions is true:
The following Cluster Autoscaler diagram was taken from the [AWS Best Practices Guides](https://aws.github.io/aws-eks-best-practices/cluster-autoscaling/)AWS Best Practices Guides portal.
The [Amazon EKS Cluster Autoscaler](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md)Amazon EKS Cluster Autoscaler implementation relies on the EC2's Autoscaling Group capability to manage NodeGroups. That is the reason why we used the [`eksctl`](https://eksctl.io)`eksctl`` --asg-access` parameter when we created our EKS Cluster. Check the [eksctl autoscaling](https://eksctl.io/usage/autoscaling/)eksctl autoscaling page to know more.
In fact, you can check all ASGs your EKS Cluster has in place with the following command, which asks for the regular MinSize, MaxSize, and DesiredCapacity values.
aws autoscaling \
describe-auto-scaling-groups \
--region us-west-1 \
--query "AutoScalingGroups[? Tags[? (Key=='eks:cluster-name') && Value=='kong35-eks129-autoscaling']].[AutoScalingGroupName, MinSize, MaxSize, DesiredCapacity]" \
--output tableThe expected output should look like this. As shown, each NodeGroup has its own ASG defined. Of course, we are interested in the `kong` NodeGroup.
--------------------------------------------------------------------------------
| DescribeAutoScalingGroups |
+--------------------------------------------------------------+----+-----+----+
| eks-nodegroup-fortio-f6c6a3b5-d248-111b-e890-cb5114697cae | 1 | 10 | 1 |
| eks-nodegroup-httpbin-72c6a3b5-c047-c19d-fde0-2646caa270df | 1 | 10 | 1 |
| eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb | 1 | 10 | 1 |
+--------------------------------------------------------------+----+-----+----+EKS Cluster Autoscaler is deployed and it runs as a Kubernetes Deployment. As such, it's recommended to use [IRSA (IAM Roles for Service Accounts)](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)IRSA (IAM Roles for Service Accounts) to associate the Service Account that the Cluster Autoscaler Deployment runs as with an IAM role that is able to perform these functions.
IRSA requires the EKS Cluster to be associated with an IAM OpenID Connect Provider. This can be done with a specific `eksctl` command:
eksctl utils associate-iam-oidc-provider --cluster kong35-eks129-autoscaling --region=us-west-1 --approveYou can check the IAM OIDC provider with the following command. You should see your EKS Cluster OIDC Issuer endpoint listed as an ARN.
aws iam list-open-id-connect-providersIf you will, you can check the OIDC Issuer endpoint with:
aws eks describe-cluster --name kong35-eks129-autoscaling --region us-west-1 | jq -r ".cluster.identity"IRSA is based on a Kubernetes Service Account and IAM Role pair. In turn, the Kubernetes Deployment should refer to the Service Account, which has the IAM Role set as an Annotation. Finally, the IAM Role allows the Deployment to access AWS Services, including, in our case, Auto Scaling Group (ASG) services.
Here is an IAM Policy example that allows Roles to access the ASG Services:
aws iam create-policy \
--policy-name AmazonEKSClusterAutoscalerPolicy \
--policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeScalingActivities",
"autoscaling:DescribeTags",
"ec2:DescribeInstanceTypes",
"ec2:DescribeLaunchTemplateVersions"
],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": [
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"ec2:DescribeImages",
"ec2:GetInstanceTypesFromInstanceRequirements",
"eks:DescribeNodegroup"
],
"Resource": ["*"]
}
]
}'With the IAM Policy created we can run another `eksctl` command to create both Kubernetes Service Account and IAM Role:
eksctl create iamserviceaccount \
--name cluster-autoscaler \
--namespace kube-system \
--cluster kong35-eks129-autoscaling \
--region us-west-1 \
--approve \
--role-name kong35-eks129-autoscaling-role \
--override-existing-serviceaccounts \
--attach-policy-arn $(aws iam list-policies --query 'Policies[?PolicyName==`AmazonEKSClusterAutoscalerPolicy`].Arn' --output text)If you check the Service Account, you will see it has the required annotation referring to the Role created by the `iamserviceaccount` command:
kubectl describe sa cluster-autoscaler -n kube-systemYou can check the Role also.
aws iam get-role --role kong35-eks129-autoscaling-roleAs expected the Role refers to our Policy:
aws iam list-attached-role-policies --role kong35-eks129-autoscaling-roleNow we are ready to install Cluster Autoscaler in our EKS Cluster. This can be done using its [Helm Charts](https://artifacthub.io/packages/helm/cluster-autoscaler/cluster-autoscaler)Helm Charts. Note the command refers to the Service Account and IAM Role created previously.
helm repo add autoscaler https://kubernetes.github.io/autoscaler
helm repo update
helm install cluster-autoscaler autoscaler/cluster-autoscaler \
-n kube-system \
--set autoDiscovery.clusterName=kong35-eks129-autoscaling \
--set cloudProvider=aws \
--set awsRegion=us-west-1 \
--set image.tag=v1.29.0 \
--set rbac.serviceAccount.create=false \
--set rbac.serviceAccount.name=cluster-autoscaler \
--set rbac.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::<your_aws_account>:role/kong35-eks129-autoscaling-roleYou can check the Cluster Autoscaler log files to see how the installation went:
kubectl logs -f -l app.kubernetes.io/name=aws-cluster-autoscaler -n kube-systemWith the existing Load Generator still running (if you don't have it, please start one), you should see a new status for both ASG and the Konnect Data Plane Kubernetes Deployment:
aws autoscaling \
describe-auto-scaling-groups \
--region us-west-1 \
--query "AutoScalingGroups[? Tags[? (Key=='eks:cluster-name') && Value=='kong35-eks129-autoscaling']].[AutoScalingGroupName, MinSize, MaxSize, DesiredCapacity]" \
--output table
--------------------------------------------------------------------------------
| DescribeAutoScalingGroups |
+--------------------------------------------------------------+----+-----+----+
| eks-nodegroup-fortio-f6c6a3b5-d248-111b-e890-cb5114697cae | 1 | 10 | 1 |
| eks-nodegroup-httpbin-72c6a3b5-c047-c19d-fde0-2646caa270df | 1 | 10 | 1 |
| eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb | 1 | 10 | 2 |
+--------------------------------------------------------------+----+-----+----+ASG has requested a new instance of the Node:
% kubectl top node --selector='eks.amazonaws.com/nodegroup=nodegroup-kong'
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
ip-192-168-11-188.us-west-1.compute.internal 2000m 103% 6641Mi 93%
ip-192-168-53-177.us-west-1.compute.internal 54m 2% 1256Mi 17%And then, the Pending Pods, after getting scheduled, are finally running:
% kubectl get pod -n kong
NAME READY STATUS RESTARTS AGE
kong-kong-789ffd6f86-bmgh8 1/1 Running 0 15m
kong-kong-789ffd6f86-fcnts 1/1 Running 0 15m
kong-kong-789ffd6f86-k7cnm 1/1 Running 0 16m
kong-kong-789ffd6f86-nsqnn 1/1 Running 0 5m14s
kong-kong-789ffd6f86-sf65t 1/1 Running 0 7m10s
kong-kong-789ffd6f86-tv96x 1/1 Running 0 22m
kong-kong-789ffd6f86-twzd8 1/1 Running 0 16mSo, what happens if we submit our Konnect Data Plane Deployment to a longer and higher throughput? For instance, we change the HPA policy like this, allowing up to 20 replicas to run:
cat <<EOF | kubectl apply -f -
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: kong-hpa
namespace: kong
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: kong-kong
minReplicas: 1
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 75
EOFAlso, we can submit a heavier load test with:
kubectl delete pod fortio
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: fortio
labels:
app: fortio
spec:
containers:
- name: fortio
image: fortio/fortio
args: ["load", "-c", "800", "-qps", "5000", "-t", "60m", "-allow-initial-errors", "http://kong-kong-proxy.kong.svc.cluster.local:80/route1/get"]
nodeSelector:
nodegroupname: fortioThe expected result is to have extra Nodes getting created along the way. Here are the HPA, ASG and Kubernetes Nodes outputs:
% kubectl get hpa -n kong
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
kong-hpa Deployment/kong-kong 73%/75% 1 20 14 35m
aws autoscaling \
describe-auto-scaling-groups \
--region us-west-1 \
--query "AutoScalingGroups[? Tags[? (Key=='eks:cluster-name') && Value=='kong35-eks129-autoscaling']].[AutoScalingGroupName, MinSize, MaxSize, DesiredCapacity]" \
--output table
--------------------------------------------------------------------------------
| DescribeAutoScalingGroups |
+--------------------------------------------------------------+----+-----+----+
| eks-nodegroup-fortio-f6c6a3b5-d248-111b-e890-cb5114697cae | 1 | 10 | 1 |
| eks-nodegroup-httpbin-72c6a3b5-c047-c19d-fde0-2646caa270df | 1 | 10 | 1 |
| eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb | 1 | 10 | 4 |
+--------------------------------------------------------------+----+-----+----+
% kubectl top node --selector='eks.amazonaws.com/nodegroup=nodegroup-kong'
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
ip-192-168-11-188.us-west-1.compute.internal 1250m 64% 5870Mi 82%
ip-192-168-28-192.us-west-1.compute.internal 1254m 64% 6366Mi 89%
ip-192-168-48-1.us-west-1.compute.internal 839m 43% 670Mi 9%
ip-192-168-53-177.us-west-1.compute.internal 1212m 62% 6268Mi 88% The problem is that all Nodes are based on the same original `t3.large` Instance Type. You can check them out with:
% aws autoscaling describe-auto-scaling-groups --region us-west-1 | jq '.AutoScalingGroups[] | select (.Tags[0].Value == "kong35-eks129-autoscaling") | { AutoScalingGroupName }' | jq -r '.AutoScalingGroupName'
eks-nodegroup-fortio-f6c6a3b5-d248-111b-e890-cb5114697cae
eks-nodegroup-httpbin-72c6a3b5-c047-c19d-fde0-2646caa270df
eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb
% aws autoscaling describe-auto-scaling-groups --auto-scaling-group-name eks-nodegroup-kong-3ac6a3c8-86de-d97c-42e2-0d788168cbbb --region us-west-1 | jq ".AutoScalingGroups[].Instances[].InstanceType"
"t3.large"
"t3.large"
"t3.large"
"t3.large"We could address that by deploying multiple instances of Cluster Autoscaler, each configured to work with a specific set of Node Groups. However, that's not recommended, as Cluster Autoscaler was not designed for this kind of configuration and it could lead to a situation where multiple Autoscaler would attempt to schedule a Pod. Please, check the [EKS Best Practices Guide for Cluster Autoscaler](https://aws.github.io/aws-eks-best-practices/cluster-autoscaling/#sharding-across-node-groups)EKS Best Practices Guide for Cluster Autoscaler provided by AWS.
That's one of the main reasons to switch to a more flexible and smarter cluster autoscaler as Karpenter implements. Let's check it out in the next (and final) part of this series, [Kong Konnect Data Plane Node Autoscaling with Karpenter on Amazon EKS 1.29](https://konghq.com/blog/engineering/dp-node-autoscaling-with-karpenter-aws-eks-1-29)Kong Konnect Data Plane Node Autoscaling with Karpenter on Amazon EKS 1.29.
In the rapidly evolving landscape of API management, understanding the raw performance and reliability of your API gateway is not just an expectation — it's a necessity. At Kong, we're dedicated to ensuring our users have access to concrete, action
In this post, we're going to explore Karpenter, the ultimate solution for Node Autoscaling. Karpenter provides a cost-effective capability to implement your Kong Konnect Data Plane layer using the best EC2 Instances Types options available for your
In this series of posts, we will look closely at how Kong Konnect Data Planes can take advantage of Autoscalers running on Amazon Elastic Kubernetes Services (EKS) 1.29 to support the throughput the demands API consumers impose on it at the lowest c
What You'll Build To explore the new integration, I'll build a realistic API platform workflow using Konnect, Kong Gateway, and Insomnia. By the end of this tutorial, I'll have: A Konnect Control Plane (KongAir Dev) A local Kong Gateway Data Pl
Local-first: your data stays with you: Insomnia stores everything on your machine by default. No forced cloud sync, no account needed just to send a request. This is helpful if privacy or working in a regulated environment is a priority for you Fre
You invested in Kong Konnect. You centralized your API governance, locked down your routing, enforced your security policies, and gave your organization a single source of truth for every API in your estate. That work is done, and it is working. So
Have you ever…. Copied an API spec out of Kong Konnect, or where you manage your APIs, pasted it into your API client, and immediately wondered if it’s the latest version? Sent an email to your platform team with the subject line “ which endpoint sh