Node Autoscaling in Kubernetes
Demystifying Kubernetes Node Autoscaling
In the world of cloud infrastructure, autoscaling is often simplified as a feedback loop: traffic rises, CPU spikes, more servers appear. While this works in traditional EC2 environments, Kubernetes changes the paradigm entirely.
🔹 The Core Myth: ASG Scales on Real-Time CPU & Memory Metrics
An AWS Auto Scaling Group (ASG) has no visibility into Kubernetes internals such as pod definitions, scheduling constraints, or application traffic distribution. Attaching CloudWatch alarms directly to ASGs can cause ghost scaling and destructive scale-downs.
- Premature/Ghost Scaling: A single pod spike triggers new EC2 nodes that remain idle.
- Destructive Scale-Downs: ASG terminates nodes without cordoning or respecting PodDisruptionBudgets.
🔹 The Paradigm Shift: Utilization vs. Allocation
| Dimension | Horizontal Pod Autoscaler (HPA) | Cluster Autoscaler (CA) |
|---|---|---|
| Layer of Operation | Pod / Application Layer | EC2 / Infrastructure Layer |
| Primary Scaling Triggers | Observed Metrics (CPU, Memory, HTTP RPS) | Pending / Unschedulable Pods |
| Target Resource | Replica Count | ASG Desired Capacity |
| Decision Frequency | Every 15s | Every 10s |
| Responsibility | Scale applications | Provision compute slots |
🔹 Behind the Scenes: The Complete Scale-Up Sequence
- HPA scales replicas based on metrics.
- Scheduler marks pods as Pending when nodes lack allocatable capacity.
- Cluster Autoscaler simulates scheduling and increases ASG desired capacity.
- ASG provisions EC2 nodes, bootstraps them, and registers them with the cluster.
- Scheduler binds pending pods to new nodes, kubelet executes containers.
🔹 Scaling on HTTP Requests: The Two-Tiered Chain Reaction
Scaling on application metrics requires coordination between HPA and CA. HPA reacts to HTTP RPS, while CA provisions nodes for unschedulable pods.
- Traffic surge detected via Prometheus/KEDA.
- HPA scales replicas from 10 ➔ 120.
- Scheduler marks 80 pods Pending.
- CA calculates need for 8 new nodes.
- ASG scales from 6 ➔ 14 instances.
🔹 The Critical Link: Why Container Resource Requests Are Mandatory
resources: requests: cpu: "500m" memory: "1Gi" limits: cpu: "1000m" memory: "2Gi"Without resource requests, the scheduler assumes pods consume zero resources, leading to OOM crashes and preventing Cluster Autoscaler from triggering.
🔹 Safe Scale-Down: Graceful Eviction and PodDisruptionBudgets
- CA detects underutilized nodes.
- Runs eviction simulation.
- Validates PodDisruptionBudgets.
- Cordons and drains nodes gracefully.
- Terminates EC2 instance via ASG API.
🔹 Modern Evolution: Cluster Autoscaler vs. Karpenter
| Feature | Cluster Autoscaler + ASG | Karpenter |
|---|---|---|
| AWS Integration | Wraps ASGs | Direct EC2 Fleet API |
| Provisioning Latency | 2–5 minutes | 30–60 seconds |
| Instance Selection | Pre-configured ASG templates | Dynamic optimal instance types |
| Workloads | Multiple ASGs required | Single NodePool handles heterogeneous workloads |
| Bin Packing | Basic scale-down | Continuous consolidation and rightsizing |
🔹 Architectural Takeaways & Production Checklist
- Never attach CloudWatch scaling policies to EKS ASGs.
- Always define container resource requests.
- Decouple application scaling (HPA/KEDA) from node scaling (CA).
- Implement PodDisruptionBudgets for safe scale-down.
- Tag ASGs with `k8s.io/cluster-autoscaler/enabled` and cluster name.
🔹 Closing Thoughts
Autoscaling in Kubernetes is a distributed conversation between application-layer controllers and cloud APIs. ASGs provide raw compute, while Kubernetes orchestrates intelligent scaling for resilience and efficiency.