Scaling CPU-Intensive Workloads in Kubernetes

Scaling CPU-Intensive Workloads in Kubernetes—When CPU Metrics Actually Work

When developers first set up a Horizontal Pod Autoscaler (HPA) in Kubernetes, they almost always start with the default example based on average CPU utilization. For a certain class of applications, this works flawlessly. But for others, it is a disaster. Understanding when CPU-based scaling is appropriate comes down to a fundamental architectural question: Is your application's primary bottleneck the CPU core?

The Anatomy of a CPU-Bound Application

A CPU-bound service is one where throughput is directly constrained by how fast the CPU can execute instructions. The threads are rarely waiting for network packets, database locks, or disk I/O; they are actively computing.

Prime Examples of CPU-Bound Workloads

• Image and Video Processing: Resizing images, encoding MP4s, or generating thumbnails.
• Cryptography & Auth: Hashing passwords (bcrypt/Argon2), validating JWTs, or encrypting tokens.
• Data Transformation: Parsing large JSON/XML payloads, running machine learning inferences (on CPU), or executing complex mathematical algorithms.
• PDF / Report Generation: Rendering HTML templates to PDF documents using headless browsers or rendering engines.

In these workloads, as incoming requests rise, CPU utilization climbs predictably toward 100%. As a result, CPU utilization serves as a reliable direct proxy for overall workload load.

Why CPU Scaling Works Great Here

• Linear Relationship: Double the requests approximately equals double the CPU instructions.
• Predictable Saturation: If CPU hits 90%, you know the pod is throttling and requests are queuing for CPU execution time.
• No Hidden Blocking: Because threads are actively using the CPU, the OS scheduler accurately reflects the load in metrics exposed to the Kubernetes Metrics Server.

Production Configuration Best Practices

When configuring CPU-based scaling, the biggest mistake is setting target utilization thresholds too high or failing to set precise container resource limits. Keep in mind that the HPA calculates CPU percentage based on the requested CPU, not the CPU limit limit.

Always define resource requests and limits cleanly. Avoid setting target utilization above 80% because CPU spikes occur faster than Kubernetes can spin up new pods. Keeping a 20% cushion prevents request throttling during scale-up events.

Production Kubernetes Manifest (HPA)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: image-processor-hpa
  namespace: media-services
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: image-processor-deployment
  minReplicas: 3
  maxReplicas: 30
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70 # Scale up early to absorb boot time
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100 # Double pods rapidly on sudden load bursts
        periodSeconds: 15
    scaleDown:
      stabilizationWindowSeconds: 300 # Wait 5 minutes to avoid flapping

Summary: Rules of Engagement

• Use CPU Scaling When: Your application performs heavy math, cryptography, data processing, or media encoding, and worker threads spend over 90% of their time actively executing on the core.
• Skip CPU Scaling When: Your application spends its time waiting on external APIs, database queries, or network sockets.