Kaushik's Notes

Scaling

5 min read

Overview

Three Kinds of Scaling

ScalerChangesDriven by
HPA (Horizontal Pod Autoscaler)number of replicasCPU/memory, custom or external metrics
VPA (Vertical Pod Autoscaler)requests/limits of containersobserved usage history
Cluster Autoscaler / Karpenternumber of nodesPods stuck Pending for lack of room, or empty nodes
  • They stack. HPA adds Pods, and if they don’t fit, the node autoscaler adds nodes.
  • Everything depends on resource requests. The HPA computes utilisation relative to requests, and node autoscalers decide “doesn’t fit” based on requests.

Recipes

Recipe 9.1 — Scale a Deployment Manually

  • Problem: Load dropped and five replicas is too many.
  • Solution:
kubectl scale deployment fancyapp --replicas=3
kubectl get deploy fancyapp
  • Gotchas: If an HPA manages the Deployment, it will overwrite your manual number. And if replicas: is in Git and your GitOps tool re-applies it, it fights the HPA. Leave replicas out of the manifest when an HPA owns it.

Source: K8s Cookbook, Recipe 9.1

Recipe 9.2 / 9.3 — Autoscale the Cluster’s Nodes

  • Problem: Add nodes when Pods can’t be scheduled, and remove them when they’re idle.
  • Solution (managed clouds):
# GKE (Autopilot does this for you automatically)
gcloud container clusters update demo --enable-autoscaling \
  --node-pool=default-pool --min-nodes=1 --max-nodes=5 --region=europe-west1
 
# AKS
az aks update -g k8s -n demo --enable-cluster-autoscaler --min-count 1 --max-count 5
 
# EKS: Karpenter (provisions right-sized nodes directly), or Cluster Autoscaler + ASGs
  • Discussion:
    • Cluster Autoscaler grows and shrinks node groups when Pods are Pending for lack of capacity, or nodes are underused.
    • Karpenter (AWS-born, now also on Azure) skips node groups. It picks an instance type that fits the pending Pods.
    • Test it by scaling a Deployment past what the current nodes can hold, then kubectl get nodes -w.
  • Gotchas:
    • Scale-down evicts Pods, so protect critical apps with a PodDisruptionBudget.
    • Pods with local storage or safe-to-evict: "false" block scale-down.
    • New nodes take minutes, so HPA plus slack capacity beats relying on node scaling for sudden spikes.
    • Watch your cloud quota for max-nodes.

Changed since the book:

The AWS recipe used a Helm 2 chart (Tiller). Today it’s Karpenter, or the Cluster Autoscaler chart via Helm 3. GKE’s gcloud beta is no longer needed.

Source: K8s Cookbook, Recipes 9.2 & 9.3 · Best Practices book, Ch. 8

Recipe 9.4 — Horizontal Pod Autoscaling

  • Problem: Automatically add and remove replicas with load.
  • Solution:
    1. Make sure metrics-server is running (kubectl top pods works). On minikube: minikube addons enable metrics-server.
    2. Give the target CPU requests.
    3. Create the HPA.
kubectl create deployment appserver --image=registry.k8s.io/hpa-example --port=80
kubectl set resources deployment appserver --requests=cpu=200m
kubectl expose deployment appserver --port=80
kubectl autoscale deployment appserver --cpu-percent=40 --min=1 --max=5
kubectl get hpa -w
 
# load generator in another terminal
kubectl run -it --rm loadgen --image=busybox:1.36 --restart=Never -- \
  sh -c "while true; do wget -q -O- http://appserver; done"

Same thing declaratively, with autoscaling/v2:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: appserver
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: appserver
  minReplicas: 1
  maxReplicas: 5
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 40       # % of the CPU *request*
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300  # wait 5 min before shrinking (avoid flapping)
  • Discussion: Roughly, desired = ceil(current × currentMetric / target). The controller checks every ~15 s. autoscaling/v2 also supports memory, custom metrics (Prometheus Adapter: requests/s, queue depth) and external metrics. KEDA extends this to event sources (Kafka lag, SQS depth, cron) and can scale to zero.
  • Gotchas:
    • TARGETS <unknown> means no metrics-server, or no CPU request on the containers.
    • CPU is often a poor proxy for load. App-level metrics (RPS, latency, queue length) scale better, as the book also hints.

Changed since the book:

Heapster is gone → metrics-server (the metrics.k8s.io API). HPA API is autoscaling/v2 (GA 1.23) with a behavior block. kubectl run --requests --expose no longer exists. The hpa-example image moved to registry.k8s.io.

Source: K8s Cookbook, Recipe 9.4 · Best Practices book, Ch. 8

Modern Extra — Vertical Pod Autoscaler (VPA)

  • What: Watches real usage and recommends (or applies) better requests/limits. The book skipped it (“still under development, Jan 2018”). It’s now a mature add-on (not built in, so install it from kubernetes/autoscaler).
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: appserver-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: appserver
  updatePolicy:
    updateMode: "Off"        # just recommend; check with kubectl describe vpa
  • Gotchas: Don’t let VPA (in Auto mode) and an HPA both act on CPU/memory for the same workload, because they fight. A common combo is VPA Off (for sizing advice) + HPA (for scaling).
  • In-place Pod resize (resizePolicy, beta and on by default since 1.33) lets requests and limits change without restarting the Pod, so VPA no longer has to evict.

Summary

  • Manual: kubectl scale. Automatic Pods: HPA (autoscaling/v2) fed by metrics-server. Sizing: VPA. Nodes: Cluster Autoscaler / Karpenter.
  • No resource requests = no sensible autoscaling of any kind.
  • Remove replicas from Git when an HPA owns it. Add behavior.scaleDown stabilisation to stop flapping.
  • Use PodDisruptionBudgets so node scale-down doesn’t take out all replicas at once.

Graph View

Backlinks