Autoscaling and Resource Management in Kubernetes

This is the fifth and final post in our Kubernetes fundamentals series, building on Pods and Deployments, networking, ConfigMaps and Secrets, and storage.

A Kubernetes cluster with no resource requests or limits configured on any of its Pods will, eventually, run into a specific and unpleasant failure mode: one Pod consumes more memory than the node actually has available, and the node itself becomes unstable — not just that one Pod. Requests and limits are how you prevent that, and they're also the foundation everything about autoscaling is built on.

Requests and limits: two different guarantees

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: my-app:1.0
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"

These two fields answer genuinely different questions:

requests is what the scheduler uses to decide which node a Pod can even be placed on — it will only schedule a Pod onto a node that has at least this much CPU and memory currently unreserved. This is a scheduling guarantee, not a hard ceiling.
limits is the actual ceiling enforced at runtime. Exceed the CPU limit, and the container is throttled — slowed down, not killed. Exceed the memory limit, specifically, and the container is killed outright (an "OOMKilled" event), because unlike CPU, memory can't be safely throttled after the fact.

kubectl get pod web-7d9f8c6b5d-2xk9p

NAME                   READY   STATUS      RESTARTS   AGE
web-7d9f8c6b5d-2xk9p   0/1     OOMKilled   1          2m

Why "no limits set" is not the same as "unlimited, and safe"

A Pod with no requests set gets scheduled onto any node, regardless of actual available capacity, because the scheduler has nothing to reason about. A handful of such Pods landing on the same node, all growing their memory usage simultaneously under real load, can exhaust that node's actual physical memory — at which point the Linux kernel's OOM killer starts terminating processes on that node somewhat unpredictably, potentially including Pods that had nothing to do with the actual problem. Setting requests and limits deliberately on every Pod is what keeps one workload's resource usage from being able to destabilize everything else sharing that node.

The Horizontal Pod Autoscaler: more replicas under load

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

The Horizontal Pod Autoscaler (HPA) watches a metric — CPU utilization here, though custom metrics like requests-per-second are equally common — and adjusts the Deployment's replica count within the given range to keep that metric near the target. Average CPU usage across all web Pods climbing above 70% triggers additional replicas, up to maxReplicas; usage dropping well below target scales back down, no lower than minReplicas.

This directly depends on requests being set correctly. "70% utilization" is measured relative to the CPU request — a Pod with no request set, or a wildly inaccurate one, gives the HPA a meaningless number to scale against, and it will make correspondingly bad scaling decisions.

The Vertical Pod Autoscaler: right-sizing the request itself, over time

Where the HPA changes how many replicas exist, the Vertical Pod Autoscaler (VPA) adjusts the requests and limits values themselves, based on a Pod's actual observed usage history — useful for workloads where the right resource request isn't obvious upfront, or drifts over time as the application evolves.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  updatePolicy:
    updateMode: "Auto"

The real caveat: HPA and VPA both actively managing CPU/memory for the same workload conflict with each other — VPA changing a Pod's resource requests can trigger the Pod being recreated, right as HPA is independently trying to reason about replica count based on those same values changing underneath it. Most real deployments pick one axis to automate — horizontal scaling for stateless, request-driven services (the common case), vertical for a smaller set of workloads with genuinely unpredictable per-instance resource needs — rather than running both simultaneously on the same Deployment.

Cluster Autoscaler: the layer above both of these

HPA and VPA both operate within the existing set of nodes. If every node is already fully allocated and a new Pod (or a scaled-up existing one) has nowhere to fit, the Cluster Autoscaler is what actually provisions a new node from the cloud provider to make room — and scales nodes back down when they're sitting mostly idle. This is the layer that connects Kubernetes' scheduling decisions to real infrastructure cost, and it's why requests accuracy matters beyond just the HPA: overly generous requests waste real money on unnecessarily provisioned nodes; overly stingy ones risk the instability covered earlier in this post.

What to actually remember from this series

requests drive scheduling; limits are the enforced ceiling — exceeding a memory limit kills the container; exceeding a CPU limit throttles it.
Unset resource requests aren't "unlimited and safe" — they're a genuine risk to overall node stability under real load.
The HPA scales replica count based on a metric measured relative to requests — inaccurate requests produce bad scaling decisions, not just bad scheduling.
HPA and VPA generally shouldn't manage the same workload simultaneously; Cluster Autoscaler operates a layer above both, provisioning actual nodes when existing capacity runs out.

That closes out our Kubernetes fundamentals series — from Pods and Deployments through networking, ConfigMaps and Secrets, storage, and now resource management. If your team is running — or planning to run — real production workloads on Kubernetes, this is exactly the kind of hands-on work we build our cloud and DevOps training around.