Creating & Modifying Workloads

Overview

• Chapters 4 and 7 of the Kubernetes Cookbook: Pods and Deployments (long-running apps), then the specialised controllers for Jobs, CronJobs, DaemonSets, StatefulSets and init containers.
• My basics: Pods (ephemeral, unique IP), Deployments (ReplicaSets, thrashing, CrashLoopBackOff). This note covers the how plus the other workload types.
• Back to the hub: Kubernetes Cookbook

Which Controller Do I Want?

Workload shape	Controller	Example
Long-running, identical, stateless replicas	Deployment	web API, frontend
Run to completion once	Job	DB migration, batch convert
Run to completion on a schedule	CronJob	nightly backup
Exactly one per node	DaemonSet	log shipper, node exporter, CNI agent
Stable identity + own disk per replica	StatefulSet	Postgres, Kafka, CockroachDB
Just one throwaway container	bare Pod	kubectl run tmp --rm -it … debugging

Changed since the book:

All these controllers are GA in apps/v1 (Deployment, DaemonSet, StatefulSet, ReplicaSet) or batch/v1 (Job, CronJob). extensions/v1beta1, apps/v1beta1/2 and batch/v2alpha1 are gone. In apps/v1, spec.selector is required and must match the template labels.

---

Recipes — Fundamental Workloads (Ch. 4)

Recipe 4.1 — Create a Deployment from the CLI

• Problem: Quickly launch a long-running app.
• Solution:

kubectl create deployment ghost --image=ghost:5 --port=2368 --replicas=2
kubectl create deployment mysql --image=mysql:8.4
kubectl set env deployment/mysql MYSQL_ROOT_PASSWORD=root   # demo only, use a Secret for real
kubectl expose deployment ghost --port=2368                 # Service in one line
kubectl run myshell --image=busybox:1.36 -it --rm --restart=Never -- sh   # one-off Pod

• Gotchas: create deployment has no --env, so add env vars with kubectl set env or in YAML.

Changed since the book:

Every kubectl run … in the book that creates a Deployment (--replicas, --expose, --requests) no longer works that way. run now makes one Pod, and most of those flags were removed.

Source: K8s Cookbook, Recipe 4.1

Recipe 4.2 — Create Objects from Manifests

• Problem: You want the object’s properties written down, not typed.
• Solution: kubectl apply -f file.yaml, -f dir/, -f https://…/x.yaml, or -k overlay/ for kustomize.
• Discussion: create -f fails if the object exists. apply is idempotent (create-or-update), so it’s the one to use in scripts and GitOps.

Source: K8s Cookbook, Recipe 4.2

Recipe 4.3 — Write a Pod Manifest from Scratch

• Problem: Understand the minimal Pod.
• Solution:

apiVersion: v1
kind: Pod
metadata:
  name: oreilly
  labels:
    app: oreilly
spec:
  containers:
    - name: web
      image: nginx:1.27
      ports:
        - containerPort: 80
    - name: cache          # a second container shares network + volumes
      image: redis:7

• Discussion: Containers in one Pod share localhost and can share volumes. That’s what enables sidecars and adapters.
• Gotchas: Don’t run bare Pods in real life. Nothing restarts them if the node dies. Wrap them in a Deployment.

Source: K8s Cookbook, Recipe 4.3

Recipe 4.4 — Launch a Deployment from a Manifest

• Problem: Full control over how an app is launched and supervised.
• Solution:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fancyapp
  labels:
    app: fancy
spec:
  replicas: 5
  selector:
    matchLabels:
      app: fancy
  template:
    metadata:
      labels:
        app: fancy
        env: development
    spec:
      containers:
        - name: sise
          image: mhausenblas/simpleservice:0.5.0
          ports:
            - containerPort: 9876
          env:
            - name: SIMPLE_SERVICE_VERSION
              value: "0.9"
          resources:
            requests:
              cpu: 100m
              memory: 64Mi
            limits:
              memory: 128Mi

• Discussion: Deployment → ReplicaSet → Pods (the picture in Deployments). Each template change creates a new ReplicaSet, and the old ones are kept (revisionHistoryLimit, default 10) so you can roll back.
• Gotchas: Delete the Deployment, not its Pods. Always set requests (the scheduler needs them) and a memory limit (Best Practices).

Changed since the book:

extensions/v1beta1 → apps/v1 with a mandatory selector. To generate this skeleton use kubectl create deployment fancyapp --image=nginx --dry-run=client -o yaml. The book’s plain --dry-run is gone.

Source: K8s Cookbook, Recipe 4.4

Recipe 4.5 — Update (and Roll Back) a Deployment

• Problem: Roll out a new version safely, and undo it if it’s broken.
• Solution:

kubectl set image deployment/sise sise=mhausenblas/simpleservice:0.5.0
kubectl annotate deployment/sise kubernetes.io/change-cause="image 0.5.0"
kubectl rollout status deployment/sise
kubectl rollout history deployment/sise
kubectl rollout undo deployment/sise --to-revision=2
kubectl rollout restart deployment/sise     # re-create Pods, same spec (e.g. after a ConfigMap change)
kubectl rollout pause|resume deployment/sise

• Tune the rollout (in the Deployment spec):

spec:
  strategy:
    type: RollingUpdate        # or Recreate (kill all, then start: has downtime)
    rollingUpdate:
      maxSurge: 25%            # extra Pods allowed during rollout
      maxUnavailable: 0        # never drop below desired count
  minReadySeconds: 10          # Pod must stay Ready this long to count
  progressDeadlineSeconds: 600 # mark rollout as failed after this

• Discussion: Only changes to .spec.template (image, env, ports, resources, annotations on the template) trigger a rollout. Scaling doesn’t.
• Gotchas: Rolling updates are only as safe as your readiness probe. Without one, Kubernetes sends traffic to Pods that aren’t ready (see Monitoring & Logging). For canary or blue-green, use Argo Rollouts / Flagger or a second Deployment behind the same Service.

Changed since the book:

--record is deprecated, so use the change-cause annotation. kubectl rollout restart (added 1.15) is the clean way to “bounce” a Deployment.

Source: K8s Cookbook, Recipe 4.5

---

Recipes — Specialised Workloads (Ch. 7)

Recipe 7.1 — Run a Batch Job

• Problem: Run something to completion: a migration, a backup, a conversion.
• Solution:

apiVersion: batch/v1
kind: Job
metadata:
  name: counter
spec:
  completions: 1
  parallelism: 1
  backoffLimit: 3                # retries before the Job is marked failed
  activeDeadlineSeconds: 300     # hard timeout
  ttlSecondsAfterFinished: 600   # auto-clean the finished Job + Pods
  template:
    spec:
      restartPolicy: Never       # Jobs need Never or OnFailure
      containers:
        - name: counter
          image: busybox:1.36
          command: ["sh", "-c", "for i in 1 2 3; do echo $i; done"]

kubectl apply -f job.yaml
kubectl wait --for=condition=complete job/counter --timeout=120s
kubectl logs job/counter

• Discussion: Raise completions/parallelism for work queues. completionMode: Indexed gives each Pod a JOB_COMPLETION_INDEX.
• Gotchas: Without ttlSecondsAfterFinished, finished Jobs and their Pods pile up forever.

Changed since the book:

The TTL-after-finished controller is GA, so the manual cleanup the book mentions isn’t needed.

Source: K8s Cookbook, Recipe 7.1

Recipe 7.2 — Run a Task on a Schedule (CronJob)

• Problem: Run a Job every hour, night or week.
• Solution:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: hourly-date
spec:
  schedule: "0 * * * *"
  timeZone: "Europe/London"        # GA since 1.27
  concurrencyPolicy: Forbid        # skip if the previous run is still going
  startingDeadlineSeconds: 120
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: date
              image: busybox:1.36
              command: ["sh", "-c", "date"]

• Discussion: Trigger a run by hand with kubectl create job --from=cronjob/hourly-date manual-1. Suspend with kubectl patch cronjob hourly-date -p '{"spec":{"suspend":true}}'.
• Gotchas: Jobs can occasionally run twice (or be skipped), so make them idempotent.

Changed since the book:

batch/v2alpha1 → batch/v1 (GA in 1.21), plus the timeZone field.

Source: K8s Cookbook, Recipe 7.2

Recipe 7.3 — One Pod per Node (DaemonSet)

• Problem: Run an agent (log collector, monitoring exporter) on every node.
• Solution:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluent-bit
  namespace: logging
spec:
  selector:
    matchLabels:
      app: fluent-bit
  template:
    metadata:
      labels:
        app: fluent-bit
    spec:
      tolerations:
        - operator: Exists           # also run on tainted/control-plane nodes
      containers:
        - name: fluent-bit
          image: cr.fluentbit.io/fluent/fluent-bit:3.1
          volumeMounts:
            - name: varlog
              mountPath: /var/log
              readOnly: true
      volumes:
        - name: varlog
          hostPath:
            path: /var/log

• Discussion: New nodes get a Pod automatically. Limit to some nodes with nodeSelector or affinity.
• Gotchas: kubectl drain needs --ignore-daemonsets, because DaemonSet Pods aren’t evicted.

Changed since the book:

extensions/v1beta1 → apps/v1. The book’s fluentd-elasticsearch image is long gone. Fluent Bit is now the usual node log agent.

Source: K8s Cookbook, Recipe 7.3

Recipe 7.4 — Stateful and Leader/Follower Apps (StatefulSet)

• Problem: Your replicas aren’t interchangeable. They need stable names and their own disks (“pets, not cattle”).
• Solution:

apiVersion: v1
kind: Service
metadata:
  name: db
spec:
  clusterIP: None              # headless: gives each Pod a DNS name
  selector:
    app: db
  ports:
    - port: 5432
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: db
spec:
  serviceName: db
  replicas: 3
  selector:
    matchLabels:
      app: db
  template:
    metadata:
      labels:
        app: db
    spec:
      containers:
        - name: postgres
          image: postgres:16
          env:
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: db-secret
                  key: password
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 10Gi

• Discussion: Pods are named db-0, db-1, db-2 and reachable at db-0.db.<ns>.svc.cluster.local. They’re created in order and deleted in reverse, and each keeps its own PVC across restarts.
• Gotchas: You must create the headless Service yourself. Deleting the StatefulSet doesn’t delete the PVCs (by design, unless you set persistentVolumeClaimRetentionPolicy). StatefulSet ≠ replication: Postgres doesn’t magically become a cluster. For real databases use an operator (CloudNativePG, Strimzi for Kafka) or a managed DB. See Extending Kubernetes.

Changed since the book:

apps/v1beta1 → apps/v1. Added persistentVolumeClaimRetentionPolicy and minReadySeconds. The book’s CockroachDB example URL in kubernetes/kubernetes/examples no longer exists.

Source: K8s Cookbook, Recipe 7.4 · Best Practices book, Ch. 16

Recipe 7.5 — Control Startup Order (Init Containers)

• Problem: Your Pod shouldn’t start until a dependency (a backend, a DB) is reachable or a setup step has run.
• Solution:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      initContainers:
        - name: checkbackend
          image: busybox:1.36
          command: ["sh", "-c", "until nslookup backend; do echo waiting; sleep 2; done"]
      containers:
        - name: webserver
          image: nginx:1.27
          ports:
            - containerPort: 80

• Discussion: Init containers run sequentially, to completion, before the app containers start. Debug them with kubectl logs <pod> -c checkbackend.
• Native sidecars: an init container with restartPolicy: Always starts first and keeps running alongside the app. Use it for proxies or log shippers that must start before the app and stop after it.
• Gotchas: Prefer apps that retry their dependencies over hard ordering, because dependencies can go away after startup too.

Changed since the book:

apps/v1beta1 → apps/v1. Native sidecar containers were added (beta, on by default in 1.29, GA in 1.33).

Source: K8s Cookbook, Recipe 7.5

---

Summary

• Deployment for stateless, Job/CronJob for run-to-completion, DaemonSet per node, StatefulSet for identity + storage.
• Everything is apps/v1 or batch/v1 now, with a required selector that matches the template labels.
• Rollouts trigger on .spec.template changes. Use rollout status/history/undo/restart, and record why with change-cause.
• Clean Jobs up with ttlSecondsAfterFinished, and make CronJobs idempotent.
• Init containers gate startup. Native sidecars (restartPolicy: Always) run alongside.