Kaushik's Notes

Security

7 min read

Overview

  • Chapter 10 of the Kubernetes Cookbook (ServiceAccounts, RBAC, security contexts), plus Recipe 14.6 (encrypted secrets in Git) and the modern pieces the book predates: Pod Security Admission, NetworkPolicy and admission policies.
  • Related: Volumes & Configuration (Secrets), Best Practices (RBAC, policy, multi-tenancy). Back to the hub: Kubernetes Cookbook

Who Can Do What: the Request Path

  1. Authentication: who are you? A user (via OIDC, cloud IAM or client certs; Kubernetes has no User object) or a ServiceAccount (an app identity, namespaced).
  2. Authorization: are you allowed? RBAC.
  3. Admission: is this object acceptable, and should it be tweaked? Pod Security Admission, ValidatingAdmissionPolicy, webhooks (Kyverno, Gatekeeper).

Recipes

Recipe 10.1 — Give an App Its Own Identity (ServiceAccount)

  • Problem: An app talks to the Kubernetes API and needs a fine-grained identity.
  • Solution:
kubectl create serviceaccount myappsa
kubectl create token myappsa --duration=1h      # mint a short-lived token by hand
apiVersion: v1
kind: Pod
metadata:
  name: myapp
spec:
  serviceAccountName: myappsa
  automountServiceAccountToken: true     # set false for apps that never call the API
  containers:
    - name: main
      image: busybox:1.36
      command: ["sleep", "3600"]
  • Discussion: The token is mounted at /var/run/secrets/kubernetes.io/serviceaccount/token. The identity string is system:serviceaccount:<ns>:<name>. Every namespace has a default SA, which Pods get if you don’t specify one.
  • Gotchas: Most apps never call the Kubernetes API. Turn the token off for them (automountServiceAccountToken: false), because a leaked token is a foothold.

Changed since the book:

Tokens are now bound, projected, short-lived (auto-rotated, about 1h, tied to the Pod). Since 1.24 no long-lived token Secret is created per ServiceAccount, so the book’s kubectl describe secret myappsa-token-… step no longer exists. Use kubectl create token. Cloud workload identity (GKE Workload Identity, EKS Pod Identity/IRSA, AKS Workload Identity) maps SAs to cloud IAM.

Source: K8s Cookbook, Recipe 10.1

Recipe 10.2 — See What Access You Have

  • Problem: Can I (or this ServiceAccount) do X?
  • Solution:
kubectl auth can-i create deployments -n team-a
kubectl auth can-i list secrets --as=system:serviceaccount:sec:myappsa -n sec
kubectl auth can-i --list -n sec
kubectl auth whoami                         # who does the API server think I am?
kubectl get roles,rolebindings -n sec
kubectl get clusterroles | grep -v system:  # admin, edit, view, cluster-admin
kubectl describe clusterrole view
  • Discussion: RBAC’s moving parts:
    • Subject: a user, group or ServiceAccount.
    • Role / ClusterRole: a list of verbs on resources (get list watch create update patch delete).
    • RoleBinding / ClusterRoleBinding: glue a role to subjects.
  • Gotchas: The book’s tip of binding cluster-admin to kube-system:default to make the dashboard work is a terrible idea. It gives every Pod using that SA full control of the cluster. Never do it.

Changed since the book:

RBAC is on by default everywhere (minikube included, no --extra-config needed). kubectl auth whoami is new (GA 1.28).

Source: K8s Cookbook, Recipe 10.2

Recipe 10.3 — Control Access to Resources (Role + RoleBinding)

  • Problem: Let an app read Pods in its namespace and nothing else.
  • Solution:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: podreader
  namespace: sec
rules:
  - apiGroups: [""]            # "" = core group
    resources: ["pods"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: podreaderbinding
  namespace: sec
subjects:
  - kind: ServiceAccount
    name: myappsa
    namespace: sec
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: podreader
# imperative equivalent
kubectl create role podreader --verb=get,list,watch --resource=pods -n sec
kubectl create rolebinding podreaderbinding --role=podreader --serviceaccount=sec:myappsa -n sec
  • Which combo?
WantUse
Namespaced resources in one namespaceRole + RoleBinding
Same permissions reused in many namespacesClusterRole + RoleBinding (per namespace)
Cluster-scoped things (nodes, PVs) or all namespacesClusterRole + ClusterRoleBinding
  • Gotchas:
    • list/watch on Secrets = read all secrets in that namespace.
    • create pods is effectively “run as any SA in the namespace”.
    • roleRef is immutable, so delete and recreate the binding to change it.

Changed since the book:

rbac.authorization.k8s.io/v1beta1 → v1. The book’s --role=sec:podreader should just be --role=podreader.

Source: K8s Cookbook, Recipe 10.3 · Best Practices book, Ch. 4 & 17

Recipe 10.4 — Secure Pods (securityContext + Pod Security Admission)

  • Problem: Run as non-root, drop privileges, and enforce that cluster-wide.
  • Solution, part 1: harden the Pod (this passes the restricted profile):
apiVersion: v1
kind: Pod
metadata:
  name: secpod
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 5000
    fsGroup: 5000
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: shell
      image: busybox:1.36
      command: ["sleep", "3600"]
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities:
          drop: ["ALL"]
  • Solution, part 2: enforce it per namespace with Pod Security Admission (built in, label-driven):
apiVersion: v1
kind: Namespace
metadata:
  name: payments
  labels:
    pod-security.kubernetes.io/enforce: restricted   # reject violators
    pod-security.kubernetes.io/warn: restricted      # warn on kubectl apply
    pod-security.kubernetes.io/audit: restricted     # log to audit
  • Discussion: Three profiles: privileged (anything goes, for system namespaces), baseline (blocks the known escalations: hostPath, privileged, hostNetwork…) and restricted (the hardened list above). Try a label in dry-run first: kubectl label --dry-run=server --overwrite ns payments pod-security.kubernetes.io/enforce=restricted prints which existing Pods would violate it.
  • Gotchas: Many public images run as root and break under restricted. Use non-root images (distroless, -nonroot tags) or set runAsUser. PSA only covers Pods. For custom rules (“images must come from our registry”, “every Deployment needs an owner label”) use ValidatingAdmissionPolicy (CEL, built in, GA 1.30), Kyverno or OPA Gatekeeper.

Changed since the book:

PodSecurityPolicy (PSP) was removed in 1.25. The book’s PSP pointer (and the Best Practices book’s whole PSP section) is obsolete, and Pod Security Admission is the replacement. seccompProfile became a proper field (it used to be an annotation).

Source: K8s Cookbook, Recipe 10.4 · Best Practices book, Ch. 10 & 11

Modern Extra — NetworkPolicy (Pod Firewall)

  • Problem: By default every Pod can talk to every Pod. You want default-deny plus explicit allows.
  • Solution:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: payments
spec:
  podSelector: {}               # all Pods in the namespace
  policyTypes: ["Ingress"]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-api
  namespace: payments
spec:
  podSelector:
    matchLabels:
      app: api
  policyTypes: ["Ingress"]
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress   # auto-label on every namespace
      ports:
        - protocol: TCP
          port: 8080
  • Gotchas: Policies are only enforced if your CNI supports them (Calico, Cilium yes, plain Flannel no). Rules are additive, with no “deny” rules. If you add egress default-deny, remember to allow DNS (UDP/TCP 53 to kube-system) or everything breaks mysteriously. Start with ingress rules only (Best Practices book, Ch. 9).

Recipe 14.6 — Encrypted Secrets in Git

  • Problem: You want everything in Git (GitOps), including Secrets, without leaking them.
  • Solution options:

    • Sealed Secrets: encrypt with the controller’s public key. Only the in-cluster controller can decrypt.

      kubectl create secret generic db --from-literal=password=s3cr3t \
  --dry-run=client -o yaml | kubeseal --format yaml > db-sealed.yaml
kubectl apply -f db-sealed.yaml      # controller creates the real Secret

    • External Secrets Operator: Secrets live in AWS Secrets Manager, GCP Secret Manager, Azure Key Vault or HashiCorp Vault. An ExternalSecret CR syncs them in, so nothing secret is in Git at all.

    • SOPS (with age or KMS keys): encrypt values in YAML files. Flux and Argo CD plugins decrypt at deploy time.

    • Secrets Store CSI Driver: mount secrets from the vault as files without creating a Secret object.

  • Gotchas: A SealedSecret is bound to its namespace and name by default. Back up the controller’s private key, or you can’t decrypt after a cluster rebuild.

Changed since the book:

--dry-run → --dry-run=client, kubeseal --format yaml, and install via Helm. The v1alpha1 sealed-secrets API is current but the project is much more mature. ESO didn’t exist in 2018.

Source: K8s Cookbook, Recipe 14.6 · Best Practices book, Ch. 4

Summary

  • Identity: ServiceAccounts with short-lived projected tokens (kubectl create token). Disable automount when an app doesn’t need the API.
  • Authorization: RBAC v1, least privilege, and check with kubectl auth can-i. Never hand out cluster-admin casually.
  • Pod hardening: runAsNonRoot, drop ALL capabilities, no privilege escalation, RuntimeDefault seccomp. Enforce with Pod Security Admission labels (PSP is gone).
  • Network: default-deny NetworkPolicies + explicit allows (needs a policy-capable CNI).
  • Secrets in Git: Sealed Secrets / External Secrets / SOPS, never plain base64.

Graph View

Backlinks