Config, secrets, and persistent storage

By now your pods run, restart themselves, get a stable address, and answer to a hostname through Traefik. There is one thing they still cannot do well: keep anything. The image is read-only and identical everywhere, the filesystem evaporates when the pod restarts, and you have been baking config and passwords straight into manifests where they do not belong. This part fixes all three — the non-secret config that changes per environment, the secret config that must never leak, and the data that has to outlive the pod.

level:Intermediate verified:Jun 2026

ℹ️ Hands-on — you'll create files and run kubectl against your own k3s cluster. Set that up first.

Why config does not belong in the image

A container image should be a build artifact: the same bytes in dev, staging, and production. The moment you bake a database URL or a log level into it, you need a different image per environment, and the whole point of "build once, run anywhere" is gone. Kubernetes solves this by keeping configuration as separate cluster objects that get injected into the pod at runtime — as environment variables or as files on disk. There are two such objects, split by sensitivity: ConfigMap for the harmless stuff and Secret for the dangerous stuff.

flowchart LR
  A["ConfigMap<br/><i>non-secret key/values</i>"] -->|"env or file"| C["Pod<br/><i>container</i>"]
  B["Secret<br/><i>sensitive key/values</i>"] -->|"env or file"| C
  D["Container image<br/><i>same bytes everywhere</i>"] -->|"pulled into"| C

The image stays generic; the cluster injects what makes this instance different. ConfigMaps decouple environment-specific configuration from container images.

ConfigMaps: the non-secret config

A ConfigMap is an API object used to store non-confidential data in key-value pairs. A pod can consume it three ways: as individual environment variables, as a whole set of variables via envFrom, or as configuration files in a volume.

By the end of this post you will have created three small files:

.
├── configmap.yaml   # non-secret settings
├── secret.yaml      # sensitive values
└── pvc.yaml         # a request for disk

Here is a small ConfigMap holding two settings:

configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  LOG_LEVEL: "info"
  GREETING: "hello from k3s"
  app.properties: |
    cache.ttl=300
    feature.beta=false

Save it and apply:

kubectl apply -f configmap.yaml
kubectl get configmap app-config

NAME         DATA   AGE
app-config   3      5s

Consuming the scalar keys as environment variables in a Deployment looks like this — just to show the shape:

spec:
  containers:
    - name: web
      image: my-app:1.4.0
      envFrom:
        - configMapRef:
            name: app-config

Or mounting the whole ConfigMap as files, where each key becomes a file whose contents are the value — again, just the shape:

spec:
  containers:
    - name: web
      image: my-app:1.4.0
      volumeMounts:
        - name: config
          mountPath: /etc/app
  volumes:
    - name: config
      configMap:
        name: app-config

The choice between the two has a real consequence. Changes to a ConfigMap mounted as a volume propagate to the running pod after the next kubelet sync, but ConfigMap data consumed as environment variables is not updated until the pod is restarted. If you want live reloads without a redeploy, mount as a file.

💡 Tip Keep ConfigMaps small and single-purpose. A ConfigMap is limited to 1 MiB of data; if you are trying to stuff a large blob in one, it belongs in a volume or an object store instead.

ℹ️ Note A ConfigMap and the pods that use it must live in the same namespace. There is no cross-namespace referencing — config is scoped to where the workload runs.

Secrets: like a ConfigMap, but for things that leak

A Secret is an object for storing a small amount of sensitive data such as a password, token, or key. It is consumed exactly like a ConfigMap — env vars or mounted files — and that symmetry is deliberate. The crucial difference is how Kubernetes treats it, and the most common misconception in the entire ecosystem lives right here.

secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
type: Opaque
stringData:
  DB_PASSWORD: "s3cr3t-do-not-commit"

Using stringData lets you write the plain value and Kubernetes base64-encodes it for you. Save and apply:

kubectl apply -f secret.yaml
kubectl get secret db-credentials

NAME             TYPE     DATA   AGE
db-credentials   Opaque   1      5s

Mounting or injecting it works the same as a ConfigMap — just to show the shape:

spec:
  containers:
    - name: web
      image: my-app:1.4.0
      env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: DB_PASSWORD

⚠️ Warning A Secret is base64-encoded, not encrypted. The values are stored unencrypted in the API server's underlying datastore (etcd) by default, and base64 is an encoding, not encryption — it provides no confidentiality whatsoever. Anyone who can read etcd, or run kubectl get secret -o yaml, can recover the value with a single base64 -d. Treating "it's a Secret" as "it's safe" is the mistake.

So what makes it actually safe? Three layers, in increasing order of effort:

flowchart TD
  A["Secret in etcd<br/><i>base64 only — readable</i>"] -->|"step 1"| B["Encryption at rest<br/><i>EncryptionConfiguration</i>"]
  B -->|"step 2"| C["RBAC<br/><i>restrict who can read</i>"]
  C -->|"step 3"| D["Sealed Secrets / External Secrets<br/><i>never commit plaintext</i>"]

Each layer closes a different hole: encryption protects the disk, RBAC protects the API, and the external tooling keeps plaintext out of Git in the first place.

• Encryption at rest. All Kubernetes APIs that persist resource data support at-rest encryption, and you can enable it for Secrets by giving the API server an EncryptionConfiguration that wraps the data with a provider (AES-CBC, AES-GCM, or an external KMS) before it lands in etcd.
• Restrict access. Anyone with API access or etcd access can read or modify a Secret, so RBAC rules that limit who can get Secrets matter as much as the encryption.
• Keep plaintext out of Git. Since you store manifests in version control, never commit a raw Secret. Tools like Sealed Secrets encrypt the value so only your cluster can decrypt it (safe to commit), or External Secrets pulls live values from a vault at runtime so they are never in Git at all.

⛔ Danger Never put real credentials in a stringData block and push it to a public repo. People scan GitHub for exactly this, and a leaked database password is leaked the moment the commit lands — rewriting history later does not un-leak it. Use Sealed Secrets or an external store for anything that touches version control.

Storage: making data outlive the pod

Config is one half of "stateful". The other half is the bytes your app writes — a database file, uploaded images, a cache. A pod's container filesystem is ephemeral: it resets to the image on every restart. Volumes are how you attach storage to a pod, and they come in two flavours separated by lifecycle.

Volumes vs PersistentVolumes

A plain volume like emptyDir is created when a pod is assigned to a node and its lifespan is tied to that pod — when the pod is removed, the data is deleted and lost. That is fine for scratch space, but useless for a database. For data that must survive the pod, Kubernetes provides the PersistentVolume subsystem, whose two API objects abstract storage provisioning from consumption:

• A PersistentVolume (PV) is a piece of storage in the cluster whose lifecycle is independent of any pod that uses it.
• A PersistentVolumeClaim (PVC) is a request for storage by a user — it asks for a size, an access mode, and a class, and the pod references the claim, not the volume.
• A StorageClass describes the class of storage on offer and names the provisioner that creates volumes on demand.

The flow is: a pod references a PVC, the PVC asks a StorageClass for storage, the StorageClass's provisioner creates a PV to satisfy it, and the PVC binds to that PV.

flowchart LR
  A["Pod<br/><i>references the claim</i>"] -->|"mounts"| B["PVC<br/><i>request: 5Gi, RWO</i>"]
  B -->|"uses class"| C["StorageClass<br/><i>provisioner</i>"]
  C -->|"dynamically creates"| D["PV<br/><i>actual disk</i>"]
  B -.->|"binds to"| D

The pod never names a disk directly. It asks for a claim; the StorageClass provisions a matching PV behind it and binds the two. Dynamic provisioning creates storage on demand when a PVC is created.

A PVC is short:

pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  # storageClassName omitted -> uses the cluster default

Save and apply it, then check that it bound to a volume:

kubectl apply -f pvc.yaml
kubectl get pvc data

NAME   STATUS   VOLUME                  CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data   Bound    pvc-3f1a...-local-path  5Gi        RWO            local-path     8s

And a pod mounts it by name — just to show the shape:

spec:
  containers:
    - name: db
      image: postgres:16
      volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: data

ℹ️ Note ReadWriteOnce means the volume mounts read-write on a single node at a time — the common case, and exactly what local-path storage supports. Multi-node read-write (ReadWriteMany) needs a networked filesystem like NFS or Longhorn, which is a later problem.

The k3s local-path provisioner

On a managed cloud, the default StorageClass provisions a cloud disk. On k3s there is no cloud, so it ships its own: k3s comes with Rancher's Local Path Provisioner out of the box, which lets you create PersistentVolumeClaims using local storage on the node. It is installed as the default StorageClass, so the PVC above just works with no extra setup — that empty storageClassName resolves to local-path.

When a pod claims storage, the provisioner creates a directory under /var/lib/rancher/k3s/storage on the node and binds the PV to it. The k3s manifest also marks it default via the storageclass.kubernetes.io/is-default-class: "true" annotation, re-applied on every reboot.

The catch is in the name: storage is local. Because data is on a specific node's disk, a pod using a local-path PVC is effectively pinned to that node — it cannot reschedule to another machine and still see its data. That is fine for a single-server homelab and for the database in the next part, but it is the wall you hit when you want true high availability, and the point where Longhorn or NFS enters the picture.

💡 Tip Run kubectl get storageclass and look for the one marked (default). On a fresh k3s cluster that is local-path. Knowing your default class tells you exactly where a PVC with no storageClassName will land.

A short close

Three objects, one principle: keep the image generic and inject what changes. ConfigMaps carry the harmless settings, Secrets carry the dangerous ones — base64-encoded by default, so you reach for encryption at rest and Sealed/External Secrets to make them genuinely safe — and PersistentVolumeClaims, backed by the k3s local-path provisioner, give a pod disk that outlives it. You now have every primitive a real application needs.

That is exactly where the series goes next: deploying a real app on k3s ties the Deployment, Service, Ingress, PVC, ConfigMap, and Secret into one manifest set. If you need the pieces you just configured into, revisit Services and networking and Ingress with Traefik, and the series hub has the whole path. This site's own state lives on K3s with storage exactly like this — and if the name still puzzles you, why Kubernetes is called k8s is the short version.