From e25d9add7722fc2f1217e87d39846c85b2d2107d Mon Sep 17 00:00:00 2001 From: Adam Carr Date: Sun, 20 Sep 2026 16:58:45 -0700 Subject: [PATCH] Backups: etcd snapshots (6h/14d), NAS btrfs daily snapshots, secrets off-laptop --- docs/PLAN.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/PLAN.md b/docs/PLAN.md index 3b297fa..9c19253 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -180,6 +180,31 @@ TFTP_PREFIX=1): postmortem; its Synology services linger (tftp disabled by design). 2. **Cluster runtime** — k3s, 3 servers (002-004) + 17 workers. +## Backups (established 2026-09-20) + +Three layers: + +1. **etcd snapshots** (cluster state — all k3s objects, secrets, config): + every 6h on each server node, 14-day retention + (`/etc/rancher/k3s/config.yaml`: `etcd-snapshot-schedule-cron: "0 */6 * * *"`). + Files at `/var/lib/rancher/k3s/server/db/snapshots/`. Manual test snapshot: + `sudo k3s etcd-snapshot save --name test`. Restore: stop k3s on all + servers, `sudo k3s server --cluster-init --cluster-reset + --cluster-reset-restore-path=` on one, then start all. +2. **NAS btrfs snapshots** (Gitea repos, k8s-volumes, netboot artifacts): + `/usr/local/bin/planck-backup.sh` on the Synology, daily at 3am, 14-day + retention, read-only btrfs snapshots of the `plancknetboot` share + (`plancknetboot@daily-YYYY-MM-DD`). Manual run: the script itself. + Restore: `btrfs subvolume snapshot /volume1/plancknetboot@daily- /volume1/restore`. +3. **Laptop-only secret files** (node/gitea/admin passwords, authelia + session/jwt secrets — the gitignored ones): tarball at + `/volume1/plancknetboot/cluster-secrets/secrets.tar.gz` (root-only). + Refresh whenever a password is created/changed. + +Grafana dashboards and all manifests live in git (Gitea + laptop), so they +are covered by layers 1-3 transitively. Prometheus history (15d) is +deliberately not backed up — monitoring data is expendable. + ## Open items (next sessions) - Synology NFS StorageClass (survives node swaps; local-path is current default)