From 282fd3db8c5e8d940f02b706d75ff5f352ff5975 Mon Sep 17 00:00:00 2001 From: Adam Carr Date: Sat, 19 Sep 2026 20:59:19 -0700 Subject: [PATCH] Documentation audit: refresh README, resolve stale PLAN.md items, list open items --- README.md | 47 +++++++++++++++++++++++++++++++++++------------ docs/PLAN.md | 35 +++++++++++++++++++++-------------- 2 files changed, 56 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 89a0abc..4bbce6a 100644 --- a/README.md +++ b/README.md @@ -1,20 +1,43 @@ # planck — 20-node Raspberry Pi 4 home lab -20× Raspberry Pi 4 Model B (8GB), each booting a 240GB USB SSD. Named -`planck001`–`planck020`. Network: Ubiquiti UDM Pro, 2 switches, one router. -Synology NAS on the same LAN for shared storage. +20× Raspberry Pi 4 (8GB, ARM64), each booting a 240GB NVMe-in-USB stick. +Fresh Raspberry Pi OS Lite (Debian 13) everywhere, running a k3s Kubernetes +cluster: servers planck002/003/004 (embedded etcd), workers 001 + 005–020. -## Docs +**Home page: https://carr.pub** (served by this cluster). -- `docs/INVENTORY.md` — hardware inventory (regenerate: `./scripts/inventory.sh > docs/INVENTORY.md`) -- `docs/MACS.txt` — eth0 MAC addresses (for DHCP reservations) -- `docs/PLAN.md` — the rebuild plan and decisions log +## The map -## Scripts +| Path | What it is | +|------|-----------| +| `docs/PLAN.md` | Full history: what was built, every bug and its fix, postmortems. READ FIRST | +| `docs/IPS.md` | Node ↔ IP ↔ MAC table (static IPs, pinned on nodes) | +| `docs/INVENTORY.md` | Hardware inventory per node | +| `scripts/flash-one.sh` | Reimage a node's NVMe stick (run on laptop via pkexec) | +| `scripts/k3s-join.sh` | Join servers/agents to the cluster (needs K3S_TOKEN) | +| `scripts/loadtest.sh` | Spawn a k6 load swarm across N nodes | +| `scripts/inventory.sh` | Regenerate hardware inventory | +| `manifests/monitoring/` | Prometheus, node-exporter, Grafana, DDNS CronJob | +| `manifests/loadtest/` | k6 rig + practice target | +| `manifests/web/` | The carr.pub site + Ingress | +| `manifests/cluster-issuer.yaml` | Let's Encrypt issuer | +| `configs/` | Node password (gitignored), installer internals, retired netboot servers | -- `scripts/inventory.sh` — collects hardware/network inventory from all nodes +## Daily operations -## Node access +- kubectl on the laptop: `~/.local/bin/kubectl` (kubeconfig `~/.kube/config`) +- SSH any node: `ssh adamcarr@planck0NN.local` (passwordless sudo) +- Grafana: `kubectl -n monitoring port-forward svc/grafana 3000:3000` +- Load test: `scripts/loadtest.sh [pods] [vus-per-pod] [duration]` +- New public site: Ingress in `web` namespace on `.carr.pub` with the + `cert-manager.io/cluster-issuer: letsencrypt-prod` annotation + DNSimple + record -SSH: `ssh adamcarr@planck0NN.local` (key auth set up 2026-09-19). mDNS -resolves `planck0NN.local` across subnets. +Agent documentation lives in `~/.agents/skills/planck-cluster/SKILL.md` +(loaded automatically by opencode sessions). + +## Safety rules + +- Never reboot more than ONE of planck002/003/004 at once (etcd quorum) +- Images must be linux/arm64 multi-arch +- The cluster room switch shares a PSU with nodes 011–015 diff --git a/docs/PLAN.md b/docs/PLAN.md index f648646..3b297fa 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -70,7 +70,11 @@ Fully unattended network reimage, no per-node physical access: - NOTE: Pi firmware injects `cgroup_disable=memory` — the node cmdline.txt overrides with `cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory` (already applied fleet-wide; required for k3s). -- Domain for websites: carr.pub (DNS not yet pointed; pending) +- Domain: carr.pub is LIVE — DNSimple apex A record (id 84723206) → + 50.46.44.67, UDM port-forwards 80/443 → planck002, Traefik ingress, + cert-manager Let's Encrypt issuer `letsencrypt-prod`, DDNS CronJob + `ddns-carr-pub` keeps DNS fresh. DNSimple token: k8s secret + `dnsimple-token` in namespace `monitoring` - Next up: monitoring (light Prometheus+Grafana), Synology NFS storageclass, carr.pub DNS + cert-manager, load-test rig, agent harness namespaces @@ -165,18 +169,21 @@ TFTP_PREFIX=1): - [x] Phase 0 — Access + inventory (SSH keys on all 20, docs/) - [ ] Phase 1 — Network design: static IPs / reservations in UniFi - [ ] Phase 2 — Reimage all 20 nodes with fresh Raspberry Pi OS -- [ ] Phase 3 — Cluster runtime install (candidates: k3s vs Docker Swarm) -- [ ] Phase 4 — Storage integration with Synology (NFS CSI) -- [ ] Phase 5 — Apps: monitoring, Portainer/UI, whatever the lab is for +- [x] Phase 3 — Cluster runtime install (k3s — decided and live) +- [x] Phase 4 — Fixed IPs pinned on nodes (Synology NFS storage class still open) +- [x] Phase 5 — Monitoring (Prometheus/Grafana), load-test rig, https://carr.pub -## Open decisions +## Resolved decisions (were "open" on day one) -1. **Reimage method** — options: - a. PXE/network boot: EEPROM boot-order set to netboot; DHCP next-server - points at a TFTP/HTTP installer. Zero-touch per node, reusable forever. - Installer server can run on the Synology (Docker) or a laptop. - b. SD-card bootstrap installer: flash one SD image, insert per node, it - wipes + installs the USB SSD, removes itself. Simple, physical walk. - c. Pull SSDs, image on a PC with Raspberry Pi Imager ×20. Most manual. -2. **Cluster runtime** — k3s (lighter, k8s-compatible, we manage it with - automation) vs Docker Swarm + Portainer (simplest). +1. **Reimage method** — pull-and-image on the laptop won (`flash-one.sh`). + The network-boot experiment is retired but documented below as a + postmortem; its Synology services linger (tftp disabled by design). +2. **Cluster runtime** — k3s, 3 servers (002-004) + 17 workers. + +## Open items (next sessions) + +- Synology NFS StorageClass (survives node swaps; local-path is current default) +- Grafana dashboards worth looking at (stack up, dashboards not built) +- Rotate the Grafana admin password (`planck-lab-admin` is a committed + placeholder in manifests/monitoring/grafana.yaml) +- UniFi DHCP reservations for the 20 node IPs (see docs/IPS.md)