diff --git a/docs/PLAN.md b/docs/PLAN.md index 4968599..22c58e7 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -58,7 +58,45 @@ Fully unattended network reimage, no per-node physical access: - UniFi MFA blocks API access; DHCP reservations deferred — mDNS (`planck0NN.local`) works across subnets today. -## Current debug state (paused here, swap to stronger model) +## REBUILD COMPLETE (2026-09-19) + +All 20 nodes re-imaged with Raspberry Pi OS Lite (Debian 13 trixie), via +pull-and-image on the laptop (`scripts/flash-one.sh`, one NVMe stick at a +time). Every node verified: correct hostname, SSH key auth, passwordless +sudo, 235G root filesystem, timezone America/Los_Angeles. + +- Node password: `configs/node-password.txt` (gitignored) +- planck019 + planck020 boot chips updated to 2026-05-17 firmware +- planck020 boot chip cleaned of the netboot-test config (BOOT_ORDER=0xf14, + no TFTP settings) +- Other 18 nodes: 2023-01-11 firmware, fine; 2026 firmware is staged on + their boot partitions and will apply on their next natural reboot +- The netboot/TFTP experiment is retired: planck-tftp service on the + Synology is stopped+disabled; planck-netboot (HTTP helper, :8000) still + running and harmless + +### Postmortem notes (for future reflashes) + +1. The OS image ships a locked `pi` user (uid 1000). Always remove or + rename it before adding your own user, or sudo breaks silently. +2. The desktop auto-mounter (udisks2) grabs partitions mid-flash; it also + reactivates itself when stopped. Mask it (`systemctl mask udisks2`) + during flashing sessions. +3. Stick device names bounce between sda/sdb — always look up the device + by size/model, never assume. +4. One stick locked up its JMicron USB bridge mid-write (e2fsck hung, 0 + IOPS, kernel I/O errors). Unplug/replug fixed it. If a stick hangs the + flash, reseat it first, suspect the stick second. +5. Boot-chip updates (EEPROM) have a timestamp guard: a staged update + older-or-equal to what's already flashed is silently ignored. Also, + network-delivered EEPROM updates change the FIRMWARE but preserve the + CONFIG. `bootloader_update=1` in config.txt is required for the + bootloader to look for updates on the boot medium. +6. If a node ever 7-blinks its green LED: kernel not found — check for + leftover `recovery.bin`/`pieeprom.upd`/`pieeprom.sig` on the boot + partition (staged-but-rejected updates block boot). + +## Old debug state (superseded) First live test on planck020 (EEPROM install-mode staged, works — confirmed via `vcgencmd bootloader_config` showing BOOT_ORDER=0xf142, TFTP_IP,