x‑hakt

Seaworthy enough to launch

The restore system finally got the part that matters: a button that reaches into a live database or folder and puts a named backup back, with the undo taken automatically before it touches anything. Once that stopped being a hope and started being a mechanism, the project itself was steady enough for the other thing that had been sitting on the list: a real version number, and a container someone other than me can pull.

infrastructure control

dockerpostgressshghcrpodman

Restoring, not just proving you could

An earlier note built the weekly restore test: every backup, once a week, put back into a scratch nobody else can see, counted, and thrown away. That answered “is the copy any good”, which was the right first question. It left the harder one alone: if a project’s actual, serving database needed to come back right now, could I press one thing and trust what happened, or would I be improvising pg_restore flags from memory with something on fire.

This week that stopped being a hope. fleet-restore.sh is wired to a button on each project’s Backups page, and it does the restore for real, into the store that is actually live.

Four ways to put something back, one rule that never changes

The script dispatches on what it is restoring. A database in a container gets its schema reset and the archive pg_restore’d back in. One reached over gets the same treatment on the far end of a tunnel. A bind-mounted folder gets untarred over itself by a throwaway root container, ownership preserved. A named volume gets the same, against the volume instead of a path.

Redis gets none of that. It rewrites its own append-only file on shutdown, so a live swap underneath it just gets overwritten by the thing it was supposed to replace, no error, no restore. I could have built something clever around that. Instead the pane shows redis as manual and prints the three commands that actually work. You’re killing me, smalls, but at least it says so instead of pretending.

What every mode shares is the one rule I was not willing to bend on: before it touches anything, it takes a snapshot of what was there first. Not optional, not a flag you can skip in a hurry. If that snapshot fails to write, the restore refuses to start.

the live storepre-restore snapshotalways taken firstthe picked archiverestored in, livenow liveundo: restore this instead
Every mode takes a snapshot of the live store before it changes anything. That snapshot is the undo, not a separate feature.

Seeing a restore before you need one

Two smaller pieces make the button worth having rather than worth dreading. The pane lists every archive it can see for a store, not just the newest, so a bad backup taken an hour ago doesn’t strand you: pick the one from yesterday instead. And a check now flags any project whose weekly restore test has gone quiet past its schedule, the same way a missing spec or a missing git remote does. A backup you have not proven you can restore is a claim, not a fact, and it was too easy for that claim to go stale without anyone noticing.

The other change is smaller and meaner: a scheduled backup or restore test that silently stops running now says so, loudly, instead of the dashboard just quietly having nothing to show. Silence was the failure mode that actually happened before, more than once.

The other overdue thing

Once the part that scared me was done, the project itself was in good enough shape to do the thing I’d been putting off for a different reason entirely: give it a real release.

git tag v0.1.0 && git push --tags now triggers a build that publishes ghcr.io/x-hakt/bosun-x-dashboard under three tags at once: the exact version, the minor line, and latest. That is doing the boring, correct thing a container registry is for. Anyone who wants this dashboard for their own fleet can now pull an image instead of cloning a private working repo and hoping the parts that were specific to my setup got left out.

git tag v0.1.0git push —tagsbuildghcr.io:v0.1.0:0.1:latest
A pushed tag is the whole release process now. The workflow does the rest.

The rest of the release is honest housekeeping rather than new work. It honours now, which turned out to need nothing clever since Podman was built to answer to the same commands. The half-built systemd unit stayed half-built and got closed rather than finished: I have no project that actually needs a non-Docker way to run this yet, and shipping a path nobody has tested is worse than not shipping it. And a request to add a public share link per project, so a hull could be shown off without a login, got a plain no for now. Not every door is worth building just because someone could walk through it.

None of that changes what shipped. A restore that reaches into the live store and takes its own undo first, and a project real enough that pulling a container is now a genuine option instead of a joke about my folder structure.

-x