Commit graph

5 commits

Author SHA1 Message Date
acamilo
f205d95e5e loop recovery: review-ladder fixes, root never runs a fly-writable binary
B1: fly-loop-reset runs only /opt/fly/sbin/flysim, a root-owned copy
05-deploy installs from the release tarball after checking it against
the tarball's MANIFEST; fixed paths, fly.env parsed as data, env -i.
B2: the wrapper pauses fly-watchdog.timer (and waits out a running
probe) for the reset, and starts flysim and the timer on every exit.
H1: a step that raises or times out is a failed step; the ladder state
is saved before the step runs. H2: one --list call computes the build's
compatibility once; TimeoutStartSec 25 min. H3: level 2+ resets to the
rung below the best or restarts, never lower.
Tests run the wrapper against a fake flysim, systemctl and archives.
2026-09-28 21:34:32 +00:00
acamilo
9c9cec49a9 loop recovery: reset only to an archive the running build can restore
fly-reset-to-milestone does not check compatibility and a flysim that
refuses every checkpoint does not start. fly-loop-reset --check applies
05-deploy's rule (identical, or an adapter-only difference named in
FLY_ACCEPT_ADAPTERS); the ladder picks the highest restorable rung and
falls back to a restart when there is none.
2026-09-28 21:25:58 +00:00
acamilo
63ecc32b2f loop recovery: an escalation ladder that unsticks a trap on its own
Restart flysim, then reset to the current rung's milestone, then to the
archive below the best rung (never lower), one step per confirmed trap
that outlives the previous one. Two resets a day, three-hour restarts
once they are spent; the ladder starts over at a new best rung or after
six quiet hours. State and history live in the unit's StateDirectory so
a reboot does not forget where the ladder stood.

A router model list confirms each step; a 'not stuck' answer delays it
at most three probes and no answer leaves the watchdog to decide alone.
Each step is announced 60 s ahead in /run/fly/wd/recovery-notice.json
for the stage's recovery splash.

fly-loop-reset is the one new root surface (a sudoers line); 05-deploy
now converges config/fly-sudoers so a release can add it.
2026-09-28 21:23:33 +00:00
acamilo
1bd2223431 Require complete router configuration before veto 2026-09-26 01:23:31 +00:00
acamilo
432991cbc5 Add conservative automated loop recovery timer 2026-09-26 01:12:46 +00:00