Commit graph

3 commits

Author SHA1 Message Date
acamilo
f205d95e5e loop recovery: review-ladder fixes, root never runs a fly-writable binary
B1: fly-loop-reset runs only /opt/fly/sbin/flysim, a root-owned copy
05-deploy installs from the release tarball after checking it against
the tarball's MANIFEST; fixed paths, fly.env parsed as data, env -i.
B2: the wrapper pauses fly-watchdog.timer (and waits out a running
probe) for the reset, and starts flysim and the timer on every exit.
H1: a step that raises or times out is a failed step; the ladder state
is saved before the step runs. H2: one --list call computes the build's
compatibility once; TimeoutStartSec 25 min. H3: level 2+ resets to the
rung below the best or restarts, never lower.
Tests run the wrapper against a fake flysim, systemctl and archives.
2026-09-28 21:34:32 +00:00
acamilo
63ecc32b2f loop recovery: an escalation ladder that unsticks a trap on its own
Restart flysim, then reset to the current rung's milestone, then to the
archive below the best rung (never lower), one step per confirmed trap
that outlives the previous one. Two resets a day, three-hour restarts
once they are spent; the ladder starts over at a new best rung or after
six quiet hours. State and history live in the unit's StateDirectory so
a reboot does not forget where the ladder stood.

A router model list confirms each step; a 'not stuck' answer delays it
at most three probes and no answer leaves the watchdog to decide alone.
Each step is announced 60 s ahead in /run/fly/wd/recovery-notice.json
for the stage's recovery splash.

fly-loop-reset is the one new root surface (a sudoers line); 05-deploy
now converges config/fly-sudoers so a release can add it.
2026-09-28 21:23:33 +00:00
acamilo
432991cbc5 Add conservative automated loop recovery timer 2026-09-26 01:12:46 +00:00