Merge feat/recovery-splash: an on-stream splash while recovery acts

apps/stage/README.md

@@ -47,12 +47,15 @@ are a `chromium --kiosk` line in a systemd unit and a Playwright test.
 | `feed` | ws URL | Feed override for `mode=live`. Default `ws://127.0.0.1:7400/feed`. |
 | `gain`, `gamegain`, `sfxgain` | 0..1 | Master / game / SFX gain. Defaults 0.9 / 0.8 / 0.5. |
 | `audio` | `0` | Do not create an AudioContext at all. |
+| `recovery` | `1`, `0` | Poll `/recovery-notice.json` for the auto-recovery splash. Defaults to on in `live` mode and off in `player` mode, so a fixture screenshot never picks up a stray notice file. |
 
 `window.__stage` exposes the operator surface: `metrics()` (per-stage paint timings), `audio()`
 (context state, ring fill, underruns, drops), `health()` (accepted snapshots, feed gaps, decode
 errors), `manifest()`, `seek(seconds)`, `stopFeed()`, `gameScale()`, `fly()` (renderer mode, gait
 phase, leg tips, proboscis extension), `motion()` (which tab and why, the moment on stage and its
 phase, the queue depth, live particles), `pam()` (the PAM centroid the flare spreads from), and
+`recovery(notice, nowS?)` (put a recovery notice on the splash by hand, validated like a polled
+one, with its clock optionally pinned; `null` clears it), and
 `fire(type, label, detail)` — the one deliberate way to drive the moment catalogue by hand, which
 is what the moment mockups and the moment assertions use instead of waiting for a fixture to
 contain one of each.
@@ -95,7 +98,8 @@ Builds, serves, and writes sixteen PNGs to `mockups/`, at 1920x1080 and DPR 1:
 `steady-t1-{senses,connectome,ladder}` and `describe` for the four tabs, `big-moment-t1` 2.5 s into that
 fixture's milestone, `moment-{milestone,badge,sugar,rollback}` shot 300 ms after the trigger (the
 middle of every arrival in the catalogue), `macros-{overworld,running,outcome,battle,indoors}` for
-the macro strip's five states, and the two `fly-*` review crops at 2x. `--only <substring>` shoots
+the macro strip's five states, `recovery-<phase>-<action>` for the auto-recovery splash, and the
+two `fly-*` review crops at 2x. `--only <substring>` shoots
 just the ones whose name contains it, which is how one panel gets re-reviewed without rewriting
 every committed PNG.
 
@@ -194,6 +198,7 @@ Four rail panels instead of layout v1's five, on a 12 px gutter; the left column
 | CHAT | 1012x244 | The last seven chat lines, or nothing at all | `panels/ChatPanel.tsx` |
 | Moment layer | — | Caption band, rail flash, particles, day slide | `panels/MomentLayer.tsx` |
 | Stale feed banner | 1824x40 | Over the title strip after 2 s of silence | `panels/StaleBanner.tsx` |
+| Recovery splash | over the game | The auto-recovery notice: a text box, or the whole game while flysim restarts | `panels/RecoverySplash.tsx` |
 
 ### The progress cluster
 
@@ -264,6 +269,48 @@ Bot lines green, names amber, text ink: a viewer has to be able to tell the brid
 replies from a person at a glance, because the bridge is the only thing on this stream that can be
 made to say something by accident.
 
+### Recovery splash
+
+`infra/bin/fly-loop-recover` (`infra/docs/loop-recovery.md`) unsticks a confirmed macro loop by
+restarting flysim or resetting the run to an earlier rung. Without a word on screen, viewers see
+the game freeze and jump. The splash says what is happening, in the Game Boy's own four greens so
+it reads as the cartridge's text box, not as a rail panel or an alarm:
+
+| Phase | Layout | Reset copy | Restart copy |
+|---|---|---|---|
+| `countdown` | Text box over the bottom ~40% of the game, big `M:SS` to `executeAt`, the loop and how long it was stuck. The stuck loop stays visible above it | The fly is stuck in a loop! / Rewinding to PEWTER CITY in 0:42 | … / Shaking it off in 0:42 |
+| `acting` | Covers the whole 800x720 game panel, which is frozen or blank while flysim is down; a stepped progress bar | Rewinding… / Back to PEWTER CITY | Shaking it off… / Same place, fresh start |
+| `done` | Text box, 8 s after the helper's `updatedAt` | Back at PEWTER CITY! | All shaken off! |
+| `failed` | Text box, 20 s | That didn't work / A human will take a look | same |
+
+**Where the notice comes from.** Not the feed: flysim is the thing being restarted. The helper
+writes `/run/fly/wd/recovery-notice.json` atomically (tmp + rename); `flystage-web`
+(`infra/config/serve.mjs`, the page's own static server, User=fly, same container, independent of
+flysim) serves it at `/recovery-notice.json` — 200 with the bytes, or 204 when there is no file
+— and the page polls that once a second (`src/lib/recovery-poll.ts`). `FLY_RECOVERY_NOTICE`
+overrides the path, for `serve.mjs` and for the Vite dev and preview servers alike, which serve
+the same route.
+
+**It can never break the stream.** `src/lib/recovery.ts` is pure and stateless: a missing,
+malformed, oversized (> 16 KiB) or wrong-version file is no splash; a notice whose `updatedAt` is
+more than 15 minutes old is ignored; `acting` stops covering the game 10 minutes after its last
+write even if the helper never follows up, so a dead helper cannot hide the game; and every
+string the helper wrote is clipped to letters, digits, spaces and `.,'&:#/-` at a fixed length
+before it can reach the screen. Because the view is a function of the file and the wall clock
+alone, a page that reloads mid-recovery shows exactly what one that watched it all would.
+
+To look at it by hand:
+
+```sh
+export FLY_RECOVERY_NOTICE=$PWD/.recovery-notice.json      # any writable path
+npm run dev -w @flybrain/stage                               # http://127.0.0.1:5273/?recovery=1
+npm run recovery-notice -w @flybrain/stage -- --phase countdown --action reset --in 45
+npm run recovery-notice -w @flybrain/stage -- --demo restart   # 20 s countdown, 15 s acting, done
+npm run recovery-notice -w @flybrain/stage -- --clear
+```
+
+`npm run mockups -- --only recovery` writes the eight `mockups/recovery-<phase>-<action>.png`.
+
 ### Moments
 
 `docs/design/animation.md`'s catalogue, wired. The engine (`src/motion/`, landed separately) owns

apps/stage/package.json

@@ -14,7 +14,8 @@
     "test:e2e": "playwright test",
     "mockups": "tsx tools/mockup.mts",
     "fonts": "tsx tools/font-compare.mts",
-    "record": "tsx tools/record-fixture.mts"
+    "record": "tsx tools/record-fixture.mts",
+    "recovery-notice": "tsx tools/recovery-notice.mts"
   },
   "dependencies": {
     "@flybrain/brain": "*",

apps/stage/src/App.tsx

@@ -50,6 +50,8 @@ import {
   boxStyle,
 } from '@/lib/geometry';
 import { stageOptions } from '@/lib/query';
+import { parseRecoveryNotice, recoveryView, type RecoveryView } from '@/lib/recovery';
+import { RecoveryPoller } from '@/lib/recovery-poll';
 import { Director, FLY_HEAD_ANCHOR } from '@/motion/director';
 import { MotionEngine } from '@/motion/engine';
 import type { MomentType } from '@/motion/moments';
@@ -68,6 +70,7 @@ import { FlyStrip } from '@/panels/FlyStrip';
 import { GamePanel } from '@/panels/GamePanel';
 import { MomentLayer } from '@/panels/MomentLayer';
 import { ProgressCluster } from '@/panels/ProgressCluster';
+import { RecoverySplash, nowSeconds, useRecovery } from '@/panels/RecoverySplash';
 import { StaleBanner } from '@/panels/StaleBanner';
 import { TabSlot } from '@/panels/TabSlot';
 import { TitleStrip } from '@/panels/TitleStrip';
@@ -632,6 +635,14 @@ export function App() {
 
     loop.start();
 
+    // -- The auto-recovery splash's notice ----------------------------------------------------
+    // Polled from `flystage-web`, not the feed: flysim is what the helper restarts
+    // (`src/lib/recovery-poll.ts`). Off in player mode unless `?recovery=1`.
+    const recoveryPoller = options.recovery
+      ? new RecoveryPoller({ onChange: (notice) => useRecovery.setState({ notice }) })
+      : null;
+    recoveryPoller?.start();
+
     // -- Test and operator surface ------------------------------------------------------------
     window.__stage = {
       options,
@@ -687,6 +698,19 @@ export function App() {
         ingest.commit(hot.lastSnapshotMs, true);
         return useStage.getState().chat.length;
       },
+      /**
+       * Put a notice on the recovery splash by hand, the same way `fire` drives a moment: for the
+       * splash mockups and e2e assertions. The raw object goes through the same validation as a
+       * polled file, the poller stops so it cannot overwrite it, and `nowS` pins the splash's clock
+       * so a countdown screenshot is reproducible. `null` clears it. Returns what the splash shows.
+       */
+      recovery: (raw, nowS) => {
+        recoveryPoller?.stop();
+        const notice = raw === null ? null : parseRecoveryNotice(raw);
+        const pinnedNowS = nowS ?? null;
+        useRecovery.setState({ notice, pinnedNowS });
+        return recoveryView(notice, nowSeconds(pinnedNowS));
+      },
       fly: () => ({
         // The renderer that is actually drawing, which is not always the one that was asked for:
         // `webgl` falls back to `paper` on a host with no usable GL context (`src/fly/index.ts`).
@@ -714,6 +738,7 @@ export function App() {
       if (readyTimer !== null) clearTimeout(readyTimer);
       loop.stop();
       source?.stop();
+      recoveryPoller?.stop();
       fly?.dispose();
       worker.terminate();
       void engine?.stop();
@@ -739,6 +764,7 @@ export function App() {
       <ChatPanel source={options.chat} />
 
       <MomentLayer particleRef={particleCanvas} />
+      <RecoverySplash />
       <StaleBanner />
     </div>
   );
@@ -764,6 +790,8 @@ declare global {
       fire: (type: MomentType, label?: string, detail?: string) => number | null;
       /** Replace the held feed's chat ring by hand; returns how many lines the panel accepted. */
       chat: (lines: readonly { by: string; text: string; bot?: boolean }[]) => number;
+      /** Show a recovery notice by hand (validated like a polled one); null clears it. */
+      recovery: (notice: unknown, nowS?: number) => RecoveryView | null;
       fly: () => {
         mode: string;
         requested: string;

apps/stage/src/index.css

@@ -3,6 +3,7 @@
 @import './theme/panels.css';
 @import './theme/rail.css';
 @import './theme/motion.css';
+@import './theme/recovery.css';
 
 /**
  * Self-hosted OFL faces. `font-display: block` with the default 3 s block period is deliberate:

apps/stage/src/lib/query.ts

@@ -57,6 +57,12 @@ export interface StageOptions {
   audio: boolean;
   /** `?metrics=1` keeps the paint-stage histogram and prints it on demand. */
   metrics: boolean;
+  /**
+   * Poll `flystage-web` for the auto-recovery notice (`src/lib/recovery.ts`). On by default in
+   * `live` mode and off in `player` mode, so a fixture screenshot can never pick up a notice file
+   * that happens to exist on the machine; `?recovery=1` / `?recovery=0` force it either way.
+   */
+  recovery: boolean;
 }
 
 const THEMES: readonly StageTheme[] = ['t1', 't2', 't3'];
@@ -114,6 +120,7 @@ export function parseStageOptions(search: string, defaultFeedUrl = 'ws://127.0.0
     },
     audio: flag(params, 'audio', true),
     metrics: flag(params, 'metrics', false),
+    recovery: flag(params, 'recovery', mode === 'live'),
   };
 }
 

apps/stage/src/lib/recovery-poll.ts

@@ -0,0 +1,88 @@
+/**
+ * Polls `flystage-web` for the recovery notice (`src/lib/recovery.ts`).
+ *
+ * Deliberately not the feed: flysim is the thing being restarted, so during `acting` there is no
+ * feed and no control API to ask. `flystage-web` is the page's own static server, in the same
+ * container and independent of flysim, and serves the helper's file at
+ * {@link RECOVERY_NOTICE_ROUTE} (`infra/config/serve.mjs`).
+ *
+ * Once a second, with a short timeout, and every failure — no server, a 404 from an older
+ * `serve.mjs`, a 204 for "no file", a half-written body, a hung request — is "no notice". The
+ * poller never throws into the page and never retries faster than its interval.
+ */
+import { RECOVERY_NOTICE_ROUTE, parseRecoveryBody, type RecoveryNotice } from './recovery';
+
+export const RECOVERY_POLL_MS = 1000;
+const TIMEOUT_MS = 2500;
+
+type Fetch = (input: string, init?: RequestInit) => Promise<Response>;
+
+export interface RecoveryPollerOptions {
+  url?: string;
+  intervalMs?: number;
+  fetch?: Fetch;
+  onChange: (notice: RecoveryNotice | null) => void;
+}
+
+export class RecoveryPoller {
+  private readonly url: string;
+  private readonly intervalMs: number;
+  private readonly fetchFn: Fetch;
+  private readonly onChange: (notice: RecoveryNotice | null) => void;
+  private timer: ReturnType<typeof setTimeout> | null = null;
+  private stopped = true;
+  /** The last body seen, so an unchanged file does not re-render the panel. */
+  private lastKey = '\u0000';
+
+  constructor(options: RecoveryPollerOptions) {
+    this.url = options.url ?? RECOVERY_NOTICE_ROUTE;
+    this.intervalMs = options.intervalMs ?? RECOVERY_POLL_MS;
+    this.fetchFn = options.fetch ?? ((input, init) => fetch(input, init));
+    this.onChange = options.onChange;
+  }
+
+  start(): void {
+    if (!this.stopped) return;
+    this.stopped = false;
+    void this.tick();
+  }
+
+  stop(): void {
+    this.stopped = true;
+    if (this.timer !== null) clearTimeout(this.timer);
+    this.timer = null;
+  }
+
+  /** One poll. Public so the unit tests can drive it without timers. */
+  async poll(): Promise<RecoveryNotice | null> {
+    let body = '';
+    const abort = new AbortController();
+    const timeout = setTimeout(() => abort.abort(), TIMEOUT_MS);
+    try {
+      const response = await this.fetchFn(this.url, { cache: 'no-store', signal: abort.signal });
+      if (response.status === 200) body = await response.text();
+    } catch {
+      body = '';
+    } finally {
+      clearTimeout(timeout);
+    }
+    const notice = parseRecoveryBody(body);
+    const key = notice === null ? '' : JSON.stringify(notice);
+    if (key !== this.lastKey) {
+      this.lastKey = key;
+      try {
+        this.onChange(notice);
+      } catch (error) {
+        console.warn(`recovery splash: ${(error as Error).message}`);
+      }
+    }
+    return notice;
+  }
+
+  private async tick(): Promise<void> {
+    if (this.stopped) return;
+    await this.poll();
+    if (this.stopped) return;
+    this.timer = setTimeout(() => void this.tick(), this.intervalMs);
+  }
+}

apps/stage/src/lib/recovery.ts

@@ -0,0 +1,286 @@
+/**
+ * The recovery splash's model: the notice file's contract, its validation, and what the splash
+ * shows at a given wall-clock second.
+ *
+ * `infra/bin/fly-loop-recover` (the auto-unstick helper) writes one small JSON file while it acts
+ * on a confirmed macro loop — restarting flysim, or resetting the run to an earlier rung — and
+ * `flystage-web` serves it to this page at {@link RECOVERY_NOTICE_ROUTE}. Viewers otherwise see
+ * the game freeze or jump for no reason; the splash says what is happening.
+ *
+ * Everything here is pure: the poller (`src/lib/recovery-poll.ts`) fetches, the panel
+ * (`src/panels/RecoverySplash.tsx`) renders, and this file decides. The one rule it exists to
+ * keep is that **the notice can never break the stream**: a missing, malformed, oversized, future
+ * or stale file is no notice at all, and every string the helper wrote is clipped to a closed
+ * character set before it can reach the screen.
+ */
+
+/** Where `flystage-web` (and the Vite dev/preview servers) serve the notice file. */
+export const RECOVERY_NOTICE_ROUTE = '/recovery-notice.json';
+
+/** Ignore a notice whose `updatedAt` is older than this (the contract: 15 minutes). */
+export const RECOVERY_STALE_S = 15 * 60;
+/** How long `done` stays up after the helper wrote it. */
+export const RECOVERY_DONE_S = 8;
+/** How long `failed` stays up after the helper wrote it. */
+export const RECOVERY_FAILED_S = 20;
+/**
+ * Stop covering the game this long after an `acting` write, even if the helper never follows up.
+ * A helper that died mid-recovery must not leave the game hidden behind a splash; with the game
+ * uncovered again the page's own STALE FEED banner is the honest state.
+ */
+export const RECOVERY_ACTING_MAX_S = 10 * 60;
+/** Tolerated clock skew between the helper's `updatedAt` and the page's clock. */
+const FUTURE_SKEW_S = 120;
+
+export type RecoveryPhase = 'countdown' | 'acting' | 'done' | 'failed';
+export type RecoveryAction = 'restart' | 'reset';
+
+/** A validated notice. Field names are the file's own; see `infra/docs/loop-recovery.md`. */
+export interface RecoveryNotice {
+  id: string;
+  phase: RecoveryPhase;
+  action: RecoveryAction;
+  fromRung: number | null;
+  fromLabel: string;
+  /** Only for `reset`; null otherwise. */
+  toRung: number | null;
+  toLabel: string;
+  reason: string;
+  loop: string[];
+  stuckSeconds: number | null;
+  announcedAt: number;
+  executeAt: number;
+  updatedAt: number;
+}
+
+const PHASES: readonly RecoveryPhase[] = ['countdown', 'acting', 'done', 'failed'];
+const ACTIONS: readonly RecoveryAction[] = ['restart', 'reset'];
+
+/** Longest place name kept (the ladder's longest is 15 characters). */
+const LABEL_MAX = 24;
+/** Longest macro name kept (the pad's are at most 12, e.g. `GO OBJECTIVE`). */
+const MACRO_MAX = 16;
+/** At most this many macros of the loop are shown. */
+const LOOP_MAX = 4;
+
+/**
+ * Clip a helper-written string to what the splash may draw: printable ASCII letters, digits,
+ * spaces and a little punctuation (place names like `MT. MOON` or `S.S. ANNE`, `ROUTE 22`), with
+ * runs of whitespace collapsed, capped at `max` characters. Anything else is dropped, so a notice
+ * cannot put markup, control characters or an unbounded line on air.
+ */
+export function cleanText(value: unknown, max: number): string {
+  if (typeof value !== 'string') return '';
+  return value
+    .replace(/\s+/g, ' ')
+    .replace(/[^A-Za-z0-9 .,'&:#/-]/g, '')
+    .replace(/ {2,}/g, ' ')
+    .trim()
+    .slice(0, max)
+    .trim();
+}
+
+function finite(value: unknown): number | null {
+  return typeof value === 'number' && Number.isFinite(value) ? value : null;
+}
+
+function rung(value: unknown): number | null {
+  const n = finite(value);
+  return n !== null && Number.isInteger(n) && n >= 0 && n < 1000 ? n : null;
+}
+
+/**
+ * Validate a parsed JSON value as a v1 notice, or return null.
+ *
+ * Required: `v === 1`, `id`, a known `phase` and `action`, and finite `announcedAt`, `executeAt`,
+ * `updatedAt`. Everything else is optional and degrades the copy rather than the notice: a reset
+ * without a `toLabel` still says it is rewinding, just not to where.
+ */
+export function parseRecoveryNotice(raw: unknown): RecoveryNotice | null {
+  if (raw === null || typeof raw !== 'object' || Array.isArray(raw)) return null;
+  const o = raw as Record<string, unknown>;
+  if (o.v !== 1) return null;
+  const id = typeof o.id === 'string' ? o.id.slice(0, 64) : '';
+  if (id === '') return null;
+  if (!PHASES.includes(o.phase as RecoveryPhase)) return null;
+  if (!ACTIONS.includes(o.action as RecoveryAction)) return null;
+  const announcedAt = finite(o.announcedAt);
+  const executeAt = finite(o.executeAt);
+  const updatedAt = finite(o.updatedAt);
+  if (announcedAt === null || executeAt === null || updatedAt === null) return null;
+
+  const action = o.action as RecoveryAction;
+  const loop = Array.isArray(o.loop)
+    ? o.loop
+        .map((step) => cleanText(step, MACRO_MAX).toUpperCase())
+        .filter((step) => step !== '')
+        .slice(0, LOOP_MAX)
+    : [];
+  const stuck = finite(o.stuckSeconds);
+
+  return {
+    id,
+    phase: o.phase as RecoveryPhase,
+    action,
+    fromRung: rung(o.fromRung),
+    fromLabel: cleanText(o.fromLabel, LABEL_MAX).toUpperCase(),
+    toRung: action === 'reset' ? rung(o.toRung) : null,
+    toLabel: action === 'reset' ? cleanText(o.toLabel, LABEL_MAX).toUpperCase() : '',
+    reason: cleanText(o.reason, 32),
+    loop,
+    stuckSeconds: stuck !== null && stuck >= 0 ? stuck : null,
+    announcedAt,
+    executeAt,
+    updatedAt,
+  };
+}
+
+/** Parse the body the server returned. Never throws. */
+export function parseRecoveryBody(body: string): RecoveryNotice | null {
+  if (body.length === 0 || body.length > 16_384) return null;
+  try {
+    return parseRecoveryNotice(JSON.parse(body));
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * How the splash sits on the stage: `box` is the Pokémon-style text box along the bottom of the
+ * game, leaving the stuck loop visible above it; `cover` fills the game panel, which is frozen or
+ * blank while flysim restarts.
+ */
+export type RecoveryLayout = 'box' | 'cover';
+
+/** What the splash draws. Every string is already final copy. */
+export interface RecoveryView {
+  id: string;
+  phase: RecoveryPhase;
+  action: RecoveryAction;
+  layout: RecoveryLayout;
+  /** The small chip in the frame's top edge. */
+  chip: string;
+  /** The first line, Pokémon-dialogue style. */
+  headline: string;
+  /** The second line. */
+  body: string;
+  /** `M:SS` during the countdown, otherwise empty. */
+  countdown: string;
+  /** `GO OBJECTIVE > GO WARP`, or empty. */
+  loop: string;
+  /** `STUCK 30 MIN`, or empty. */
+  stuck: string;
+  /** Whether to animate the trailing dots / blinking arrow. */
+  busy: boolean;
+}
+
+/** `0:42`, `1:00`, `12:05`. Negative clamps to `0:00`. */
+export function formatCountdown(seconds: number): string {
+  const s = Math.max(0, Math.ceil(seconds));
+  const m = Math.floor(s / 60);
+  return `${m}:${String(s % 60).padStart(2, '0')}`;
+}
+
+/** `STUCK 30 MIN`, `STUCK 2 H`, `STUCK 1 H 5 MIN`; empty under a minute or when unknown. */
+export function formatStuck(seconds: number | null): string {
+  if (seconds === null || seconds < 60) return '';
+  const minutes = Math.round(seconds / 60);
+  if (minutes < 60) return `STUCK ${minutes} MIN`;
+  const h = Math.floor(minutes / 60);
+  const m = minutes % 60;
+  return m === 0 ? `STUCK ${h} H` : `STUCK ${h} H ${m} MIN`;
+}
+
+/**
+ * Whether the notice is showing at `nowS` (epoch seconds), per the contract's lifetimes.
+ *
+ * Stateless on purpose: a page that reloads mid-recovery, or a Chromium the watchdog restarted,
+ * draws exactly what a page that watched the whole thing would, from the file alone.
+ */
+export function recoveryVisible(notice: RecoveryNotice, nowS: number): boolean {
+  const age = nowS - notice.updatedAt;
+  if (age > RECOVERY_STALE_S) return false;
+  if (age < -FUTURE_SKEW_S) return false;
+  switch (notice.phase) {
+    case 'countdown':
+      return true;
+    case 'acting':
+      return age <= RECOVERY_ACTING_MAX_S;
+    case 'done':
+      return age <= RECOVERY_DONE_S;
+    case 'failed':
+      return age <= RECOVERY_FAILED_S;
+  }
+}
+
+/** The splash at `nowS`, or null when nothing should be on screen. */
+export function recoveryView(notice: RecoveryNotice | null, nowS: number): RecoveryView | null {
+  if (notice === null || !recoveryVisible(notice, nowS)) return null;
+  const reset = notice.action === 'reset';
+  const to = notice.toLabel;
+  const base = {
+    id: notice.id,
+    phase: notice.phase,
+    action: notice.action,
+    loop: notice.loop.join(' > '),
+    stuck: formatStuck(notice.stuckSeconds),
+  };
+
+  switch (notice.phase) {
+    case 'countdown': {
+      const remaining = notice.executeAt - nowS;
+      const now = remaining <= 0;
+      return {
+        ...base,
+        layout: 'box',
+        chip: 'AUTO RECOVERY',
+        headline: 'The fly is stuck in a loop!',
+        body: reset
+          ? now
+            ? to !== '' ? `Rewinding to ${to} now` : 'Rewinding now'
+            : to !== '' ? `Rewinding to ${to} in` : 'Rewinding to the last milestone in'
+          : now
+            ? 'Shaking it off now'
+            : 'Shaking it off in',
+        countdown: formatCountdown(remaining),
+        busy: now,
+      };
+    }
+    case 'acting':
+      return {
+        ...base,
+        layout: 'cover',
+        chip: 'AUTO RECOVERY',
+        headline: reset ? 'Rewinding' : 'Shaking it off',
+        body: reset
+          ? to !== '' ? `Back to ${to}` : 'Back to the last milestone'
+          : 'Same place, fresh start',
+        countdown: '',
+        busy: true,
+      };
+    case 'done':
+      return {
+        ...base,
+        layout: 'box',
+        chip: 'AUTO RECOVERY',
+        headline: reset ? (to !== '' ? `Back at ${to}!` : 'Rewound!') : 'All shaken off!',
+        body: "Go get 'em, little fly",
+        countdown: '',
+        loop: '',
+        stuck: '',
+        busy: false,
+      };
+    case 'failed':
+      return {
+        ...base,
+        layout: 'box',
+        chip: 'AUTO RECOVERY',
+        headline: "That didn't work",
+        body: 'A human will take a look',
+        countdown: '',
+        loop: '',
+        stuck: '',
+        busy: false,
+      };
+  }
+}

apps/stage/src/panels/RecoverySplash.tsx

@@ -0,0 +1,129 @@
+import { useEffect, useState } from 'react';
+import { create } from 'zustand';
+
+import { LAYOUT, boxStyle } from '@/lib/geometry';
+import { recoveryView, type RecoveryNotice, type RecoveryView } from '@/lib/recovery';
+
+/**
+ * The auto-recovery splash, over the game panel.
+ *
+ * `infra/bin/fly-loop-recover` announces a recovery a minute before it acts, then restarts flysim
+ * or resets the run to an earlier rung; without this, viewers see the game freeze and jump for no
+ * reason. The model is `src/lib/recovery.ts`, the data path `src/lib/recovery-poll.ts`.
+ *
+ * Two layouts, both in the Game Boy's own four greens, so it reads as the cartridge's text box
+ * rather than as one more rail panel:
+ *
+ * - **box** (countdown, done, failed): a Pokémon-style text box across the bottom of the game,
+ *   with the countdown beside it. The top two thirds of the game stay visible, because the stuck
+ *   loop is the thing the countdown is about.
+ * - **cover** (acting): the whole game panel, because the picture there is frozen or blank while
+ *   flysim restarts.
+ *
+ * It re-renders at 4 Hz while a notice is held — the stage's own React cadence — and not at all
+ * otherwise. The countdown is wall-clock arithmetic against the helper's `executeAt`, which is why
+ * the page needs nothing but the file: it is right after a reload, and while the feed is down.
+ */
+export const useRecovery = create<{ notice: RecoveryNotice | null; pinnedNowS: number | null }>(() => ({
+  notice: null,
+  pinnedNowS: null,
+}));
+
+const TICK_MS = 250;
+
+export function nowSeconds(pinned: number | null): number {
+  return pinned ?? Date.now() / 1000;
+}
+
+export function RecoverySplash() {
+  const notice = useRecovery((state) => state.notice);
+  const pinned = useRecovery((state) => state.pinnedNowS);
+  const [nowS, setNowS] = useState(() => nowSeconds(pinned));
+
+  useEffect(() => {
+    setNowS(nowSeconds(pinned));
+    if (notice === null || pinned !== null) return;
+    const timer = setInterval(() => setNowS(nowSeconds(null)), TICK_MS);
+    return () => clearInterval(timer);
+  }, [notice, pinned]);
+
+  let view: RecoveryView | null = null;
+  try {
+    view = recoveryView(notice, nowS);
+  } catch {
+    view = null;
+  }
+  if (view === null) return null;
+
+  return (
+    <div
+      className="recovery"
+      data-testid="recovery-splash"
+      data-phase={view.phase}
+      data-action={view.action}
+      data-layout={view.layout}
+      style={{ ...boxStyle(LAYOUT.game), zIndex: 50 }}
+    >
+      {view.layout === 'cover' ? <Cover view={view} /> : <Box view={view} />}
+    </div>
+  );
+}
+
+function LoopLine({ view }: { view: RecoveryView }) {
+  if (view.loop === '' && view.stuck === '') return null;
+  return (
+    <div className="recovery__loop" data-testid="recovery-loop">
+      {view.loop !== '' ? <span className="recovery__loop-macros">{view.loop}</span> : null}
+      {view.loop !== '' && view.stuck !== '' ? <span className="recovery__sep" aria-hidden /> : null}
+      {view.stuck !== '' ? <span>{view.stuck}</span> : null}
+    </div>
+  );
+}
+
+function Box({ view }: { view: RecoveryView }) {
+  return (
+    <div className="recovery__box">
+      <span className="recovery__chip">{view.chip}</span>
+      <div className="recovery__row">
+        <div className="recovery__text">
+          <p className="recovery__headline" data-testid="recovery-headline">
+            {view.headline}
+          </p>
+          <p className="recovery__body" data-testid="recovery-body">
+            {view.body}
+            {view.busy ? <span className="recovery__dots" aria-hidden /> : null}
+          </p>
+        </div>
+        {view.countdown !== '' ? (
+          <div className="recovery__countdown" data-testid="recovery-countdown">
+            {view.countdown}
+          </div>
+        ) : null}
+      </div>
+      <LoopLine view={view} />
+      <span className="recovery__arrow" aria-hidden />
+    </div>
+  );
+}
+
+function Cover({ view }: { view: RecoveryView }) {
+  return (
+    <div className="recovery__cover">
+      <span className="recovery__chip recovery__chip--cover">{view.chip}</span>
+      <p className="recovery__big" data-testid="recovery-headline">
+        {view.headline}
+        <span className="recovery__dots" aria-hidden />
+      </p>
+      <p className="recovery__body recovery__body--cover" data-testid="recovery-body">
+        {view.body}
+      </p>
+      <div className={`recovery__bar recovery__bar--${view.action}`} aria-hidden>
+        {Array.from({ length: 8 }, (_, index) => (
+          <span key={index} style={{ animationDelay: `${index * 150}ms` }} />
+        ))}
+      </div>
+      <div className="recovery__was">{view.loop !== '' ? 'IT WAS LOOPING ON' : null}</div>
+      <LoopLine view={view} />
+    </div>
+  );
+}

apps/stage/src/theme/recovery.css

@@ -0,0 +1,230 @@
+/**
+ * The auto-recovery splash (`src/panels/RecoverySplash.tsx`), over the 800x720 game panel.
+ *
+ * Its own palette on purpose: the Game Boy's four greens, so the splash reads as the cartridge's
+ * own text box / screen and never as a rail panel or an alarm. Square corners, a 4 px outer frame
+ * and a 2 px inner line, like every other box on the page (`tokens.css`). Type sits on the page's
+ * floors: Press Start 2P for the dialogue line and the countdown, VT323 for the rest.
+ */
+.recovery {
+  --gb-0: #e0f0c8; /* lightest: the paper */
+  --gb-1: #8bac0f;
+  --gb-2: #306230;
+  --gb-3: #0f380f; /* darkest: the ink */
+  pointer-events: none;
+}
+
+/* -- Box: countdown, done, failed ---------------------------------------------------------- */
+
+.recovery__box {
+  position: absolute;
+  left: 16px;
+  right: 16px;
+  bottom: 16px;
+  padding: 30px 28px 20px;
+  background: var(--gb-0);
+  color: var(--gb-3);
+  border: var(--border-w) solid var(--gb-3);
+  outline: var(--border-inner-w) solid var(--gb-0);
+  outline-offset: calc(-1 * var(--border-w) - 6px);
+  box-shadow: inset 0 0 0 10px var(--gb-0), inset 0 0 0 12px var(--gb-2);
+  animation: recovery-rise 320ms var(--ease-pixel, steps(8, end)) both;
+}
+
+.recovery__chip {
+  position: absolute;
+  top: -18px;
+  left: 24px;
+  padding: 2px 12px;
+  font-family: var(--font-label);
+  font-size: var(--fs-body);
+  line-height: 28px;
+  background: var(--gb-3);
+  color: var(--gb-0);
+  letter-spacing: 0.04em;
+}
+
+.recovery__row {
+  display: flex;
+  align-items: center;
+  gap: 24px;
+}
+
+.recovery__text {
+  flex: 1 1 auto;
+  min-width: 0;
+}
+
+.recovery__headline {
+  margin: 0 0 14px;
+  font-family: var(--font-pixel);
+  font-size: 24px;
+  line-height: 1.45;
+}
+
+.recovery__body {
+  margin: 0;
+  font-family: var(--font-text);
+  font-size: 40px;
+  line-height: 1.05;
+  color: var(--gb-2);
+}
+
+.recovery__countdown {
+  flex: 0 0 auto;
+  font-family: var(--font-pixel);
+  font-size: 52px;
+  line-height: 1;
+  padding: 16px 14px 12px;
+  background: var(--gb-3);
+  color: var(--gb-0);
+  font-variant-numeric: tabular-nums;
+}
+
+.recovery__loop {
+  margin-top: 14px;
+  font-family: var(--font-text);
+  font-size: 30px;
+  line-height: 1;
+  color: var(--gb-2);
+  white-space: nowrap;
+  overflow: hidden;
+  text-overflow: ellipsis;
+}
+
+.recovery__loop-macros {
+  color: var(--gb-3);
+}
+
+.recovery__sep {
+  display: inline-block;
+  width: 8px;
+  height: 8px;
+  margin: 0 14px;
+  vertical-align: middle;
+  background: var(--gb-1);
+}
+
+/* The Pokémon "more text" arrow, blinking in the corner. Drawn, not typed: no glyph to miss. */
+.recovery__arrow {
+  position: absolute;
+  right: 22px;
+  bottom: 16px;
+  width: 0;
+  height: 0;
+  border-left: 10px solid transparent;
+  border-right: 10px solid transparent;
+  border-top: 12px solid var(--gb-3);
+  animation: recovery-blink 1s steps(1, end) infinite;
+}
+
+/* Three dots that count up, for "working on it". Steps, not a fade. */
+.recovery__dots::after {
+  content: '...';
+  display: inline-block;
+  width: 3ch;
+  overflow: hidden;
+  vertical-align: bottom;
+  text-align: left;
+  animation: recovery-dots 1.2s steps(4, jump-none) infinite;
+}
+
+/* -- Cover: acting --------------------------------------------------------------------------- */
+
+.recovery__cover {
+  position: absolute;
+  inset: 0;
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  justify-content: center;
+  gap: 28px;
+  padding: 40px;
+  background: var(--gb-0);
+  color: var(--gb-3);
+  border: var(--border-w) solid var(--gb-3);
+  box-shadow: inset 0 0 0 12px var(--gb-0), inset 0 0 0 16px var(--gb-2);
+  text-align: center;
+}
+
+.recovery__chip--cover {
+  position: static;
+}
+
+.recovery__big {
+  margin: 0;
+  font-family: var(--font-pixel);
+  font-size: 44px;
+  line-height: 1.3;
+}
+
+.recovery__body--cover {
+  font-size: 48px;
+}
+
+.recovery__was {
+  margin-top: 8px;
+  font-family: var(--font-label);
+  font-size: var(--fs-body);
+  color: var(--gb-2);
+  min-height: 1px;
+}
+
+.recovery__cover .recovery__loop {
+  margin-top: 0;
+  max-width: 100%;
+}
+
+/* Eight cells lighting in turn: right-to-left for a rewind, left-to-right for a restart. */
+.recovery__bar {
+  display: flex;
+  gap: 8px;
+}
+
+.recovery__bar--reset {
+  flex-direction: row-reverse;
+}
+
+.recovery__bar span {
+  width: 40px;
+  height: 24px;
+  background: var(--gb-1);
+  animation: recovery-cell 1.2s steps(1, end) infinite;
+}
+
+@keyframes recovery-rise {
+  from {
+    transform: translateY(24px);
+    opacity: 0;
+  }
+  to {
+    transform: none;
+    opacity: 1;
+  }
+}
+
+@keyframes recovery-blink {
+  50% {
+    visibility: hidden;
+  }
+}
+
+@keyframes recovery-dots {
+  from {
+    width: 0;
+  }
+  to {
+    width: 3ch;
+  }
+}
+
+@keyframes recovery-cell {
+  0%,
+  12% {
+    background: var(--gb-3);
+  }
+  13%,
+  100% {
+    background: var(--gb-1);
+  }
+}

apps/stage/tests/e2e/recovery.spec.ts

@@ -0,0 +1,138 @@
+/**
+ * The auto-recovery splash, against the real build.
+ *
+ * The first two tests go through the page's real data path — the poller fetching
+ * `/recovery-notice.json` — with Playwright answering the route, which is exactly what
+ * `flystage-web` does with the helper's file. The rest drive the splash through
+ * `window.__stage.recovery` with a pinned clock, for the geometry and the text floor.
+ */
+import { expect, test, type Page } from '@playwright/test';
+
+import { stageUrl } from './stage';
+
+const now = (): number => Math.floor(Date.now() / 1000);
+
+function notice(overrides: Record<string, unknown> = {}): Record<string, unknown> {
+  const t = now();
+  return {
+    v: 1,
+    id: `${t}-reset`,
+    phase: 'countdown',
+    action: 'reset',
+    fromRung: 12,
+    fromLabel: 'MT. MOON',
+    toRung: 11,
+    toLabel: 'PEWTER CITY',
+    reason: 'unrewarded',
+    loop: ['GO OBJECTIVE', 'GO WARP'],
+    stuckSeconds: 1800,
+    announcedAt: t,
+    executeAt: t + 45,
+    updatedAt: t,
+    ...overrides,
+  };
+}
+
+async function open(page: Page): Promise<void> {
+  await page.goto(`${stageUrl({ t: 95, tab: 'senses', audio: false })}&recovery=1`);
+  await page.waitForSelector('html[data-ready="1"]', { timeout: 60_000 });
+}
+
+test('a polled notice shows the countdown, follows the phases and clears', async ({ page }) => {
+  let body: string | null = JSON.stringify(notice());
+  await page.route('**/recovery-notice.json', (route) =>
+    body === null
+      ? route.fulfill({ status: 204 })
+      : route.fulfill({ status: 200, contentType: 'application/json', body }),
+  );
+  await open(page);
+
+  const splash = page.getByTestId('recovery-splash');
+  await expect(splash).toBeVisible({ timeout: 10_000 });
+  await expect(splash).toHaveAttribute('data-phase', 'countdown');
+  await expect(splash).toHaveAttribute('data-layout', 'box');
+  await expect(page.getByTestId('recovery-countdown')).toHaveText(/^0:4\d$/);
+  await expect(page.getByTestId('recovery-body')).toContainText('Rewinding to PEWTER CITY');
+  await expect(page.getByTestId('recovery-loop')).toContainText('GO OBJECTIVE > GO WARP');
+
+  body = JSON.stringify(notice({ phase: 'acting' }));
+  await expect(splash).toHaveAttribute('data-layout', 'cover', { timeout: 5_000 });
+
+  body = JSON.stringify(notice({ phase: 'done' }));
+  await expect(splash).toHaveAttribute('data-phase', 'done', { timeout: 5_000 });
+
+  // The file going away is also an end: the page never holds a notice the server no longer has.
+  body = null;
+  await expect(splash).toHaveCount(0, { timeout: 5_000 });
+});
+
+test('stale, malformed and missing notices never show', async ({ page }) => {
+  const bodies = [
+    JSON.stringify(notice({ updatedAt: now() - 16 * 60 })),
+    '{"v":1,"phase":"countdown"',
+    JSON.stringify(notice({ v: 2 })),
+    JSON.stringify(notice({ phase: 'done', updatedAt: now() - 60 })),
+  ];
+  let index = 0;
+  await page.route('**/recovery-notice.json', (route) => {
+    const body = bodies[Math.min(index, bodies.length - 1)] as string;
+    index += 1;
+    return route.fulfill({ status: 200, contentType: 'application/json', body });
+  });
+  await open(page);
+  await expect.poll(() => index, { timeout: 10_000 }).toBeGreaterThan(bodies.length);
+  await expect(page.getByTestId('recovery-splash')).toHaveCount(0);
+});
+
+test('player mode does not poll unless asked', async ({ page }) => {
+  let hits = 0;
+  await page.route('**/recovery-notice.json', (route) => {
+    hits += 1;
+    return route.fulfill({ status: 204 });
+  });
+  await page.goto(stageUrl({ t: 95, tab: 'senses', audio: false }));
+  await page.waitForSelector('html[data-ready="1"]', { timeout: 60_000 });
+  await page.waitForTimeout(2500);
+  expect(hits).toBe(0);
+});
+
+test('the box leaves the top of the game clear; the cover stays inside the game', async ({ page }) => {
+  await open(page);
+  const game = await page.getByTestId('game').boundingBox();
+  if (!game) throw new Error('no game box');
+  const t = 1_790_629_095;
+  const base = notice({ announcedAt: t, executeAt: t + 60, updatedAt: t });
+
+  await page.evaluate(([raw, nowS]) => window.__stage?.recovery(raw, nowS as number), [base, t + 18] as const);
+  // Measure after the 320 ms rise, not during it.
+  await page.locator('.recovery__box').evaluate((element) =>
+    Promise.all(element.getAnimations().map((animation) => animation.finished)).then(() => undefined),
+  );
+  const box = await page.locator('.recovery__box').boundingBox();
+  if (!box) throw new Error('no text box');
+  expect(box.y).toBeGreaterThan(game.y + game.height / 2);
+  expect(box.x).toBeGreaterThanOrEqual(game.x);
+  expect(box.x + box.width).toBeLessThanOrEqual(game.x + game.width);
+  expect(box.y + box.height).toBeLessThanOrEqual(game.y + game.height);
+  await expect(page.getByTestId('recovery-countdown')).toHaveText('0:42');
+
+  await page.evaluate(
+    ([raw, nowS]) => window.__stage?.recovery(raw, nowS as number),
+    [{ ...base, phase: 'acting', updatedAt: t + 60 }, t + 70] as const,
+  );
+  const cover = await page.locator('.recovery__cover').boundingBox();
+  if (!cover) throw new Error('no cover');
+  expect(cover).toEqual(game);
+
+  // Every run of text on the splash sits on the page's body floor.
+  const sizes = await page.evaluate(() =>
+    [...document.querySelectorAll('[data-testid="recovery-splash"] *')]
+      .filter((element) => [...element.childNodes].some((node) => node.nodeType === 3 && node.textContent?.trim()))
+      .map((element) => Number.parseFloat(getComputedStyle(element).fontSize)),
+  );
+  expect(sizes.length).toBeGreaterThan(3);
+  for (const size of sizes) expect(size).toBeGreaterThanOrEqual(24);
+
+  await page.evaluate(() => window.__stage?.recovery(null));
+  await expect(page.getByTestId('recovery-splash')).toHaveCount(0);
+});

apps/stage/tests/unit/recovery.test.ts

@@ -0,0 +1,261 @@
+/**
+ * The recovery splash's model, poller and server route.
+ *
+ * The one property every test here protects: the notice file can never break the stream. A
+ * missing, malformed, stale or hostile file is no splash, and a good one shows exactly the
+ * contract's lifetimes (countdown until acting, acting while flysim is down, done ~8 s, failed
+ * briefly) from the file alone.
+ */
+import assert from 'node:assert/strict';
+import { spawn } from 'node:child_process';
+import { mkdtemp, rm, writeFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { dirname, join, resolve } from 'node:path';
+import test from 'node:test';
+import { fileURLToPath } from 'node:url';
+
+import { parseStageOptions } from '../../src/lib/query';
+import {
+  RECOVERY_ACTING_MAX_S,
+  RECOVERY_DONE_S,
+  RECOVERY_FAILED_S,
+  RECOVERY_STALE_S,
+  cleanText,
+  formatCountdown,
+  formatStuck,
+  parseRecoveryBody,
+  parseRecoveryNotice,
+  recoveryView,
+} from '../../src/lib/recovery';
+import { RecoveryPoller } from '../../src/lib/recovery-poll';
+
+const T = 1_790_629_095;
+
+function notice(overrides: Record<string, unknown> = {}): Record<string, unknown> {
+  return {
+    v: 1,
+    id: `${T}-reset`,
+    phase: 'countdown',
+    action: 'reset',
+    fromRung: 12,
+    fromLabel: 'MT. MOON',
+    toRung: 11,
+    toLabel: 'PEWTER CITY',
+    reason: 'unrewarded',
+    loop: ['GO OBJECTIVE', 'GO WARP'],
+    stuckSeconds: 1800,
+    announcedAt: T,
+    executeAt: T + 60,
+    updatedAt: T,
+    ...overrides,
+  };
+}
+
+const view = (raw: Record<string, unknown>, now: number) => recoveryView(parseRecoveryNotice(raw), now);
+
+test('the contract example parses', () => {
+  const parsed = parseRecoveryNotice(notice());
+  assert.ok(parsed);
+  assert.equal(parsed.phase, 'countdown');
+  assert.equal(parsed.action, 'reset');
+  assert.equal(parsed.toLabel, 'PEWTER CITY');
+  assert.equal(parsed.fromLabel, 'MT. MOON');
+  assert.deepEqual(parsed.loop, ['GO OBJECTIVE', 'GO WARP']);
+});
+
+test('anything malformed is no notice at all', () => {
+  const bad: unknown[] = [
+    null,
+    42,
+    'x',
+    [],
+    notice({ v: 2 }),
+    notice({ v: '1' }),
+    notice({ id: '' }),
+    notice({ id: 7 }),
+    notice({ phase: 'panic' }),
+    notice({ action: 'reboot' }),
+    notice({ executeAt: 'soon' }),
+    notice({ updatedAt: Number.NaN }),
+    notice({ announcedAt: undefined }),
+  ];
+  for (const raw of bad) assert.equal(parseRecoveryNotice(raw), null, JSON.stringify(raw));
+  for (const body of ['', '{', 'null', '{"v":1}', 'x'.repeat(20_000)]) assert.equal(parseRecoveryBody(body), null);
+  assert.ok(parseRecoveryBody(JSON.stringify(notice())));
+});
+
+test('optional fields degrade the copy, not the notice', () => {
+  const bare = view({ v: 1, id: 'a', phase: 'countdown', action: 'reset', announcedAt: T, executeAt: T + 30, updatedAt: T }, T);
+  assert.ok(bare);
+  assert.equal(bare.body, 'Rewinding to the last milestone in');
+  assert.equal(bare.loop, '');
+  assert.equal(bare.stuck, '');
+});
+
+test('helper strings are clipped to a closed character set', () => {
+  assert.equal(cleanText('<img src=x onerror=alert(1)>', 24), 'img srcx onerroralert1');
+  assert.equal(cleanText('S.S. ANNE', 24), 'S.S. ANNE');
+  assert.equal(cleanText('  ROUTE\n\t22  ', 24), 'ROUTE 22');
+  assert.equal(cleanText('A'.repeat(100), 24).length, 24);
+  assert.equal(cleanText(12, 24), '');
+  const parsed = parseRecoveryNotice(notice({ loop: ['go objective', 5, '', 'A', 'B', 'C', 'D'], toLabel: 'pewter\u0000city' }));
+  assert.ok(parsed);
+  assert.deepEqual(parsed.loop, ['GO OBJECTIVE', 'A', 'B', 'C']);
+  assert.equal(parsed.toLabel, 'PEWTERCITY');
+  // A restart has no destination, whatever the file says.
+  assert.equal(parseRecoveryNotice(notice({ action: 'restart' }))?.toLabel, '');
+});
+
+test('countdown: the text box with M:SS to executeAt, reset and restart copy differ', () => {
+  const reset = view(notice(), T + 18);
+  assert.ok(reset);
+  assert.equal(reset.layout, 'box');
+  assert.equal(reset.countdown, '0:42');
+  assert.equal(reset.headline, 'The fly is stuck in a loop!');
+  assert.equal(reset.body, 'Rewinding to PEWTER CITY in');
+  assert.equal(reset.loop, 'GO OBJECTIVE > GO WARP');
+  assert.equal(reset.stuck, 'STUCK 30 MIN');
+
+  const restart = view(notice({ action: 'restart' }), T);
+  assert.ok(restart);
+  assert.equal(restart.body, 'Shaking it off in');
+  assert.equal(restart.countdown, '1:00');
+
+  // Past executeAt but not yet acting: say "now", never a negative clock.
+  const late = view(notice(), T + 75);
+  assert.ok(late);
+  assert.equal(late.countdown, '0:00');
+  assert.equal(late.body, 'Rewinding to PEWTER CITY now');
+  assert.equal(late.busy, true);
+});
+
+test('acting: covers the game while flysim is down, and gives up after a cap', () => {
+  const acting = notice({ phase: 'acting', updatedAt: T + 60 });
+  const reset = view(acting, T + 90);
+  assert.ok(reset);
+  assert.equal(reset.layout, 'cover');
+  assert.equal(reset.headline, 'Rewinding');
+  assert.equal(reset.body, 'Back to PEWTER CITY');
+  const restart = view({ ...acting, action: 'restart' }, T + 90);
+  assert.equal(restart?.headline, 'Shaking it off');
+  assert.equal(view(acting, T + 60 + RECOVERY_ACTING_MAX_S + 1), null);
+});
+
+test('done shows about 8 s, failed a little longer, then both clear', () => {
+  const done = notice({ phase: 'done', updatedAt: T + 120 });
+  assert.equal(view(done, T + 120)?.headline, 'Back at PEWTER CITY!');
+  assert.equal(view({ ...done, action: 'restart' }, T + 121)?.headline, 'All shaken off!');
+  assert.ok(view(done, T + 120 + RECOVERY_DONE_S));
+  assert.equal(view(done, T + 120 + RECOVERY_DONE_S + 1), null);
+
+  const failed = notice({ phase: 'failed', updatedAt: T + 120 });
+  assert.equal(view(failed, T + 125)?.body, 'A human will take a look');
+  assert.equal(view(failed, T + 120 + RECOVERY_FAILED_S + 1), null);
+});
+
+test('a notice older than 15 minutes, or from the far future, is ignored', () => {
+  assert.ok(view(notice(), T + RECOVERY_STALE_S));
+  assert.equal(view(notice(), T + RECOVERY_STALE_S + 1), null);
+  assert.equal(view(notice(), T - 3600), null);
+});
+
+test('formatting', () => {
+  assert.equal(formatCountdown(59.2), '1:00');
+  assert.equal(formatCountdown(5), '0:05');
+  assert.equal(formatCountdown(725), '12:05');
+  assert.equal(formatCountdown(-3), '0:00');
+  assert.equal(formatStuck(null), '');
+  assert.equal(formatStuck(30), '');
+  assert.equal(formatStuck(1800), 'STUCK 30 MIN');
+  assert.equal(formatStuck(7200), 'STUCK 2 H');
+  assert.equal(formatStuck(3900), 'STUCK 1 H 5 MIN');
+});
+
+test('the page polls in live mode, not in player mode, unless told', () => {
+  assert.equal(parseStageOptions('?mode=live').recovery, true);
+  assert.equal(parseStageOptions('').recovery, false);
+  assert.equal(parseStageOptions('?recovery=1').recovery, true);
+  assert.equal(parseStageOptions('?mode=live&recovery=0').recovery, false);
+});
+
+test('the poller turns every failure into "no notice" and reports changes once', async () => {
+  const seen: (string | null)[] = [];
+  let reply: () => Promise<Response> = () => Promise.resolve(new Response(JSON.stringify(notice()), { status: 200 }));
+  const poller = new RecoveryPoller({
+    fetch: () => reply(),
+    onChange: (n) => seen.push(n === null ? null : n.phase),
+  });
+
+  assert.equal((await poller.poll())?.phase, 'countdown');
+  await poller.poll(); // unchanged: no second callback
+  reply = () => Promise.resolve(new Response(null, { status: 204 }));
+  assert.equal(await poller.poll(), null);
+  reply = () => Promise.resolve(new Response('{"v":1,', { status: 200 }));
+  assert.equal(await poller.poll(), null);
+  reply = () => Promise.resolve(new Response('not found', { status: 404 }));
+  assert.equal(await poller.poll(), null);
+  reply = () => Promise.reject(new Error('ECONNREFUSED'));
+  assert.equal(await poller.poll(), null);
+  reply = () => Promise.resolve(new Response(JSON.stringify(notice({ phase: 'acting' })), { status: 200 }));
+  assert.equal((await poller.poll())?.phase, 'acting');
+
+  assert.deepEqual(seen, ['countdown', null, 'acting']);
+});
+
+test('the poller survives a throwing listener', async () => {
+  const poller = new RecoveryPoller({
+    fetch: () => Promise.resolve(new Response(JSON.stringify(notice()), { status: 200 })),
+    onChange: () => {
+      throw new Error('boom');
+    },
+  });
+  const warn = console.warn;
+  console.warn = () => {};
+  try {
+    assert.equal((await poller.poll())?.phase, 'countdown');
+  } finally {
+    console.warn = warn;
+  }
+});
+
+test('flystage-web serves the notice file at /recovery-notice.json, 204 without one', async () => {
+  const here = dirname(fileURLToPath(import.meta.url));
+  const serve = resolve(here, '../../../../infra/config/serve.mjs');
+  const dir = await mkdtemp(join(tmpdir(), 'recovery-notice-'));
+  const file = join(dir, 'recovery-notice.json');
+  const child = spawn(process.execPath, [serve, dir, '127.0.0.1', '0'], {
+    env: { ...process.env, FLY_RECOVERY_NOTICE: file },
+    stdio: ['ignore', 'pipe', 'inherit'],
+  });
+  try {
+    const base = await new Promise<string>((done, fail) => {
+      let out = '';
+      child.stdout.on('data', (chunk: Buffer) => {
+        out += chunk.toString();
+        const match = /(http:\/\/127\.0\.0\.1:\d+)\//.exec(out);
+        if (match?.[1]) done(match[1]);
+      });
+      child.on('exit', () => fail(new Error(`serve.mjs exited: ${out}`)));
+    });
+
+    const missing = await fetch(`${base}/recovery-notice.json`);
+    assert.equal(missing.status, 204);
+    assert.equal(missing.headers.get('cache-control'), 'no-store');
+
+    const body = JSON.stringify(notice());
+    await writeFile(file, body);
+    const present = await fetch(`${base}/recovery-notice.json?t=1`);
+    assert.equal(present.status, 200);
+    assert.equal(await present.text(), body);
+
+    await writeFile(file, 'x'.repeat(20_000));
+    assert.equal((await fetch(`${base}/recovery-notice.json`)).status, 204, 'oversized files are not served');
+
+    // The static half still works, and the notice is not reachable any other way.
+    await writeFile(join(dir, 'index.html'), '<p>stage</p>');
+    assert.equal(await (await fetch(`${base}/`)).text(), '<p>stage</p>');
+  } finally {
+    child.kill('SIGTERM');
+    await rm(dir, { recursive: true, force: true });
+  }
+});

apps/stage/tools/mockup.mts

@@ -24,6 +24,8 @@
  *     macros-center.png          wide pad with both of its columns filled
  *     macros-tab.png
  *     pad-strip.png
+ *     recovery-{countdown,acting,done,failed}-{reset,restart}.png
+ *                                the auto-recovery splash, each phase and both actions
  *
  * T1 Instrument is the chosen theme (`docs/stream-mvp-plan.md`, decisions 2026-09-15 evening), so
  * the theme sweep the first gate needed is gone: what these images are for now is the *layout* and
@@ -142,6 +144,34 @@ const MOMENTS: { name: string; type: string; label: string; detail: string; tab?
   { name: 'moment-rollback', type: 'rollback', label: 'Rolled back to the archived frame', detail: 'try 3' },
 ];
 
+/**
+ * The recovery splash shots (`src/panels/RecoverySplash.tsx`): every phase for both actions,
+ * driven through `window.__stage.recovery` with the splash's clock pinned 18 s into the countdown,
+ * so the clock reads 0:42 on every machine. The notice is the contract's own example.
+ */
+const RECOVERY_T = 1_790_629_095;
+const RECOVERY_SHOTS = (['countdown', 'acting', 'done', 'failed'] as const).flatMap((phase) =>
+  (['reset', 'restart'] as const).map((action) => ({
+    name: `recovery-${phase}-${action}`,
+    nowS: phase === 'countdown' ? RECOVERY_T + 18 : RECOVERY_T + 63,
+    notice: {
+      v: 1,
+      id: `${RECOVERY_T}-${action}`,
+      phase,
+      action,
+      fromRung: 12,
+      fromLabel: 'MT. MOON',
+      ...(action === 'reset' ? { toRung: 11, toLabel: 'PEWTER CITY' } : {}),
+      reason: 'unrewarded',
+      loop: ['GO OBJECTIVE', 'GO WARP'],
+      stuckSeconds: 1800,
+      announcedAt: RECOVERY_T,
+      executeAt: RECOVERY_T + 60,
+      updatedAt: phase === 'countdown' ? RECOVERY_T : RECOVERY_T + 60,
+    },
+  })),
+);
+
 function run(command: string, args: string[]): Promise<void> {
   return new Promise((done, fail) => {
     const child = spawn(command, args, { cwd: appDir, stdio: 'inherit' });
@@ -264,6 +294,23 @@ async function main(): Promise<void> {
       console.log(`  ${path}  (${String(state?.moment)} ${String(state?.momentPhase)}, ${String(state?.particles)} particles, tab ${String(state?.tab)})`);
     }
 
+    for (const shot of RECOVERY_SHOTS) {
+      if (!wanted(shot.name)) continue;
+      await page.goto(url(baseUrl, { fixture: 'steady', t: 95, tab: 'senses' }));
+      await settle(page);
+      const shown = await page.evaluate(([notice, nowS]) => window.__stage?.recovery(notice, nowS as number) ?? null, [
+        shot.notice,
+        shot.nowS,
+      ] as const);
+      if (shown === null) throw new Error(`${shot.name}: the splash refused its notice`);
+      // The text box rises in over 320 ms (`src/theme/recovery.css`).
+      await page.waitForTimeout(500);
+      const path = resolve(outDir, `${shot.name}.png`);
+      await page.screenshot({ path, clip: FRAME });
+      written.push(path);
+      console.log(`  ${path}  (${shown.layout}: ${shown.headline} / ${shown.body} ${shown.countdown})`);
+    }
+
     // Two extra review shots: the fly strip alone, at 2x, once per renderer. The strip is 800x220
     // in a 1920x1080 frame, which is too small to judge an animal in; these are what a human looks
     // at to decide whether it reads.

apps/stage/tools/recovery-notice.mts

@@ -0,0 +1,92 @@
+/**
+ * Write a fake auto-recovery notice, for looking at the recovery splash by hand.
+ *
+ * The file is the one `infra/bin/fly-loop-recover` writes (contract: `infra/docs/loop-recovery.md`,
+ * "Stream notice"), written the same way — a temporary file renamed over the target, mode 0644 —
+ * to `--out`, else `$FLY_RECOVERY_NOTICE`. Point the page's server at the same path:
+ *
+ * ```sh
+ * export FLY_RECOVERY_NOTICE=$PWD/.recovery-notice.json   # any writable path
+ * npm run dev -w @flybrain/stage                           # serves it at /recovery-notice.json
+ * #   http://127.0.0.1:5273/?mode=player&recovery=1
+ *
+ * npm run recovery-notice -w @flybrain/stage -- --phase countdown --action reset --in 45
+ * npm run recovery-notice -w @flybrain/stage -- --phase acting --action restart
+ * npm run recovery-notice -w @flybrain/stage -- --demo reset   # countdown 20 s, acting 15 s, done
+ * npm run recovery-notice -w @flybrain/stage -- --clear
+ * ```
+ *
+ * On the release container the deployed `serve.mjs` reads `/run/fly/wd/recovery-notice.json`
+ * unless its unit sets `FLY_RECOVERY_NOTICE`; this tool is for a dev machine, not for faking a
+ * recovery on air.
+ */
+import { chmod, rename, rm, writeFile } from 'node:fs/promises';
+import { dirname, join } from 'node:path';
+
+type Phase = 'countdown' | 'acting' | 'done' | 'failed';
+type Action = 'restart' | 'reset';
+
+function arg(name: string): string | null {
+  const at = process.argv.indexOf(`--${name}`);
+  return at === -1 ? null : (process.argv[at + 1] ?? null);
+}
+
+const out = arg('out') ?? process.env.FLY_RECOVERY_NOTICE ?? '';
+if (out === '') {
+  console.error('recovery-notice: pass --out <path> or set FLY_RECOVERY_NOTICE');
+  process.exit(2);
+}
+
+async function write(phase: Phase, action: Action, id: string, announcedAt: number, executeAt: number): Promise<void> {
+  const notice = {
+    v: 1,
+    id,
+    phase,
+    action,
+    fromRung: Number(arg('from-rung') ?? 12),
+    fromLabel: arg('from-label') ?? 'MT. MOON',
+    ...(action === 'reset' ? { toRung: Number(arg('to-rung') ?? 11), toLabel: arg('to-label') ?? 'PEWTER CITY' } : {}),
+    reason: arg('reason') ?? 'unrewarded',
+    loop: (arg('loop') ?? 'GO OBJECTIVE,GO WARP').split(',').map((step) => step.trim()),
+    stuckSeconds: Number(arg('stuck') ?? 1800),
+    announcedAt,
+    executeAt,
+    updatedAt: Math.floor(Date.now() / 1000),
+  };
+  const tmp = join(dirname(out), `.recovery-notice.${process.pid}.tmp`);
+  await writeFile(tmp, `${JSON.stringify(notice, null, 2)}\n`);
+  await chmod(tmp, 0o644);
+  await rename(tmp, out);
+  console.log(`${phase} ${action} -> ${out}`);
+}
+
+const sleep = (s: number): Promise<void> => new Promise((done) => setTimeout(done, s * 1000));
+
+async function main(): Promise<void> {
+  if (process.argv.includes('--clear')) {
+    await rm(out, { force: true });
+    console.log(`removed ${out}`);
+    return;
+  }
+  const now = Math.floor(Date.now() / 1000);
+  const demo = arg('demo');
+  if (demo !== null) {
+    const action = (demo === 'restart' ? 'restart' : 'reset') as Action;
+    const id = `${now}-${action}`;
+    await write('countdown', action, id, now, now + 20);
+    await sleep(20);
+    await write('acting', action, id, now, now + 20);
+    await sleep(15);
+    await write(process.argv.includes('--fail') ? 'failed' : 'done', action, id, now, now + 20);
+    return;
+  }
+  const phase = (arg('phase') ?? 'countdown') as Phase;
+  const action = (arg('action') ?? 'reset') as Action;
+  const lead = Number(arg('in') ?? 60);
+  await write(phase, action, `${now}-${action}`, now, now + lead);
+}
+
+main().catch((error: unknown) => {
+  console.error(error);
+  process.exit(1);
+});

apps/stage/vite.config.ts

@@ -16,7 +16,8 @@
  */
 import { execFileSync } from 'node:child_process';
 import { createReadStream } from 'node:fs';
-import { cp, stat } from 'node:fs/promises';
+import { cp, readFile, stat } from 'node:fs/promises';
+import type { IncomingMessage, ServerResponse } from 'node:http';
 import { dirname, join, normalize, resolve } from 'node:path';
 import { fileURLToPath } from 'node:url';
 import tailwindcss from '@tailwindcss/vite';
@@ -87,8 +88,41 @@ function datasetArtifacts(): Plugin {
   };
 }
 
+/**
+ * `GET /recovery-notice.json` in dev and preview, the same route `infra/config/serve.mjs` gives
+ * the deployed page: the auto-recovery helper's notice file (`FLY_RECOVERY_NOTICE`, default
+ * `/run/fly/wd/recovery-notice.json`), or 204 when there is none. What makes
+ * `npm run recovery-notice` (`tools/recovery-notice.mts`) work against a local page.
+ */
+function recoveryNotice(): Plugin {
+  const serve = async (req: IncomingMessage, res: ServerResponse, next: () => void): Promise<void> => {
+    if ((req.url ?? '').split('?')[0] !== '/recovery-notice.json') return next();
+    const path = process.env.FLY_RECOVERY_NOTICE || '/run/fly/wd/recovery-notice.json';
+    res.setHeader('cache-control', 'no-store');
+    try {
+      const info = await stat(path);
+      if (!info.isFile() || info.size === 0 || info.size > 16_384) throw new Error('no notice');
+      const body = await readFile(path);
+      res.setHeader('content-type', 'application/json; charset=utf-8');
+      res.end(body);
+    } catch {
+      res.statusCode = 204;
+      res.end();
+    }
+  };
+  return {
+    name: 'flystage-recovery-notice',
+    configureServer(server) {
+      server.middlewares.use((req, res, next) => void serve(req, res, next));
+    },
+    configurePreviewServer(server) {
+      server.middlewares.use((req, res, next) => void serve(req, res, next));
+    },
+  };
+}
+
 export default defineConfig(({ command }) => ({
-  plugins: [react(), tailwindcss(), datasetArtifacts()],
+  plugins: [react(), tailwindcss(), datasetArtifacts(), recoveryNotice()],
   define: {
     __STAGE_VERSION__: JSON.stringify(stageVersion(command)),
   },

infra/config/serve.mjs

@@ -9,6 +9,14 @@
 // on flystage-web.service). Deliberately not a general-purpose server:
 // no directory listing, no symlink following outside root, no range
 // requests (the page is a handful of static assets, not video).
+//
+// One route is not a file under root: GET /recovery-notice.json returns the
+// auto-recovery helper's notice (infra/bin/fly-loop-recover writes it to
+// /run/fly/wd/recovery-notice.json, FLY_RECOVERY_NOTICE overrides), which the
+// page's recovery splash polls once a second. It is served from here and not
+// from flysim because flysim is the thing being restarted. 200 with the file's
+// bytes, or 204 when there is no (readable, small, regular) file; the page does
+// all of the validation (apps/stage/src/lib/recovery.ts).
 import { createServer } from 'node:http';
 import { readFile, stat } from 'node:fs/promises';
 import { join, normalize, extname } from 'node:path';
@@ -17,6 +25,27 @@ const root = process.argv[2] ?? new URL('.', import.meta.url).pathname;
 const host = process.argv[3] ?? '127.0.0.1';
 const port = Number(process.argv[4] ?? 7402);
 
+const RECOVERY_ROUTE = '/recovery-notice.json';
+const RECOVERY_PATH = process.env.FLY_RECOVERY_NOTICE || '/run/fly/wd/recovery-notice.json';
+const RECOVERY_MAX_BYTES = 16384;
+
+async function serveRecoveryNotice(res) {
+  const headers = { 'cache-control': 'no-store' };
+  const st = await stat(RECOVERY_PATH).catch(() => null);
+  if (!st?.isFile() || st.size === 0 || st.size > RECOVERY_MAX_BYTES) {
+    res.writeHead(204, headers).end();
+    return;
+  }
+  const body = await readFile(RECOVERY_PATH).catch(() => null);
+  if (!body) { res.writeHead(204, headers).end(); return; }
+  res.writeHead(200, {
+    ...headers,
+    'content-type': 'application/json; charset=utf-8',
+    'content-length': body.length,
+  });
+  res.end(body);
+}
+
 const TYPES = {
   '.html': 'text/html; charset=utf-8', '.js': 'text/javascript; charset=utf-8',
   '.mjs': 'text/javascript; charset=utf-8', '.css': 'text/css; charset=utf-8',
@@ -27,6 +56,7 @@ const TYPES = {
 
 const server = createServer(async (req, res) => {
   try {
+    if (req.url.split('?')[0] === RECOVERY_ROUTE) { await serveRecoveryNotice(res); return; }
     const urlPath = normalize(decodeURIComponent(req.url.split('?')[0]));
     if (urlPath.includes('..')) { res.writeHead(400).end('bad path'); return; }
     let path = join(root, urlPath === '/' ? 'index.html' : urlPath);
@@ -47,5 +77,5 @@ const server = createServer(async (req, res) => {
 });
 
 server.listen(port, host, () => {
-  console.log(`flystage-web: serving ${root} on http://${host}:${port}/`);
+  console.log(`flystage-web: serving ${root} on http://${host}:${server.address().port}/`);
 });

infra/docs/loop-recovery.md

@@ -63,8 +63,13 @@ reads. It holds the contract below, written atomically, mode 0644:
  "announcedAt": 1790629095, "executeAt": 1790629155, "updatedAt": 1790629160}
 ```
 
-`toRung`/`toLabel` are present for a reset only. Consumers ignore a notice whose `updatedAt` is
-more than 15 minutes old.
+`toRung`/`toLabel` are present for a reset only; every write refreshes `updatedAt`, and the
+helper never deletes the file. `flystage-web` serves it at `/recovery-notice.json` (204 when
+absent) and the page's recovery splash polls it once a second (`apps/stage/README.md`, "Recovery
+splash"). The page ignores a notice more than 15 minutes old, shows `done` for 8 s and `failed`
+for 20 s, and stops covering the game 10 minutes after an `acting` write that nothing followed.
+A milestone step pauses the watchdog, so Chromium is not restarted under the splash; a plain
+restart is short enough that a watchdog pass rarely lands in it.
 
 ## Model confirmation