# infra/units/flybridge.service — pushed to /etc/systemd/system/flybridge.service. # docs/design/infra.md section 3: "flybridge.service Wants, never Requires # (chat must not be able to stop the sim)" — see infra/units/fly.target, # which lists this unit in Wants= only. # # ExecStart runs the BUILT bundle, not the TypeScript sources. # services/bridge is ESM TypeScript with extensionless relative imports and a # workspace dependency (@flybrain/feed) whose package `main` is itself a .ts # file, so there is nothing here `node` can execute directly and `tsc` cannot # emit anything it could either (its output would keep the extensionless # specifiers, which Node's ESM resolver rejects). `npm run build -w # @flybrain/bridge` therefore bundles src/index.ts with esbuild into one # ES module, dist/index.js, which infra/build/build-bridge.sh lays out as the # bridge/ directory that infra/build/package-release.sh ships and this line # executes. `npm start` (tsx) stays the dev path and is deliberately NOT what # runs in production: it would put a TypeScript compiler in the process that # holds the channel's chat connection. [Unit] Description=flybridge: Twitch chat and Channel Points bridge After=network-online.target flysim.service Wants=network-online.target # Before the bridge is built and authorized there is neither an index.js, a # twitch-app credential nor a tokens.json, and without these conditions this # unit restart-looped every 10 s forever: # `Failed at step CREDENTIALS spawning /usr/local/bin/node: Protocol error` # (LoadCredentialEncrypted= on a credential file that does not exist), twice # in the journal per attempt, 6 attempts a minute, on every container, for # weeks. systemd's default start limit never catches it because RestartSec=10 # spaces the attempts wider than DefaultStartLimitIntervalSec. That buries the # journal — which is the watchdog's only notification channel # (docs/design/infra.md section 3, "Notification") — in the one noise source # guaranteed to be present. Measured on the P0 spike run 2. # # Conditions, not a start limit: a failed Condition leaves the unit cleanly # inactive with no error and no restart attempts, and the moment the release # carries a built bridge, 06-secrets.sh has installed the credential and the # tokens file is in place, it starts normally. A start limit would have been # the wrong tool — flypush's own header explains why an outage must not # permanently defeat restarts, and the same argument applies here. ConditionPathExists=/opt/fly/current/bridge/index.js ConditionPathExists=/etc/fly/creds/twitch-app.cred # The tokens file is the third thing that cannot be synthesised: src/auth.ts # refuses to start without it and would restart-loop on the same 10 s cadence. # This path tracks TOKENS_FILE, which 05-deploy.sh writes into /etc/fly/fly.env # from the same default — if you ever point TOKENS_FILE somewhere else, move # this condition with it or the unit will stay silently inactive. ConditionPathExists=/var/lib/flybridge/tokens.json # The bridge now exits ITSELF (code 75) when its channel.chat.message EventSub # subscription has been missing for EVENTSUB_GRACE_MS — see # services/bridge/src/subscription-health.ts and the 2026-09-16 incident it # records: the sockets dropped, Twitch refused the re-created subscription with # "number of websocket transports limit exceeded", and the process sat there # active (running), NRestarts=0, with a dead channel for an hour. # # That makes the restart path load-bearing, so the start limit is disabled # outright rather than left at the default. A bad half hour on Twitch's side can # legitimately produce a dozen exits in a row, and DefaultStartLimitBurst=5 # would park the unit in `failed` state exactly when the channel needs it most — # the same argument flypush's own header makes about an outage never being # allowed to defeat restarts permanently. Nothing here loops for free either: # each attempt costs the 60 s grace plus RestartSec, i.e. one attempt about # every 75 s, each leaving one clear FATAL line in the journal. StartLimitIntervalSec=0 [Service] Type=simple User=fly LoadCredentialEncrypted=twitch-app:/etc/fly/creds/twitch-app.cred # CHANNEL, BOT_USER, GAME_TITLE, TOKENS_FILE, SIM_CONTROL_URL and the FEATURE_* # flags all come from here (generated by 05-deploy.sh out of the env file # source). services/bridge/src/config.ts collects every missing one and refuses # to start naming all of them at once. EnvironmentFile=/etc/fly/fly.env Environment=NODE_ENV=production # /var/lib/flybridge, owned by this unit's User=, for tokens.json and the # redemption intent log. systemd creates and chowns it on every start, so it # survives a container restart and needs no provisioning step; the tokens file # itself is pushed by hand (a secret, never in a release artifact). StateDirectory=flybridge StateDirectoryMode=0700 ExecStart=/usr/local/bin/node /opt/fly/current/bridge/index.js Restart=always # 15 s, not 10, and the number is a measurement rather than a taste: Twitch had # not yet reaped the stale websocket transports when twurple re-subscribed one # second after the 1006 disconnect on 2026-09-16, and refused again ten seconds # later. The self-healing exit exists to come back with a FRESH transport, so # the restart has to wait long enough for the old ones to be gone. Do not lower # it below 15 s without re-measuring against the per-user transport limit of 3. RestartSec=15 MemoryMax=512M [Install] WantedBy=fly.target