flybrain/docs/design/room-escape.md

Room escape: straighter runs and boundary rewards

Decided 2026-09-16 (the operator): the fly struggles with rooms. Two levers, both honest under the doctrine (buttons always the fly's; readout fixed and global; rewards are design):

1. Straighter runs (readout)

The Game Boy preset commits to a direction for 400 ms with hysteresis 1.15 and fatigue 0.08/0.8. One overworld step is about 270 ms, so runs are one to two steps and the walker jitters. Raise the direction hold and decision period together (candidates 600, 800, 1000 ms), keep hysteresis, and lower fatigue gain so a run can persist (0.04). Choose by measurement, then bake the value into the preset as gameboyDecoderConfig() (TS and Rust twins, golden regenerated). Decoder version note and the compatibility string are unaffected (readout state is not in the neural checkpoint) but the readout doc records the change and the date.

Measurement: the random-walker harness (services/flysim/crates/flysim/examples/random_walker.rs pattern, on the Pokémon ROM) started from the archived bedroom snapshot and from the house 1F snapshot, 20 seeded runs per hold value, 10 brain minutes each, unthrottled. Report median time to leave the map and the fraction that leave at all. Then confirm with the real brain for the chosen value (5 runs) so the number is not a random-walker artefact.

2. Boundary rewards (adapter)

Pay a small bonus the first time per map the fly stands on a tile adjacent to a warp (door, stairs, map edge exit) and again the first time it stands on the warp tile itself. Warps come from the current map's warp table in WRAM (wNumberOfWarps, wWarpEntries at the pinned pokered commit; verify names and layout: y, x, destination warp id, destination map). Map-edge exits are the connection tiles (wMapConnections); pay the same for the first step onto a route boundary tile. Values: adjacent 0.05, on-warp 0.10, capped once per (map, warp) for the lifetime ledger so oscillating cannot farm. Kind boundary, feed reward kind mapping explore (no new feed kind), label "found an exit". Adapter version pokered-unique8-v5; old checkpoints refused as before, and the deploy-time compatibility check archives state.

Doctrine note for the honesty panel and README: the reward catalog now includes exits; still no button path, still no map knowledge in the readout.

3. v0.1.1: the ground floor, diagnosed from the live fly

Decided 2026-09-16 (the operator), after the release box (the release container, twitch.tv/<twitch-channel>, v0.1.0) spent forty minutes on rung 2 with the whole of section 2's boundary reward already collected: "rework reward and/or movement and push out 0.1.1".

Section 1's measurement was a random walker from a freshly booted cartridge. That instrument cannot see this bug, and said so at the time: every candidate left both rooms in every one of 480 runs, so the leave fraction saturated and only the time to leave could be compared. It also measured the wrong thing, which nobody noticed: on the ground floor "left the map" is the staircase, which is one step and lands in the other room of the same house.

The v0.1.1 measurement therefore starts from the live fly instead — generation 525 of the release run, pulled read-only off the release container — through FLY_ESCAPE_CHECKPOINT in the same harness, and reports leaving the house. That state carries what a fresh boot does not: 43 brain minutes of plasticity, a calibrated baseline the network has since drifted away from, and a reward ledger with every payout in the house already spent. infra/docs/room-escape.md holds the numbers.

What the room actually is

survey in the harness walks map 37 with real button presses on throwaway emulators. Red's ground floor has 48 reachable tiles, and exactly six presses leave it:

from	press	to
(6, 1)	right	map 38, the bedroom
(7, 2)	up	map 38, the bedroom
(2, 6)	down	map 0, Pallet Town
(3, 6)	down	map 0, Pallet Town
(2, 7)	down	map 0, Pallet Town
(3, 7)	down	map 0, Pallet Town

Two facts follow that the design did not have when section 2 was written, and both matter more than anything in it:

• Leaving the map is not leaving the house. Two of the six exits are the staircase, which warps the instant it is stepped on and lands the fly in a room it has also exhausted.
• The front door needs DOWN, and only DOWN. The two doormats are on the bottom row and the two tiles above them are walkable, so four tiles are one press from outside; on every one of them that press is down. A sideways step onto a mat does nothing, which is why the live ledger holds boundary:37:7:2:on and boundary:37:7:3:on — paid 0.10 each, at 355 s — from a fly that never went outside.

The four hypotheses, answered

• (a) A persistent rate bias makes one direction win almost always, and fatigue 0.04 lets it stick. Half right, and the half that is wrong matters. There is a persistent bias — down won 14 to 18% of decisions against left's 32 to 34%, run after run — but no direction wins almost always, because the bias is small and fatigue does rotate the winner every four decisions. The bias is a standing preference order, not a lock. What made it fatal is that the direction at the bottom of the order is the only one that opens the door.
• (b) The door needs DOWN from the tile above the mat while the fly's DOWN drive is weak. Yes, and it is the proximate cause. Four tiles, one press, and that press is the least likely of the four. Over fifteen ten-minute runs the fly spent 123 frames of 537,000 — 0.02% of its life — on the two tiles above the mats.
• (c) Hysteresis 1.15 plus long holds create wall-hugging loops. Yes, and it is the biggest number in the report: nearly two of every three 800 ms holds moved the player not one tile, for up to 34 seconds at a stretch. The lock is arithmetic — the four direction scores sit inside a 9% spread and a challenger needs 15% — so the winner could only ever be changed by habituation. The fly orbited the walls, and the staircase is on that orbit while the front door is not.
• (d) The stall rollback resets to the same room and wastes attempts. Not what happened. The ratchet fired zero recoveries in 43 live brain minutes and zero in 150 measured ones, with 0 of 3 attempts spent at rank 2. The stall window never matures: Ratchet::observe pushes lastProgress forward on any unsafe stretch longer than unsafeResetMs (1 s), and a fly walking and bumping produces those constantly. The objection is still true in principle — the archived snapshot for rank 2 is a ground-floor state — but it is not this bug and nothing here changes it. Recorded as a known limitation.

The fix

Three numbers in the Game Boy preset, all readout, no reward rule and no button path:

	v0.1.0	v0.1.1
holdMs = decisionMs	800	800
fatigueGain	0.04	0.08
hysteresis	1.15	1.05
blockedFatigue / blockedMs	off	0.35 / 800

Out of the house within ten brain minutes, fifteen runs each from generation 525: v0.1.0 got 1 out, this gets 14, and 13 of those inside five minutes against 0. infra/docs/room-escape.md has the candidate table, including the two that were tried and rejected (hysteresis 1.00, and hold 600) and the interaction that decides the third number: the blocked-direction cooldown made things worse at hysteresis 1.15 and is decisive at 1.05.

The blocked-direction cooldown is the one new mechanism, and it is deliberately the narrowest input that fixes a wall bump: the sim loop tells the readout the name of a direction it has watched produce no movement for a whole hold, and the readout habituates that channel. It says "that button did nothing" — not where the fly is, not where the door is, not which button would have worked. docs/readout.md specifies it and docs/rewards-learning.md discloses it on the honesty panel.

What was not changed, and why

The reward side. Section 2's boundary rule pays once per exit per lifetime, and the design's own v0.1.1 candidate was a small decaying bonus for each new visit to a warp-adjacent tile after 60 s. It was not needed and, on these numbers, would not have worked: the fly spent 0.02% of its frames on a door-approach tile, so a repeat bonus would have paid at most once or twice in a ten-minute run — too sparse for the eligibility traces to build anything from, and the runs that most needed help never got near the door at all. The readout change fixes what was broken, and a reward gradient is worth reconsidering now that the fly reliably reaches the tiles it could be paid on. REWARD_ADAPTER stays at pokered-unique8-v5 and STATE_VERSION at 4.

For the deploy: because readout state is not part of the neural checkpoint and the adapter version is untouched, the compatibility string does not move. v0.1.1 restores the live fly's generation 525 and simply decodes on the new numbers. The run does not reset and the ladder does not rebaseline.

The direction-run cap. The design listed "after N consecutive holds of the same direction, that direction's fatigue jumps" as a candidate. It is already what fatigueGain does, one linear step at a time instead of one jump, and unlike a counter it needs no new field in DecoderState and no version bump. Raising the gain from 0.04 to 0.08 halves N, which is the same lever with none of the cost.

Recalibration. calibrate() runs once, at warm-up, so the entire 9% spread between the four direction scores is drift since then, frozen in. Re-tying them periodically would dissolve the preference order at its root, but running once is what makes the readout fixed (docs/readout.md, "The readout is fixed") — a doctrine commitment, not a tuning knob. Left for The operator; docs/limitations.md records it.

Verification

Synthetic WRAM traces: warp table parsing, adjacency, once-per-warp, edge exits, no payout inside a map already baselined; ROM-gated: from the bedroom archive, the first payouts are boundary events near the stairs. Random-walker report committed under infra/docs/room-escape.md with the chosen hold value.

For section 3: the blocked-direction cooldown has unit tests on both decoder twins (one report unseats an incumbent that habituation would have taken three more decisions to shift; repeated reports inside one hold are one penalty; a report never lowers fatigue and is capped at 1; a report is ignored when the rule is off or names a channel outside the group), and GameAdapter::location has one on the adapter (the same tile in two rooms is two locations, which is what stops a warp reading as standing still). The preset's own numbers are pinned separately, in flybrain-core/tests/decoder.rs and packages/brain/tests/readout.test.ts, so a preset edit fails a test. The live-state measurement itself is not a test: it needs a checkpoint that is not in the repository and a quarter of an hour, and it is reproduced by hand from infra/docs/room-escape.md.