flybrain/infra/docs/macros-bench.md

Macro palette: the smoke run

Measured 2026-09-16 on the WSL development box for docs/design/macros.md section 7.

This is not the gate. Section 7 asks for 6 brain hours per arm from the same archived checkpoint, three runs, with palette mode shipping as the default only if it reaches rung 6 or higher in fewer brain hours in 3 of 3. This is one run of one brain hour per arm from a fresh boot, because there is no live checkpoint on this box: the release box's durable state has not been pulled here, and FLY_MACRO_CHECKPOINT was therefore unset. It establishes that the wiring works and what it costs; it decides nothing.

What was run

services/flysim/crates/flysim/examples/palette_bench.rs at feat/macros-simloop, on the pinned Pokémon Red cartridge, unthrottled, both arms side by side with 4 sweep threads each.

FLY_ROM=".../Pokemon Red (U) [S][BF].gb" FLY_MACRO_BRAIN=data/fafb-v783 \
  FLY_MACRO_HOURS=1 FLY_MACRO_DIGEST=1 FLY_MACRO_THREADS=4 \
  cargo run --release -p flysim --example palette_bench

Both arms are the sim loop's own frame order over the real connectome (data/fafb-v783), the real Game Boy readout preset with nothing overridden, the real Pokémon adapter paying the real reward catalog, and the real ratchet on the adapter's own recovery policy. The only difference between them is flysim::macros::MacroLayer, built from the configuration the way Sim::boot builds it, so the raw arm has no layer rather than a disabled one.

The harness drove the cartridge intro with scripted presses and both arms started from that one identical playable frame in Red's bedroom (map 38) with a freshly warmed-up brain. The fly's own readout was not asked to sit through the naming screens, which is also why this run says nothing about palette mode on the title screen — that path is raw by design and is covered by tests/integration.rs instead.

Both arms, 1 brain hour each

measure	raw	palette
brain hours	1.00	1.00
frames	215,020	215,020
wall seconds	1,449	1,449
realtime factor	2.48x	2.48x
rung reached	5 (GOT A STARTER)	4 (OAK'S LAB)
reward total	9.200	6.650
reward per brain hour	9.200	6.650
new locations	141	244
recoveries	0	0
frames with nothing pressed	0 (0%)	134,446 (63%)
frames a macro owned the buttons	—	118,500 (55%)
macros started	0	3,355
trajectory digest	a25dbaef740223f0	0358f5d151216668

Brain hours at each rung, from the shared starting frame:

rung	label	raw	palette
0	BOOT	0.000	0.000
1	BEDROOM	0.000	0.000
2	DOWNSTAIRS	0.008	0.001
3	PALLET TOWN	0.148	0.013
4	OAK'S LAB	0.156	0.249
5	GOT A STARTER	0.185	—

Macro outcomes over the palette hour: 3,355 started, 3,334 done, 21 blocked, 0 timeout, 0 refused. Payouts by kind — raw: boundary 11, exploration 16, map 3, milestone 5, species 4; palette: boundary 32, exploration 29, map 4, milestone 2. Frames by scene in the palette arm: overworld 66%, dialog 15%, menu 13%, unknown 6%.

What this establishes

• The wiring works end to end. 3,355 macros ran in an hour and 99.4% of them finished on their own script rather than being cut off. Nothing timed out, and nothing was refused: the palette's preconditions and the executor's route search agreed on every one of 3,355 starts, which is the path that was most likely to be noisy in production.
• It costs nothing measurable in throughput. Both arms stepped 215,020 frames in the same 1,449 s at 2.48x realtime, on the same box at the same time. Scene detection and Palette::for_scene run every frame in palette mode, and at this resolution they do not show up against a 139,255-neuron brain step. That is the number the release box cares about, since it has to hold 1.0x.
• Palette mode leaves a room much faster and then stalls. Out of the bedroom in 0.001 brain hours against 0.008, and out of the house into Pallet Town in 0.013 against 0.148 — an order of magnitude, and exactly what GO EXIT is for. It then took longer to reach Oak's lab (0.249 against 0.156) and never got the starter inside the hour, while covering nearly twice as much ground (244 new locations against 141).
• And it earned less. 6.65 reward per brain hour against 9.20. The two arms earn from different rules: palette doubled the boundary and exploration payouts (32 and 29 against 11 and 16) and lost the milestone and species ones, which are the large values. Covering ground is not the same as climbing, and this hour is the clearest statement of that difference so far.

What it does not establish

• Nothing about the gate. One run, one seed, one brain hour, a fresh boot rather than an archived checkpoint, and palette mode came out behind on the rung that matters. Section 7's bar is rung 6 in fewer brain hours in 3 of 3 runs; this is not evidence for or against that, because an hour from a fresh brain is the noisiest part of a run — a single early dialogue box the fly answers differently moves both arms by more than the gap between them.
• Nothing about the fly. It measures what a scene-appropriate action palette does to a run.
• Nothing about a long run. The reward attribution the design is actually for — "the interval between a macro decision and its payout is now one macro long" — is a learning effect, and an hour with 0 recoveries is not long enough to see one.

Plan mode, one brain hour, 2026-09-16

The third arm (docs/design/macros.md section 9, FLY_MACRO_ARMS=plan), on the same box, the same fresh boot and the same seed as smoke run 1, so the row below sits beside the two above. One arm rather than three: the raw and palette numbers are the ones already recorded, and re-running them would have measured the box's load rather than the mode. Re-run after section 9.1's TALK amendment; the pre-amendment run is the "what it does not establish" note below.

FLY_ROM=".../Pokemon Red (U) [S][BF].gb" FLY_MACRO_BRAIN=data/fafb-v783 \
  FLY_MACRO_HOURS=1 FLY_MACRO_DIGEST=1 FLY_MACRO_THREADS=4 FLY_MACRO_SEED=20260916 \
  FLY_MACRO_ARMS=plan cargo run --release -p flysim --example palette_bench

measure	raw	palette	plan
brain hours	1.00	1.00	1.00
frames	215,020	215,020	215,020
wall seconds	1,449	1,449	1,033
realtime factor	2.48x	2.48x	3.49x
rung reached	5 (GOT A STARTER)	4 (OAK'S LAB)	4 (OAK'S LAB)
reward total	9.200	6.650	4.650
reward per brain hour	9.200	6.650	4.650
new locations	141	244	63
recoveries	0	0	0
frames with nothing pressed	0 (0%)	134,446 (63%)	157,519 (73%)
frames a macro owned the buttons	—	118,500 (55%)	90,989 (42%)
macros started	0	3,355	3,403
trajectory digest	a25dbaef740223f0	0358f5d151216668	61f77acca870656e

Brain hours at each rung, from the shared starting frame:

rung	label	raw	palette	plan
0	BOOT	0.000	0.000	0.000
1	BEDROOM	0.000	0.000	0.000
2	DOWNSTAIRS	0.008	0.001	0.001
3	PALLET TOWN	0.148	0.013	0.003
4	OAK'S LAB	0.156	0.249	0.005
5	GOT A STARTER	0.185	—	—

Macro outcomes over the plan hour: 3,403 started, 3,401 done, 2 blocked, 0 timeout, 0 refused. Payouts by kind — boundary 22, exploration 6, map 4, milestone 2. Frames by scene: overworld 65%, dialog 22%, unknown 12%. Silent frames by cause: decision cooldown 68%, macro pressing nothing 23%, readout silent 8%, plan skipped 647 frames, no bound slot 0, macro refused 0.

The wall-clock column is not comparable: the raw and palette arms ran side by side on this box and the plan arm ran alone, which is the whole of the 2.48x against 3.49x. Nothing about the layer changed the per-frame cost — 215,020 frames in an hour in all three arms, as before.

What this establishes

• The early ladder is a walk now, not a search. Out of the bedroom in 0.001 brain hours, into Pallet Town in 0.003 against palette's 0.013 and raw's 0.148, and into Oak's lab in 0.005 against 0.249 and 0.156 — fifty times faster than palette mode on the rung palette mode was slowest on. That is GO OBJECTIVE reading the rung catalog's places and the drive doing nothing cleverer than "next step".
• TALK is in the plan and firing. Dialog is 22% of the hour's frames, against none before the amendment, and tests/rom_plan.rs shows the same drive taking a starter from Oak's lab in 3.5 brain minutes when the balls are on the table. Nothing refused and two blocked in 3,403.
• And the hour still ends on rung 4, for a different reason than before. The starter is gated behind Oak's own script: PalletTownDefaultScript triggers on wYCoord == 1, Oak intercepts the player at the north edge of the town and walks them into the lab, and until that has run there are no Pokéballs on the table to talk to. The plan pulls the fly into the lab — rung 4's place is the lab, and rung 5's is the same room — so it does not spend time at the north edge, and nothing in the plan is aimed at a script trigger. The ROM test drives that one stretch itself and says so. This is the next thing to decide, and it is a contract question like the last one: either the rung catalog carries a place for rung 5 that is the trigger rather than the room, or the plan needs an entry for "the map edge the story is behind".
• It earns least of the three. 4.650 reward per brain hour against palette's 6.650 and raw's 9.200, on a quarter of the ground covered (63 new locations against 244 and 141). The plan walks to the places the ladder names and stops exploring, so the boundary and exploration payouts that carried palette mode's total do not arrive. Reaching rungs faster and earning less in the same hour is exactly the tension section 7's gate is about, and one hour of one seed does not settle it.

What it does not establish

Nothing about the gate — one run, one seed, one brain hour, a fresh boot rather than the archived checkpoint, and no raw or palette arm run beside it on the same box at the same time. Nothing about the fly. Nothing about a long run: the reward attribution plan mode is for is a learning effect and an hour with 0 recoveries cannot show one.

Two earlier plan hours were run and discarded, and both are worth knowing about rather than hiding: the first (digest c9da99631adf4071) predates the section 9.1 demotion and bounced between Oak's lab and Pallet Town once per hold; the second (2c0e08fe5586d4bb) predates the TALK amendment and so could not press A at anything. Neither is evidence about anything except the two faults it found.

plan (biased), one brain hour, 2026-09-16

docs/design/macros.md section 10's blend, on the same box, the same fresh boot and the same seed as smoke run 1 and the plan row above, so all four compare. One arm again, for the same reason: the other numbers are already recorded and re-running them would measure the box's load.

FLY_ROM=".../Pokemon Red (U) [S][BF].gb" FLY_MACRO_BRAIN=data/fafb-v783 \
  FLY_MACRO_HOURS=1 FLY_MACRO_DIGEST=1 FLY_MACRO_THREADS=4 FLY_MACRO_SEED=20260916 \
  FLY_MACRO_ARMS=plan cargo run --release -p flysim --example palette_bench

Weights as shipped (SceneBias::default): overworld 0.3, dialog 0.95, menu 0.9, battle 0.9, battle-switch 0.95, shop 0.9, pc 0.9, unknown 0.5. fly[slot] is the spread across the bound slots (section 10 as amended the same day; the run on the first reading of that sentence is the note at the end of this section).

measure	raw	palette	plan	plan (biased)
brain hours	1.00	1.00	1.00	1.00
frames	215,020	215,020	215,020	215,020
wall seconds	1,449	1,449	1,033	1,063
realtime factor	2.48x	2.48x	3.49x	3.39x
rung reached	5 (GOT A STARTER)	4	4	4 (OAK'S LAB)
reward per brain hour	9.200	6.650	4.650	4.450
new locations	141	244	63	59
recoveries	0	0	0	0
frames with nothing pressed	0 (0%)	134,446 (63%)	157,519 (73%)	156,768 (73%)
frames a macro owned the buttons	—	118,500 (55%)	90,989 (42%)	84,708 (39%)
macros started	0	3,355	3,403	4,041
trajectory digest	a25dbaef740223f0	0358f5d151216668	61f77acca870656e	ca2b9c96fdad264f

Brain hours at each rung, from the shared starting frame:

rung	label	raw	palette	plan	plan (biased)
0	BOOT	0.000	0.000	0.000	0.000
1	BEDROOM	0.000	0.000	0.000	0.000
2	DOWNSTAIRS	0.008	0.001	0.001	0.001
3	PALLET TOWN	0.148	0.013	0.003	0.003
4	OAK'S LAB	0.156	0.249	0.005	0.025
5	GOT A STARTER	0.185	—	—	—

Macro outcomes: 4,041 started, 3,600 done, 440 blocked, 0 timeout, 0 refused. Payouts by kind — boundary 20, exploration 5, map 4, milestone 2. Frames by scene: dialog 66%, overworld 34%. Silent frames by cause: decision cooldown 50%, readout silent 31%, macro pressing nothing 19%, no bound slot 0, macro refused 0.

Macros chosen by rank: rank 0 3,482 (86%), rank 2 455 (11%), rank 4 93 (2%), rank 3 11 (0%). That is the row section 10's gate asks for, and it is a blend.

What this establishes

• The blend blends. The fly took 14% of the decisions off the plan's head — 559 macros — and the plan kept the other 86%. Both halves are doing what the weights say: the overworld is a third of the hour's frames and w = 0.3 there, dialog is two thirds and w = 0.95, so a rank-0-heavy overall split with the fly winning in the rooms is exactly the shape the operator's numbers describe. Rank 1 never won, which is the prior ladder's own doing: rank 1's prior is half rank 0's, so a fly that leans on rank 1 has to beat 0.3 with 0.7 x 1.0 + 0.15 — it does, whenever it leans — and the reason the count is 0 rather than small is that the policy rarely offers a second entry the fly's leading channel points at. Worth watching over a longer run.
• The early ladder holds. Downstairs 0.001 and Pallet Town 0.003, the same as the cursor drive. Oak's lab took 0.025 brain hours against the cursor drive's 0.005: five times slower and still six times faster than raw mode's 0.156 and ten times faster than palette mode's 0.249. That is the cost of letting the fly overrule the map on a third of its decisions, and it is the trade section 10 is asking for rather than a regression.
• 440 blocked, against 2 on the cursor drive. A blocked macro is one whose route stopped producing movement, and the fly interrupting a walk it did not choose is how that happens: the fly's slot wins a decision, its macro starts, and the next decision goes elsewhere. Nothing refused and nothing timed out in 4,041 starts, so the executor and the palette still agree on every start.
• The fly is still the only thing that starts a macro. 31% of the silent frames are "readout silent" — every bound row at or under the readout's activity floor, or the spread flat — so a third of the doing-nothing is the fly saying nothing, and the scene never acted alone in it.

What it does not establish

Nothing about the gate: one run, one seed, one brain hour, a fresh boot rather than the archived checkpoint, no other arm beside it on the same box at the same time, and the box was also running cartridge tests during this hour, so the wall-clock and realtime columns are worth even less than usual. Nothing about the fly. Nothing about a long run — and the reward attribution plan mode is for is a learning effect, which an hour with 0 recoveries cannot show.

The run that forced the amendment. The same hour on the first reading of "the channel's normalized score in [0, 1]" — each channel against its own baseline, clamp(score - 1, 0, 1) — chose rank 0 for all 4,399 of its macros (digest 6abcbcbd815646fd, 3.650 reward per brain hour, 36 new locations, rungs at 0.001/0.003/0.005). The fly decided whether a macro started and never which one, because the readout's score is a ratio around 1 whose directions sit inside a 9% spread (docs/readout.md): every vote landed in 0..0.1 while beating rank 0 at w = 0.3 needs 0.214. Same weights, same seed, same code otherwise — the whole difference between a blend and the plan alone was the unit the fly's vote is measured in.

And one thing neither hour establishes: the starter. Section 9.1's TALK amendment relied on section 9's "a plan whose head changes puts the drive back at the top", and section 10 has no cursor to reset — TALK is rank 0, rank 0 is the UP channel, so the press that takes a Pokéball needs the fly to lean on UP while facing one. The cartridge test (services/flysim/crates/flysim/tests/rom_plan.rs) still reaches Oak's lab and its rung from the bedroom on a game-blind stub readout, in 1.6 brain minutes on 10 macros, but no stub took a starter under either reading of fly[slot]: every slot equal pressed A 615 times at one shelf, one hot slot per burst wandered out of the lab 367 times, and a committing rotation did the same over five times the frames (4,904 macros, TALK once, identical under both mappings). The starter leg is therefore off that test, with the finding recorded in it. This is the next thing to decide.

Smoke run 2: not yet measured

The two causes smoke run 1 pointed at are fixed on feat/macros-lab (2026-09-16) — slot 2 is GO ITEM instead of the duplicate LOOK, and GO EXIT reads the adapter's boundary ledger through GameAdapter::exit_visited — and the harness now attributes every silent frame to its cause. The re-run was deferred, so there is no smoke-2 table here yet. Do not read the fixes as measured: all that is established is that they are wired, unit-tested and confirmed on the cartridge (tests/rom_macros.rs: GO ITEM walks to a starter ball in Oak's lab and TALK there hands over a Charmander; GO ROUTE -- GO EXIT when that run was recorded -- in Pallet Town leaves by an unvisited door rather than the one it came out of).

When it is run, it is one brain hour per arm from a fresh boot on the same seed as the record above, so that the two tables compare:

FLY_ROM=".../Pokemon Red (U) [S][BF].gb" FLY_MACRO_BRAIN=data/fafb-v783 \
  FLY_MACRO_HOURS=1 FLY_MACRO_DIGEST=1 FLY_MACRO_THREADS=4 FLY_MACRO_SEED=20260916 \
  cargo run --release -p flysim --example palette_bench

The palette arm now prints one extra table, which is the question smoke run 1 could not answer about its own 63% of frames pressing nothing:

silent frame	meaning
macro pressing nothing	a macro owned the buttons and its script held nothing: a settle, the gap between two pulses, a wait for a menu cursor, or the frame it finished on
readout silent	no channel was active, so the fly asked for nothing
no bound slot	a channel was active and its slot is unbound in this scene
decision cooldown	a channel was active and bound, but the last decision is less than one holdMs old
macro refused	a bound macro's precondition had lapsed, or it found no route

"readout silent" means something slightly different in plan mode since section 10, because the channels there are scores rather than a set: it is a readout with every bound row's channel at or below the readout's own activity floor, i.e. a fly with nothing to say. The sixth cause, "plan skipped", is gone with the B-skips rule section 10 replaced.

The split is what decides whether the palette is too small or never reached, and the two are different fixes: the first is a binding change in docs/design/macros.md section 3, the second is not. A trajectory digest that differs from smoke run 1's is expected and is not a harness fault — the fixes change which buttons get pressed, which is the point.

Running the real thing

1. Pull the release box's durable checkpoint read-only (/srv/fly/state/<gen>.checkpoint) onto the dev box. Do not run this against the release container's live state directory.
2. FLY_MACRO_CHECKPOINT=/path/to/<gen>.checkpoint FLY_MACRO_HOURS=6 with three different FLY_MACRO_SEED values, one run per seed, both arms each.
3. Record all three tables here with the checkpoint's generation and brain hours, and compare the brain hours at rung 6 and above. FLY_MACRO_DIGEST=1 on every run: two runs of one arm with one seed on one box must print the same digest, and a digest that moves between runs means something in the harness is not deterministic and the numbers cannot be compared.
4. Palette mode stays off (FLY_MACRO_MODE=raw, which is the default and what every env file carries) until that passes and the operator has seen the screen.

The one-hour smoke run above takes about 24 minutes of wall time for both arms together; six hours per arm is about two and a half hours, so the three runs are most of a working day on this box.