docs: the engagement rewards, and the v6 migration in deploy examples

rewards-learning.md carries the v7 table, the definition of indoors, the four
conditions a conversation pays under, the item and hidden-item reads, the seed,
boundary's indoor rule with the warp-transition frame it was measured on, the
feed-kind choice and the ticker word, and the honesty note on paying for a
conversation. flysim.md records the v6 -> v7 migration; macros-wram.md the six
new reads; the architecture tour and integration.md point at the live catalog.
The deploy comments, example.env and the runbook name FLY_ACCEPT_ADAPTERS=
pokered-unique8-v6 for the v0.6.0 deploy.
This commit is contained in:
acamilo 2026-09-23 11:50:08 +00:00
parent 3cf706c96a
commit 8d1d5c44f5
8 changed files with 157 additions and 10 deletions

View file

@ -89,7 +89,9 @@ decoder preset to use.
disassembly at commit 0cd19d3 (`symbols.rs`), gates rewards on a playable state, baselines already
achieved flags on the first sample so a restore never replays them, and pays only positive
rewards: story flags, exploration coverage (capped per map), new areas, Pokédex entries, trainer
flags, decaying wild wins, badges. Version `pokered-unique8-v5`.
flags, decaying wild wins, badges, exits found outdoors, catches, conversations indoors and items
picked up (`docs/rewards-learning.md`). Version `pokered-unique8-v7`; a deploy that names the
previous version in `FLY_ACCEPT_ADAPTERS` migrates its checkpoints instead of refusing them.
- Ratchet (`ratchet.rs`): a 38-rung ladder (boot, bedroom, Pallet Town, Oak's lab, starter, parcel,
Pokédex, each town, each badge, the Elite Four, Champion). On first reaching a higher rung in a
safe state it archives the emulator snapshot; on a stall (120 s without new exploration) or a game

View file

@ -387,6 +387,17 @@ on-screen ticker cannot disagree with what the sim did.
absent from a `v5` state and restores empty, which is the truth about a run that was never paid
for a catch. `STATE_VERSION` does not move, because the schema did not.
**`v6` -> `v7`** (2026-09-23, the engagement rewards) is the same migration for the next pair,
and `pokered-unique8-v7`'s `migrates_from()` is `["pokered-unique8-v6"]` and nothing else -- `v5`
is no longer migrated, because the live run is `v6`. The deploy that ships it sets
`FLY_ACCEPT_ADAPTERS=pokered-unique8-v6`. No field is added this time: `talk` and `item` key
their ledgers into the existing `seen` array, as `boundary` did, so a `v6` state restores
unchanged with no `talk:` keys. The one step is at the first sample after the restore, not in
`import_state`: a ledger without the `items:seeded` key writes an `item:`/`hidden:` key for every
item the cartridge already shows as taken, pays for none, and marks the seed, so a rollback to a
slot from before a `v6`-era pickup cannot pay for it (`docs/rewards-learning.md`, "The seed").
`STATE_VERSION` stays 4.
- **Restarting a run from an earlier rung** (2026-09-22). `FLY_RESET_STATE=1` throws the run away;
`infra/bin/fly-reset-to-milestone <N>` keeps it and rewinds it. It archives both stores to a
dated directory, rewrites `milestone-<N>.checkpoint` with the ratchet's `attempts` and

View file

@ -133,6 +133,28 @@ a trade and a Pokémon withdrawn from the PC. Reading a catch off it would need
tell those apart. The cartridge's own flag needs none, which is why the row above is the one the
adapter reads.
### The engagement rewards' reads (2026-09-23)
**New 2026-09-23** (`talk` and `item`, `docs/rewards-learning.md`, "Engagement rewards"). All six
were resolved by `services/flysim/tools/resolve_wram.py` from `ram/wram.asm` at the pinned commit
and are bracketed by addresses `symbols.rs` already carried; two of the brackets needed the tool
to count `NUM_STATS` and `NUM_CITY_MAPS`, which the decomp defines as `const_value` over an
enumeration.
| what | symbol | address | notes | verified |
| --- | --- | --- | --- | --- |
| the text's subject | `wSpriteIndex` | `$cf13` | `DisplayTextID` copies its argument here: a sprite slot up to `wNumSprites`, else a text id. It arrives **about twenty frames after** `wFontLoaded` bit 0 rises, because `DisplayTextIDInit` loads the font's tiles first; until then it still holds the previous text's subject. | survey (`tests/rom_engage.rs`: the Viridian Forest north gate, the old man at slot 2, font bit on frame 820 and the argument on frame 840) |
| mid-step | `wWalkCounter` | `$cfc5` | non-zero for the frames of a step; the overworld only reads A at zero. Right after `wFontLoaded` in `ram/wram.asm`. | ROM, trace |
| an item ball's item | `wMapSpriteExtraData` | `$d504` | two bytes per sprite slot (slot 1 first): `(item id, 0)` for an `ITEM` `object_event`, `(trainer class, trainer number)` for a `TRAINER` one, zeroes otherwise -- `LoadMapHeader`'s `.itemBallSprite` / `.trainerSprite` / `.regularSprite` | survey (the forest's Antidote ball read `(11, 0)`) |
| taken or hidden, per object | `wToggleableObjectFlags` | `$d5a6` | `flag_array $100`, one bit per global toggleable index (`constants/toggle_constants.asm`); `PickUpItem`'s `HideObject` sets an item ball's bit after `GiveItem` succeeded | survey (the forest's Antidote ball's bit rose on the pickup frame) |
| this map's toggleables | `wToggleableObjectList` | `$d5ce` | up to sixteen `(sprite slot, global index)` pairs, `$ff`-terminated, written by `MarkTownVisitedAndLoadToggleableObjects` | survey |
| hidden items found | `wObtainedHiddenItemsFlags` | `$d6f0` | `flag_array MAX_HIDDEN_ITEMS` (112); `FoundHiddenItemText` sets the bit after `GiveItem` succeeded, and nothing else writes it | ROM (disassembly), trace |
Not used, and why: `hJoyPressed`/`hJoyHeld` would say "A was pressed" directly, but they are HRAM,
which neither `gen_symbols.py` nor `resolve_wram.py` resolves, and a hand-written address is the one
thing those tools exist to refuse. "The fly had the joypad and was standing still on the frame
before the box opened, and the box is about the thing it faces" is the same fact read out of WRAM.
### Battle menu and cursor, own turn against forced switch
`HandleMenuInput` is shared by every menu in the game, so which menu is up is read from where it

View file

@ -177,6 +177,10 @@ achievements.
| Wild win | +0.1, +0.05, +0.0333 | at most three observed wild KOs per `(map,species,level)` |
| Badge | +3 | each newly set badge bit |
That is the prototype's catalog as it shipped, kept here as history. The live one is
`docs/rewards-learning.md`: `pokered-unique8-v7` adds `boundary` (v5), `catch` (v6), and `talk` and
`item` with no `boundary` payout indoors (v7, the operator's decision of 2026-09-23).
Every value is positive; there were no loss or blackout penalties. Values in a frame summed into
`R`, then `m = tanh(R)`. PAM stimulation ran 80 to 400 ms depending on reward kind, with
overlapping pulses taking their maximum.

View file

@ -1,6 +1,6 @@
# Rewards and learning
The live reward catalog of the Pokémon Red adapter, `pokered-unique8-v6`. The code of record is
The live reward catalog of the Pokémon Red adapter, `pokered-unique8-v7`. The code of record is
`services/flysim/crates/flybrain-gb/src/pokemon_red/` (`catalog.rs` holds the values, `mod.rs` the
gates and the rules); this page says what each rule pays for and why it is allowed to. The
prototype's own `docs/rewards-learning.md` in `fly-plays-pokemon` is where the first seven rules
@ -23,8 +23,10 @@ change what the fly can do.
| `trainer` | `trainer` | +0.5 | 200 ms | Each named `EVENT_BEAT_*` flag once, except the flags classified as story milestones |
| `battle` | `wildwin` | +0.1, +0.05, +0.0333 | 100 ms | At most three observed wild KOs per `(map, species, level)` |
| `badge` | `badge` | +3 | 400 ms | Each newly set badge bit |
| `boundary` | `explore` | +0.05, +0.10 | 100 ms | First tile adjacent to one of the map's exits, and the exit tile itself; once per `(map, exit)` for the lifetime of the ledger |
| `boundary` | `explore` | +0.05, +0.10 | 100 ms | First tile adjacent to one of the map's exits, and the exit tile itself; once per `(map, exit)` for the lifetime of the ledger. **Nothing on an indoor map** (since v7): the exit is still recorded, and pays 0 |
| `catch` | `wildwin` | +0.30, +0.10 | 150 ms | A wild Pokémon kept by a ball: +0.30 for a species this run had never owned, +0.10 for a repeat; at most three payouts per species for the lifetime of the ledger |
| `talk` | `explore` | +0.10 | 100 ms | A conversation the fly opened with a person or a sign **indoors**, paid when its box closes; once per `(map, sprite slot or sign text id)` for the lifetime of the ledger |
| `item` | `explore` | +0.15 | 120 ms | An item ball or a hidden item picked up, on any map; once per item for the lifetime of the ledger |
Every value is positive: there are no loss or blackout penalties, and `catalog::rule("blackout")`
is `None` by test. The values in one frame sum into `R`, and the network reinforces once with
@ -43,6 +45,18 @@ currently reads "wild win". The event's own label, `CAUGHT #<species>`, is what
log, `/status` and the checkpoint. Changing the ticker copy means opening the feed's closed kind
set, which this rule deliberately did not do.
`talk` and `item` publish on `explore`, for the same reason `boundary` does: each is the fly
finding what is in a place -- new ground, a door, a person or sign it opened, an item it picked up
-- at the same quiet scale (0.05 to 0.15). Not `area`, which counts maps and is a notable row; not
`story`, which is the plot; not `wildwin`, which is a battle. No feed kind was added, so
`docs/feed-protocol.md` and the stage's switch statements did not move. What did move is the one
word that would have been untrue: the Pokémon Red ticker's `explore` row said "new place", which is
not what a conversation or an item is, and now says **"new find"** ("3 new finds" collapsed), which
is true of all four. The event labels -- `TALKED TO #<slot> IN AREA <map>`, `READ SIGN #<id> IN
AREA <map>`, `FOUND ITEM #<item>`, `FOUND A HIDDEN ITEM` -- reach the event log, `/status` and the
checkpoint. Both also reset the stage's stall meter, which counts `explore`: engaging with a
building is progress in the sense the operator asked for.
## Catch rewards
The operator's decision of 2026-09-22: the fly is paid for *keeping* a wild Pokémon, not only for
@ -82,6 +96,86 @@ well below a badge (3.0). A catch of a new species pays 0.80 across two kinds, w
a story flag and a badge -- deliberately, because it is the one event that is both a discovery and
a thing the fly had to do on purpose.
## Engagement rewards
The operator's decision of 2026-09-23, recorded with the port decisions: reward the fly for
engaging *inside* buildings and stop paying it for leaving them. It was chosen over a pad rule and
over weighting the choice, and it is a catalog change -- an operator decision, like the catch
reward -- not a loop-review fix (`docs/loop-review.md`). It answers a shape the loop reviews kept
finding in Pewter: `GO OBJECTIVE` into a building and `GO OUT` straight back, paid for the door on
the way out and for nothing inside.
**Indoors** is two of the cartridge's own tables, and nothing hand-classified
(`pokemon_red/engage.rs`, `indoor`). `CheckIfInOutsideMap` (`home/overworld.asm`) is the game's
outdoor test -- tileset `OVERWORLD` or `PLATEAU` -- and `WarpFound2` labels its other branch
`.indoorMaps`; on its own that would call Viridian Forest and every cave indoors, and their exits
are how the fly gets anywhere. `BikeRidingTilesets` (`data/tilesets/bike_riding_tilesets.asm`) is
the list of places the bicycle may be ridden -- `OVERWORLD`, `FOREST`, `UNDERGROUND`, `SHIP_PORT`,
`CAVERN` -- and the bike is the one thing the cartridge refuses inside a building by rule. A map is
indoors when its `wCurMapTileset` is in neither: every house, mart, Pokémon Center, gym, gate, lab
and museum, the S.S. Anne, Silph Co., the Pokémon Tower, the Mansion, the Rocket Hideout and the
Indigo Plateau's rooms. Not the forest, a cave, the Underground Path or Vermilion's dock.
**`talk`, +0.10.** Paid on the sample the text box closes, for a conversation that
1. *the fly opened*: on the last sample before the font bit (`wFontLoaded` bit 0) rose, the fly had
the joypad -- no `wJoyIgnore`, no simulated input, no scripted movement -- was standing still
(`wWalkCounter` zero, the only state the overworld reads A in) and stood where it stands now. A
script's text opens with the joypad taken, or on the frame a step onto a trigger tile ends;
2. *is with the thing in front of it*: `DisplayTextID` copies its argument into `wSpriteIndex` --
a sprite slot up to `wNumSprites`, or a text id -- and the sprite must stand on the tile the
player faces (or one further, across a counter, on a tileset that has counter tiles, which is
`IsSpriteOrSignInFrontOfPlayer`'s own long reach), or the text id must be the sign's on that
tile. The byte arrives about **twenty frames after** the font bit (measured on the cartridge:
`DisplayTextIDInit` loads the font's tiles first) and until then still names the previous
text's subject, so it is read once it has changed or 45 samples have passed, and only while the
bottom dialogue box is drawn -- the start menu draws its own box elsewhere. An item ball is a
sprite but not a person, and pays `item`;
3. *opened indoors*, on the map the box opened on;
4. *finished*: the box closed on the same map. A conversation that ends in a warp, a rollback or a
restore pays nothing.
The ledger is the adapter's lifetime `seen` set, keyed `talk:<map>:sprite:<slot>` or
`talk:<map>:sign:<text id>` -- the same "map and object index" the macros' `talked` ledger uses,
but **not** that ledger: the macros' ledger is session state and is thrown away on a restore; this
one is checkpointed and survives a rollback, so talking to the same person again, after a restore
or not, pays nothing. The trainer the fly speaks to before a battle is a person and pays once; the
nurse, a clerk and a sign each pay once per map.
**`item`, +0.15.** Read from the cartridge's own "this one has been taken" bits, on any map.
An item ball is one of the map's toggleable sprites (`wToggleableObjectList`, sprite slot and
global index) whose `wMapSpriteExtraData` is `(item id, 0)` -- the shape `LoadMapHeader` writes for
an `ITEM` `object_event` and for nothing else (a trainer's is `(class, number)` with numbers from
1, a person's two zeroes); `PickUpItem` sets its global bit in `wToggleableObjectFlags` through
`HideObject`, and only after `GiveItem` succeeded, so a full bag pays nothing. A hidden item is a
bit of `wObtainedHiddenItemsFlags`, set by `FoundHiddenItemText` after `GiveItem` and by nothing
else; hidden *coins* have their own bitset and are not items. Either pays when its bit rises
between two playable samples, keyed `item:<global index>` or `hidden:<index>`, once for the life
of the ledger. A gift item from a script (the Old Amber, a TM from a person) is not an item ball:
the conversation pays `talk`, and the item nothing.
**The seed.** The first playable sample that finds the key `items:seeded` absent -- a fresh
adapter, or a `v6` ledger restored under `v7` -- writes a key for every bit the game already shows
as taken and pays for none of them, so a rollback to a slot from before a `v6`-era pickup cannot
pay for taking it again. The two item balls a script *reveals* are left out of the seed, because
their bits are set from a new game until Giovanni's defeat clears them: the Rocket Hideout's Silph
Scope and Lift Key (`$87`, `$88`), the only `ITEM` entries `data/maps/toggleable_objects.asm` starts
`OFF`.
**`boundary` indoors.** Every exit on an indoor map is still written to the ledger, so
`exit_visited` answers exactly what it did and the macros see no change, but nothing is paid. A
town's doors, a route's edges, the forest's gates and a cave's ladders pay as before. One
consequence, measured on the cartridge (`tests/rom_engage.rs`): for the thirty-odd frames of
`PlayMapChangeSound` the cartridge has written the destination into `wCurMap` while the tileset and
the warp table are still the map being left, so the exit the fly is standing on is classified by
the map it belongs to. Walking into a building through a town door still pays that door's on-exit
half, 0.10, once, keyed under the building's id as it always was; walking out pays nothing.
**The scale.** A building's worth of engagement -- a few people, a sign, perhaps a ball -- is
0.3 to 0.6: more than the 0.15 its door paid for being left, less than a new Pokédex entry per
person, far below a badge. Everything is once per thing for the lifetime of the ledger, so no
building can be farmed.
## Gates
Semantic rewards are enabled for exactly one cartridge, the SHA-256 in `SUPPORTED_ROM`. Any other
@ -110,6 +204,10 @@ therefore replays none of it.
## Boundary rewards
Since `pokered-unique8-v7` everything below holds **outdoors** -- towns, routes, the forest,
caves -- and on an indoor map the same keys are written and nothing is paid ("Engagement rewards"
above has the definition of indoors and the one transition frame worth knowing about).
`docs/design/room-escape.md` section 2. The rule pays 0.05 the first time the fly stands on a tile
orthogonally adjacent to one of the current map's exits, and 0.10 the first time it stands on the
exit tile itself. Both are keyed into the adapter's lifetime `seen` ledger as
@ -177,7 +275,15 @@ body picks the macro; the descending neurons press the buttons.**
## Honesty
The catalog now includes catches. The honesty panel's copy is not data-driven from the catalog --
The catalog now includes conversations and items (v7). Paying for a conversation is the closest
the catalog has come to paying for a *button*: A is what opens one. It is still a reward, not a
press. Nothing in the adapter presses A, chooses when, or tells the fly who is there; the payout is
read out of WRAM after a conversation the fly's own buttons -- or the macro the mushroom body chose
-- opened and finished, and it is once per person or sign for the life of the run, so the thing
that is learned is "the people in a building are worth a visit", not "press A". It is also why the
rule demands evidence that the fly opened the box: text a script started pays nothing.
The catalog also includes catches. The honesty panel's copy is not data-driven from the catalog --
`apps/stage/src/lib/schedule.ts`'s rotating card is four written lines and lists no kinds -- so
there was nothing to regenerate and the copy is unchanged. The sentences below are where the
argument lives.

View file

@ -300,8 +300,9 @@ if [ -n "$RELEASE_TARBALL" ]; then
#
# FLY_ACCEPT_ADAPTERS is the *other* override, and the opposite one: it keeps
# the run. It names adapter version strings whose checkpoints the new build
# may migrate — e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v5 for the deploy
# that adds the catch reward. It only applies when the adapter segment is the
# may migrate — e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v6 for the deploy
# that adds the engagement rewards (v7; v5 -> v6 was the catch reward's).
# It only applies when the adapter segment is the
# ONLY difference between the two strings and the new build's adapter says it
# can read that one; a dataset, kernel, emulator or state-format change is
# still a refusal, because none of those has a migration. The same variable is
@ -354,7 +355,7 @@ The difference is usually an adapter/ladder or dataset version bump. Three ways
* deploy a build whose string matches (check out the commit the running release was built from), or
* if the ADAPTER VERSION is the only segment that differs and the new build documents a
migration from the old one, re-run with FLY_ACCEPT_ADAPTERS set to the adapter id in the live
string (e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v5). The run is kept; flysim applies the same
string (e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v6). The run is kept; flysim applies the same
rule at restore. See docs/design/flysim.md, \"Restoring across an adapter version\", or
* accept losing everything the brain has learned and re-run with FLY_RESET_STATE=1, which
archives ${state_dir}'s checkpoints to ${state_dir}.<timestamp> (kept, not deleted) and

View file

@ -263,7 +263,7 @@ pct exec $CTID -- /opt/fly/bin/fly-reset-to-milestone $N
# (a) the running release already wrote that checkpoint -> nothing to deploy, skip to 5.
# (b) the new build bumps the ADAPTER VERSION and nothing else -> name the checkpoint's
# adapter so the gate and flysim both migrate instead of refusing:
FLY_ACCEPT_ADAPTERS=pokered-unique8-v5 infra/05-deploy.sh <release-env> <release-tarball>
FLY_ACCEPT_ADAPTERS=pokered-unique8-v6 infra/05-deploy.sh <release-env> <release-tarball>
# The gate logs "the adapter version is the only difference, and it is named; the run is KEPT
# and migrated", and writes FLY_ACCEPT_ADAPTERS into /etc/fly/fly.env so flysim applies the
# same rule at restore. Anything else about the string differing is still a refusal.

View file

@ -335,8 +335,9 @@ FLY_FEED_VIA=direct
# two strings AND the new build's adapter declares a migration from that one. A
# dataset, kernel, plasticity, emulator or state-format difference is still a
# refusal. Set it for the one deploy that needs it and leave it out afterwards;
# 05-deploy writes it into /etc/fly/fly.env only while it is set.
# FLY_ACCEPT_ADAPTERS=pokered-unique8-v5
# 05-deploy writes it into /etc/fly/fly.env only while it is set. The v0.6.0
# deploy (pokered-unique8-v7, the engagement rewards) is the one that needs:
# FLY_ACCEPT_ADAPTERS=pokered-unique8-v6
# --- push mode --------------------------------------------------------------
# local: flypush.service stays disabled, everything else identical to prod.