From 8d1d5c44f57e8ee22278acd8332c292aca88aaf0 Mon Sep 17 00:00:00 2001 From: acamilo Date: Wed, 23 Sep 2026 11:50:08 +0000 Subject: [PATCH] docs: the engagement rewards, and the v6 migration in deploy examples rewards-learning.md carries the v7 table, the definition of indoors, the four conditions a conversation pays under, the item and hidden-item reads, the seed, boundary's indoor rule with the warp-transition frame it was measured on, the feed-kind choice and the ticker word, and the honesty note on paying for a conversation. flysim.md records the v6 -> v7 migration; macros-wram.md the six new reads; the architecture tour and integration.md point at the live catalog. The deploy comments, example.env and the runbook name FLY_ACCEPT_ADAPTERS= pokered-unique8-v6 for the v0.6.0 deploy. --- docs/architecture-tour.md | 4 +- docs/design/flysim.md | 11 ++++ docs/design/macros-wram.md | 22 ++++++++ docs/integration.md | 4 ++ docs/rewards-learning.md | 112 ++++++++++++++++++++++++++++++++++++- infra/05-deploy.sh | 7 ++- infra/docs/runbook.md | 2 +- infra/env/example.env | 5 +- 8 files changed, 157 insertions(+), 10 deletions(-) diff --git a/docs/architecture-tour.md b/docs/architecture-tour.md index 9f0f01b..d121287 100644 --- a/docs/architecture-tour.md +++ b/docs/architecture-tour.md @@ -89,7 +89,9 @@ decoder preset to use. disassembly at commit 0cd19d3 (`symbols.rs`), gates rewards on a playable state, baselines already achieved flags on the first sample so a restore never replays them, and pays only positive rewards: story flags, exploration coverage (capped per map), new areas, Pokédex entries, trainer - flags, decaying wild wins, badges. Version `pokered-unique8-v5`. + flags, decaying wild wins, badges, exits found outdoors, catches, conversations indoors and items + picked up (`docs/rewards-learning.md`). Version `pokered-unique8-v7`; a deploy that names the + previous version in `FLY_ACCEPT_ADAPTERS` migrates its checkpoints instead of refusing them. - Ratchet (`ratchet.rs`): a 38-rung ladder (boot, bedroom, Pallet Town, Oak's lab, starter, parcel, Pokédex, each town, each badge, the Elite Four, Champion). On first reaching a higher rung in a safe state it archives the emulator snapshot; on a stall (120 s without new exploration) or a game diff --git a/docs/design/flysim.md b/docs/design/flysim.md index 5e7f0b5..ba17c1d 100644 --- a/docs/design/flysim.md +++ b/docs/design/flysim.md @@ -387,6 +387,17 @@ on-screen ticker cannot disagree with what the sim did. absent from a `v5` state and restores empty, which is the truth about a run that was never paid for a catch. `STATE_VERSION` does not move, because the schema did not. + **`v6` -> `v7`** (2026-09-23, the engagement rewards) is the same migration for the next pair, + and `pokered-unique8-v7`'s `migrates_from()` is `["pokered-unique8-v6"]` and nothing else -- `v5` + is no longer migrated, because the live run is `v6`. The deploy that ships it sets + `FLY_ACCEPT_ADAPTERS=pokered-unique8-v6`. No field is added this time: `talk` and `item` key + their ledgers into the existing `seen` array, as `boundary` did, so a `v6` state restores + unchanged with no `talk:` keys. The one step is at the first sample after the restore, not in + `import_state`: a ledger without the `items:seeded` key writes an `item:`/`hidden:` key for every + item the cartridge already shows as taken, pays for none, and marks the seed, so a rollback to a + slot from before a `v6`-era pickup cannot pay for it (`docs/rewards-learning.md`, "The seed"). + `STATE_VERSION` stays 4. + - **Restarting a run from an earlier rung** (2026-09-22). `FLY_RESET_STATE=1` throws the run away; `infra/bin/fly-reset-to-milestone ` keeps it and rewinds it. It archives both stores to a dated directory, rewrites `milestone-.checkpoint` with the ratchet's `attempts` and diff --git a/docs/design/macros-wram.md b/docs/design/macros-wram.md index 70bd517..13d0ae8 100644 --- a/docs/design/macros-wram.md +++ b/docs/design/macros-wram.md @@ -133,6 +133,28 @@ a trade and a Pokémon withdrawn from the PC. Reading a catch off it would need tell those apart. The cartridge's own flag needs none, which is why the row above is the one the adapter reads. +### The engagement rewards' reads (2026-09-23) + +**New 2026-09-23** (`talk` and `item`, `docs/rewards-learning.md`, "Engagement rewards"). All six +were resolved by `services/flysim/tools/resolve_wram.py` from `ram/wram.asm` at the pinned commit +and are bracketed by addresses `symbols.rs` already carried; two of the brackets needed the tool +to count `NUM_STATS` and `NUM_CITY_MAPS`, which the decomp defines as `const_value` over an +enumeration. + +| what | symbol | address | notes | verified | +| --- | --- | --- | --- | --- | +| the text's subject | `wSpriteIndex` | `$cf13` | `DisplayTextID` copies its argument here: a sprite slot up to `wNumSprites`, else a text id. It arrives **about twenty frames after** `wFontLoaded` bit 0 rises, because `DisplayTextIDInit` loads the font's tiles first; until then it still holds the previous text's subject. | survey (`tests/rom_engage.rs`: the Viridian Forest north gate, the old man at slot 2, font bit on frame 820 and the argument on frame 840) | +| mid-step | `wWalkCounter` | `$cfc5` | non-zero for the frames of a step; the overworld only reads A at zero. Right after `wFontLoaded` in `ram/wram.asm`. | ROM, trace | +| an item ball's item | `wMapSpriteExtraData` | `$d504` | two bytes per sprite slot (slot 1 first): `(item id, 0)` for an `ITEM` `object_event`, `(trainer class, trainer number)` for a `TRAINER` one, zeroes otherwise -- `LoadMapHeader`'s `.itemBallSprite` / `.trainerSprite` / `.regularSprite` | survey (the forest's Antidote ball read `(11, 0)`) | +| taken or hidden, per object | `wToggleableObjectFlags` | `$d5a6` | `flag_array $100`, one bit per global toggleable index (`constants/toggle_constants.asm`); `PickUpItem`'s `HideObject` sets an item ball's bit after `GiveItem` succeeded | survey (the forest's Antidote ball's bit rose on the pickup frame) | +| this map's toggleables | `wToggleableObjectList` | `$d5ce` | up to sixteen `(sprite slot, global index)` pairs, `$ff`-terminated, written by `MarkTownVisitedAndLoadToggleableObjects` | survey | +| hidden items found | `wObtainedHiddenItemsFlags` | `$d6f0` | `flag_array MAX_HIDDEN_ITEMS` (112); `FoundHiddenItemText` sets the bit after `GiveItem` succeeded, and nothing else writes it | ROM (disassembly), trace | + +Not used, and why: `hJoyPressed`/`hJoyHeld` would say "A was pressed" directly, but they are HRAM, +which neither `gen_symbols.py` nor `resolve_wram.py` resolves, and a hand-written address is the one +thing those tools exist to refuse. "The fly had the joypad and was standing still on the frame +before the box opened, and the box is about the thing it faces" is the same fact read out of WRAM. + ### Battle menu and cursor, own turn against forced switch `HandleMenuInput` is shared by every menu in the game, so which menu is up is read from where it diff --git a/docs/integration.md b/docs/integration.md index 00f82b3..02ae563 100644 --- a/docs/integration.md +++ b/docs/integration.md @@ -177,6 +177,10 @@ achievements. | Wild win | +0.1, +0.05, +0.0333 | at most three observed wild KOs per `(map,species,level)` | | Badge | +3 | each newly set badge bit | +That is the prototype's catalog as it shipped, kept here as history. The live one is +`docs/rewards-learning.md`: `pokered-unique8-v7` adds `boundary` (v5), `catch` (v6), and `talk` and +`item` with no `boundary` payout indoors (v7, the operator's decision of 2026-09-23). + Every value is positive; there were no loss or blackout penalties. Values in a frame summed into `R`, then `m = tanh(R)`. PAM stimulation ran 80 to 400 ms depending on reward kind, with overlapping pulses taking their maximum. diff --git a/docs/rewards-learning.md b/docs/rewards-learning.md index 1975629..b5869db 100644 --- a/docs/rewards-learning.md +++ b/docs/rewards-learning.md @@ -1,6 +1,6 @@ # Rewards and learning -The live reward catalog of the Pokémon Red adapter, `pokered-unique8-v6`. The code of record is +The live reward catalog of the Pokémon Red adapter, `pokered-unique8-v7`. The code of record is `services/flysim/crates/flybrain-gb/src/pokemon_red/` (`catalog.rs` holds the values, `mod.rs` the gates and the rules); this page says what each rule pays for and why it is allowed to. The prototype's own `docs/rewards-learning.md` in `fly-plays-pokemon` is where the first seven rules @@ -23,8 +23,10 @@ change what the fly can do. | `trainer` | `trainer` | +0.5 | 200 ms | Each named `EVENT_BEAT_*` flag once, except the flags classified as story milestones | | `battle` | `wildwin` | +0.1, +0.05, +0.0333 | 100 ms | At most three observed wild KOs per `(map, species, level)` | | `badge` | `badge` | +3 | 400 ms | Each newly set badge bit | -| `boundary` | `explore` | +0.05, +0.10 | 100 ms | First tile adjacent to one of the map's exits, and the exit tile itself; once per `(map, exit)` for the lifetime of the ledger | +| `boundary` | `explore` | +0.05, +0.10 | 100 ms | First tile adjacent to one of the map's exits, and the exit tile itself; once per `(map, exit)` for the lifetime of the ledger. **Nothing on an indoor map** (since v7): the exit is still recorded, and pays 0 | | `catch` | `wildwin` | +0.30, +0.10 | 150 ms | A wild Pokémon kept by a ball: +0.30 for a species this run had never owned, +0.10 for a repeat; at most three payouts per species for the lifetime of the ledger | +| `talk` | `explore` | +0.10 | 100 ms | A conversation the fly opened with a person or a sign **indoors**, paid when its box closes; once per `(map, sprite slot or sign text id)` for the lifetime of the ledger | +| `item` | `explore` | +0.15 | 120 ms | An item ball or a hidden item picked up, on any map; once per item for the lifetime of the ledger | Every value is positive: there are no loss or blackout penalties, and `catalog::rule("blackout")` is `None` by test. The values in one frame sum into `R`, and the network reinforces once with @@ -43,6 +45,18 @@ currently reads "wild win". The event's own label, `CAUGHT #`, is what log, `/status` and the checkpoint. Changing the ticker copy means opening the feed's closed kind set, which this rule deliberately did not do. +`talk` and `item` publish on `explore`, for the same reason `boundary` does: each is the fly +finding what is in a place -- new ground, a door, a person or sign it opened, an item it picked up +-- at the same quiet scale (0.05 to 0.15). Not `area`, which counts maps and is a notable row; not +`story`, which is the plot; not `wildwin`, which is a battle. No feed kind was added, so +`docs/feed-protocol.md` and the stage's switch statements did not move. What did move is the one +word that would have been untrue: the Pokémon Red ticker's `explore` row said "new place", which is +not what a conversation or an item is, and now says **"new find"** ("3 new finds" collapsed), which +is true of all four. The event labels -- `TALKED TO # IN AREA `, `READ SIGN # IN +AREA `, `FOUND ITEM #`, `FOUND A HIDDEN ITEM` -- reach the event log, `/status` and the +checkpoint. Both also reset the stage's stall meter, which counts `explore`: engaging with a +building is progress in the sense the operator asked for. + ## Catch rewards The operator's decision of 2026-09-22: the fly is paid for *keeping* a wild Pokémon, not only for @@ -82,6 +96,86 @@ well below a badge (3.0). A catch of a new species pays 0.80 across two kinds, w a story flag and a badge -- deliberately, because it is the one event that is both a discovery and a thing the fly had to do on purpose. +## Engagement rewards + +The operator's decision of 2026-09-23, recorded with the port decisions: reward the fly for +engaging *inside* buildings and stop paying it for leaving them. It was chosen over a pad rule and +over weighting the choice, and it is a catalog change -- an operator decision, like the catch +reward -- not a loop-review fix (`docs/loop-review.md`). It answers a shape the loop reviews kept +finding in Pewter: `GO OBJECTIVE` into a building and `GO OUT` straight back, paid for the door on +the way out and for nothing inside. + +**Indoors** is two of the cartridge's own tables, and nothing hand-classified +(`pokemon_red/engage.rs`, `indoor`). `CheckIfInOutsideMap` (`home/overworld.asm`) is the game's +outdoor test -- tileset `OVERWORLD` or `PLATEAU` -- and `WarpFound2` labels its other branch +`.indoorMaps`; on its own that would call Viridian Forest and every cave indoors, and their exits +are how the fly gets anywhere. `BikeRidingTilesets` (`data/tilesets/bike_riding_tilesets.asm`) is +the list of places the bicycle may be ridden -- `OVERWORLD`, `FOREST`, `UNDERGROUND`, `SHIP_PORT`, +`CAVERN` -- and the bike is the one thing the cartridge refuses inside a building by rule. A map is +indoors when its `wCurMapTileset` is in neither: every house, mart, Pokémon Center, gym, gate, lab +and museum, the S.S. Anne, Silph Co., the Pokémon Tower, the Mansion, the Rocket Hideout and the +Indigo Plateau's rooms. Not the forest, a cave, the Underground Path or Vermilion's dock. + +**`talk`, +0.10.** Paid on the sample the text box closes, for a conversation that + +1. *the fly opened*: on the last sample before the font bit (`wFontLoaded` bit 0) rose, the fly had + the joypad -- no `wJoyIgnore`, no simulated input, no scripted movement -- was standing still + (`wWalkCounter` zero, the only state the overworld reads A in) and stood where it stands now. A + script's text opens with the joypad taken, or on the frame a step onto a trigger tile ends; +2. *is with the thing in front of it*: `DisplayTextID` copies its argument into `wSpriteIndex` -- + a sprite slot up to `wNumSprites`, or a text id -- and the sprite must stand on the tile the + player faces (or one further, across a counter, on a tileset that has counter tiles, which is + `IsSpriteOrSignInFrontOfPlayer`'s own long reach), or the text id must be the sign's on that + tile. The byte arrives about **twenty frames after** the font bit (measured on the cartridge: + `DisplayTextIDInit` loads the font's tiles first) and until then still names the previous + text's subject, so it is read once it has changed or 45 samples have passed, and only while the + bottom dialogue box is drawn -- the start menu draws its own box elsewhere. An item ball is a + sprite but not a person, and pays `item`; +3. *opened indoors*, on the map the box opened on; +4. *finished*: the box closed on the same map. A conversation that ends in a warp, a rollback or a + restore pays nothing. + +The ledger is the adapter's lifetime `seen` set, keyed `talk::sprite:` or +`talk::sign:` -- the same "map and object index" the macros' `talked` ledger uses, +but **not** that ledger: the macros' ledger is session state and is thrown away on a restore; this +one is checkpointed and survives a rollback, so talking to the same person again, after a restore +or not, pays nothing. The trainer the fly speaks to before a battle is a person and pays once; the +nurse, a clerk and a sign each pay once per map. + +**`item`, +0.15.** Read from the cartridge's own "this one has been taken" bits, on any map. +An item ball is one of the map's toggleable sprites (`wToggleableObjectList`, sprite slot and +global index) whose `wMapSpriteExtraData` is `(item id, 0)` -- the shape `LoadMapHeader` writes for +an `ITEM` `object_event` and for nothing else (a trainer's is `(class, number)` with numbers from +1, a person's two zeroes); `PickUpItem` sets its global bit in `wToggleableObjectFlags` through +`HideObject`, and only after `GiveItem` succeeded, so a full bag pays nothing. A hidden item is a +bit of `wObtainedHiddenItemsFlags`, set by `FoundHiddenItemText` after `GiveItem` and by nothing +else; hidden *coins* have their own bitset and are not items. Either pays when its bit rises +between two playable samples, keyed `item:` or `hidden:`, once for the life +of the ledger. A gift item from a script (the Old Amber, a TM from a person) is not an item ball: +the conversation pays `talk`, and the item nothing. + +**The seed.** The first playable sample that finds the key `items:seeded` absent -- a fresh +adapter, or a `v6` ledger restored under `v7` -- writes a key for every bit the game already shows +as taken and pays for none of them, so a rollback to a slot from before a `v6`-era pickup cannot +pay for taking it again. The two item balls a script *reveals* are left out of the seed, because +their bits are set from a new game until Giovanni's defeat clears them: the Rocket Hideout's Silph +Scope and Lift Key (`$87`, `$88`), the only `ITEM` entries `data/maps/toggleable_objects.asm` starts +`OFF`. + +**`boundary` indoors.** Every exit on an indoor map is still written to the ledger, so +`exit_visited` answers exactly what it did and the macros see no change, but nothing is paid. A +town's doors, a route's edges, the forest's gates and a cave's ladders pay as before. One +consequence, measured on the cartridge (`tests/rom_engage.rs`): for the thirty-odd frames of +`PlayMapChangeSound` the cartridge has written the destination into `wCurMap` while the tileset and +the warp table are still the map being left, so the exit the fly is standing on is classified by +the map it belongs to. Walking into a building through a town door still pays that door's on-exit +half, 0.10, once, keyed under the building's id as it always was; walking out pays nothing. + +**The scale.** A building's worth of engagement -- a few people, a sign, perhaps a ball -- is +0.3 to 0.6: more than the 0.15 its door paid for being left, less than a new Pokédex entry per +person, far below a badge. Everything is once per thing for the lifetime of the ledger, so no +building can be farmed. + ## Gates Semantic rewards are enabled for exactly one cartridge, the SHA-256 in `SUPPORTED_ROM`. Any other @@ -110,6 +204,10 @@ therefore replays none of it. ## Boundary rewards +Since `pokered-unique8-v7` everything below holds **outdoors** -- towns, routes, the forest, +caves -- and on an indoor map the same keys are written and nothing is paid ("Engagement rewards" +above has the definition of indoors and the one transition frame worth knowing about). + `docs/design/room-escape.md` section 2. The rule pays 0.05 the first time the fly stands on a tile orthogonally adjacent to one of the current map's exits, and 0.10 the first time it stands on the exit tile itself. Both are keyed into the adapter's lifetime `seen` ledger as @@ -177,7 +275,15 @@ body picks the macro; the descending neurons press the buttons.** ## Honesty -The catalog now includes catches. The honesty panel's copy is not data-driven from the catalog -- +The catalog now includes conversations and items (v7). Paying for a conversation is the closest +the catalog has come to paying for a *button*: A is what opens one. It is still a reward, not a +press. Nothing in the adapter presses A, chooses when, or tells the fly who is there; the payout is +read out of WRAM after a conversation the fly's own buttons -- or the macro the mushroom body chose +-- opened and finished, and it is once per person or sign for the life of the run, so the thing +that is learned is "the people in a building are worth a visit", not "press A". It is also why the +rule demands evidence that the fly opened the box: text a script started pays nothing. + +The catalog also includes catches. The honesty panel's copy is not data-driven from the catalog -- `apps/stage/src/lib/schedule.ts`'s rotating card is four written lines and lists no kinds -- so there was nothing to regenerate and the copy is unchanged. The sentences below are where the argument lives. diff --git a/infra/05-deploy.sh b/infra/05-deploy.sh index b80557a..c6cd3db 100644 --- a/infra/05-deploy.sh +++ b/infra/05-deploy.sh @@ -300,8 +300,9 @@ if [ -n "$RELEASE_TARBALL" ]; then # # FLY_ACCEPT_ADAPTERS is the *other* override, and the opposite one: it keeps # the run. It names adapter version strings whose checkpoints the new build - # may migrate — e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v5 for the deploy - # that adds the catch reward. It only applies when the adapter segment is the + # may migrate — e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v6 for the deploy + # that adds the engagement rewards (v7; v5 -> v6 was the catch reward's). + # It only applies when the adapter segment is the # ONLY difference between the two strings and the new build's adapter says it # can read that one; a dataset, kernel, emulator or state-format change is # still a refusal, because none of those has a migration. The same variable is @@ -354,7 +355,7 @@ The difference is usually an adapter/ladder or dataset version bump. Three ways * deploy a build whose string matches (check out the commit the running release was built from), or * if the ADAPTER VERSION is the only segment that differs and the new build documents a migration from the old one, re-run with FLY_ACCEPT_ADAPTERS set to the adapter id in the live - string (e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v5). The run is kept; flysim applies the same + string (e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v6). The run is kept; flysim applies the same rule at restore. See docs/design/flysim.md, \"Restoring across an adapter version\", or * accept losing everything the brain has learned and re-run with FLY_RESET_STATE=1, which archives ${state_dir}'s checkpoints to ${state_dir}. (kept, not deleted) and diff --git a/infra/docs/runbook.md b/infra/docs/runbook.md index a1a1cc6..81b7073 100644 --- a/infra/docs/runbook.md +++ b/infra/docs/runbook.md @@ -263,7 +263,7 @@ pct exec $CTID -- /opt/fly/bin/fly-reset-to-milestone $N # (a) the running release already wrote that checkpoint -> nothing to deploy, skip to 5. # (b) the new build bumps the ADAPTER VERSION and nothing else -> name the checkpoint's # adapter so the gate and flysim both migrate instead of refusing: -FLY_ACCEPT_ADAPTERS=pokered-unique8-v5 infra/05-deploy.sh +FLY_ACCEPT_ADAPTERS=pokered-unique8-v6 infra/05-deploy.sh # The gate logs "the adapter version is the only difference, and it is named; the run is KEPT # and migrated", and writes FLY_ACCEPT_ADAPTERS into /etc/fly/fly.env so flysim applies the # same rule at restore. Anything else about the string differing is still a refusal. diff --git a/infra/env/example.env b/infra/env/example.env index 21afecc..4dae070 100644 --- a/infra/env/example.env +++ b/infra/env/example.env @@ -335,8 +335,9 @@ FLY_FEED_VIA=direct # two strings AND the new build's adapter declares a migration from that one. A # dataset, kernel, plasticity, emulator or state-format difference is still a # refusal. Set it for the one deploy that needs it and leave it out afterwards; -# 05-deploy writes it into /etc/fly/fly.env only while it is set. -# FLY_ACCEPT_ADAPTERS=pokered-unique8-v5 +# 05-deploy writes it into /etc/fly/fly.env only while it is set. The v0.6.0 +# deploy (pokered-unique8-v7, the engagement rewards) is the one that needs: +# FLY_ACCEPT_ADAPTERS=pokered-unique8-v6 # --- push mode -------------------------------------------------------------- # local: flypush.service stays disabled, everything else identical to prod.