Compare commits

...

9 commits

Author SHA1 Message Date
acamilo
55129002a4 tests: the placeholder fingerprint in the migration test no longer reads as a MAC address
Some checks are pending
ci / node 22 (test + typecheck) (push) Waiting to run
ci / rust stable (cargo test --workspace --release) (push) Waiting to run
ci / infra/tests/lint.sh (push) Waiting to run
ci / playwright apps/stage (allowed to fail) (push) Waiting to run
2026-09-22 18:59:17 +00:00
acamilo
083abe5f1f Merge feat/sf-state-01: a coherent all-participant checkpoint, group restore and a liftable fence 2026-09-22 18:57:18 +00:00
dev
388e112ad5 Merge main into feat/sf-state-01 2026-09-22 18:45:56 +00:00
dev
45621903be session: name the two ways a durable wait ends without an acknowledgment
Review follow-up on the checkpoint store.

A dropped reply channel and an expired caller budget were both reported as
ReplyLost. They are different facts -- the first means the write is over and its
outcome did not reach here, the second means the save is still going -- so they are
now separate outcomes, and the durable wait has its own budget rather than borrowing
the one that bounds a call to a participant. Both still leave durable metadata where
it was, and for both the resolution asks the store about the same checkpoint.

The writer's two bounds refuse at different moments and the comment claimed
otherwise: the outstanding-capture bound is taken before a capture is requested, and
the byte budget cannot be, because a capture's size is not known until it exists. The
byte check, the decision and the change to the byte total are now one critical
section, the peak is sampled after a superseded job's bytes are gone, and the writer's
own bookkeeping is over a type that holds only the outcomes a writer can produce.

The manifest's coordinator.eventWatermarks is {lastSourceStep, issued}; the fixture
illustrated {lastEventId, lastOrdinal}, and the illustration is what changed, because
an event id is derived from the epoch and cannot be compared across the restore that
gives the session a new one.

checkpoint-envelope-v1 section 3 also now says, under the same dated amendment, that
a required-manifest-field change must bump envelopeVersion once production files
exist: contractDigest is taken over the schema set and does not cover this manifest,
so the envelope version is the only thing that can carry such a change.
2026-09-22 18:45:51 +00:00
acamilo
55221a7c77 docs: v0.5.0 status 2026-09-22 18:41:17 +00:00
acamilo
a8afd8f448 Merge feat/catch-reward: a catch reward, adapter v6 with a v5 migration, and a rung restart 2026-09-22 18:41:15 +00:00
dev
db30708c3f Merge main into feat/sf-state-01 2026-09-22 17:55:03 +00:00
dev
6655a1b1c6 session: coherent all-participant checkpoint and recovery
STATE-01 over the FLYSESS1 envelope CONTRACT-01 specified.

fly-session gains a `state` module: the durable store with its generations, its
rotation and the commit order of checkpoint-envelope-v1 section 5, where the store
manifest rename is the durable commit point; a compatibility block whose comparison
names the identity that differs rather than one opaque digest; and a bounded writer
that owns its payload handles until the bytes are committed or the job fails.

The writer's queue slot is taken before the first State.Capture, so a saturated
writer refuses a capture rather than queueing it without bound, and the refusal is a
BUSY the stepping session survives. Capture and durability are two events: a capture
completes when an immutable capture exists, and only the store manifest rename moves
the durable mark. A lost save reply is an outcome, and the resolution asks the store
about the same checkpoint instead of saving again.

Both worker roles implement State.Capture, State.StageRestore and
State.ActivateRestore, with once-only restore tokens bound to checkpoint, scope,
payload and incarnation. A restore selects a complete compatible generation, imports
every payload as a fresh artifact, stages the group, validates the coordinator's own
ledgers, and only then activates; a failure anywhere leaves the fence closed and
records every participant that staged as one that must be replaced. The fence lifts
at Failed -> Restoring(k) -> Paused(k) and nowhere else.

The task and the action executor gain the capture/validate_restore/install_restore
interfaces workers-v1 section 4 lists, and the ledger can re-derive the event
identities it issued under another epoch, which is what lets a resumed run's
behaviour trace be compared with an uninterrupted one.

media: check_required_audio now takes the observation's provenance instead of
exempting boundary 0. A chunk is the audio of an interval, and the observation
ActivateRestore installs covers none.

checkpoint-envelope-v1 section 3 gains a dated amendment adding `environment` to the
manifest, the holder of the world's own payload, which the table named for every
other participant; `helperState`, which that table already listed, joins the
required-field set in Rust and TypeScript. The fixture was regenerated by the
existing example; the schema set and contractDigest are unchanged.

state-media-v1 section 5 gains a dated amendment for three readings this slice
enforces: the State RPCs' compatibilityDigest is the participant's, not the
manifest's composition-level block; a restored observation carries no audio chunk;
and a participant that staged into an abandoned install must be replaced.
2026-09-22 17:43:43 +00:00
alex
23a4d7379b rewards: a catch reward, adapter v6 with a v5 migration, and a rung restart
The operator's decision of 2026-09-22: pay the fly for keeping a wild Pokemon,
bump the adapter properly, and restart the live run from an early checkpoint
rather than from scratch.

The rule. `catch` is the catalog's ninth kind, appended so the key order
`counts` serializes in does not move. 0.30 for a species this run had never
owned, 0.10 for a repeat, three payouts per species for the lifetime of the
ledger; the `species` rule is untouched, so a first catch of a new species pays
0.80 across two kinds. The catch is read from `wCapturedMonSpecies` ($d11c),
whose comment in ram/wram.asm is "0 if no mon was captured": ItemUseBall zeroes
it before every throw and writes wEnemyMonSpecies into it only on the branch
that keeps the Pokemon, and UseBagItem's `.returnAfterCapturingMon` zeroes it
again and sets wBattleResult to 2 -- a value written on exactly two paths in
the game, that one and a link battle whose opponent ran. Both are required, so
a byte read out of a half-initialised battle cannot pay. Not wPartyCount: a
catch with a full party raises wBoxCount instead, and wPartyCount also rises
for a gift, a trade and a PC withdrawal.

"Never owned this run" is the `species` payout inside the same battle, because
nothing else can set a Pokedex bit during one. It is not read off the captured
species byte: that is the cartridge's internal index while the owned bitset is
by Pokedex number, and nothing in WRAM converts between them.

The address was resolved by tools/resolve_wram.py, not written by hand. The
tool needed NUM_TMS and NUM_HMS, which the decomp defines through its `const`
enumeration, so it now counts them from the file's own add_tm/add_hm
definitions and cross-checks NUM_TMS against the literal the same file
declares.

The feed's kinds are closed, so `catch` publishes on `wildwin` and nothing in
packages/feed or apps/stage changed. Deliberately not `pokedex`: the `species`
rule already pays for the bit the same catch sets. The stage's ticker copy is
keyed on the feed kind, so a catch row reads "wild win" -- stated in
docs/rewards-learning.md rather than left to be discovered.

v5 -> v6. STATE_VERSION stays 4: the rule adds one counter, `catchCounts`, and
changes nothing else, so a v5 state restores with it empty. That migration is
opt-in and needs all three of: the adapter segment being the only difference
between the two compatibility strings, the running adapter listing the
checkpoint's adapter in `migrates_from()`, and the deploy naming it in
FLY_ACCEPT_ADAPTERS. flysim applies the rule at restore and 05-deploy's gate
applies the same rule before it flips the symlink, writing the variable into
fly.env so the two cannot disagree.

The restart. infra/bin/fly-reset-to-milestone <N> archives both stores to a
dated directory, rewrites milestone-<N>.checkpoint with the ratchet's attempts
and recoveries at zero, installs it as the newest generation of both stores,
clears the milestone archives above N and the event log, and prints what it
did. It refuses while flysim is running and refuses a rung the run never
reached. The envelope work is in flysim::reset (`flysim
--reset-to-milestone N`); the shell script is the operator's wrapper.

Tests: catalog values and order; a synthetic WRAM trace of a catch (new,
repeat, cap, already-owned species, trainer/Safari/old-man/missed-ball
negatives, rollback replay); a v5 state restoring with the counter at zero; a
v5 checkpoint fixture accepted with the opt-in and refused without it; the
reset tool against copies of a state dir in temp directories; and a ROM-gated
catch from a rung-9 forest checkpoint, driven by the shipping THROW BALL macro.

The compatibility string differs from main's in exactly one segment, checked by
splitting both on `/`: pokered-unique8-v5 -> pokered-unique8-v6.
2026-09-22 17:30:00 +00:00
45 changed files with 7945 additions and 83 deletions

View file

@ -355,6 +355,50 @@ on-screen ticker cannot disagree with what the sim did.
against the prototype's WASM size and diffs a known save. If they match, prototype checkpoints
import and the segment records the shared tag; if not, milestone saves must be re-earned and that
is a stated M3 finding.
- **Restoring across an adapter version** (2026-09-22). The compatibility string is compared
whole, so bumping the reward adapter refuses every checkpoint the previous one wrote -- which is
the right default and was, until now, the only behaviour. It is the wrong default for a change
that only *adds* a rule: `pokered-unique8-v6` adds the catch reward and one counter,
`catchCounts`, and means the same thing as `v5` for every other field, so a `v5` run is
resumable and throwing it away would be a choice nobody made deliberately.
So there is one narrow, opt-in migration, `flybrain_gb::compatibility::decide`, and it requires
**all three** of:
1. the two compatibility strings differ in the adapter segment (segment 1) and **nowhere else**.
A dataset, kernel, plasticity, emulator-revision, symbol-provenance or state-format
difference is still a refusal: none of those has a migration, and a fly restored across one
is a different fly;
2. the running adapter's `migrates_from()` lists the checkpoint's adapter, so the code that will
read that state says out loud that it can. Pokémon Red's list is `["pokered-unique8-v5"]` and
nothing else -- `v4` is excluded because its ledger holds no `boundary:` keys and resuming it
would pay a second time for every exit already found, and `v3` because its stored rank is a
rung on a different ladder;
3. the deploy names the same adapter id in **`FLY_ACCEPT_ADAPTERS`** (comma- or
space-separated). Unset or empty migrates nothing, which is what every deploy before this one
did.
Condition 2 without 3 would make the migration silent; condition 3 without 2 would let an
operator wave through a pair nobody wrote a migration for. `infra/05-deploy.sh`'s compatibility
gate applies the same rule before it flips the `current` symlink, and writes the variable into
`/etc/fly/fly.env` so flysim applies it at restore -- the two must agree, or a deploy would pass
a gate that flysim then fails, which is the black stream the gate exists to prevent. The
migration itself is `PokemonRedReward::import_state` doing what it already did: `catchCounts` is
absent from a `v5` state and restores empty, which is the truth about a run that was never paid
for a catch. `STATE_VERSION` does not move, because the schema did not.
- **Restarting a run from an earlier rung** (2026-09-22). `FLY_RESET_STATE=1` throws the run away;
`infra/bin/fly-reset-to-milestone <N>` keeps it and rewinds it. It archives both stores to a
dated directory, rewrites `milestone-<N>.checkpoint` with the ratchet's `attempts` and
`recoveries` at zero (so the restarted run does not begin with its recovery budget already
spent), installs it as the newest generation of the hot and durable stores, removes the
milestone archives above N, and clears the event log -- whose id sequence the restored
checkpoint's `lastEventId` rewinds. `best` is not touched: the archive's own `best` is the rung
it was taken at, and the rank the stream shows is recomputed by the adapter from the restored
game state. The implementation is `flysim::reset` (`flysim --reset-to-milestone N`) rather than
the shell script, because two of those steps are inside the envelope. The sequence around it is
in `infra/docs/runbook.md`.
- **A running macro is not checkpointed** (2026-09-16, `docs/design/macros.md`). Palette mode's
state — the scene, the palette, the running macro, its plan and its frame count — is transient,
like the readout's blocked-direction cooldown and for the same reason: a restore that resumed a

View file

@ -118,6 +118,20 @@ Addresses are at the pinned commit. "Verified" is one of:
| which slot is out | `wPlayerMonNumber` | `$cc2f` | 0-based party slot | ROM, trace |
| the enemy | `wEnemyMonSpecies`, `wEnemyMonHP`, `wEnemyMonLevel`, `wEnemyMonMaxHP` | `$cfe5`, `$cfe6`, `$cff3`, `$cff4` | HP big-endian. Not written on the frame a battle starts — the reward adapter's own comment says the same — so the enemy is `None` for the first few hundred frames of a battle. | ROM (the rival's Squirtle, level 5, 20/20, and `None` on the first frame), trace |
| how many moves | `wNumMovesMinusOne` | `$cd6c` | the move count minus one, valid in a battle | trace |
| **a ball kept this one** | `wCapturedMonSpecies` | `$d11c` | **new 2026-09-22** (the catch reward, `docs/rewards-learning.md`). `ram/wram.asm`'s own comment is "0 if no mon was captured". `ItemUseBall` zeroes it before every throw (`.canUseBall`) and writes `wEnemyMonSpecies` into it only on the branch that keeps the Pokémon; `UseBagItem`'s `.returnAfterCapturingMon` zeroes it again and sets `wBattleResult` to 2 on the way out of the battle. It is therefore non-zero for the hundreds of frames the catch's text and Pokédex screen take, and zero everywhere else. The value is the **internal** species index, like `wEnemyMonSpecies` and unlike `wPokedexOwned`'s bit index. Address resolved by `services/flysim/tools/resolve_wram.py`, bracketed by `wFontLoaded` and `wForcePlayerToChooseMon`. | survey (`tests/rom_catch.rs`: a real wild battle from a rung-9 checkpoint, balls thrown by the `THROW BALL` macro, the byte read out of the running game), trace (`pokemon_red/tests.rs`) |
`wBattleResult` (`$cf0b`) is the second half of that row and is worth its own sentence: it is 0
for a win, 1 for a loss, and 2 on exactly two paths in the whole game -- `.returnAfterCapturingMon`
and a *link* battle whose opponent ran (`engine/battle/core.asm`), which this cartridge never has.
So "the captured-species byte was non-zero during the battle **and** the result is 2" is a catch
and nothing else. `InitBattleVariables`, `ResetStatusAndHalveMoneyOnBlackout` and
`HandleFlyWarpOrDungeonWarp` all clear it, so a stale 2 cannot survive into the next battle.
Not used for the catch, and why: `wPartyCount` (`$d163`) rises on a catch **only** when the party
has room -- a full party sends the Pokémon to `wBoxCount` instead -- and it also rises for a gift,
a trade and a Pokémon withdrawn from the PC. Reading a catch off it would need a second rule to
tell those apart. The cartridge's own flag needs none, which is why the row above is the one the
adapter reads.
### Battle menu and cursor, own turn against forced switch

View file

@ -105,6 +105,31 @@ names; a manifest missing any of them is not a complete checkpoint.
| `helperState` | External-helper state required for exact resume, as payload names |
| `payloads` | `[{name, byteLength, digest}]`, mirroring the payload table |
**Amendment, 2026-09-22 (STATE-01).** The table above names a holder for every payload except
the environment's own, although section 6's fixture has one (`world`) and a group install has
to map it by name like any other participant's. The manifest therefore also records:
| Field | Contents |
| --- | --- |
| `environment` | `{workerId, payload}`: which worker the world belonged to and the payload name holding its state |
The reference implementations' required-field set was also missing `helperState`, which this
section has listed from the start. Both are now in `REQUIRED_MANIFEST_FIELDS` in Rust and in
TypeScript, and the fixture was regenerated by the existing example. The schema set is
untouched, so `contractDigest` is unchanged.
`coordinator.eventWatermarks` is `{lastSourceStep, issued}`. The fixture illustrated
`{lastEventId, lastOrdinal}`, and it is the illustration that changed: an event id is derived
from the epoch, so a watermark spelled as one cannot be compared across the restore that
gives the session a new epoch, while a source step and an issued count can.
**A required-manifest-field change is compatibility-relevant and `contractDigest` does not
cover it.** The digest is taken over the schema set, and this manifest is not in it, so
`envelopeVersion` is the only thing that can carry such a change. It stays `1` here only
because no production `FLYSESS1` file exists yet: once one does, adding or removing a required
manifest field **must** bump `envelopeVersion`, because a reader of the older version would
otherwise accept a file it cannot completely read, or refuse one it could.
`payloads` is redundant with the table on purpose: the table is what a reader needs to map
bytes, and the manifest is what a store lists, compares and reports without opening the
payload area. A reader checks that the two agree.

View file

@ -190,6 +190,28 @@ restored time. It cannot advance gameplay to manufacture it. Capture/reconstruct
covers render/inspection state and any pending sensor pipeline. Agent state agrees with it;
do not replay reward or recalibrate merely to fill missing cached data.
**Amendment, 2026-09-22 (STATE-01).** Three readings of this section, made explicit because
they are now enforced:
- `compatibilityDigest` on `CaptureResult` and `StageRestoreParams` is the **participant's**
capture compatibility digest of [worker interfaces](workers-v1.md) section 2 -- profile,
resolved seed, numerical model version and effective instance configuration for an agent;
backend, content, patch, controller and parser identity for an environment. It is not the
manifest's `compatibility` block of section 4, which is the composition's and which the
coordinator compares before anything is asked to stage. Both exist because they answer
different questions, and a restore that passed the second could still be handing an agent
another agent's brain.
- The observation `ActivateRestore` returns ran no transition, so it carries **no audio
chunk**, and one in it is refused. Section 2's chunk is the audio of an interval and this
observation covers none; MEDIA-01 implemented that rule as "boundary 0 carries no chunk",
which is true of the only such observation that slice could produce and false of this one.
The rule is about provenance, not about the boundary number.
- A participant that staged into a group install the coordinator then abandoned must be
**replaced** before another restore, exactly as one that activated must. It is holding a
validated replacement state that nothing installed, and [session RPC](ipc-v1.md) section 6
already refuses to silently reattach such a participant to an active epoch. Without this the
group's second attempt meets its own leftovers and calls them a conflict.
If emulator validation requires mutation, stage a stopped replacement emulator. If that cannot
provide externally atomic resume, advertise episode-restart, not exact-checkpoint. After all
activation acknowledgments, install the coordinator's staged task/executor/admission state

View file

@ -1,6 +1,6 @@
# Rewards and learning
The live reward catalog of the Pokémon Red adapter, `pokered-unique8-v5`. The code of record is
The live reward catalog of the Pokémon Red adapter, `pokered-unique8-v6`. The code of record is
`services/flysim/crates/flybrain-gb/src/pokemon_red/` (`catalog.rs` holds the values, `mod.rs` the
gates and the rules); this page says what each rule pays for and why it is allowed to. The
prototype's own `docs/rewards-learning.md` in `fly-plays-pokemon` is where the first seven rules
@ -24,6 +24,7 @@ change what the fly can do.
| `battle` | `wildwin` | +0.1, +0.05, +0.0333 | 100 ms | At most three observed wild KOs per `(map, species, level)` |
| `badge` | `badge` | +3 | 400 ms | Each newly set badge bit |
| `boundary` | `explore` | +0.05, +0.10 | 100 ms | First tile adjacent to one of the map's exits, and the exit tile itself; once per `(map, exit)` for the lifetime of the ledger |
| `catch` | `wildwin` | +0.30, +0.10 | 150 ms | A wild Pokémon kept by a ball: +0.30 for a species this run had never owned, +0.10 for a repeat; at most three payouts per species for the lifetime of the ledger |
Every value is positive: there are no loss or blackout penalties, and `catalog::rule("blackout")`
is `None` by test. The values in one frame sum into `R`, and the network reinforces once with
@ -33,6 +34,54 @@ The feed-kind column is `RewardKind::from_adapter` in `services/flysim/crates/fl
`docs/feed-protocol.md` publishes seven counters, and an adapter kind that has no counter of its
own shares the nearest one. It still reaches the page as an event with its own label.
Two consequences of that sharing are worth stating rather than discovering. `catch` publishes on
`wildwin` because a catch is a wild battle the fly won by keeping the Pokémon, and *not* on
`pokedex` because the `species` rule already pays for the Pokédex bit the same catch sets --
counting it twice would be the dishonest option. And the stage's ticker copy is keyed on the feed
kind, not on the catalog kind (`apps/stage/src/games/pokemon-red.ts`), so the row for a catch
currently reads "wild win". The event's own label, `CAUGHT #<species>`, is what reaches the event
log, `/status` and the checkpoint. Changing the ticker copy means opening the feed's closed kind
set, which this rule deliberately did not do.
## Catch rewards
The operator's decision of 2026-09-22: the fly is paid for *keeping* a wild Pokémon, not only for
knocking one out. The rule is one kind with two payouts, the way `boundary` is.
**How a catch is read.** From `wCapturedMonSpecies` (`$d11c`), whose comment in `ram/wram.asm` at
the pinned commit is "0 if no mon was captured". `ItemUseBall` zeroes it before every throw
(`.canUseBall`) and writes `wEnemyMonSpecies` into it only on the branch that keeps the Pokémon;
`UseBagItem`'s `.returnAfterCapturingMon` zeroes it again and sets `wBattleResult` to 2 on the way
out of the battle. `wBattleResult` is 2 on exactly two paths in the whole game -- that one, and a
link battle whose opponent ran -- so requiring both the species and the result means a byte read
out of a half-initialised battle cannot pay. The adapter records the species during the battle and
pays on the way out, where the wild-KO payout already lives.
Not from `wPartyCount`. A catch with a full party raises `wBoxCount` instead, and `wPartyCount`
also rises for a gift, a trade and a Pokémon taken out of the PC, so it would need a second rule
to mean anything. The cartridge's own flag needs none.
**What counts as a new species.** The `species` payout inside the same battle. Nothing but a catch
can set a `wPokedexOwned` bit during a wild battle, so a `species` payout between the battle
starting and the ball keeping the Pokémon *is* that Pokémon being new to the run. It is read this
way rather than off `wCapturedMonSpecies` because that byte is the cartridge's **internal** species
index while the owned bitset is by **Pokédex number**, and nothing in WRAM converts between the two
(`docs/design/macros-wram.md` section 2, "species numbering"). A battle restored from a checkpoint
written before this rule existed carries no "species payouts when it started", which reads as
"cannot tell" and pays the repeat amount: the conservative half, and at most 0.20 once.
**The budget.** Three payouts per species for the lifetime of the ledger, the same cap and the
same reason as the wild-KO rule's three: a species the fly can find over and over is a farm, and
three is enough for the behaviour to be learned. A rollback blocks every species already paid,
exactly as it blocks every wild-KO key already paid, so the same catch cannot be replayed for
reward. A Safari Zone or old-man battle pays nothing, because the whole sample is dropped a step
earlier with a visible mode; a trainer battle pays nothing, because balls cannot be thrown in one.
**The scale.** 0.30 on its own is below a new Pokédex entry (0.50), below a story flag (1.0) and
well below a badge (3.0). A catch of a new species pays 0.80 across two kinds, which sits between
a story flag and a badge -- deliberately, because it is the one event that is both a discovery and
a thing the fly had to do on purpose.
## Gates
Semantic rewards are enabled for exactly one cartridge, the SHA-256 in `SUPPORTED_ROM`. Any other
@ -128,7 +177,19 @@ body picks the macro; the descending neurons press the buttons.**
## Honesty
The catalog now includes exits. That is worth saying plainly on the honesty panel, because paying
The catalog now includes catches. The honesty panel's copy is not data-driven from the catalog --
`apps/stage/src/lib/schedule.ts`'s rotating card is four written lines and lists no kinds -- so
there was nothing to regenerate and the copy is unchanged. The sentences below are where the
argument lives.
Paying for a catch does not move the fly: the ball is thrown by a macro the mushroom body chose
among the ones the battle scene put on the pad, and the payout is read out of WRAM after the
frame. What it does do is make one of the palette's existing macros worth choosing, which is the
same kind of pressure every other rule applies. The cap is what keeps it from becoming a farm: a
run that finds one patch of grass and throws balls at the same species all night earns 0.50 from
it and then nothing.
The catalog also includes exits. That is worth saying plainly on the honesty panel, because paying
for a door is closer to telling the fly where to go than paying for a badge is:
- **still no button path.** Nothing in the adapter chooses or biases a button. The reward is read

View file

@ -736,3 +736,11 @@ rewritten separately.
(73/73) because 82% of the fixed run is battle time; Fable shipped it on the same judgement as
v0.4.6 and started row 50 (MOVE n blocked on an unresponsive move list). The on-screen chat ring
now survives a sim restart (sidecar in the hot dir, never in the checkpoint).
- 2026-09-22 (v0.5.0, the operator's decision): the fly is paid for keeping a wild Pokémon. New
catalog kind `catch` (0.30 for a species this run never caught, 0.10 for a repeat, three payouts
per species), read from the captured-species byte and the battle result together; the existing
species rule still pays on top. Adapter `pokered-unique8-v6`; the compatibility string differs
in the adapter segment only, and a deploy with `FLY_ACCEPT_ADAPTERS=pokered-unique8-v5` migrates
a v5 checkpoint instead of refusing it. `fly-reset-to-milestone <N>` restarts the run from a
ladder rung (archives both stores first). The live run restarts from rung 7 with this release, so
the ladder is climbed again with the catch reward and the row-54 walks in place.

63
infra/05-deploy.sh Executable file → Normal file
View file

@ -186,6 +186,36 @@ else
log "05-deploy: CPUSET unset — heavy in-container steps run unpinned (no partition configured)"
fi
# Whether the only difference between two compatibility strings is the adapter
# segment, and FLY_ACCEPT_ADAPTERS names the adapter the live checkpoints carry.
#
# The bash half of flybrain_gb::compatibility::decide, which is what flysim
# itself applies at restore. Both have to agree: a gate that let a deploy
# through and a flysim that then refused every checkpoint would be the black
# stream this whole section exists to prevent. The string is
# {kernel}/{adapter}/{fingerprint}/{plasticity}/binjgb:{rev}/pokered:{commit}/statefmt:{id},
# so the adapter is segment 1 and nothing else may move.
adapter_migration_accepted() {
local live="$1" new="$2" accepted="$3"
local -a live_parts new_parts
IFS='/' read -r -a live_parts <<< "$live"
IFS='/' read -r -a new_parts <<< "$new"
[ "${#live_parts[@]}" -eq "${#new_parts[@]}" ] || return 1
local i differing=0 index=-1
for ((i = 0; i < ${#live_parts[@]}; i++)); do
if [ "${live_parts[$i]}" != "${new_parts[$i]}" ]; then
differing=$((differing + 1))
index=$i
fi
done
[ "$differing" -eq 1 ] && [ "$index" -eq 1 ] || return 1
local entry
for entry in ${accepted//,/ }; do
[ "$entry" = "${live_parts[1]}" ] && return 0
done
return 1
}
# cpu_pin CMD [ARGS...] — run CMD inside the container on the page cpus.
# Falls through to a plain ct_exec when no partition is configured, so this is
# a no-op on an unpartitioned container rather than a new failure mode (a
@ -262,6 +292,16 @@ if [ -n "$RELEASE_TARBALL" ]; then
# checkpoints and the on-screen chat ring's sidecar — so the new build warms
# up fresh. Everything learned so far is thrown away, which is why it is not
# the default.
#
# FLY_ACCEPT_ADAPTERS is the *other* override, and the opposite one: it keeps
# the run. It names adapter version strings whose checkpoints the new build
# may migrate — e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v5 for the deploy
# that adds the catch reward. It only applies when the adapter segment is the
# ONLY difference between the two strings and the new build's adapter says it
# can read that one; a dataset, kernel, emulator or state-format change is
# still a refusal, because none of those has a migration. The same variable is
# written into /etc/fly/fly.env below, so flysim applies the same rule at
# restore that this gate applied at deploy.
# -----------------------------------------------------------------------
state_dir="${FLY_STATE_DIR:-/srv/fly/state}"
hot_dir="${FLY_STATE_HOT_DIR:-/run/fly/state}"
@ -287,6 +327,11 @@ if [ -n "$RELEASE_TARBALL" ]; then
log "05-deploy: no decodable checkpoint in ${state_dir} — nothing to compare, continuing"
elif [ "$new_compat" = "$live_compat" ]; then
log "05-deploy: checkpoint compatibility matches the live state, the new build will restore it"
elif [ -n "${FLY_ACCEPT_ADAPTERS:-}" ] \
&& adapter_migration_accepted "$live_compat" "$new_compat" "$FLY_ACCEPT_ADAPTERS"; then
log "05-deploy: FLY_ACCEPT_ADAPTERS=${FLY_ACCEPT_ADAPTERS} — the adapter version is the only difference, and it is named; the run is KEPT and migrated"
log "05-deploy: live: $live_compat"
log "05-deploy: new: $new_compat"
elif [ "${FLY_RESET_STATE:-0}" = 1 ]; then
archive="${state_dir}.$(date -u +%Y%m%d%H%M%S)"
log "05-deploy: FLY_RESET_STATE=1 — compatibility CHANGED, archiving the durable state to ${archive} and clearing the hot ring"
@ -300,8 +345,12 @@ if [ -n "$RELEASE_TARBALL" ]; then
die "05-deploy: REFUSING to deploy release ${version}: its checkpoint compatibility string does not match the live state in ${state_dir}, so flysim would refuse every checkpoint there and then refuse to start at all — a black stream.
live state: ${live_compat}
new build: ${new_compat}
The difference is usually an adapter/ladder or dataset version bump. Two ways forward:
The difference is usually an adapter/ladder or dataset version bump. Three ways forward:
* deploy a build whose string matches (check out the commit the running release was built from), or
* if the ADAPTER VERSION is the only segment that differs and the new build documents a
migration from the old one, re-run with FLY_ACCEPT_ADAPTERS set to the adapter id in the live
string (e.g. FLY_ACCEPT_ADAPTERS=pokered-unique8-v5). The run is kept; flysim applies the same
rule at restore. See docs/design/flysim.md, \"Restoring across an adapter version\", or
* accept losing everything the brain has learned and re-run with FLY_RESET_STATE=1, which
archives ${state_dir}'s checkpoints to ${state_dir}.<timestamp> (kept, not deleted) and
clears ${hot_dir} so the new build warms up fresh.
@ -457,6 +506,16 @@ trap 'rm -f "$tmp_fly_env" "$tmp_flypush_env"' EXIT
if [[ -n "${FLY_MACRO_BLOCKED_MINUTES:-}" ]]; then
echo "FLY_MACRO_BLOCKED_MINUTES=${FLY_MACRO_BLOCKED_MINUTES}"
fi
# Adapter versions whose checkpoints this build may migrate
# (flybrain_gb::compatibility, docs/design/flysim.md "Restoring across an
# adapter version"). Only written when it is set, because the safe state is
# absent: an empty or missing variable migrates nothing, which is what every
# deploy before 2026-09-22 did. It stays in fly.env for as long as the
# operator leaves it on the deploy command line, so removing the opt-in is
# one deploy without it.
if [[ -n "${FLY_ACCEPT_ADAPTERS:-}" ]]; then
echo "FLY_ACCEPT_ADAPTERS=${FLY_ACCEPT_ADAPTERS}"
fi
# flybridge (services/bridge/src/config.ts). Nothing wrote these before, so
# flybridge.service had no EnvironmentFile= at all and the service refused to
# start with "CHANNEL is required / BOT_USER is required / GAME_TITLE is
@ -607,7 +666,7 @@ fi
# ---------------------------------------------------------------------------
log "05-deploy: converging bin/ helpers to /opt/fly/bin"
ct_exec "$CTID" -- mkdir -p /opt/fly/bin
for name in fly-watchdog fly-recap fly-retention flypush flystage-launch flycast-launch wait-for-x wait-for-stage wait-for-health; do
for name in fly-watchdog fly-recap fly-retention fly-reset-to-milestone flypush flystage-launch flycast-launch wait-for-x wait-for-stage wait-for-health; do
converge_file "$CTID" "$INFRA_DIR/bin/$name" "/opt/fly/bin/$name" 0755 root:root >/dev/null
done

View file

@ -0,0 +1,77 @@
#!/usr/bin/env bash
# infra/bin/fly-reset-to-milestone — restart the run from an earlier ladder rung,
# instead of from scratch.
#
# The operator's decision of 2026-09-22: "restart the live run from an early
# checkpoint instead of from scratch". 05-deploy's FLY_RESET_STATE=1 cannot do
# that — it archives the durable state and the next start warms up a fresh fly,
# losing everything the brain has learned. This promotes one milestone archive
# (milestone-<N>.checkpoint, written at the first commit at a new best rank and
# never rotated away) to being what both stores restore.
#
# Usage: fly-reset-to-milestone <N>
# Run INSIDE the container, as root, with flysim STOPPED. It refuses
# otherwise, and it refuses a rung this run never reached.
#
# The whole sequence — stop, reset, deploy with the adapter opt-in, start,
# verify the rank — is in infra/docs/runbook.md, "Restart the run from a rung".
# Nothing here is destructive on its own: every file in both stores is copied to
# a dated directory next to the durable one before anything is rewritten.
set -euo pipefail
: "${FLY_STATE_DIR:=/srv/fly/state}"
: "${FLY_STATE_HOT_DIR:=/run/fly/state}"
: "${FLY_RELEASE_DIR:=/opt/fly/current}"
: "${FLY_SERVICE:=flysim.service}"
: "${FLY_USER:=fly}"
FLYSIM="${FLY_BIN:-${FLY_RELEASE_DIR}/flysim}"
log() { echo "fly-reset-to-milestone: $*" >&2; }
die() { log "$*"; exit 1; }
RANK="${1:-}"
if [ "$#" -ne 1 ] || ! [[ "$RANK" =~ ^[0-9]+$ ]]; then
die "usage: fly-reset-to-milestone <rung> (e.g. fly-reset-to-milestone 9)"
fi
# --- refusals ----------------------------------------------------------------
# A running flysim owns both stores: it commits a hot checkpoint every few
# seconds and a durable one every few minutes, so a reset underneath it would be
# overwritten within the minute and the tool would have lied.
if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet "$FLY_SERVICE"; then
die "$FLY_SERVICE is running. Stop it first: systemctl stop $FLY_SERVICE"
fi
[ -x "$FLYSIM" ] || die "no flysim binary at $FLYSIM (set FLY_BIN to point at one)"
milestone="${FLY_STATE_DIR}/milestone-${RANK}.checkpoint"
# The binary refuses this too, and refuses before it copies anything; checking
# here as well is what makes the message name the rungs that do exist.
if [ ! -f "$milestone" ]; then
log "no milestone archive for rung ${RANK}: $milestone does not exist."
log "rungs this run reached:"
ls -1 "${FLY_STATE_DIR}"/milestone-*.checkpoint 2>/dev/null \
| sed 's|.*/milestone-||; s|\.checkpoint$||' | sort -n | tr '\n' ' ' >&2 || true
echo >&2
exit 1
fi
# --- the reset ---------------------------------------------------------------
log "resetting to rung ${RANK} (durable ${FLY_STATE_DIR}, hot ${FLY_STATE_HOT_DIR})"
FLY_STATE="$FLY_STATE_DIR" FLY_STATE_HOT="$FLY_STATE_HOT_DIR" \
"$FLYSIM" --reset-to-milestone "$RANK"
# flysim runs unprivileged; this tool runs as root, so everything it wrote and
# everything it archived has to go back to the service account.
if command -v chown >/dev/null 2>&1 && id "$FLY_USER" >/dev/null 2>&1; then
chown -R "${FLY_USER}:${FLY_USER}" "$FLY_STATE_DIR" "$FLY_STATE_HOT_DIR" 2>/dev/null || true
for dir in "${FLY_STATE_DIR}".reset-*; do
[ -d "$dir" ] && chown -R "${FLY_USER}:${FLY_USER}" "$dir"
done
fi
log "done. Next, per infra/docs/runbook.md:"
log " 1. deploy the build whose adapter wrote that checkpoint, or deploy the new"
log " one with FLY_ACCEPT_ADAPTERS set to the checkpoint's adapter id"
log " 2. systemctl start $FLY_SERVICE"
log " 3. curl -s localhost:7401/status | grep -o '\"rank\":[0-9]*'"

View file

@ -231,6 +231,59 @@ auto-reset (`docs/design/flysim.md` section 8: "no automatic fresh start, ever")
is deliberate — a silent reset would be indistinguishable from real progress on stream.
A deliberate reset means moving `/srv/fly/state` aside by hand.
## Restart the run from a rung
When the run has to go back to an earlier milestone rather than start over — the operator's
decision of 2026-09-22 was "restart the live run from an early checkpoint instead of from
scratch". `FLY_RESET_STATE=1` is the wrong tool: it archives the durable state and the next start
warms up a fresh fly, losing everything the brain has learned.
`infra/bin/fly-reset-to-milestone <N>` promotes `milestone-<N>.checkpoint` to being what both
stores restore, with the ratchet's attempts and recoveries back at zero. It copies every file in
both stores to `/srv/fly/state.reset-<UTC>` first, so it is reversible by hand. It refuses while
flysim is running, and refuses a rung this run never reached.
The whole sequence, in order. Claim the container in the host's agent claim log first, like any
other work on it.
```
CTID=<release-ctid>
N=9 # the rung to restart from
# 1. what rungs exist at all
pct exec $CTID -- ls -1 /srv/fly/state/milestone-*.checkpoint
# 2. stop flysim (it owns both stores; a reset underneath it is overwritten within the minute)
pct exec $CTID -- systemctl stop flysim.service
# 3. the reset. Prints what it did, one line per step.
pct exec $CTID -- /opt/fly/bin/fly-reset-to-milestone $N
# 4. deploy. Two cases:
# (a) the running release already wrote that checkpoint -> nothing to deploy, skip to 5.
# (b) the new build bumps the ADAPTER VERSION and nothing else -> name the checkpoint's
# adapter so the gate and flysim both migrate instead of refusing:
FLY_ACCEPT_ADAPTERS=pokered-unique8-v5 infra/05-deploy.sh <release-env> <release-tarball>
# The gate logs "the adapter version is the only difference, and it is named; the run is KEPT
# and migrated", and writes FLY_ACCEPT_ADAPTERS into /etc/fly/fly.env so flysim applies the
# same rule at restore. Anything else about the string differing is still a refusal.
# 5. start
pct exec $CTID -- systemctl start flysim.service
# 6. verify: the rank is the rung, and the restore came from the generation the tool wrote
pct exec $CTID -- curl -s http://127.0.0.1:7401/status | jq '.milestone.rank, .game.badges, .checkpoint'
pct exec $CTID -- journalctl -u flysim -n 40 --no-pager | grep -E 'restored|migration|compatibility'
```
Step 6 is the one that must be read rather than assumed. The rank is recomputed by the adapter
from the restored game state, not taken from the ratchet, so a rank that is *not* N means the
milestone archive was taken somewhere other than where its name says — stop and look before
starting a stream on it.
To undo: stop flysim, move the contents of `/srv/fly/state.reset-<UTC>/durable` back into
`/srv/fly/state`, delete the generation the tool wrote, and start again.
## Restore from the backup host
```

14
infra/env/example.env vendored
View file

@ -317,6 +317,20 @@ FLY_MACRO_MODE=raw
# target once more. Unset means the default, 10.
# FLY_MACRO_BLOCKED_MINUTES=10
# --- restoring across an adapter version ------------------------------------
# Adapter version strings whose checkpoints this build may migrate, comma- or
# space-separated (docs/design/flysim.md, "Restoring across an adapter
# version"). Unset -- the default, and what every deploy before 2026-09-22 did
# -- migrates nothing: a build whose compatibility string differs from the live
# state's is refused by 05-deploy's gate and by flysim at restore.
#
# It applies only when the ADAPTER segment is the only difference between the
# two strings AND the new build's adapter declares a migration from that one. A
# dataset, kernel, plasticity, emulator or state-format difference is still a
# refusal. Set it for the one deploy that needs it and leave it out afterwards;
# 05-deploy writes it into /etc/fly/fly.env only while it is set.
# FLY_ACCEPT_ADAPTERS=pokered-unique8-v5
# --- push mode --------------------------------------------------------------
# local: flypush.service stays disabled, everything else identical to prod.
# twitch: flypush.service is enabled by 07-enable.sh.

View file

@ -223,7 +223,12 @@ export function decode(input: Uint8Array): Envelope {
};
}
/** The manifest fields state-media-v1 section 4 requires. */
/**
* The manifest fields state-media-v1 section 4 requires.
*
* `helperState` and `environment` join the list under the 2026-09-22 amendment to
* checkpoint-envelope-v1 section 3.
*/
export const REQUIRED_MANIFEST_FIELDS = [
'envelopeVersion',
'checkpointId',
@ -236,6 +241,8 @@ export const REQUIRED_MANIFEST_FIELDS = [
'compatibility',
'agents',
'coordinator',
'environment',
'helperState',
'payloads',
] as const;

View file

@ -197,8 +197,9 @@ fn checkpoint_envelope() -> String {
"priorInspection": "prior-inspection",
"executorState": [{"agentId": "fly-a", "payload": "executor-fly-a"}],
"admissionState": null,
"eventWatermarks": {"lastEventId": "evt-1", "lastOrdinal": "7"},
"eventWatermarks": {"lastSourceStep": "42", "issued": "7"},
},
"environment": {"workerId": "arena", "payload": "world"},
"helperState": [],
"payloads": payload_table(),
});

View file

@ -59,10 +59,14 @@
],
"admissionState": null,
"eventWatermarks": {
"lastEventId": "evt-1",
"lastOrdinal": "7"
"lastSourceStep": "42",
"issued": "7"
}
},
"environment": {
"workerId": "arena",
"payload": "world"
},
"helperState": [],
"payloads": [
{
@ -115,49 +119,49 @@
}
],
"envelope": {
"base64": "RkxZU0VTUzEBAAAAIAAAAAAIAAAFAAAAIAgAAAAAAAB7ImFnZW50cyI6W3siYWdlbnRJZCI6ImZseS1hIiwiYnJhaW5UaWNrcyI6IjI1MzQiLCJkYXRhc2V0RGlnZXN0IjoiNmMwYWYxZjA3ODRlZjYzYTM5M2VlNzdkNjE0ZTgyNDZjNjI1MDUxMzYwZjNmMWE0ODgzODM3NGM1ZDM1NWI1MiIsIm1vZGVsVmVyc2lvbiI6ImxpZi0xbXMtZjY0LXYyIiwicGF5bG9hZCI6ImFnZW50LWZseS1hIiwicGxhc3RpY2l0eVZlcnNpb24iOiJmbHkta2MtbWJvbi1yc3RkcC12MiIsInByb2ZpbGVEaWdlc3QiOiIxOTAwZWFiNmMwMjg0ODNkNzEyNjU5OWVlNmY1MGRlMGQyNzkwN2I1YzY1ZmE5MDUyNDU4MGI0YjBmOTg1MmIwIiwicmVtYWluZGVyIjp7ImRlbm9taW5hdG9yIjoiMyIsIm51bWVyYXRvciI6IjEwMDAwMDAifSwic2VlZCI6LTE4NDk0NjA2M31dLCJjaGVja3BvaW50SWQiOiJja3B0LTEiLCJjb21wYXRpYmlsaXR5Ijp7ImJhY2tlbmREaWdlc3QiOiIxMGUwOGE0MTllODUwZWJhMWViYmExOGZkZDI4ZWI3ZWMxYjdlOGJhYTliY2MzYjk3M2UyYjg4OTFlYzcyNmJlIiwiY29udGVudERpZ2VzdCI6ImVkNzAwMmI0MzllOWFjODQ1ZjIyMzU3ZDgyMmJhYzE0NDQ3MzBmYmRiNjAxNmQzZWM5NDMyMjk3YjllYzlmNzMiLCJjb250cm9sbGVyRGlnZXN0IjoiYzE0NzIxMzViMTRjNzdjOGJlZjk4ZTczZjcwMjA4MzI1ZmEwZGNmMWU2YmQ2NjhhZTliMzFhOWNlYTI5NWZlNyIsInBhcnNlckRpZ2VzdCI6ImIxN2Q0NTEyMTE1MDkyOGYyMTQ2YWY0OWUxOTVlZmYxZWVmNWQ2NzMyNWJlMjczYTczM2ZiNzRhY2FkYWEzNDIiLCJwYXRjaERpZ2VzdCI6ImE0ODk1ZWI0NGFmYzMzNmZlY2JiYTZlNTIwY2Q2N2UxNzhkYWNlMDI3NjY1NWQxMDJmY2VmZmE4ZTVmNzA1NzAiLCJzdGF0ZUZvcm1hdElkIjoiZmx5c2Vzcy0xIn0sImNvbXBvc2l0aW9uRGlnZXN0IjoiNzMwZDcyNWM4YTU5ZDNhNzMwM2RlZjJiZWQwNDFhNTc3ZWRiNDI1NWFhYmQ0ODg5Y2UxMjkxODMxMWQ5NTJmMCIsImNvb3JkaW5hdG9yIjp7ImFkbWlzc2lvblN0YXRlIjpudWxsLCJldmVudFdhdGVybWFya3MiOnsibGFzdEV2ZW50SWQiOiJldnQtMSIsImxhc3RPcmRpbmFsIjoiNyJ9LCJleGVjdXRvclN0YXRlIjpbeyJhZ2VudElkIjoiZmx5LWEiLCJwYXlsb2FkIjoiZXhlY3V0b3ItZmx5LWEifV0sInByaW9ySW5zcGVjdGlvbiI6InByaW9yLWluc3BlY3Rpb24iLCJ0YXNrTGVkZ2VyIjoidGFzay1sZWRnZXIifSwiZW52ZWxvcGVWZXJzaW9uIjoxLCJlcGlzb2RlSWQiOiJlcGlzb2RlLTEiLCJoZWxwZXJTdGF0ZSI6W10sInBheWxvYWRzIjpbeyJieXRlTGVuZ3RoIjoiMTciLCJkaWdlc3QiOiIxMzIxZGZmYjBjZGM2ZjkwOTJjYmY3ZmEyYTVmYzY4YmJlZDEyYzk5M2Q1YWQzOTgyNjQwMTI4MTBjZTliZjkzIiwibmFtZSI6ImFnZW50LWZseS1hIn0seyJieXRlTGVuZ3RoIjoiMTQiLCJkaWdlc3QiOiIzYWVlNjBkZjdlMjllZmViYTdmNWY5OWZjNTg2NzY0N2IzNmFlYmZmMWQ1ZDNjODM4ZGJmZjMyMzEyMmU2NDYyIiwibmFtZSI6ImV4ZWN1dG9yLWZseS1hIn0seyJieXRlTGVuZ3RoIjoiMTEiLCJkaWdlc3QiOiI0MGIwMGVkMmJiYmE5MDFkNjgyMDVmZjcxYjA0YTQ0YjllZTUzYzUxY2IzMTA5YWEyY2VhYTQ0ZjFjNDU3MjdlIiwibmFtZSI6InRhc2stbGVkZ2VyIn0seyJieXRlTGVuZ3RoIjoiMTAiLCJkaWdlc3QiOiIyYzEzYjdiNGQ5YTk5MTY4MDFhYjkxOTFjMzE0ZjMxYjA0NWU5YjljNWI2NjlhNmMwNDc0ZjAyMTdlZjc1YmY1IiwibmFtZSI6InByaW9yLWluc3BlY3Rpb24ifSx7ImJ5dGVMZW5ndGgiOiI2NCIsImRpZ2VzdCI6ImY1YTVmZDQyZDE2YTIwMzAyNzk4ZWY2ZWQzMDk5NzliNDMwMDNkMjMyMGQ5ZjBlOGVhOTgzMWE5Mjc1OWZiNGIiLCJuYW1lIjoid29ybGQifV0sInBvcnRNYXAiOlt7ImFnZW50SWQiOiJmbHktYSIsInBvcnRJZCI6InBvcnQtMSJ9XSwic2NoZWR1bGVySWQiOiJsb2Nrc3RlcC12MSIsInNvdXJjZVNjb3BlIjp7ImVwb2NoIjoiZXBvY2gtMSIsInNlc3Npb25JZCI6ImRlbW8iLCJzdGVwIjoiNDIifSwid29ybGRUaW1lIjp7ImRlbm9taW5hdG9yIjoiMSIsIm51bWVyYXRvciI6IjcwMDAwMDAwMCJ9fWFnZW50LWZseS1hAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQCgAAAAAAABEAAAAAAAAAEyHf+wzcb5CSy/f6Kl/Gi77RLJk9WtOYJkASgQzpv5NleGVjdXRvci1mbHktYQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAaAoAAAAAAAAOAAAAAAAAADruYN9+Ke/rp/X5n8WGdkezauv/HV08g42/8yMSLmRidGFzay1sZWRnZXIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHgKAAAAAAAACwAAAAAAAABAsA7Su7qQHWggX/cbBKRLnuU8UcsxCaos6qRPHEVyfnByaW9yLWluc3BlY3Rpb24AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACICgAAAAAAAAoAAAAAAAAALBO3tNmpkWgBq5GRwxTzGwRem5xbZppsBHTwIX73W/V3b3JsZAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAmAoAAAAAAABAAAAAAAAAAPWl/ULRaiAwJ5jvbtMJl5tDAD0jINnw6OqYMaknWftLYWdlbnQgc3RhdGUgYnl0ZXMAAAAAAAAAZXhlY3V0b3Igc3RhdGUAAHsicmFuayI6MTB9AAAAAAB7Im1hcCI6NDB9AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgLAAAAAAAAq++fEx+FvDZho/eB4imbENN4HZrGNC2OCAsI7/gp9r5GTFlTRVNTRg==",
"byteLength": 2824,
"base64": "RkxZU0VTUzEBAAAAIAAAADAIAAAFAAAAUAgAAAAAAAB7ImFnZW50cyI6W3siYWdlbnRJZCI6ImZseS1hIiwiYnJhaW5UaWNrcyI6IjI1MzQiLCJkYXRhc2V0RGlnZXN0IjoiNmMwYWYxZjA3ODRlZjYzYTM5M2VlNzdkNjE0ZTgyNDZjNjI1MDUxMzYwZjNmMWE0ODgzODM3NGM1ZDM1NWI1MiIsIm1vZGVsVmVyc2lvbiI6ImxpZi0xbXMtZjY0LXYyIiwicGF5bG9hZCI6ImFnZW50LWZseS1hIiwicGxhc3RpY2l0eVZlcnNpb24iOiJmbHkta2MtbWJvbi1yc3RkcC12MiIsInByb2ZpbGVEaWdlc3QiOiIxOTAwZWFiNmMwMjg0ODNkNzEyNjU5OWVlNmY1MGRlMGQyNzkwN2I1YzY1ZmE5MDUyNDU4MGI0YjBmOTg1MmIwIiwicmVtYWluZGVyIjp7ImRlbm9taW5hdG9yIjoiMyIsIm51bWVyYXRvciI6IjEwMDAwMDAifSwic2VlZCI6LTE4NDk0NjA2M31dLCJjaGVja3BvaW50SWQiOiJja3B0LTEiLCJjb21wYXRpYmlsaXR5Ijp7ImJhY2tlbmREaWdlc3QiOiIxMGUwOGE0MTllODUwZWJhMWViYmExOGZkZDI4ZWI3ZWMxYjdlOGJhYTliY2MzYjk3M2UyYjg4OTFlYzcyNmJlIiwiY29udGVudERpZ2VzdCI6ImVkNzAwMmI0MzllOWFjODQ1ZjIyMzU3ZDgyMmJhYzE0NDQ3MzBmYmRiNjAxNmQzZWM5NDMyMjk3YjllYzlmNzMiLCJjb250cm9sbGVyRGlnZXN0IjoiYzE0NzIxMzViMTRjNzdjOGJlZjk4ZTczZjcwMjA4MzI1ZmEwZGNmMWU2YmQ2NjhhZTliMzFhOWNlYTI5NWZlNyIsInBhcnNlckRpZ2VzdCI6ImIxN2Q0NTEyMTE1MDkyOGYyMTQ2YWY0OWUxOTVlZmYxZWVmNWQ2NzMyNWJlMjczYTczM2ZiNzRhY2FkYWEzNDIiLCJwYXRjaERpZ2VzdCI6ImE0ODk1ZWI0NGFmYzMzNmZlY2JiYTZlNTIwY2Q2N2UxNzhkYWNlMDI3NjY1NWQxMDJmY2VmZmE4ZTVmNzA1NzAiLCJzdGF0ZUZvcm1hdElkIjoiZmx5c2Vzcy0xIn0sImNvbXBvc2l0aW9uRGlnZXN0IjoiNzMwZDcyNWM4YTU5ZDNhNzMwM2RlZjJiZWQwNDFhNTc3ZWRiNDI1NWFhYmQ0ODg5Y2UxMjkxODMxMWQ5NTJmMCIsImNvb3JkaW5hdG9yIjp7ImFkbWlzc2lvblN0YXRlIjpudWxsLCJldmVudFdhdGVybWFya3MiOnsiaXNzdWVkIjoiNyIsImxhc3RTb3VyY2VTdGVwIjoiNDIifSwiZXhlY3V0b3JTdGF0ZSI6W3siYWdlbnRJZCI6ImZseS1hIiwicGF5bG9hZCI6ImV4ZWN1dG9yLWZseS1hIn1dLCJwcmlvckluc3BlY3Rpb24iOiJwcmlvci1pbnNwZWN0aW9uIiwidGFza0xlZGdlciI6InRhc2stbGVkZ2VyIn0sImVudmVsb3BlVmVyc2lvbiI6MSwiZW52aXJvbm1lbnQiOnsicGF5bG9hZCI6IndvcmxkIiwid29ya2VySWQiOiJhcmVuYSJ9LCJlcGlzb2RlSWQiOiJlcGlzb2RlLTEiLCJoZWxwZXJTdGF0ZSI6W10sInBheWxvYWRzIjpbeyJieXRlTGVuZ3RoIjoiMTciLCJkaWdlc3QiOiIxMzIxZGZmYjBjZGM2ZjkwOTJjYmY3ZmEyYTVmYzY4YmJlZDEyYzk5M2Q1YWQzOTgyNjQwMTI4MTBjZTliZjkzIiwibmFtZSI6ImFnZW50LWZseS1hIn0seyJieXRlTGVuZ3RoIjoiMTQiLCJkaWdlc3QiOiIzYWVlNjBkZjdlMjllZmViYTdmNWY5OWZjNTg2NzY0N2IzNmFlYmZmMWQ1ZDNjODM4ZGJmZjMyMzEyMmU2NDYyIiwibmFtZSI6ImV4ZWN1dG9yLWZseS1hIn0seyJieXRlTGVuZ3RoIjoiMTEiLCJkaWdlc3QiOiI0MGIwMGVkMmJiYmE5MDFkNjgyMDVmZjcxYjA0YTQ0YjllZTUzYzUxY2IzMTA5YWEyY2VhYTQ0ZjFjNDU3MjdlIiwibmFtZSI6InRhc2stbGVkZ2VyIn0seyJieXRlTGVuZ3RoIjoiMTAiLCJkaWdlc3QiOiIyYzEzYjdiNGQ5YTk5MTY4MDFhYjkxOTFjMzE0ZjMxYjA0NWU5YjljNWI2NjlhNmMwNDc0ZjAyMTdlZjc1YmY1IiwibmFtZSI6InByaW9yLWluc3BlY3Rpb24ifSx7ImJ5dGVMZW5ndGgiOiI2NCIsImRpZ2VzdCI6ImY1YTVmZDQyZDE2YTIwMzAyNzk4ZWY2ZWQzMDk5NzliNDMwMDNkMjMyMGQ5ZjBlOGVhOTgzMWE5Mjc1OWZiNGIiLCJuYW1lIjoid29ybGQifV0sInBvcnRNYXAiOlt7ImFnZW50SWQiOiJmbHktYSIsInBvcnRJZCI6InBvcnQtMSJ9XSwic2NoZWR1bGVySWQiOiJsb2Nrc3RlcC12MSIsInNvdXJjZVNjb3BlIjp7ImVwb2NoIjoiZXBvY2gtMSIsInNlc3Npb25JZCI6ImRlbW8iLCJzdGVwIjoiNDIifSwid29ybGRUaW1lIjp7ImRlbm9taW5hdG9yIjoiMSIsIm51bWVyYXRvciI6IjcwMDAwMDAwMCJ9fWFnZW50LWZseS1hAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACACgAAAAAAABEAAAAAAAAAEyHf+wzcb5CSy/f6Kl/Gi77RLJk9WtOYJkASgQzpv5NleGVjdXRvci1mbHktYQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAmAoAAAAAAAAOAAAAAAAAADruYN9+Ke/rp/X5n8WGdkezauv/HV08g42/8yMSLmRidGFzay1sZWRnZXIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKgKAAAAAAAACwAAAAAAAABAsA7Su7qQHWggX/cbBKRLnuU8UcsxCaos6qRPHEVyfnByaW9yLWluc3BlY3Rpb24AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC4CgAAAAAAAAoAAAAAAAAALBO3tNmpkWgBq5GRwxTzGwRem5xbZppsBHTwIX73W/V3b3JsZAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAyAoAAAAAAABAAAAAAAAAAPWl/ULRaiAwJ5jvbtMJl5tDAD0jINnw6OqYMaknWftLYWdlbnQgc3RhdGUgYnl0ZXMAAAAAAAAAZXhlY3V0b3Igc3RhdGUAAHsicmFuayI6MTB9AAAAAAB7Im1hcCI6NDB9AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADgLAAAAAAAAX4L9WkdX0MViD3h5YJgf7VQgwocWU4XJoKX6MUwl+8hGTFlTRVNTRg==",
"byteLength": 2872,
"layout": {
"headerBytes": 32,
"manifestOffset": "32",
"manifestBytes": 2048,
"tableOffset": "2080",
"manifestBytes": 2096,
"tableOffset": "2128",
"tableEntryBytes": 112,
"entries": [
{
"name": "agent-fly-a",
"offset": "2640",
"offset": "2688",
"byteLength": "17",
"digest": "1321dffb0cdc6f9092cbf7fa2a5fc68bbed12c993d5ad398264012810ce9bf93"
},
{
"name": "executor-fly-a",
"offset": "2664",
"offset": "2712",
"byteLength": "14",
"digest": "3aee60df7e29efeba7f5f99fc5867647b36aebff1d5d3c838dbff323122e6462"
},
{
"name": "task-ledger",
"offset": "2680",
"offset": "2728",
"byteLength": "11",
"digest": "40b00ed2bbba901d68205ff71b04a44b9ee53c51cb3109aa2ceaa44f1c45727e"
},
{
"name": "prior-inspection",
"offset": "2696",
"offset": "2744",
"byteLength": "10",
"digest": "2c13b7b4d9a9916801ab9191c314f31b045e9b9c5b669a6c0474f0217ef75bf5"
},
{
"name": "world",
"offset": "2712",
"offset": "2760",
"byteLength": "64",
"digest": "f5a5fd42d16a20302798ef6ed309979b43003d2320d9f0e8ea9831a92759fb4b"
}
],
"footerOffset": "2776",
"footerOffset": "2824",
"footerBytes": 48,
"totalBytes": "2824"
"totalBytes": "2872"
}
},
"corruption": [
@ -178,17 +182,17 @@
},
{
"name": "a flipped payload byte",
"offset": 2640,
"offset": 2688,
"reason": "every payload carries its own digest"
},
{
"name": "a flipped footer digest byte",
"offset": 2784,
"offset": 2832,
"reason": "the footer digest must match the contents"
},
{
"name": "a flipped footer magic byte",
"offset": 2816,
"offset": 2864,
"reason": "a truncated file cannot look complete"
}
]

View file

@ -296,6 +296,11 @@ pub fn decode(bytes: &[u8]) -> Result<Envelope> {
/// The manifest fields state-media-v1 section 4 requires, checked as a set: a manifest that
/// omits one of them is not a complete checkpoint.
///
/// `helperState` and `environment` join the list under the 2026-09-22 amendment to
/// checkpoint-envelope-v1 section 3: the first has been in that section's table from the
/// start and was missing here, and the second is the holder of the world's own payload, which
/// the table named for every other participant and not for the environment.
pub const REQUIRED_MANIFEST_FIELDS: &[&str] = &[
"envelopeVersion",
"checkpointId",
@ -308,6 +313,8 @@ pub const REQUIRED_MANIFEST_FIELDS: &[&str] = &[
"compatibility",
"agents",
"coordinator",
"environment",
"helperState",
"payloads",
];

View file

@ -44,6 +44,7 @@ Ready(k) ─ Prepare all agents concurrently ───────────
| `metrics` | Latency percentiles and the machine's core and memory counters |
| `measure` | The execution-mode comparison of the guide's section 5 |
| `cli` | The binary's subcommands: `agent`, `environment`, `measure` |
| `state` | The durable checkpoint store over `FLYSESS1`: compatibility, generations, the bounded writer |
| `harness` | The runnable composition: router, the flies, one arena, one coordinator |
## Execution modes and the launcher
@ -178,6 +179,44 @@ harness.shutdown().await;
event ids derived from epoch, source step, rule and ordinal.
- **Executors.** The stateless identity executor only, as v1 specifies.
## Checkpoints and recovery
The durable store is `state`, over the `FLYSESS1` layout the contract crate owns.
- **One boundary, every participant.** `Coordinator::capture` runs at `Ready(k)` or
`Paused(k)` only. It takes its queue slot *before* the first `State.Capture`, so a saturated
writer refuses the capture rather than queueing it without bound, and the refusal is a
`BUSY` a stepping session survives rather than an epoch failure.
- **Capture and durability are two events.** `State.Capture` completes when an immutable
capture exists; `Coordinator::await_durable` completes when the store manifest rename has
happened, which is the durable commit point. Only the second moves the durable mark, and the
three ways it can end without one are told apart: `Failed` (the write stopped),
`ReplyLost` (the write finished and the acknowledgment did not arrive) and
`DeadlineExpired` (the caller's own budget ran out while the save was still going).
`Coordinator::resolve_durable` then asks the store about the *same* checkpoint instead of
saving again.
- **The writer is bounded twice**, and the two bounds refuse at different moments. The
outstanding-capture bound is taken before a capture is requested; the byte budget cannot be,
because a capture's size is not known until it exists, so it refuses at submit and releases
the payloads with the refusal. The writer owns its payload handles until the bytes are
committed or the job fails. A queued *replaceable* capture is superseded by a later one,
releasing its holds; a durable one never is.
- **The install is a group.** A restore selects a complete compatible generation, imports its
payloads as fresh artifacts, stages every participant, validates the coordinator's own
ledgers, and only then activates. A failure anywhere leaves the fence closed, and every
participant that got as far as staging is recorded as one that must be replaced before
another restore is attempted.
- **The fence lifts once.** `Failed -> Restoring(k) -> Paused(k)`, at the end of a complete
install and nowhere else. A fenced session takes no step, publishes nothing, captures
nothing and holds no artifact handle.
- **Nothing old crosses.** The fence drops every media handle; the restore imports fresh
artifacts; the environment re-renders its pending sensor pipeline from recorded
reconstruction inputs; and the new epoch's first audio chunk resumes the preserved sample
position and marks the discontinuity.
- **Epoch metadata in a trace.** `scope.epoch`, the batch id and every task event id are
derived from the epoch, so a resumed run's behaviour is compared through
`EpochRebase`, which rewrites exactly those and fails on anything it does not recognise.
## Where this crate narrows or adds to the contract crate
- **Required views.** `WorldObservation::validate_against` checks the views a result carries
@ -196,9 +235,9 @@ harness.shutdown().await;
- **Fake workers.** There is no neural model and no emulator. What is modelled exactly is the
ordering, the identity rules and the retry rules, not any numerical behaviour.
- **No state methods.** `State.Capture`, `State.StageRestore` and `State.ActivateRestore` are
STATE-01. The phase machine has their edges (`Capturing`, `Restoring`) and the workers do not
advertise them as implemented methods.
- **One environment, one task.** A checkpoint records the composition it was taken from, and a
restore refuses one taken under another backend, content, patch, controller or parser
identity. It does not migrate between compositions, and it does not try.
- **No audience input.** The admitted pre-step stimulation list exists and is always empty.
- **Pacing is coarse.** The pacing deadline rounds one step to whole nanoseconds for sleeping
only; simulation time stays rational and that rounding never re-enters the accumulator.
@ -259,6 +298,9 @@ The three integration suites do not all run over both transports, and cannot:
- `tests/processes.rs` runs over the Unix socket only, in all three execution modes. A
participant in a process of its own has no in-memory transport to reach the router by, so
the mode is the axis that suite varies and the transport is fixed.
- `tests/media.rs` and `tests/state.rs` run over both transports *and* in all three execution
modes: each acceptance body is written once and registered twice, by `both_transports!` in
the in-process composition and by `all_modes!` over the socket.
- `tests/session.rs`: one world advance per complete batch; every agent Prepared before the
advance; one task evaluation per transition; every agent committed before the next Prepare or
@ -275,6 +317,14 @@ The three integration suites do not all run over both transports, and cannot:
allocation -- plus the sequential/reversed/parallel trace comparison across all three modes
and the two process-mode section 4 rows: a router restart during a world advance, and an old
worker's reply after a restart.
- `tests/state.rs`: the STATE-01 acceptance bullets -- an uninterrupted run and a resumed run
committing the same behaviour once the epoch metadata is rebased, a corrupt payload failing
the install as a group for every participant and for the coordinator's own ledger, a lost
save reply and an uncommitted store manifest both leaving the durable mark where it was, a
refused activation resuming no part of the world, the capture queue staying bounded under a
stalled writer, and old media and another parser's state failing to cross a recovery --
plus the once-only restore token, the superseded replaceable capture, and the fence that
lifts only through a complete restore.
- `tests/failures.rs`: a duplicate Prepare after a lost reply; a duplicate Commit; the same
batch with altered controls; a lost Advance result; a cached artifact consumed by its first
caller; one Commit failing after another succeeded; a replaced registration; a reply from

View file

@ -7,7 +7,7 @@
//! stimulation, then reinforces once, and executes no tick at all. Every mutating step bumps
//! one counter, which is how a test proves a duplicate request changed nothing.
use std::collections::BTreeMap;
use std::collections::{BTreeMap, BTreeSet};
use serde_json::Value;
@ -193,6 +193,12 @@ pub struct AgentFaults {
pub prepare_delay_ms: u64,
/// Hold `Agent.Commit` open for this long.
pub commit_delay_ms: u64,
/// Refuse `State.StageRestore`, so a group install meets one participant that will not
/// validate while the others already have.
pub fail_stage_restore: bool,
/// Refuse `State.ActivateRestore` after this worker has already staged, so a group meets
/// a failure halfway through activation.
pub fail_activate_restore: bool,
}
/// One fake agent worker's configuration.
@ -234,6 +240,10 @@ pub struct FakeAgentWorker {
context: Option<TypedValue>,
context_digest: Option<Digest>,
prepared: Option<(DomainRequestId, PreparedDecision)>,
/// A validated replacement state that the live session cannot see yet.
staged: Option<StagedAgent>,
/// Restore tokens this worker has activated. A token activates once.
activated: BTreeSet<Id>,
}
impl FakeAgentWorker {
@ -248,10 +258,17 @@ impl FakeAgentWorker {
context: None,
context_digest: None,
prepared: None,
staged: None,
activated: BTreeSet::new(),
config,
}
}
/// True while a validated replacement state is staged and not yet activated.
pub fn has_staged_restore(&self) -> bool {
self.staged.is_some()
}
pub fn status(&self) -> StatusCell {
self.status.clone()
}
@ -657,7 +674,11 @@ impl WorkerEndpoint for FakeAgentWorker {
}
fn capabilities(&self) -> Vec<Id> {
vec![id("agent-step-v1"), id("pixel-observation-v1")]
vec![
id("agent-step-v1"),
id("pixel-observation-v1"),
id(crate::state::CHECKPOINT_CAPABILITY),
]
}
fn status_cell(&self) -> StatusCell {
@ -669,7 +690,14 @@ impl WorkerEndpoint for FakeAgentWorker {
}
fn methods(&self) -> Vec<&'static str> {
vec!["Agent.Initialize", "Agent.Prepare", "Agent.Commit"]
vec![
"Agent.Initialize",
"Agent.Prepare",
"Agent.Commit",
"State.Capture",
"State.StageRestore",
"State.ActivateRestore",
]
}
fn handle<'a>(&'a mut self, ctx: HandlerCtx<'a>) -> BoxFuture<'a, DomainResult<HandlerReply>> {
@ -678,6 +706,9 @@ impl WorkerEndpoint for FakeAgentWorker {
"Agent.Initialize" => self.initialize(&ctx).await,
"Agent.Prepare" => self.prepare(&ctx).await,
"Agent.Commit" => self.commit(&ctx).await,
"State.Capture" => self.state_capture(&ctx).await,
"State.StageRestore" => self.state_stage_restore(&ctx).await,
"State.ActivateRestore" => self.state_activate_restore(&ctx).await,
other => Err(DomainError::before(
ErrorCode::Unsupported,
format!("{other} is not an agent method"),
@ -690,7 +721,12 @@ impl WorkerEndpoint for FakeAgentWorker {
/// The retention class table an agent endpoint follows, for a caller that wants it.
pub fn agent_op_class(method: &str) -> Option<OpClass> {
match method {
"Agent.Initialize" => Some(OpClass::Lifecycle),
// `ipc-v1` section 5: lifecycle *and capture* replies are retained until
// `Worker.Acknowledge`, which is also what lets a duplicate restore request replay
// its cached reply rather than staging or activating twice.
"Agent.Initialize" | "State.Capture" | "State.StageRestore" | "State.ActivateRestore" => {
Some(OpClass::Lifecycle)
}
"Agent.Prepare" | "Agent.Commit" => Some(OpClass::StepMutation),
_ => None,
}
@ -712,3 +748,532 @@ pub fn synthetic_profile(agent_id: &Id, tick_duration: &RationalNs, warmup_ticks
/// The per-agent contexts a bootstrap produced, keyed by agent id.
pub type Contexts = BTreeMap<Id, TypedValue>;
// -------------------------------------------------------------------------------------------
// STATE-01: capture and restore
/// The numerical model version this worker implements. It is part of a capture's
/// compatibility identity: the same profile and seed under another model is not the same
/// state (`workers-v1` section 2).
pub const MODEL_VERSION: &str = "fake-lcg-v1";
/// The plasticity rule version, for the same reason.
pub const PLASTICITY_VERSION: &str = "fake-reinforce-v1";
/// The version this payload layout is written and read under.
pub const AGENT_PAYLOAD_VERSION: u64 = 1;
/// The dataset identity a synthetic agent resolves.
///
/// There is no connectome dataset behind this worker, and a checkpoint says so with a stable
/// identity rather than omitting the field: "no dataset" has to be distinguishable from "the
/// dataset was not recorded".
pub fn dataset_digest() -> Digest {
digest_of_bytes(b"fly-session/no-dataset-v1")
}
/// The capture compatibility digest of one agent (`workers-v1` section 2).
///
/// The profile digest identifies the profile definition; this additionally covers the
/// resolved seed, the numerical model version and the plasticity rule, because two agents
/// with the same profile digest and different seeds hold state that is not interchangeable.
/// Every field it covers is one the checkpoint manifest already records in that agent's row,
/// so a restore derives the expected digest from the manifest rather than from the payload it
/// is about to validate.
pub fn agent_compatibility_digest(
agent_id: &Id,
profile_digest: &Digest,
dataset_digest: &Digest,
model_version: &str,
plasticity_version: &str,
seed: i32,
) -> Digest {
let value = serde_json::json!({
"agentId": agent_id.as_str(),
"profileDigest": profile_digest.as_str(),
"datasetDigest": dataset_digest.as_str(),
"modelVersion": model_version,
"plasticityVersion": plasticity_version,
"seed": seed,
});
digest_of(&value).expect("an agent compatibility block canonicalizes")
}
impl FakeModel {
/// Every field of the model, so a resumed agent is this agent and not a fresh one.
fn capture(&self) -> Value {
serde_json::json!({
"seed": self.seed,
"state": self.state.to_string(),
"mutations": self.mutations.to_string(),
"ticks": self.ticks.to_string(),
"stimulations": self.stimulations.to_string(),
"reinforcements": self.reinforcements.to_string(),
"learningEnabled": self.learning_enabled,
"learningUpdates": self.learning_updates.to_string(),
"learningChanged": self.learning_changed.to_string(),
"lastSignal": self.last_signal,
"inputValue": self.input_value.to_string(),
"inputInstalls": self.input_installs.to_string(),
})
}
fn restored(value: &Value) -> DomainResult<FakeModel> {
let number = |key: &str| -> DomainResult<u64> {
value
.get(key)
.and_then(Value::as_str)
.ok_or_else(|| incompatible(format!("the agent payload has no {key}")))?
.parse::<u64>()
.map_err(|_| incompatible(format!("the agent payload's {key} is not a U64")))
};
let seed = value
.get("seed")
.and_then(Value::as_i64)
.and_then(|v| i32::try_from(v).ok())
.ok_or_else(|| incompatible("the agent payload has no seed"))?;
let input_value = value
.get("inputValue")
.and_then(Value::as_str)
.ok_or_else(|| incompatible("the agent payload has no inputValue"))?
.parse::<i64>()
.map_err(|_| incompatible("the agent payload's inputValue is not an integer"))?;
let last_signal = value
.get("lastSignal")
.and_then(Value::as_f64)
.filter(|v| v.is_finite())
.ok_or_else(|| incompatible("the agent payload's lastSignal is not finite"))?;
let learning_enabled = value
.get("learningEnabled")
.and_then(Value::as_bool)
.ok_or_else(|| incompatible("the agent payload has no learningEnabled"))?;
Ok(FakeModel {
seed,
state: number("state")?,
mutations: number("mutations")?,
ticks: number("ticks")?,
stimulations: number("stimulations")?,
reinforcements: number("reinforcements")?,
learning_enabled,
learning_updates: number("learningUpdates")?,
learning_changed: number("learningChanged")?,
last_signal,
input_value,
input_installs: number("inputInstalls")?,
})
}
}
fn incompatible(message: impl std::fmt::Display) -> DomainError {
DomainError::before(ErrorCode::IncompatibleState, message)
}
/// One staged restore, held outside the live agent until it is activated.
struct StagedAgent {
token: Id,
checkpoint_id: Id,
scope: Scope,
model: FakeModel,
accumulator: TickAccumulator,
context: TypedValue,
profile: AssetRef,
committed_step: u64,
}
impl FakeAgentWorker {
/// This worker's own compatibility identity, from its configuration and a resolved seed.
fn compatibility_digest(&self, profile: &AssetRef, seed: i32) -> Digest {
agent_compatibility_digest(
&self.config.agent_id,
&profile.digest,
&dataset_digest(),
MODEL_VERSION,
PLASTICITY_VERSION,
seed,
)
}
/// `State.Capture`: an immutable snapshot of this agent at its committed boundary.
///
/// It is allowed at `Ready(k)` only. A Prepared agent holds half a transition, and there
/// is no coherent boundary to file that under.
async fn state_capture(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
let scope = ctx.scope()?.clone();
self.check_epoch(&scope)?;
let AgentPhase::Ready(k) = self.phase.clone() else {
return Err(DomainError::before(
ErrorCode::InvalidPhase,
format!(
"State.Capture needs a quiescent Ready(k); this worker is {:?}",
self.phase
),
));
};
if scope.step != k {
return Err(DomainError::before(
if scope.step < k { ErrorCode::StaleStep } else { ErrorCode::FutureStep },
"State.Capture names a boundary this worker is not at",
));
}
let params: CaptureParams = ctx.params()?;
let profile = self.profile.clone().expect("initialized");
let context = self.context.clone().expect("initialized");
let accumulator = self.accumulator.as_ref().expect("initialized");
let previous = self.status.state();
self.status.set_state(WorkerState::Capturing);
let payload = serde_json::json!({
"payloadVersion": AGENT_PAYLOAD_VERSION,
"kind": "agent",
"agentId": self.config.agent_id.as_str(),
"checkpointId": params.checkpoint_id.as_str(),
"sourceScope": scope.to_json(),
"committedStep": k.to_string(),
"profile": profile.to_json(),
"modelVersion": MODEL_VERSION,
"plasticityVersion": PLASTICITY_VERSION,
"datasetDigest": dataset_digest().as_str(),
"model": self.model.capture(),
"accumulator": {
"tickDuration": accumulator.tick_duration().to_json(),
"remainder": accumulator.remainder().to_json(),
"executedTicks": accumulator.executed_ticks().to_string(),
"warmupOffset": accumulator.warmup_offset().to_string(),
},
"context": context.to_json(),
});
let bytes = canonicalize(&payload)
.map_err(|e| DomainError::invalid(format!("State.Capture: {}", e.0)))?
.into_bytes();
let digest = digest_of_bytes(&bytes);
let artifact = crate::state::seal_payload(ctx.client, &bytes, &digest).await?;
// Capture is a read of the model, not a mutation of it: nothing above changed a
// counter, and the worker goes back to the boundary it was already at.
self.status.set_state(previous);
let result = CaptureResult {
checkpoint_id: params.checkpoint_id,
boundary: k,
compatibility_digest: self.compatibility_digest(&profile, self.model.seed()),
payload: artifact.reference().clone(),
};
Ok(HandlerReply::with_artifacts(
object(result.to_json()),
vec![(crate::state::PAYLOAD_ATTACHMENT.to_owned(), artifact)],
))
}
/// `State.StageRestore`: validate a replacement state into a staging slot.
///
/// Nothing the live session can see changes here, and the worker keeps whatever state it
/// had. It is allowed on an uninitialized replacement or a quiescent worker only; a
/// failed one is neither, which is why a group that failed is replaced rather than
/// reused.
async fn state_stage_restore(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
let scope = ctx.scope()?.clone();
if scope.session_id != self.config.session_id {
return Err(DomainError::before(
ErrorCode::IdentityMismatch,
"this worker belongs to another session",
));
}
match &self.phase {
AgentPhase::Uninitialized | AgentPhase::Ready(_) => {}
other => {
return Err(DomainError::before(
ErrorCode::InvalidPhase,
format!(
"State.StageRestore needs an uninitialized replacement or a quiescent \
worker; this worker is {other:?}"
),
));
}
}
if let Some(epoch) = &self.epoch
&& *epoch == scope.epoch
{
return Err(DomainError::before(
ErrorCode::StaleEpoch,
"State.StageRestore proposes the epoch this worker is already running",
));
}
let params: StageRestoreParams = ctx.params()?;
if params.source_scope.step != scope.step {
return Err(DomainError::invalid(
"State.StageRestore's scope step must be the source boundary",
));
}
let artifact = ctx.artifact(crate::state::PAYLOAD_ATTACHMENT)?;
if artifact.reference() != &params.payload {
return Err(DomainError::before(
ErrorCode::BufferInvalid,
"the staged payload attachment is not the artifact the request names",
));
}
let bytes = artifact.read_all().await.map_err(|e| {
DomainError::before(
ErrorCode::BufferInvalid,
format!("the staged payload could not be read: {}", e.message),
)
})?;
let declared = params
.payload
.digest
.clone()
.ok_or_else(|| incompatible("a checkpoint payload must carry a content digest"))?;
let actual = digest_of_bytes(&bytes);
if actual != declared || bytes.len() as u64 != params.payload.byte_length {
return Err(incompatible(
"the staged payload is not the content the request declares",
));
}
let value: Value = serde_json::from_slice(&bytes)
.map_err(|e| DomainError::invalid(format!("the staged payload is not JSON: {e}")))?;
let text = |key: &str| -> DomainResult<String> {
value
.get(key)
.and_then(Value::as_str)
.map(str::to_owned)
.ok_or_else(|| incompatible(format!("the agent payload has no {key}")))
};
if value.get("payloadVersion").and_then(Value::as_u64) != Some(AGENT_PAYLOAD_VERSION) {
return Err(incompatible("the agent payload is another payload version"));
}
if text("kind")? != "agent" {
return Err(incompatible("this payload is not an agent's state"));
}
if text("agentId")? != self.config.agent_id {
return Err(DomainError::before(
ErrorCode::IdentityMismatch,
"the staged payload belongs to another agent",
));
}
if text("checkpointId")? != params.checkpoint_id {
return Err(incompatible("the staged payload belongs to another checkpoint"));
}
if text("modelVersion")? != MODEL_VERSION || text("plasticityVersion")? != PLASTICITY_VERSION
{
return Err(incompatible(
"the staged payload was captured under another numerical model",
));
}
let source_scope = Scope::from_json(
value
.get("sourceScope")
.ok_or_else(|| incompatible("the agent payload has no sourceScope"))?,
)
.map_err(|e| incompatible(format!("the agent payload's sourceScope: {}", e.0)))?;
if source_scope != params.source_scope {
return Err(incompatible(
"the staged payload was captured at another source scope",
));
}
let committed_step: u64 = text("committedStep")?
.parse()
.map_err(|_| incompatible("the agent payload's committedStep is not a U64"))?;
if committed_step != params.source_scope.step {
return Err(incompatible(
"the staged payload's committed step is not the source boundary",
));
}
let profile = AssetRef::from_json(
value
.get("profile")
.ok_or_else(|| incompatible("the agent payload has no profile"))?,
)
.map_err(|e| incompatible(format!("the agent payload's profile: {}", e.0)))?;
let model = FakeModel::restored(
value
.get("model")
.ok_or_else(|| incompatible("the agent payload has no model"))?,
)?;
// The compatibility digest is recomputed from this worker's own configuration and the
// identity the payload declares. A capture of the same profile under another seed, or
// of another agent's brain, fails here and never reaches activation.
let computed = self.compatibility_digest(&profile, model.seed());
if computed != params.compatibility_digest {
return Err(incompatible(format!(
"the staged state's compatibility {computed} is not the {} the restore \
requires",
params.compatibility_digest
)));
}
let accumulator_value = value
.get("accumulator")
.ok_or_else(|| incompatible("the agent payload has no accumulator"))?;
let rational = |key: &str| -> DomainResult<RationalNs> {
RationalNs::from_json(
accumulator_value
.get(key)
.ok_or_else(|| incompatible(format!("the accumulator has no {key}")))?,
)
.map_err(|e| incompatible(format!("the accumulator's {key}: {}", e.0)))
};
let counter = |key: &str| -> DomainResult<u64> {
accumulator_value
.get(key)
.and_then(Value::as_str)
.ok_or_else(|| incompatible(format!("the accumulator has no {key}")))?
.parse::<u64>()
.map_err(|_| incompatible(format!("the accumulator's {key} is not a U64")))
};
let tick_duration = rational("tickDuration")?;
if tick_duration != self.config.tick_duration {
return Err(incompatible(
"the staged state was captured at another model tick duration",
));
}
let accumulator = TickAccumulator::restored(
tick_duration,
rational("remainder")?,
counter("executedTicks")?,
counter("warmupOffset")?,
)
.map_err(incompatible)?;
let context = TypedValue::from_json(
value
.get("context")
.ok_or_else(|| incompatible("the agent payload has no context"))?,
)
.map_err(|e| incompatible(format!("the agent payload's context: {}", e.0)))?;
FakeAgentWorker::available_actions(&context)?;
if self.config.faults.fail_stage_restore {
// The row where a group validates three participants and the fourth does not.
// Nothing is staged here and nothing is staged anywhere else either: the
// coordinator abandons the whole install.
return Err(incompatible(
"injected staging refusal: this participant's replacement state does not \
validate",
));
}
// One staged restore at a time. A second proposal replaces nothing silently.
if let Some(staged) = &self.staged {
return Err(DomainError::before(
ErrorCode::Conflict,
format!(
"this worker already holds the staged restore {} for checkpoint {}",
staged.token, staged.checkpoint_id
),
));
}
let token = restore_token(&params.checkpoint_id, &scope, &actual, &self.config.incarnation_id);
if self.activated.contains(&token) {
return Err(DomainError::before(
ErrorCode::Conflict,
"this exact restore was already activated on this worker",
));
}
self.staged = Some(StagedAgent {
token: token.clone(),
checkpoint_id: params.checkpoint_id.clone(),
scope: scope.clone(),
model,
accumulator,
context,
profile,
committed_step,
});
self.status.set_state(WorkerState::StagedRestore);
let result = StageRestoreResult {
checkpoint_id: params.checkpoint_id,
restore_token: token,
};
Ok(HandlerReply::from(&result))
}
/// `State.ActivateRestore`: install the staged state under its new scope, without a tick.
///
/// The token activates once. A duplicate domain request replays the cached reply through
/// the shell's result cache; a fresh request naming an already activated token is a
/// conflict, which is what stops a second group from being resumed from the same bytes.
async fn state_activate_restore(
&mut self,
ctx: &HandlerCtx<'_>,
) -> DomainResult<HandlerReply> {
let params: ActivateRestoreParams = ctx.params()?;
if self.activated.contains(&params.restore_token) {
return Err(DomainError::before(
ErrorCode::Conflict,
"this restore token has already been activated",
));
}
let Some(staged) = self.staged.take() else {
return Err(DomainError::before(
ErrorCode::InvalidPhase,
"this worker holds no staged restore",
));
};
if staged.token != params.restore_token {
// Put it back: naming another token is not a reason to discard this one.
let token = staged.token.clone();
self.staged = Some(staged);
return Err(DomainError::before(
ErrorCode::IdentityMismatch,
format!("this worker's staged restore is {token}, not {}", params.restore_token),
));
}
if self.config.faults.fail_activate_restore {
let token = staged.token.clone();
self.staged = Some(staged);
self.status.set_state(WorkerState::Failed);
return Err(DomainError::new(
ErrorCode::BackendFailure,
format!("injected activation failure; {token} stays staged and unresumed"),
MutationCertainty::None,
));
}
self.status.set_state(WorkerState::Restoring);
let StagedAgent {
token,
checkpoint_id,
scope,
model,
accumulator,
context,
profile,
committed_step,
} = staged;
self.model = model;
self.accumulator = Some(accumulator);
self.context_digest = Some(context.digest());
self.context = Some(context);
self.profile = Some(profile);
self.epoch = Some(scope.epoch.clone());
self.prepared = None;
self.phase = AgentPhase::Ready(committed_step);
self.activated.insert(token);
self.status.set_state(WorkerState::Ready);
self.status.set_scope(Some(scope_at(
&scope.session_id,
&scope.epoch,
committed_step,
)));
self.status.advance_to(self.model.mutations());
let result = ActivateRestoreResult {
committed_step,
checkpoint_id,
// An agent returns a null observation; the environment returns the world's.
observation: None,
};
result
.validate_for_role(Role::Agent)
.map_err(|e| DomainError::invalid(e.0))?;
Ok(HandlerReply::from(&result))
}
}
/// A restore token bound to the checkpoint, the proposed scope, the payload bytes and the
/// worker incarnation staging them.
///
/// `state-media-v1` section 5 binds a token to scope, payload and checkpoint. Binding it to
/// the incarnation as well is what keeps a token minted by a worker that has since been
/// replaced from activating anything on its replacement.
pub fn restore_token(checkpoint_id: &Id, scope: &Scope, payload_digest: &Digest, incarnation: &Id) -> Id {
let digest = digest_of_bytes(
format!(
"fly-session/restore-token-v1\n{checkpoint_id}\n{}\n{}\n{}\n{payload_digest}\n{incarnation}\n",
scope.session_id, scope.epoch, scope.step
)
.as_bytes(),
);
parse_id(&format!("rt-{}", &digest[..32])).expect("a hex suffix is an Id")
}

View file

@ -46,8 +46,10 @@ Worker options (agent and environment):
agent: --agent ID --port ID --tick-numerator N --tick-denominator N
--warmup-ticks N [--prepare-delay-ms N] [--commit-delay-ms N]
[--fail-commit-at-step N]
[--fail-stage-restore 0|1] [--fail-activate-restore 0|1]
environment: --worker ID --ports p1,p2 --step-numerator N --step-denominator N
[--advance-delay-ms N] [--omit-view-at-boundary N]
[--fail-stage-restore 0|1] [--fail-activate-restore 0|1]
Measure options:
--steps N transitions per run (default 200)
@ -145,6 +147,17 @@ impl Options {
}
}
/// A flag whose value is `0` or `1`. Anything else is an error naming it, so a
/// mistyped injection is a failed launch rather than a fault that never fires.
fn flag(&self, name: &str) -> Result<bool, String> {
match self.0.get(name) {
None => Ok(false),
Some(value) if value == "0" => Ok(false),
Some(value) if value == "1" => Ok(true),
Some(value) => Err(format!("--{name}: {value:?} is not 0 or 1")),
}
}
fn opt_u64(&self, name: &str) -> Result<Option<u64>, String> {
match self.0.get(name) {
None => Ok(None),
@ -197,6 +210,8 @@ fn serve(role: &str, options: &Options) -> Result<(), String> {
fail_commit_at_step: options.opt_u64(flags::FAIL_COMMIT_AT_STEP)?,
prepare_delay_ms: options.u64(flags::PREPARE_DELAY_MS, 0)?,
commit_delay_ms: options.u64(flags::COMMIT_DELAY_MS, 0)?,
fail_stage_restore: options.flag(flags::FAIL_STAGE_RESTORE)?,
fail_activate_restore: options.flag(flags::FAIL_ACTIVATE_RESTORE)?,
},
client_id: client_id.clone(),
service: service.clone(),
@ -219,6 +234,8 @@ fn serve(role: &str, options: &Options) -> Result<(), String> {
omit_audio_at_boundary: options.opt_u64(flags::OMIT_AUDIO_AT_BOUNDARY)?,
overlapping_audio_at_boundary: options
.opt_u64(flags::OVERLAPPING_AUDIO_AT_BOUNDARY)?,
fail_stage_restore: options.flag(flags::FAIL_STAGE_RESTORE)?,
fail_activate_restore: options.flag(flags::FAIL_ACTIVATE_RESTORE)?,
},
client_id: client_id.clone(),
service: service.clone(),

View file

@ -29,6 +29,30 @@ impl TickAccumulator {
})
}
/// The exact accumulator a capture recorded.
///
/// The remainder is restored, never rounded or reset: a resumed agent that started its
/// first interval from zero would drift away from the run it is supposed to continue.
pub fn restored(
tick_duration: RationalNs,
remainder: RationalNs,
executed_ticks: u64,
warmup_offset: u64,
) -> Result<TickAccumulator, String> {
let mut accumulator = TickAccumulator::new(tick_duration)?;
remainder.validate().map_err(|e| e.0)?;
if remainder >= tick_duration {
return Err("a captured remainder is not below one model tick".to_owned());
}
if warmup_offset > executed_ticks {
return Err("a captured warm-up offset exceeds the executed tick count".to_owned());
}
accumulator.remainder = remainder;
accumulator.executed_ticks = executed_ticks;
accumulator.warmup_offset = warmup_offset;
Ok(accumulator)
}
pub fn tick_duration(&self) -> RationalNs {
self.tick_duration
}

File diff suppressed because it is too large Load diff

View file

@ -13,6 +13,8 @@
use std::collections::BTreeSet;
use serde_json::Value;
use crate::media::{self, AudioSource, RenderCounter, ViewPipeline};
use crate::task::{controller_schema_ref, inspection, inspection_schema};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
@ -48,6 +50,12 @@ pub struct EnvironmentFaults {
pub omit_audio_at_boundary: Option<u64>,
/// Emit an audio chunk that starts before the previous chunk ended.
pub overlapping_audio_at_boundary: Option<u64>,
/// Refuse `State.StageRestore`, so a group install meets a participant that will not
/// validate.
pub fail_stage_restore: bool,
/// Refuse `State.ActivateRestore` after staging, so a group meets a failure halfway
/// through activation.
pub fail_activate_restore: bool,
}
#[derive(Clone, Debug)]
@ -86,6 +94,10 @@ pub struct CounterEnvironment {
audio: Option<AudioSource>,
/// The frame served at the previous boundary, kept only so a fault can serve it again.
previous_view: Option<(ViewRef, flybus::Artifact)>,
/// A validated replacement world the live session cannot see yet.
staged: Option<StagedWorld>,
/// Restore tokens this world has activated. A token activates once.
activated: BTreeSet<Id>,
}
impl CounterEnvironment {
@ -104,10 +116,17 @@ impl CounterEnvironment {
pipeline: None,
audio: None,
previous_view: None,
staged: None,
activated: BTreeSet::new(),
config,
}
}
/// True while a validated replacement world is staged and not yet activated.
pub fn has_staged_restore(&self) -> bool {
self.staged.is_some()
}
pub fn status(&self) -> StatusCell {
self.status.clone()
}
@ -477,7 +496,7 @@ impl WorkerEndpoint for CounterEnvironment {
vec![
id("world-step-v1"),
id("pixel-observation-v1"),
id("checkpoint-v1"),
id(crate::state::CHECKPOINT_CAPABILITY),
]
}
@ -490,7 +509,13 @@ impl WorkerEndpoint for CounterEnvironment {
}
fn methods(&self) -> Vec<&'static str> {
vec!["Environment.Initialize", "Environment.Advance"]
vec![
"Environment.Initialize",
"Environment.Advance",
"State.Capture",
"State.StageRestore",
"State.ActivateRestore",
]
}
fn handle<'a>(&'a mut self, ctx: HandlerCtx<'a>) -> BoxFuture<'a, DomainResult<HandlerReply>> {
@ -498,6 +523,9 @@ impl WorkerEndpoint for CounterEnvironment {
match ctx.method {
"Environment.Initialize" => self.initialize(&ctx).await,
"Environment.Advance" => self.advance(&ctx).await,
"State.Capture" => self.state_capture(&ctx).await,
"State.StageRestore" => self.state_stage_restore(&ctx).await,
"State.ActivateRestore" => self.state_activate_restore(&ctx).await,
other => Err(DomainError::before(
ErrorCode::Unsupported,
format!("{other} is not an environment method"),
@ -516,3 +544,488 @@ pub fn synthetic_asset(asset_id: &str, body: &str) -> AssetRef {
format: id("fly-config-v1"),
}
}
// -------------------------------------------------------------------------------------------
// STATE-01: capture and restore
/// The version this payload layout is written and read under.
pub const WORLD_PAYLOAD_VERSION: u64 = 1;
fn incompatible(message: impl std::fmt::Display) -> DomainError {
DomainError::before(ErrorCode::IncompatibleState, message)
}
/// One staged restore, held outside the live world until it is activated.
struct StagedWorld {
token: Id,
checkpoint_id: Id,
scope: Scope,
episode_id: Id,
descriptor: EnvironmentDescriptor,
boundary: u64,
counter: i64,
world_time: RationalNs,
advances: u64,
frames: Vec<(u64, i64)>,
audio_next_sample: u64,
audio_phase: u64,
audio_accumulator: u128,
audio_denominator: u128,
}
impl CounterEnvironment {
/// `State.Capture`: the world at its committed boundary, including its pending sensor
/// pipeline.
///
/// The pipeline is recorded as reconstruction inputs -- the producing boundary and the
/// world counter of every retained frame -- and never as an artifact identity: a
/// transient artifact belongs to the router that is running now, and a checkpoint outlives
/// it.
async fn state_capture(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
let scope = ctx.scope()?.clone();
let Some(descriptor) = self.descriptor.clone() else {
return Err(DomainError::before(
ErrorCode::InvalidPhase,
"this environment is uninitialized",
));
};
if scope.session_id != self.config.session_id {
return Err(DomainError::before(
ErrorCode::IdentityMismatch,
"this environment belongs to another session",
));
}
match &self.epoch {
Some(epoch) if *epoch == scope.epoch => {}
_ => {
return Err(DomainError::before(
ErrorCode::StaleEpoch,
"State.Capture names an epoch this environment has left",
));
}
}
if scope.step != self.boundary {
return Err(DomainError::before(
if scope.step < self.boundary {
ErrorCode::StaleStep
} else {
ErrorCode::FutureStep
},
"State.Capture must name the boundary the world is at",
));
}
let params: CaptureParams = ctx.params()?;
let pipeline = self
.pipeline
.as_ref()
.ok_or_else(|| DomainError::before(ErrorCode::InvalidPhase, "no view pipeline"))?;
let audio = self
.audio
.as_ref()
.ok_or_else(|| DomainError::before(ErrorCode::InvalidPhase, "no audio source"))?;
let (accumulator, denominator) = audio.accumulator();
let previous = self.status.state();
self.status.set_state(WorkerState::Capturing);
let payload = serde_json::json!({
"payloadVersion": WORLD_PAYLOAD_VERSION,
"kind": "world",
"workerId": self.config.worker_id.as_str(),
"checkpointId": params.checkpoint_id.as_str(),
"sourceScope": scope.to_json(),
"episodeId": self.episode_id.clone().expect("initialized").as_str(),
"committedStep": self.boundary.to_string(),
"counter": self.counter.to_string(),
"worldTime": self.world_time.to_json(),
"advances": self.advances.to_string(),
"descriptor": descriptor.to_json(),
"pipeline": {
// The declared delay's whole queue, oldest first.
"frames": pipeline
.retained()
.into_iter()
.map(|(boundary, counter)| serde_json::json!({
"boundary": boundary.to_string(),
"counter": counter.to_string(),
}))
.collect::<Vec<_>>(),
},
"audio": {
"nextSample": audio.next_sample().to_string(),
"phase": audio.phase().to_string(),
"accumulator": accumulator.to_string(),
"denominator": denominator.to_string(),
"chunks": audio.chunks().to_string(),
},
});
let bytes = canonicalize(&payload)
.map_err(|e| DomainError::invalid(format!("State.Capture: {}", e.0)))?
.into_bytes();
let digest = digest_of_bytes(&bytes);
let artifact = crate::state::seal_payload(ctx.client, &bytes, &digest).await?;
// A capture reads the world; it does not advance it.
self.status.set_state(previous);
let result = CaptureResult {
checkpoint_id: params.checkpoint_id,
boundary: self.boundary,
compatibility_digest: crate::state::Compatibility::of(&descriptor).digest(),
payload: artifact.reference().clone(),
};
Ok(HandlerReply::with_artifacts(
object(result.to_json()),
vec![(crate::state::PAYLOAD_ATTACHMENT.to_owned(), artifact)],
))
}
/// `State.StageRestore`: validate a replacement world into a staging slot.
async fn state_stage_restore(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
let scope = ctx.scope()?.clone();
if scope.session_id != self.config.session_id {
return Err(DomainError::before(
ErrorCode::IdentityMismatch,
"this environment belongs to another session",
));
}
if let Some(epoch) = &self.epoch
&& *epoch == scope.epoch
{
return Err(DomainError::before(
ErrorCode::StaleEpoch,
"State.StageRestore proposes the epoch this environment is already running",
));
}
if self.descriptor.is_some() {
// A world that is already running a boundary is not a quiescent replacement: the
// group replaces it rather than restoring over a live one.
return Err(DomainError::before(
ErrorCode::InvalidPhase,
"State.StageRestore needs an uninitialized replacement environment",
));
}
let params: StageRestoreParams = ctx.params()?;
if params.source_scope.step != scope.step {
return Err(DomainError::invalid(
"State.StageRestore's scope step must be the source boundary",
));
}
let artifact = ctx.artifact(crate::state::PAYLOAD_ATTACHMENT)?;
if artifact.reference() != &params.payload {
return Err(DomainError::before(
ErrorCode::BufferInvalid,
"the staged payload attachment is not the artifact the request names",
));
}
let bytes = artifact.read_all().await.map_err(|e| {
DomainError::before(
ErrorCode::BufferInvalid,
format!("the staged payload could not be read: {}", e.message),
)
})?;
let declared = params
.payload
.digest
.clone()
.ok_or_else(|| incompatible("a checkpoint payload must carry a content digest"))?;
let actual = digest_of_bytes(&bytes);
if actual != declared || bytes.len() as u64 != params.payload.byte_length {
return Err(incompatible(
"the staged payload is not the content the request declares",
));
}
let value: Value = serde_json::from_slice(&bytes)
.map_err(|e| DomainError::invalid(format!("the staged payload is not JSON: {e}")))?;
let text = |key: &str| -> DomainResult<String> {
value
.get(key)
.and_then(Value::as_str)
.map(str::to_owned)
.ok_or_else(|| incompatible(format!("the world payload has no {key}")))
};
let number = |key: &str| -> DomainResult<u64> {
text(key)?
.parse::<u64>()
.map_err(|_| incompatible(format!("the world payload's {key} is not a U64")))
};
if value.get("payloadVersion").and_then(Value::as_u64) != Some(WORLD_PAYLOAD_VERSION) {
return Err(incompatible("the world payload is another payload version"));
}
if text("kind")? != "world" {
return Err(incompatible("this payload is not a world's state"));
}
if text("workerId")? != self.config.worker_id {
return Err(DomainError::before(
ErrorCode::IdentityMismatch,
"the staged payload belongs to another world",
));
}
if text("checkpointId")? != params.checkpoint_id {
return Err(incompatible("the staged payload belongs to another checkpoint"));
}
let source_scope = Scope::from_json(
value
.get("sourceScope")
.ok_or_else(|| incompatible("the world payload has no sourceScope"))?,
)
.map_err(|e| incompatible(format!("the world payload's sourceScope: {}", e.0)))?;
if source_scope != params.source_scope {
return Err(incompatible(
"the staged payload was captured at another source scope",
));
}
let committed_step = number("committedStep")?;
if committed_step != params.source_scope.step {
return Err(incompatible(
"the staged payload's committed step is not the source boundary",
));
}
let descriptor = EnvironmentDescriptor::from_json(
value
.get("descriptor")
.ok_or_else(|| incompatible("the world payload has no descriptor"))?,
)
.map_err(|e| incompatible(format!("the world payload's descriptor: {}", e.0)))?;
// The replacement builds the descriptor it would advertise and compares. A world
// started with other ports, another cadence or another declared render delay is a
// different backend, not this one resumed.
let live = self.build_descriptor()?;
if descriptor != live {
return Err(incompatible(
"the staged world was captured under another environment descriptor",
));
}
let expected = crate::state::Compatibility::of(&descriptor).digest();
if expected != params.compatibility_digest {
return Err(incompatible(format!(
"the staged world's compatibility {expected} is not the {} the restore requires",
params.compatibility_digest
)));
}
let counter: i64 = text("counter")?
.parse()
.map_err(|_| incompatible("the world payload's counter is not an integer"))?;
let world_time = RationalNs::from_json(
value
.get("worldTime")
.ok_or_else(|| incompatible("the world payload has no worldTime"))?,
)
.map_err(|e| incompatible(format!("the world payload's worldTime: {}", e.0)))?;
let pipeline_value = value
.get("pipeline")
.and_then(|p| p.get("frames"))
.and_then(Value::as_array)
.ok_or_else(|| incompatible("the world payload has no pipeline frames"))?;
let mut frames = Vec::with_capacity(pipeline_value.len());
for frame in pipeline_value {
let boundary = frame
.get("boundary")
.and_then(Value::as_str)
.ok_or_else(|| incompatible("a captured frame has no boundary"))?
.parse::<u64>()
.map_err(|_| incompatible("a captured frame's boundary is not a U64"))?;
let frame_counter = frame
.get("counter")
.and_then(Value::as_str)
.ok_or_else(|| incompatible("a captured frame has no counter"))?
.parse::<i64>()
.map_err(|_| incompatible("a captured frame's counter is not an integer"))?;
frames.push((boundary, frame_counter));
}
match frames.last() {
Some((boundary, _)) if *boundary == committed_step => {}
_ => {
return Err(incompatible(
"the captured pipeline does not end at the committed boundary",
));
}
}
let audio_value = value
.get("audio")
.ok_or_else(|| incompatible("the world payload has no audio state"))?;
let audio_number = |key: &str| -> DomainResult<u128> {
audio_value
.get(key)
.and_then(Value::as_str)
.ok_or_else(|| incompatible(format!("the captured audio state has no {key}")))?
.parse::<u128>()
.map_err(|_| incompatible(format!("the captured audio {key} is not a number")))
};
let audio_next_sample = u64::try_from(audio_number("nextSample")?)
.map_err(|_| incompatible("the captured audio position is outside U64"))?;
let audio_phase = u64::try_from(audio_number("phase")?)
.map_err(|_| incompatible("the captured audio phase is outside U64"))?;
if self.config.faults.fail_stage_restore {
return Err(incompatible(
"injected staging refusal: this participant's replacement state does not \
validate",
));
}
if let Some(staged) = &self.staged {
return Err(DomainError::before(
ErrorCode::Conflict,
format!(
"this environment already holds the staged restore {} for checkpoint {}",
staged.token, staged.checkpoint_id
),
));
}
let token = crate::agent::restore_token(
&params.checkpoint_id,
&scope,
&actual,
&self.config.incarnation_id,
);
if self.activated.contains(&token) {
return Err(DomainError::before(
ErrorCode::Conflict,
"this exact restore was already activated on this environment",
));
}
self.staged = Some(StagedWorld {
token: token.clone(),
checkpoint_id: params.checkpoint_id.clone(),
scope,
episode_id: parse_id(&text("episodeId")?)
.map_err(|e| incompatible(format!("the world payload's episodeId {e}")))?,
descriptor,
boundary: committed_step,
counter,
world_time,
advances: number("advances")?,
frames,
audio_next_sample,
audio_phase,
audio_accumulator: audio_number("accumulator")?,
audio_denominator: audio_number("denominator")?,
});
self.status.set_state(WorkerState::StagedRestore);
let result = StageRestoreResult {
checkpoint_id: params.checkpoint_id,
restore_token: token,
};
Ok(HandlerReply::from(&result))
}
/// `State.ActivateRestore`: install the staged world and return its coherent observation.
///
/// Nothing advances. The pipeline's frames are rendered again into fresh artifacts of the
/// current store, which is what "the durable store imports fresh immutable bus artifacts"
/// means on the producing side, and the observation carries no audio chunk because no
/// interval was played.
async fn state_activate_restore(
&mut self,
ctx: &HandlerCtx<'_>,
) -> DomainResult<HandlerReply> {
let params: ActivateRestoreParams = ctx.params()?;
if self.activated.contains(&params.restore_token) {
return Err(DomainError::before(
ErrorCode::Conflict,
"this restore token has already been activated",
));
}
let Some(staged) = self.staged.take() else {
return Err(DomainError::before(
ErrorCode::InvalidPhase,
"this environment holds no staged restore",
));
};
if staged.token != params.restore_token {
let token = staged.token.clone();
self.staged = Some(staged);
return Err(DomainError::before(
ErrorCode::IdentityMismatch,
format!(
"this environment's staged restore is {token}, not {}",
params.restore_token
),
));
}
if self.config.faults.fail_activate_restore {
let token = staged.token.clone();
self.staged = Some(staged);
self.status.set_state(WorkerState::Failed);
return Err(DomainError::new(
ErrorCode::BackendFailure,
format!("injected activation failure; {token} stays staged and unresumed"),
MutationCertainty::None,
));
}
self.status.set_state(WorkerState::Restoring);
let mut pipeline = ViewPipeline::new(
CounterEnvironment::view_descriptor(self.config.observation_delay_steps),
self.config.renders.clone(),
);
pipeline.restore(ctx.client, &staged.frames).await?;
let audio = AudioSource::restored_from(
CounterEnvironment::audio_descriptor(),
staged.audio_next_sample,
staged.audio_phase,
staged.audio_accumulator,
staged.audio_denominator,
)?;
self.epoch = Some(staged.scope.epoch.clone());
self.episode_id = Some(staged.episode_id.clone());
self.descriptor = Some(staged.descriptor.clone());
self.boundary = staged.boundary;
self.counter = staged.counter;
self.world_time = staged.world_time;
self.advances = staged.advances;
// Batch ids are unique within an epoch, and this is a new one. Keeping the old set
// would refuse nothing extra: a request under the old epoch is already refused by its
// scope.
self.batches.clear();
self.pipeline = Some(pipeline);
self.audio = Some(audio);
self.previous_view = None;
self.activated.insert(staged.token);
self.status.set_state(WorkerState::Ready);
self.status.set_scope(Some(scope_at(
&staged.scope.session_id,
&staged.scope.epoch,
staged.boundary,
)));
let (observation, attachments) = self.restored_observation()?;
let result = ActivateRestoreResult {
committed_step: staged.boundary,
checkpoint_id: staged.checkpoint_id,
observation: Some(observation),
};
result
.validate_for_role(Role::Environment)
.map_err(|e| DomainError::invalid(e.0))?;
let mut reply = HandlerReply::from(&result);
reply.artifacts = attachments;
Ok(reply)
}
/// The observation the restored world is already at: no render, no advance, no audio.
fn restored_observation(
&mut self,
) -> DomainResult<(WorldObservation, Vec<(String, flybus::Artifact)>)> {
let boundary = self.boundary;
let counter = self.counter;
let pipeline = self
.pipeline
.as_ref()
.ok_or_else(|| DomainError::before(ErrorCode::InvalidPhase, "no view pipeline"))?;
let (view, artifact) = pipeline.at(boundary).ok_or_else(|| {
incompatible("the restored pipeline holds no frame for the restored boundary")
})?;
self.previous_view = Some((view.clone(), artifact.clone()));
let observation = WorldObservation {
boundary,
world_time: self.world_time,
engine_frame: Some(boundary.to_string()),
sensory_views: vec![view.clone()],
inspection: inspection(counter, boundary),
broadcast_views: vec![view.clone()],
// No interval was played, so there is no chunk. A chunk here would be an old
// epoch's audio offered as current.
audio: Vec::new(),
};
Ok((
observation,
vec![(media::view_attachment(&view.view_id), artifact)],
))
}
}

View file

@ -26,6 +26,7 @@ use crate::media::{RenderCounter, SensorLog};
use crate::launcher::{
AgentLaunch, EnvironmentLaunch, Launcher, ReapOutcome, SUPERVISOR_CLIENT, ThreadBudget,
};
use crate::state::{CheckpointStore, CheckpointWriter, StoreConfig, StoreFaults, WriterConfig, WriterFaults};
use crate::task::{ActionExecutor, CounterTask, IdentityExecutor, Terminal};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
// glob keeps the contract's own names in sight instead of restating them.
@ -85,6 +86,14 @@ pub struct HarnessConfig {
/// The threads reserved for the coordinator, its router and its store.
pub coordinator_threads: usize,
pub environment_threads: usize,
/// How many committed generations the durable checkpoint store keeps.
pub store: StoreConfig,
/// The durable write faults this composition injects.
pub store_faults: StoreFaults,
/// The checkpoint queue's bounds.
pub writer: WriterConfig,
/// The writer faults this composition injects.
pub writer_faults: WriterFaults,
}
impl Default for HarnessConfig {
@ -107,6 +116,10 @@ impl Default for HarnessConfig {
thread_budget: None,
coordinator_threads: 1,
environment_threads: 1,
store: StoreConfig::default(),
store_faults: StoreFaults::default(),
writer: WriterConfig::default(),
writer_faults: WriterFaults::default(),
}
}
}
@ -134,6 +147,14 @@ const ENV_SERVICE: &str = "env.arena";
const ENV_CLIENT: &str = "environment";
const ENV_WORKER: &str = "arena";
const COORDINATOR_CLIENT: &str = "coordinator";
/// The checkpoint writer's own bus identity. It publishes checkpoint events and nothing else.
const WRITER_CLIENT: &str = "checkpoint-writer";
/// How many times one participant may be replaced in a composition.
///
/// Each replacement connects under its own client id, so a restart is visibly a new
/// participant rather than a silent reattachment, and the policy has to name them all.
const MAX_GENERATIONS: u32 = 8;
fn agent_service(agent_id: &Id) -> String {
format!("agent.{agent_id}")
@ -172,6 +193,10 @@ pub struct SessionHarness {
/// The supervisor. It owns every participant's lifetime and thread allocation.
pub launcher: Launcher,
observers: Mutex<Vec<Client>>,
/// Which generation of each participant is running: 1 is the one the composition started.
generations: BTreeMap<Id, u32>,
/// Where the durable checkpoint store lives, for a test that reads the files themselves.
checkpoint_root: std::path::PathBuf,
}
impl SessionHarness {
@ -204,12 +229,16 @@ impl SessionHarness {
g.call = vec![Pattern::prefix("agent."), Pattern::prefix("env.")];
}),
)
// The writer publishes the checkpoint events and never calls a participant.
.client(WRITER_CLIENT, grants(|g| g.publish = vec![Pattern::prefix("session.")]))
.client(ENV_CLIENT, grants(|g| g.register = vec![Pattern::exact(ENV_SERVICE)]))
.client(
&format!("{ENV_CLIENT}-r2"),
grants(|g| g.register = vec![Pattern::exact(ENV_SERVICE)]),
)
.client("observer", grants(|g| g.subscribe = vec![Pattern::prefix("session.")]));
for generation in 2..=MAX_GENERATIONS {
policy = policy.client(
&format!("{ENV_CLIENT}-r{generation}"),
grants(|g| g.register = vec![Pattern::exact(ENV_SERVICE)]),
);
}
for spec in &config.agents {
let service = agent_service(&spec.agent_id);
policy = policy.client(
@ -218,10 +247,12 @@ impl SessionHarness {
);
// A replacement worker connects under its own client id, so a restart is visibly a
// new participant rather than a silent reattachment to the active epoch.
policy = policy.client(
&format!("{}-r2", agent_client(&spec.agent_id)),
grants(|g| g.register = vec![Pattern::exact(&service)]),
);
for generation in 2..=MAX_GENERATIONS {
policy = policy.client(
&format!("{}-r{generation}", agent_client(&spec.agent_id)),
grants(|g| g.register = vec![Pattern::exact(&service)]),
);
}
}
let mut router_config = RouterConfig::new(&store_root);
router_config.policy = policy;
@ -299,6 +330,21 @@ impl SessionHarness {
}
let coordinator_client = launcher.connect(COORDINATOR_CLIENT).await?;
// The durable store lives beside the router's artifact store and never inside it: a
// committed generation is outside the bus's ephemeral collection.
let checkpoint_root = root.join("checkpoints");
let mut store = CheckpointStore::open(&checkpoint_root, config.store).map_err(refusal)?;
*store.faults_mut() = config.store_faults.clone();
let writer_client = launcher.connect(WRITER_CLIENT).await?;
let writer = CheckpointWriter::start(
store,
config.writer,
config.writer_faults.clone(),
Some((
writer_client,
format!("session.{}.checkpoints", config.session_id),
)),
);
let executors: BTreeMap<Id, Box<dyn ActionExecutor>> = config
.agents
.iter()
@ -316,6 +362,8 @@ impl SessionHarness {
Box::new(CounterTask::new(&config.epoch, config.terminal)),
executors,
);
let mut coordinator = coordinator;
coordinator.attach_store(writer);
Ok(SessionHarness {
coordinator,
@ -326,9 +374,16 @@ impl SessionHarness {
sensors,
launcher,
observers: Mutex::new(Vec::new()),
generations: BTreeMap::new(),
checkpoint_root,
})
}
/// Where the durable checkpoint store's generations and store manifest live.
pub fn checkpoint_root(&self) -> &std::path::Path {
&self.checkpoint_root
}
pub fn router(&self) -> &Router {
self.launcher.router()
}
@ -373,10 +428,11 @@ impl SessionHarness {
.find(|spec| spec.agent_id == *agent_id)
.expect("a configured agent")
.clone();
let generation = self.next_generation(agent_id)?;
self.launcher.kill(agent_id).await;
let tick_duration = millis(self.config.tick_ms).expect("a positive tick");
let incarnation_id =
parse_id(&format!("{agent_id}-inc-2")).expect("an agent id plus a suffix is an Id");
let incarnation_id = parse_id(&format!("{agent_id}-inc-{generation}"))
.expect("an agent id plus a suffix is an Id");
self.launcher
.launch_agent(AgentLaunch {
session_id: self.config.session_id.clone(),
@ -390,7 +446,7 @@ impl SessionHarness {
// predecessor wrote, so a restore's sensory input is visible beside it.
sensors: self.sensors.get(agent_id).cloned().unwrap_or_default(),
faults: spec.faults.clone(),
client_id: format!("{}-r2", agent_client(agent_id)),
client_id: format!("{}-r{generation}", agent_client(agent_id)),
service: agent_service(agent_id),
})
.await
@ -403,6 +459,102 @@ impl SessionHarness {
})
}
/// Replaces the environment with a fresh, uninitialized incarnation, as a restore needs.
pub async fn restart_environment(&mut self) -> Result<Restarted, flybus::BusError> {
let worker_id = id(ENV_WORKER);
let generation = self.next_generation(&worker_id)?;
self.launcher.kill(&worker_id).await;
let step_duration = hz(self.config.step_hz).expect("a positive cadence");
let incarnation_id = parse_id(&format!("arena-inc-{generation}"))
.expect("a worker id plus a suffix is an Id");
self.launcher
.launch_environment(EnvironmentLaunch {
session_id: self.config.session_id.clone(),
worker_id: worker_id.clone(),
incarnation_id: incarnation_id.clone(),
step_duration,
ports: self.config.agents.iter().map(|a| a.port_id.clone()).collect(),
worker_threads: self.config.environment_threads,
observation_delay_steps: self.config.observation_delay_steps,
renders: self.renders.clone(),
faults: self.config.environment_faults.clone(),
client_id: format!("{ENV_CLIENT}-r{generation}"),
service: ENV_SERVICE.to_owned(),
})
.await
.map_err(refusal)?;
let worker = self.launcher.worker(&worker_id).expect("just launched");
Ok(Restarted {
service: worker.identity.service.clone(),
service_incarnation: worker.service_incarnation.clone(),
incarnation_id,
})
}
fn next_generation(&mut self, worker_id: &Id) -> Result<u32, flybus::BusError> {
let slot = self.generations.entry(worker_id.clone()).or_insert(1);
if *slot >= MAX_GENERATIONS {
return Err(flybus::BusError::new(
flybus::ErrorCode::QuotaExceeded,
format!(
"{worker_id} has used all {MAX_GENERATIONS} configured client identities; a composition declares how many replacements it allows"
),
));
}
*slot += 1;
Ok(*slot)
}
/// Replaces every participant and points the fenced coordinator at the replacements.
///
/// This is what a recovery does before it restores: the old participants belong to an
/// invalid epoch, and the references the coordinator pinned are exchanged deliberately.
pub async fn replace_all_participants(&mut self) -> Result<(), flybus::BusError> {
let environment = self.environment_id();
self.restart_environment().await?;
let worker = self
.launcher
.worker(&environment)
.expect("just launched")
.worker_ref();
self.coordinator
.replace_participant(&environment, worker)
.map_err(|e| refusal(e.error))?;
for agent_id in self.config.agents.iter().map(|a| a.agent_id.clone()).collect::<Vec<_>>() {
self.restart_agent(&agent_id).await?;
let worker = self
.launcher
.worker(&agent_id)
.expect("just launched")
.worker_ref();
self.coordinator
.replace_participant(&agent_id, worker)
.map_err(|e| refusal(e.error))?;
}
Ok(())
}
/// Changes one agent's injected faults, so the replacement the next restart launches is
/// a participant without them.
///
/// A fault is launch configuration, so clearing one is a relaunch and not a live change:
/// the worker running now keeps whatever it was started with.
pub fn set_agent_faults(&mut self, agent_id: &Id, faults: AgentFaults) {
if let Some(spec) = self
.config
.agents
.iter_mut()
.find(|spec| spec.agent_id == *agent_id)
{
spec.faults = faults;
}
}
/// Changes the environment's injected faults, with the same relaunch rule.
pub fn set_environment_faults(&mut self, faults: EnvironmentFaults) {
self.config.environment_faults = faults;
}
/// Ends one participant without asking it, as a crash would.
pub async fn kill(&mut self, worker_id: &Id) -> ReapOutcome {
self.launcher.kill(worker_id).await
@ -466,7 +618,10 @@ impl SessionHarness {
/// Reaps every participant and closes the router.
pub async fn shutdown(self) {
let SessionHarness { coordinator, mut launcher, observers, .. } = self;
let SessionHarness { mut coordinator, mut launcher, observers, .. } = self;
// The writer task owns artifact handles and a blocking store. Leaving it running
// would leave both behind.
coordinator.shutdown_store().await;
drop(coordinator);
launcher.reap_all(&id("shutdown")).await;
for observer in observers.into_inner().expect("not poisoned") {

View file

@ -1231,6 +1231,8 @@ pub(crate) mod flags {
pub const PREPARE_DELAY_MS: &str = "prepare-delay-ms";
pub const COMMIT_DELAY_MS: &str = "commit-delay-ms";
pub const FAIL_COMMIT_AT_STEP: &str = "fail-commit-at-step";
pub const FAIL_STAGE_RESTORE: &str = "fail-stage-restore";
pub const FAIL_ACTIVATE_RESTORE: &str = "fail-activate-restore";
pub const WORKER: &str = "worker";
pub const PORTS: &str = "ports";
@ -1271,6 +1273,8 @@ pub(crate) mod flags {
PREPARE_DELAY_MS,
COMMIT_DELAY_MS,
FAIL_COMMIT_AT_STEP,
FAIL_STAGE_RESTORE,
FAIL_ACTIVATE_RESTORE,
];
/// What only the environment is given, media options included.
pub const ENVIRONMENT_ONLY: &[&str] = &[
@ -1285,6 +1289,8 @@ pub(crate) mod flags {
TRUNCATED_VIEW_AT_BOUNDARY,
OMIT_AUDIO_AT_BOUNDARY,
OVERLAPPING_AUDIO_AT_BOUNDARY,
FAIL_STAGE_RESTORE,
FAIL_ACTIVATE_RESTORE,
];
/// What a measurement run or one of its row children is given.
pub const MEASURE: &[&str] = &[MODE, AGENTS, STEPS, WARMUP_STEPS, WORKER_THREADS, MODES];
@ -1327,6 +1333,11 @@ impl Started {
arg(flags::WARMUP_TICKS, spec.warmup_ticks),
arg(flags::PREPARE_DELAY_MS, spec.faults.prepare_delay_ms),
arg(flags::COMMIT_DELAY_MS, spec.faults.commit_delay_ms),
arg(flags::FAIL_STAGE_RESTORE, u64::from(spec.faults.fail_stage_restore)),
arg(
flags::FAIL_ACTIVATE_RESTORE,
u64::from(spec.faults.fail_activate_restore),
),
];
if let Some(step) = spec.faults.fail_commit_at_step {
args.push(arg(flags::FAIL_COMMIT_AT_STEP, step));
@ -1345,6 +1356,11 @@ impl Started {
// The media options a world in another process needs to be exactly this
// world. Its render counter and its agents' sensor logs stay there.
arg(flags::OBSERVATION_DELAY_STEPS, spec.observation_delay_steps),
arg(flags::FAIL_STAGE_RESTORE, u64::from(spec.faults.fail_stage_restore)),
arg(
flags::FAIL_ACTIVATE_RESTORE,
u64::from(spec.faults.fail_activate_restore),
),
];
for (flag, boundary) in [
(flags::OMIT_VIEW_AT_BOUNDARY, spec.faults.omit_view_at_boundary),
@ -1458,6 +1474,8 @@ mod flag_tests {
truncated_view_at_boundary: Some(3),
omit_audio_at_boundary: Some(4),
overlapping_audio_at_boundary: Some(5),
fail_stage_restore: true,
fail_activate_restore: true,
}
}
@ -1491,6 +1509,8 @@ mod flag_tests {
fail_commit_at_step: Some(2),
prepare_delay_ms: 1,
commit_delay_ms: 2,
fail_stage_restore: true,
fail_activate_restore: true,
},
client_id: "worker-fly-a".to_owned(),
service: "agent.fly-a".to_owned(),
@ -1535,6 +1555,8 @@ mod flag_tests {
flags::TRUNCATED_VIEW_AT_BOUNDARY,
flags::OMIT_AUDIO_AT_BOUNDARY,
flags::OVERLAPPING_AUDIO_AT_BOUNDARY,
flags::FAIL_STAGE_RESTORE,
flags::FAIL_ACTIVATE_RESTORE,
] {
assert!(written.contains(&format!("--{flag}")), "--{flag} is not written");
}

View file

@ -34,6 +34,7 @@ pub mod media;
pub mod metrics;
pub mod phase;
pub mod rpc;
pub mod state;
pub mod task;
pub mod worker;

View file

@ -133,7 +133,14 @@ pub fn arena_frame(descriptor: &ViewDescriptor, counter: i64, boundary: u64) ->
/// cannot be served an arbitrary stale image.
pub struct ViewPipeline {
descriptor: ViewDescriptor,
frames: VecDeque<(u64, flybus::Artifact)>,
/// Each retained frame: its producing boundary, the world counter it was rendered from
/// and the owned handle on its immutable bytes.
///
/// The counter is kept because it is the whole of the reconstruction input: a checkpoint
/// records `(boundary, counter)` per retained frame and a restore re-renders them into
/// fresh artifacts of the current store, rather than persisting a transient artifact
/// identity that cannot survive a router restart.
frames: VecDeque<(u64, i64, flybus::Artifact)>,
renders: RenderCounter,
}
@ -160,7 +167,7 @@ impl ViewPipeline {
let bytes = arena_frame(&self.descriptor, counter, boundary);
let artifact = seal(client, FRAME_CONTENT_TYPE, &bytes).await?;
self.renders.bump();
self.frames.push_back((boundary, artifact));
self.frames.push_back((boundary, counter, artifact));
// Keep exactly the frames a declared delay can still require.
while self.frames.len() > self.descriptor.observation_delay_steps as usize + 1 {
self.frames.pop_front();
@ -168,6 +175,50 @@ impl ViewPipeline {
Ok(())
}
/// The reconstruction inputs of every retained frame, oldest first.
///
/// This is what a checkpoint records for the pending sensor pipeline: the producing
/// boundary and the world counter, never an artifact identity.
pub fn retained(&self) -> Vec<(u64, i64)> {
self.frames
.iter()
.map(|(boundary, counter, _)| (*boundary, *counter))
.collect()
}
/// Rebuilds the pipeline from recorded reconstruction inputs, into fresh artifacts.
///
/// Every frame is rendered again in the current store, so nothing a fence dropped is
/// expected to come back and no old artifact identity crosses the recovery.
pub async fn restore(
&mut self,
client: &flybus::Client,
frames: &[(u64, i64)],
) -> DomainResult<()> {
if frames.len() > self.descriptor.observation_delay_steps as usize + 1 {
return Err(media_error(format!(
"a captured pipeline of {} frames does not fit a declared delay of {}",
frames.len(),
self.descriptor.observation_delay_steps
)));
}
for window in frames.windows(2) {
if window[1].0 != window[0].0 + 1 {
return Err(media_error(
"a captured pipeline's producing boundaries are not consecutive",
));
}
}
self.frames.clear();
for (boundary, counter) in frames {
let bytes = arena_frame(&self.descriptor, *counter, *boundary);
let artifact = seal(client, FRAME_CONTENT_TYPE, &bytes).await?;
self.renders.bump();
self.frames.push_back((*boundary, *counter, artifact));
}
Ok(())
}
/// Seals a frame of the wrong length, which is what a broken backend produces. The
/// reference it returns describes the artifact honestly, so the shape check is the thing
/// under test rather than a lie in the payload.
@ -181,7 +232,7 @@ impl ViewPipeline {
bytes.truncate(bytes.len() - self.descriptor.row_stride as usize);
let artifact = seal(client, FRAME_CONTENT_TYPE, &bytes).await?;
self.renders.bump();
self.frames.push_back((boundary, artifact));
self.frames.push_back((boundary, counter, artifact));
while self.frames.len() > self.descriptor.observation_delay_steps as usize + 2 {
self.frames.pop_front();
}
@ -198,8 +249,8 @@ impl ViewPipeline {
pub fn frame_produced_at(&self, produced: u64) -> Option<(ViewRef, flybus::Artifact)> {
self.frames
.iter()
.find(|(step, _)| *step == produced)
.map(|(step, artifact)| {
.find(|(step, _, _)| *step == produced)
.map(|(step, _, artifact)| {
(
ViewRef {
view_id: self.descriptor.view_id.clone(),
@ -257,6 +308,52 @@ impl AudioSource {
source
}
/// The exact state a capture recorded: sample position, waveform phase and the
/// unconsumed fraction of a frame.
///
/// Restoring the position alone would restart the waveform and round the remainder away,
/// which is a resample the restore rules refuse. The first chunk of the new epoch marks
/// the discontinuity the recovery established.
pub fn restored_from(
descriptor: AudioDescriptor,
next_sample: u64,
phase: u64,
accumulator: u128,
denominator: u128,
) -> DomainResult<AudioSource> {
if denominator == 0 {
return Err(DomainError::invalid(
"audio: a captured accumulator denominator of zero",
));
}
if accumulator >= denominator {
return Err(DomainError::invalid(
"audio: a captured accumulator is not below one whole frame",
));
}
if phase >= descriptor.sample_rate {
return Err(DomainError::invalid(
"audio: a captured phase is not below the sample rate",
));
}
let mut source = AudioSource::new(descriptor, next_sample);
source.discontinuous = true;
source.phase = phase;
source.accumulator = accumulator;
source.denominator = denominator;
Ok(source)
}
/// The waveform phase, for a capture.
pub fn phase(&self) -> u64 {
self.phase
}
/// The unconsumed fraction of a frame and the denominator it is over, for a capture.
pub fn accumulator(&self) -> (u128, u128) {
(self.accumulator, self.denominator)
}
pub fn descriptor(&self) -> &AudioDescriptor {
&self.descriptor
}
@ -439,18 +536,47 @@ pub fn check_required_views(
Ok(())
}
/// Every declared audio stream produces exactly one chunk per transition.
/// Where an observation came from.
///
/// `state-media-v1` section 2 makes a chunk the audio of an *interval*, so whether an
/// observation must carry one is a question about its provenance and not about its boundary
/// number. MEDIA-01 wrote the rule as "boundary 0 carries no chunk", which is true of the one
/// observation that slice could produce without a transition and false of the other one:
/// `State.ActivateRestore` installs a coherent observation at boundary `k` without advancing
/// gameplay, and it covers no interval either. Naming the provenance is the fix; exempting
/// the restored observation from the validator instead would have left "must a chunk exist"
/// unanswered exactly where a stale chunk would do the most damage.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum ObservationOrigin {
/// The observation a completed transition produced. Its interval has audio.
Transition,
/// An observation established at a boundary without running a transition:
/// `Environment.Initialize`'s `O[0]` and `State.ActivateRestore`'s restored observation.
/// It covers no interval, so it carries no chunk and one in it is refused.
Installed,
}
/// Every declared audio stream produces exactly one chunk per transition, and none at all in
/// an observation that is not one.
///
/// The contract states the shape and the ordering of chunks, not whether one has to exist, so
/// this is MEDIA-01's choice and it is deliberate: a session that tolerates a silently missing
/// chunk cannot tell "this world produced no audio for this interval" from "the chunk was
/// lost", and the second is the case the retention rules care about. Boundary 0 has no
/// preceding interval and so carries no chunk.
/// lost", and the second is the case the retention rules care about. The mirror of that, which
/// STATE-01 needs, is that an installed observation carrying a chunk is a stale chunk being
/// offered as current, and is refused for the same reason.
pub fn check_required_audio(
descriptor: &EnvironmentDescriptor,
observation: &WorldObservation,
origin: ObservationOrigin,
) -> DomainResult<()> {
if observation.boundary == 0 {
if origin == ObservationOrigin::Installed {
if let Some(chunk) = observation.audio.first() {
return Err(media_error(format!(
"audio stream {} produced a chunk for an observation that ran no transition",
chunk.stream_id
)));
}
return Ok(());
}
for stream in &descriptor.audio {

File diff suppressed because it is too large Load diff

View file

@ -39,6 +39,16 @@ pub fn episode_schema() -> SchemaRef {
synthetic_schema("arena.episode.v1", 1)
}
/// The schema of a captured task ledger.
pub fn ledger_schema() -> SchemaRef {
synthetic_schema("arena.ledger.v1", 1)
}
/// The schema of a captured action-executor state.
pub fn executor_schema() -> SchemaRef {
synthetic_schema("arena.executor.v1", 1)
}
pub fn controller_schema_ref() -> SchemaRef {
synthetic_schema("arena.controller.v1", 1)
}
@ -86,6 +96,29 @@ pub trait Task: Send {
/// How many times `evaluate_transition` has run. A transition must evaluate once.
fn evaluations(&self) -> u64;
/// The checkpointable ledger at a committed boundary (`workers-v1` section 4).
fn capture(&self) -> DomainResult<TypedValue>;
/// Validates a captured ledger without installing it, so a group install can fail before
/// anything is changed.
fn validate_restore(&self, state: &TypedValue) -> DomainResult<()>;
/// Installs a validated ledger under `epoch`. Event identity is derived from the epoch,
/// so the new one is part of the install rather than something the ledger keeps from the
/// epoch it was captured in.
fn install_restore(&mut self, epoch: &Id, state: &TypedValue) -> DomainResult<()>;
/// Every event identity this ledger has issued, mapped onto the identity it would have
/// under `to_epoch`.
///
/// `workers-v1` section 4 derives an event id from the epoch, so a trace recorded in one
/// epoch cannot be compared with a trace recorded in another until these are rebased.
/// The ledger owns the derivation, so it is the only thing that can do it.
fn rebase_ids(&self, to_epoch: &Id) -> DomainResult<BTreeMap<Id, Id>>;
/// How far event identity has reached: the highest source step and the number issued.
fn event_watermarks(&self) -> (u64, u64);
}
/// Translates one selected decision into a controller intent, with no port assignment.
@ -98,6 +131,15 @@ pub trait ActionExecutor: Send {
progress: &TypedValue,
clock: &RationalNs,
) -> DomainResult<(ControllerIntent, Vec<TaskEvent>)>;
/// Per-executor state at a committed boundary (`workers-v1` section 4).
fn capture(&self) -> DomainResult<TypedValue>;
/// Validates a captured executor state without installing it.
fn validate_restore(&self, state: &TypedValue) -> DomainResult<()>;
/// Installs a validated executor state.
fn install_restore(&mut self, state: &TypedValue) -> DomainResult<()>;
}
/// The only executor v1 supports: it passes a direct-control decision through unchanged.
@ -123,6 +165,37 @@ impl ActionExecutor for IdentityExecutor {
.map_err(|e| DomainError::invalid(format!("decision: {e}")))?;
Ok((intent, Vec::new()))
}
/// The identity executor is stateless, and says so rather than capturing nothing.
///
/// An empty object would be indistinguishable from a stateful executor whose capture went
/// missing, so the capture names the executor it came from and a restore refuses any
/// other one.
fn capture(&self) -> DomainResult<TypedValue> {
TypedValue::new(executor_schema(), json!({"executor": "identity-v1"}))
.map_err(|e| DomainError::invalid(e.0))
}
fn validate_restore(&self, state: &TypedValue) -> DomainResult<()> {
if state.schema != executor_schema() {
return Err(DomainError::before(
ErrorCode::IncompatibleState,
"the captured executor state does not carry the executor schema",
));
}
match state.value.get("executor").and_then(Value::as_str) {
Some("identity-v1") => Ok(()),
other => Err(DomainError::before(
ErrorCode::IncompatibleState,
format!("the captured executor is {other:?}, not the identity executor"),
)),
}
}
fn install_restore(&mut self, state: &TypedValue) -> DomainResult<()> {
// Stateless: validation is the whole of the install, and it is not skipped.
self.validate_restore(state)
}
}
/// When the counter task asks for a terminal episode transition.
@ -146,6 +219,10 @@ pub struct CounterTask {
evaluations: u64,
total_reward: f64,
counter: i64,
/// The highest source step any issued event belongs to, and how many were issued. These
/// are the event watermarks a checkpoint records and a resumed epoch continues from.
last_source_step: u64,
issued_events: u64,
terminal: Terminal,
}
@ -159,6 +236,8 @@ impl CounterTask {
evaluations: 0,
total_reward: 0.0,
counter: 0,
last_source_step: 0,
issued_events: 0,
terminal,
}
}
@ -239,6 +318,7 @@ impl Task for CounterTask {
payload: TypedValue::new(event_schema(), json!({"counter": self.counter}))
.expect("a synthetic typed value fits the contract"),
}];
self.issued_events += events.len() as u64;
Ok(Bootstrap { contexts, progress: self.progress_value(), events })
}
@ -314,6 +394,8 @@ impl Task for CounterTask {
));
}
self.last_source_step = self.last_source_step.max(source_step);
self.issued_events += events.len() as u64;
let next_contexts = self
.agents
.iter()
@ -346,6 +428,173 @@ impl Task for CounterTask {
fn evaluations(&self) -> u64 {
self.evaluations
}
fn capture(&self) -> DomainResult<TypedValue> {
TypedValue::new(
ledger_schema(),
json!({
"epoch": self.epoch.as_str(),
"agents": self.agents.iter().map(String::as_str).collect::<Vec<_>>(),
"bindings": self
.bindings
.iter()
.map(|b| json!({"portId": b.port_id.as_str(), "agentId": b.agent_id.as_str()}))
.collect::<Vec<_>>(),
"transitions": self.transitions,
"evaluations": self.evaluations,
"totalReward": self.total_reward,
"counter": self.counter,
"lastSourceStep": self.last_source_step,
"issuedEvents": self.issued_events,
}),
)
.map_err(|e| DomainError::invalid(e.0))
}
fn validate_restore(&self, state: &TypedValue) -> DomainResult<()> {
if state.schema != ledger_schema() {
return Err(DomainError::before(
ErrorCode::IncompatibleState,
"the captured ledger does not carry this task's schema",
));
}
for field in [
"epoch",
"agents",
"bindings",
"transitions",
"evaluations",
"totalReward",
"counter",
"lastSourceStep",
"issuedEvents",
] {
if state.value.get(field).is_none() {
return Err(DomainError::before(
ErrorCode::IncompatibleState,
format!("the captured ledger has no {field}"),
));
}
}
let bindings = state
.value
.get("bindings")
.and_then(Value::as_array)
.ok_or_else(|| {
DomainError::before(
ErrorCode::IncompatibleState,
"the captured ledger's bindings are not a list",
)
})?;
if bindings.len() != self.bindings.len() && !self.bindings.is_empty() {
return Err(DomainError::before(
ErrorCode::IncompatibleState,
"the captured ledger binds another number of ports",
));
}
Ok(())
}
fn install_restore(&mut self, epoch: &Id, state: &TypedValue) -> DomainResult<()> {
self.validate_restore(state)?;
let number = |key: &str| -> DomainResult<u64> {
state.value.get(key).and_then(Value::as_u64).ok_or_else(|| {
DomainError::before(
ErrorCode::IncompatibleState,
format!("the captured ledger's {key} is not a whole number"),
)
})
};
let mut agents = Vec::new();
for value in state.value["agents"].as_array().expect("validated") {
let agent = value.as_str().ok_or_else(|| {
DomainError::before(
ErrorCode::IncompatibleState,
"the captured ledger names an agent that is not a string",
)
})?;
agents.push(parse_id(agent).map_err(|e| {
DomainError::before(ErrorCode::IncompatibleState, format!("ledger: {e}"))
})?);
}
let mut bindings = Vec::new();
for value in state.value["bindings"].as_array().expect("validated") {
let port_id = value.get("portId").and_then(Value::as_str).ok_or_else(|| {
DomainError::before(
ErrorCode::IncompatibleState,
"the captured ledger has a binding with no portId",
)
})?;
let agent_id = value.get("agentId").and_then(Value::as_str).ok_or_else(|| {
DomainError::before(
ErrorCode::IncompatibleState,
"the captured ledger has a binding with no agentId",
)
})?;
bindings.push(PortBinding {
port_id: parse_id(port_id).map_err(|e| {
DomainError::before(ErrorCode::IncompatibleState, format!("ledger: {e}"))
})?,
agent_id: parse_id(agent_id).map_err(|e| {
DomainError::before(ErrorCode::IncompatibleState, format!("ledger: {e}"))
})?,
});
}
let counter = state.value.get("counter").and_then(Value::as_i64).ok_or_else(|| {
DomainError::before(
ErrorCode::IncompatibleState,
"the captured ledger's counter is not an integer",
)
})?;
let total_reward = state
.value
.get("totalReward")
.and_then(Value::as_f64)
.filter(|v| v.is_finite())
.ok_or_else(|| {
DomainError::before(
ErrorCode::IncompatibleState,
"the captured ledger's totalReward is not a finite number",
)
})?;
// The epoch is the caller's, not the capture's: event identity belongs to the epoch
// the ledger is being installed into.
self.epoch = epoch.clone();
self.agents = agents;
self.bindings = bindings;
self.transitions = number("transitions")?;
self.evaluations = number("evaluations")?;
self.total_reward = total_reward;
self.counter = counter;
self.last_source_step = number("lastSourceStep")?;
self.issued_events = number("issuedEvents")?;
Ok(())
}
fn rebase_ids(&self, to_epoch: &Id) -> DomainResult<BTreeMap<Id, Id>> {
let mut out = BTreeMap::new();
out.insert(
event_id(&self.epoch, 0, "bootstrap", 0),
event_id(to_epoch, 0, "bootstrap", 0),
);
// The counter task issues exactly one `counter-delta` event per bound port per
// evaluated transition, in descriptor port order, so every identity it has ever
// issued is re-derivable from its ledger without keeping a list of them.
let ports = self.bindings.len() as u32;
for source_step in 1..=self.last_source_step {
for ordinal in 0..ports {
out.insert(
event_id(&self.epoch, source_step, "counter-delta", ordinal),
event_id(to_epoch, source_step, "counter-delta", ordinal),
);
}
}
Ok(out)
}
fn event_watermarks(&self) -> (u64, u64) {
(self.last_source_step, self.issued_events)
}
}
/// The inspection value the counter environment publishes.

View file

@ -8,13 +8,18 @@
//! `Id` and `Digest` are type aliases, because the shared crate carries both as validated
//! `String`s from `flybus::wire` rather than forking the encodings into newtypes.
use std::collections::BTreeMap;
use serde_json::{Map, Value};
pub use fly_session_types::ArtifactRef;
pub use fly_session_types::canonical::{
self, OperationKey, body_digest, canonicalize, digest_of, sha256_hex,
};
pub use fly_session_types::media::{AudioDescriptor, AudioRef, ViewDescriptor, ViewRef};
pub use fly_session_types::media::{
ActivateRestoreParams, ActivateRestoreResult, AudioDescriptor, AudioRef, CaptureParams,
CaptureResult, StageRestoreParams, StageRestoreResult, ViewDescriptor, ViewRef,
};
pub use fly_session_types::rpc::{
ErrorCode, MutationCertainty, SessionRpcFailure, SessionRpcOutcome, SessionRpcRequest,
SessionRpcSuccess,
@ -214,6 +219,57 @@ pub fn outcome_identity(
}
}
/// The epoch-derived identities of a behaviour trace, rewritten onto one reference epoch.
///
/// `step-v1` section 8 compares committed behaviour across runs, excluding wall time, request
/// ids "and other explicitly operational metadata". A resumed run's epoch is neither: it is
/// behaviour metadata, and `scope.epoch`, the batch id and every task event id are derived
/// from it. Comparing the two runs therefore means rewriting exactly those three things and
/// nothing else, which is what this does -- and it **fails** on anything it does not
/// recognise instead of passing it through, so a field that silently stopped being rebased
/// would fail the comparison rather than weaken it.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct EpochRebase {
pub from: Id,
pub to: Id,
/// Every event identity the task issued under `from`, and the identity it has under `to`.
pub events: BTreeMap<Id, Id>,
}
impl EpochRebase {
/// Rewrites one behaviour record. An identity this rebase does not know is an error.
pub fn apply(&self, behaviour: &TraceBehaviour) -> Result<TraceBehaviour, String> {
if behaviour.scope.epoch != self.from {
return Err(format!(
"this behaviour was recorded in epoch {}, not {}",
behaviour.scope.epoch, self.from
));
}
let mut out = behaviour.clone();
out.scope = Scope::new(&behaviour.scope.session_id, &self.to, behaviour.scope.step)
.map_err(|e| e.0)?;
let prefix = format!("batch-{}-", self.from);
let suffix = behaviour
.batch_id
.strip_prefix(&prefix)
.ok_or_else(|| format!("the batch id {} is not derived from {}", behaviour.batch_id, self.from))?;
out.batch_id = parse_id(&format!("batch-{}-{suffix}", self.to))?;
let map = |ids: &[Id]| -> Result<Vec<Id>, String> {
ids.iter()
.map(|id| {
self.events
.get(id)
.cloned()
.ok_or_else(|| format!("no rebased identity for the event {id}"))
})
.collect()
};
out.outcome_ids = map(&behaviour.outcome_ids)?;
out.event_ids = map(&behaviour.event_ids)?;
Ok(out)
}
}
/// One session phase transition, recorded whether or not it ends a step.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct PhaseTransition {
@ -253,6 +309,28 @@ impl TraceLog {
.collect()
}
/// Every transition's behaviour, rebased onto one epoch and canonicalized.
///
/// This is the comparison a resumed run is held to: the same strings as
/// [`TraceLog::behavior`], with the epoch metadata accounted for and nothing else changed.
/// A resumed run's log holds transitions from two epochs -- the ones before the checkpoint
/// and the ones after the restore -- so a transition already recorded in `rebase.to` is
/// kept as it stands and one recorded in `rebase.from` is rewritten. A transition in a
/// third epoch is an error; there is no pass-through case.
pub fn behavior_rebased(&self, rebase: &EpochRebase) -> Result<Vec<String>, String> {
self.transitions
.iter()
.map(|t| {
let behaviour = if t.behaviour.scope.epoch == rebase.to {
t.behaviour.clone()
} else {
rebase.apply(&t.behaviour)?
};
canonicalize(&behaviour.to_json()).map_err(|e| e.0)
})
.collect()
}
/// The phase path, as `from -> to` strings.
pub fn phase_path(&self) -> Vec<String> {
self.phases.iter().map(|p| format!("{} -> {}", p.from, p.to)).collect()

View file

@ -597,7 +597,17 @@ async fn execute<E: WorkerEndpoint>(
fn classify_default(method: &str) -> Option<OpClass> {
match method {
"Agent.Prepare" | "Agent.Commit" | "Environment.Advance" => Some(OpClass::StepMutation),
"Agent.Initialize" | "Environment.Initialize" => Some(OpClass::Lifecycle),
// `ipc-v1` section 5 retains lifecycle *and capture* replies until
// `Worker.Acknowledge`. The restore methods join them: their replies carry a
// once-only token and, for an environment, the restored observation's artifact, and a
// duplicate domain request must replay that reply rather than stage or activate a
// second time. They are not step mutations -- they carry no committed step of their
// own and are not keyed by one.
"Agent.Initialize"
| "Environment.Initialize"
| "State.Capture"
| "State.StageRestore"
| "State.ActivateRestore" => Some(OpClass::Lifecycle),
"Worker.Hello" | "Worker.Status" | "Worker.Acknowledge" | "Worker.Shutdown" => {
Some(OpClass::ReadOnly)
}

View file

@ -121,6 +121,8 @@ async fn a_slow_participant_is_resolved_rather_than_failed(mode: ExecutionMode)
resolve: Duration::from_secs(20),
resolve_attempts: 4096,
boot: Duration::from_secs(30),
capture: Duration::from_secs(30),
durable: Duration::from_secs(60),
};
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
let reports = within("run", f.harness.coordinator.run(2))
@ -191,6 +193,8 @@ async fn a_resolution_says_which_of_its_two_bounds_ended_it(mode: ExecutionMode)
resolve: Duration::from_millis(300),
resolve_attempts: 8192,
boot: Duration::from_secs(30),
capture: Duration::from_secs(30),
durable: Duration::from_secs(60),
};
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
let started = Instant::now();
@ -221,6 +225,8 @@ async fn a_resolution_says_which_of_its_two_bounds_ended_it(mode: ExecutionMode)
resolve: Duration::from_secs(600),
resolve_attempts: 3,
boot: Duration::from_secs(30),
capture: Duration::from_secs(30),
durable: Duration::from_secs(60),
};
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
let failure = within("step", f.harness.coordinator.step())

File diff suppressed because it is too large Load diff

View file

@ -56,7 +56,7 @@ impl MemoryReader for &mut dyn MemoryReader {
/// One reward payout in one frame.
///
/// `kind` is an adapter-owned interned name (Pokémon: `milestone`,
/// `exploration`, `map`, `species`, `trainer`, `battle`, `badge`, `boundary`); it is the
/// `exploration`, `map`, `species`, `trainer`, `battle`, `badge`, `boundary`, `catch`); it is the
/// key the statistics counters and the on-screen ticker group by. Field names
/// serialize exactly as the prototype's `RewardEvent` did, so a checkpoint
/// written by either implementation reads in the other.
@ -241,9 +241,21 @@ impl std::error::Error for AdapterError {}
/// A game, as the sim loop sees it.
pub trait GameAdapter: Send {
/// Adapter version string, pinned into the checkpoint compatibility string.
/// Pokémon: `pokered-unique8-v5`.
/// Pokémon: `pokered-unique8-v6`.
fn id(&self) -> &'static str;
/// Earlier [`GameAdapter::id`]s whose checkpoints this build can read, by a migration
/// this adapter has written down and tested.
///
/// The default is empty: an adapter migrates from nothing unless it says otherwise, which
/// is the behaviour every adapter had before this existed. It is only half of the gate --
/// [`crate::compatibility::decide`] also requires the operator to have named the same id in
/// `FLY_ACCEPT_ADAPTERS` for that deploy -- so listing an id here never migrates a live run
/// on its own.
fn migrates_from(&self) -> &'static [&'static str] {
&[]
}
/// Whether semantic rewards are enabled for this cartridge. An adapter that
/// says no must still sample without paying anything, so the stream keeps
/// running with a visible "rewards off" mode.
@ -470,6 +482,10 @@ mod tests {
let platformer =
adapter_for_with_rom_pin("platformer", Some(&"a".repeat(64))).unwrap();
assert_ne!(pokemon.id(), platformer.id());
assert!(
!platformer.migrates_from().contains(&pokemon.id()),
"a migration never crosses games"
);
assert_ne!(pokemon.symbol_provenance(), platformer.symbol_provenance());
// And two ROM revisions of the same game cannot either.
let other = adapter_for_with_rom_pin("platformer", Some(&"b".repeat(64))).unwrap();

View file

@ -30,7 +30,7 @@ pub const PROTOTYPE_PLASTICITY_VERSION: &str = "fly-kc-mbon-rstdp-v2";
pub struct Compatibility<'a> {
/// `kernelVersion(config)` from the neural library.
pub neural_kernel_version: &'a str,
/// The adapter's version string, e.g. `pokered-unique8-v5`.
/// The adapter's version string, e.g. `pokered-unique8-v6`.
pub adapter: &'a str,
/// The dataset's seven SHA-256 digests joined with `:`.
pub dataset_fingerprint: &'a str,
@ -67,6 +67,92 @@ impl Compatibility<'_> {
}
}
/// Position of the adapter's version string in [`Compatibility::string`].
///
/// `{kernel}/{adapter}/{fingerprint}/{plasticity}/binjgb:{rev}/pokered:{commit}/statefmt:{id}`,
/// so the adapter is segment one. Nothing else in the string may move for a migration to be
/// considered: a different kernel, dataset, plasticity, emulator revision, symbol provenance or
/// state format is a different *fly*, not a different reward rule.
const ADAPTER_SEGMENT: usize = 1;
/// The environment variable that opts a deploy into the adapter migration.
///
/// Read by flysim at restore and by `infra/05-deploy.sh`'s compatibility gate. Comma- or
/// whitespace-separated adapter ids, e.g. `FLY_ACCEPT_ADAPTERS=pokered-unique8-v5`.
pub const ACCEPT_ADAPTERS_ENV: &str = "FLY_ACCEPT_ADAPTERS";
/// What a build may do with a checkpoint whose compatibility string is not its own.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum RestoreDecision {
/// Byte-identical. Restore it, as every build always has.
Exact,
/// Every segment but the adapter's is identical, this build's adapter says it can migrate
/// from that one, and the operator named it in [`ACCEPT_ADAPTERS_ENV`]. Restore it.
MigrateAdapter { from: String },
/// Refuse, and say which of the three conditions failed.
Refuse(&'static str),
}
/// Decide whether `checkpoint`'s compatibility string may be restored under `current`.
///
/// Three conditions, all required, in the order they are cheapest to explain:
///
/// 1. the two strings differ in the adapter segment and **nowhere else**;
/// 2. `migrates_from` -- the running adapter's own list -- contains the checkpoint's adapter, so
/// the code that will read that state says out loud that it can;
/// 3. `accepted` -- [`ACCEPT_ADAPTERS_ENV`] as the operator set it for this deploy -- contains it
/// too, so no build ever migrates a run by itself.
///
/// Condition 2 without condition 3 would make the migration silent; condition 3 without condition
/// 2 would let an operator wave through a pair nobody wrote a migration for. Neither alone is
/// enough, which is why both are here.
pub fn decide(
checkpoint: &str,
current: &str,
migrates_from: &[&str],
accepted: &[String],
) -> RestoreDecision {
if checkpoint == current {
return RestoreDecision::Exact;
}
let old: Vec<&str> = checkpoint.split('/').collect();
let new: Vec<&str> = current.split('/').collect();
if old.len() != new.len() {
return RestoreDecision::Refuse("the two compatibility strings do not have the same shape");
}
let differing: Vec<usize> = (0..old.len()).filter(|&index| old[index] != new[index]).collect();
if differing != [ADAPTER_SEGMENT] {
return RestoreDecision::Refuse(
"more than the adapter version differs; nothing but a reward-rule change can migrate",
);
}
let from = old[ADAPTER_SEGMENT];
if !migrates_from.contains(&from) {
return RestoreDecision::Refuse("this build's adapter has no migration from that adapter");
}
if !accepted.iter().any(|name| name == from) {
return RestoreDecision::Refuse(
"the checkpoint's adapter is not in FLY_ACCEPT_ADAPTERS, so the migration was not \
asked for",
);
}
RestoreDecision::MigrateAdapter { from: from.to_string() }
}
/// Parse [`ACCEPT_ADAPTERS_ENV`]: comma- or whitespace-separated, empty entries dropped.
///
/// An unset variable and an empty one are the same thing -- no migration -- so that clearing the
/// opt-in is one edit rather than two.
pub fn accepted_adapters(value: Option<&str>) -> Vec<String> {
value
.unwrap_or_default()
.split([',', ' ', '\t', '\n'])
.map(str::trim)
.filter(|entry| !entry.is_empty())
.map(str::to_string)
.collect()
}
/// `<state size>-<target triple>`: the two things that decide whether a
/// binjgb save state written elsewhere can be memcpy'd back in here.
pub fn state_format_id() -> String {
@ -94,7 +180,7 @@ mod tests {
assert_eq!(
fixture().prototype_string(),
concat!(
"lif-1ms-f64-v2/pokered-unique8-v5/aa:bb:cc:dd:ee:ff:00/",
"lif-1ms-f64-v2/pokered-unique8-v6/aa:bb:cc:dd:ee:ff:00/",
"fly-kc-mbon-rstdp-v2/",
"binjgb:c60e138da5a795ebb55e56b11b7e90024e41112c/",
"pokered:0cd19d3b877b7dc66d12c7050bed9a7f38154d4b",
@ -102,6 +188,73 @@ mod tests {
);
}
fn with_adapter(adapter: &'static str) -> String {
Compatibility { adapter, ..fixture() }.string()
}
#[test]
fn an_identical_string_restores_without_any_opt_in() {
let current = with_adapter("pokered-unique8-v6");
assert_eq!(decide(&current, &current, &[], &[]), RestoreDecision::Exact);
}
#[test]
fn a_v5_checkpoint_restores_under_v6_only_with_the_opt_in() {
let old = with_adapter("pokered-unique8-v5");
let new = with_adapter("pokered-unique8-v6");
let migrates = ["pokered-unique8-v5"];
assert!(matches!(decide(&old, &new, &migrates, &[]), RestoreDecision::Refuse(_)));
assert_eq!(
decide(&old, &new, &migrates, &accepted_adapters(Some("pokered-unique8-v5"))),
RestoreDecision::MigrateAdapter { from: "pokered-unique8-v5".to_string() }
);
// And only for a pair the running adapter says it can migrate.
assert!(matches!(
decide(&old, &new, &[], &accepted_adapters(Some("pokered-unique8-v5"))),
RestoreDecision::Refuse(_)
));
}
#[test]
fn nothing_but_the_adapter_segment_may_move() {
let migrates = ["pokered-unique8-v5"];
let accepted = accepted_adapters(Some("pokered-unique8-v5"));
let new = with_adapter("pokered-unique8-v6");
// A different dataset, with the same adapter bump, is not a migration.
let other_dataset = Compatibility {
adapter: "pokered-unique8-v5",
dataset_fingerprint: "00:11:22:33:44:55:66",
..fixture()
}
.string();
assert!(matches!(
decide(&other_dataset, &new, &migrates, &accepted),
RestoreDecision::Refuse(_)
));
// Neither is a different kernel, and neither is a string of another shape.
let other_kernel =
Compatibility { adapter: "pokered-unique8-v5", neural_kernel_version: "lif-1ms-f64-v3", ..fixture() }
.string();
assert!(matches!(
decide(&other_kernel, &new, &migrates, &accepted),
RestoreDecision::Refuse(_)
));
assert!(matches!(decide("a/b", &new, &migrates, &accepted), RestoreDecision::Refuse(_)));
}
#[test]
fn the_opt_in_list_is_separated_by_commas_or_spaces() {
assert!(accepted_adapters(None).is_empty());
assert!(accepted_adapters(Some(" ")).is_empty());
assert_eq!(
accepted_adapters(Some("pokered-unique8-v5, pokered-unique8-v4")),
vec!["pokered-unique8-v5".to_string(), "pokered-unique8-v4".to_string()]
);
}
#[test]
fn the_state_format_segment_is_appended_not_interleaved() {
let full = fixture().string();

View file

@ -1,4 +1,4 @@
//! The `pokered-unique8-v5` reward catalog.
//! The `pokered-unique8-v6` reward catalog.
//!
//! A direct port of the prototype's `src/reward/catalog.ts`, including the
//! declaration order, which is the order `counts` and `last` serialize in.
@ -20,8 +20,18 @@ pub mod kind {
pub const BATTLE: &str = "battle";
pub const BADGE: &str = "badge";
pub const BOUNDARY: &str = "boundary";
pub const CATCH: &str = "catch";
}
/// What a `catch` of a species this run has already caught pays.
///
/// Not a multiple of the rule's catalog value, because no binary float scales 0.30 into
/// exactly 0.10: `0.3 * (1.0 / 3.0)` is `0.09999999999999999`, and that number would reach
/// the ticker, the checkpoint and `docs/rewards-learning.md`'s table as itself. `boundary`'s
/// two payouts are 0.05 and 0.10, which a scale of two does express exactly, so that rule
/// still goes through the scaling path.
pub const CATCH_REPEAT_VALUE: f64 = 0.10;
#[derive(Debug, Clone, Copy)]
pub struct RewardRule {
pub kind: &'static str,
@ -34,7 +44,7 @@ pub struct RewardRule {
pub stimulation_ms: u32,
}
pub const REWARDS: [RewardRule; 8] = [
pub const REWARDS: [RewardRule; 9] = [
RewardRule {
kind: kind::MILESTONE,
label: "Story",
@ -99,6 +109,22 @@ pub const REWARDS: [RewardRule; 8] = [
value: 0.05,
stimulation_ms: 100,
},
// The operator's decision of 2026-09-22: the fly is paid for *keeping* a wild Pokémon, not
// only for knocking one out. Appended rather than slotted next to `species` for the same
// reason `boundary` was appended -- the declaration order is the key order `counts`
// serializes in, and every checkpoint already written carries the first eight in this order.
//
// One rule, two payouts, like `boundary`: this value is what a species this run has never
// caught pays, and [`CATCH_REPEAT_VALUE`] is what a repeat pays. The existing `species`
// rule is untouched and still pays 0.50 the first time a species is owned by any means, so
// a first catch of a new species pays 0.50 + 0.30 across two kinds.
RewardRule {
kind: kind::CATCH,
label: "Catch",
trigger: "Wild Pokémon caught; 0.10 for a species already caught; max 3 per species",
value: 0.30,
stimulation_ms: 150,
},
];
/// Position of `kind` in [`REWARDS`], or `None` for an unknown kind. This is
@ -199,10 +225,34 @@ mod tests {
// separate kinds.
assert_eq!(rule(kind::BOUNDARY).unwrap().value, 0.05);
assert_eq!(rule(kind::BOUNDARY).unwrap().stimulation_ms, 100);
// Nor the prototype's: the operator's catch rule, `pokered-unique8-v6`.
assert_eq!(rule(kind::CATCH).unwrap().value, 0.30);
assert_eq!(CATCH_REPEAT_VALUE, 0.10);
assert_eq!(rule(kind::CATCH).unwrap().stimulation_ms, 150);
assert!(rule("blackout").is_none(), "the catalog has no penalties");
assert!(REWARDS.iter().all(|rule| rule.value > 0.0));
}
#[test]
fn the_catch_rule_is_last_so_the_older_key_order_does_not_move() {
let order: Vec<&str> = REWARDS.iter().map(|rule| rule.kind).collect();
assert_eq!(
order,
vec![
kind::MILESTONE,
kind::EXPLORATION,
kind::MAP,
kind::SPECIES,
kind::TRAINER,
kind::BATTLE,
kind::BADGE,
kind::BOUNDARY,
kind::CATCH,
]
);
assert_eq!(index(kind::CATCH), Some(REWARDS.len() - 1));
}
#[test]
fn empty_counts_lists_every_kind_at_zero() {
let counts = Counts::default();

View file

@ -1,11 +1,12 @@
//! The Pokémon Red reward adapter, `pokered-unique8-v5`.
//! The Pokémon Red reward adapter, `pokered-unique8-v6`.
//!
//! A port of the prototype's `src/reward/pokemon-red.ts`. The gates and budgets
//! are unchanged; `docs/rewards-learning.md` holds the live rule table and the
//! source evidence behind each gate. v4 replaced the 16-rung boot-to-badges
//! ladder with the 38 rungs of `docs/design/ladder.md`; v5 adds one reward rule,
//! `boundary` (`docs/design/room-escape.md` section 2), which pays the first step
//! next to and the first step onto each of a map's exits.
//! next to and the first step onto each of a map's exits; v6 adds `catch`, the
//! operator's decision of 2026-09-22, which pays for keeping a wild Pokémon.
pub mod catalog;
#[cfg(test)]
@ -32,13 +33,33 @@ use symbols::ram;
/// Adapter version, pinned into the checkpoint compatibility string.
///
/// `v5` is the `boundary` rule. Bumping it is what rejects every checkpoint written
/// by `v4`: the string is compared whole before a restore is attempted, so a ledger
/// that has never recorded a single `boundary:` key can never be resumed as though
/// its exits were already collected. (`v4` was the 38-rung ladder, and rejected
/// `v3` for the same reason: a stored rank that meant "4 badges" on the old ladder
/// could not be read as a rung on the new one.) Pre-launch, so no run is lost.
pub const REWARD_ADAPTER: &str = "pokered-unique8-v5";
/// `v6` is the `catch` rule. Bumping it is what makes a `v5` checkpoint a decision
/// rather than an accident: the compatibility string is compared whole before a
/// restore is attempted, so a `v5` run is refused by default and resumed only when
/// the operator names it in `FLY_ACCEPT_ADAPTERS`
/// ([`crate::compatibility::RestoreDecision`], `docs/design/flysim.md`). That
/// migration is safe in one direction only, and only for this pair: `v5`'s ledger is
/// a `v6` ledger with the catch counter absent, and an absent counter reads as zero.
///
/// (`v5` was the `boundary` rule, and rejected `v4` because a ledger that had never
/// recorded a `boundary:` key could not be resumed as though its exits were already
/// collected. `v4` was the 38-rung ladder, and rejected `v3` because a stored rank
/// that meant "4 badges" on the old ladder is not a rung on the new one. Neither of
/// those is a migration: this one is, because nothing a `v5` ledger holds means
/// something different under `v6`.)
pub const REWARD_ADAPTER: &str = "pokered-unique8-v6";
/// Adapter ids whose checkpoints `v6` can read.
///
/// Exactly one, and it is one because the `catch` rule adds a counter and changes nothing else:
/// a `v5` ledger restores as a `v6` ledger with `catchCounts` empty, and every other byte of the
/// state means what it meant. `v4` is not here -- its `seen` ledger holds no `boundary:` keys, so
/// resuming it would pay a second time for every exit the run had already found -- and neither is
/// `v3`, whose stored rank is a rung on a different ladder.
///
/// Listing an id here is necessary but not sufficient: `FLY_ACCEPT_ADAPTERS` must name it too
/// (`crate::compatibility::decide`, `docs/design/flysim.md`).
pub const MIGRATES_FROM: &[&str] = &["pokered-unique8-v5"];
/// The only cartridge semantic rewards are enabled for. Even the canonical
/// pret build stays disabled until reviewed; see `docs/rewards-learning.md`.
@ -60,8 +81,22 @@ pub const SUPPORTED_ROM: &str =
/// [`REWARD_ADAPTER`] is the gate that refuses such a checkpoint anyway, and it is
/// the right gate, because the objection to loading one is about semantics rather
/// than shape.
///
/// *Not* bumped for the `catch` rule either, and this time the answer matters,
/// because `v5` checkpoints are meant to be restorable under `v6`. The rule adds one
/// counter, `catchCounts`, and nothing else: every other field keeps its name, its
/// shape and its meaning, and a state written without the counter restores with it
/// empty, which is the truth about a run that was never paid for a catch. That is the
/// whole of the documented `v5` -> `v6` migration; see
/// [`crate::compatibility::RestoreDecision`].
pub const STATE_VERSION: u64 = 4;
/// Catch payouts one species may earn in the lifetime of a run's ledger.
///
/// The same cap and the same reason as the wild-KO rule's three: a species the fly can
/// find over and over is a farm, and three is enough for the behaviour to be learned.
const MAX_CATCH_PAYOUTS: u64 = 3;
const BADGE_NAMES: [&str; 8] = [
"BOULDER", "CASCADE", "THUNDER", "RAINBOW", "SOUL", "MARSH", "VOLCANO", "EARTH",
];
@ -325,6 +360,26 @@ struct Battle {
wild: bool,
saw_living: bool,
ko: bool,
/// Lifetime `species` payouts when this battle started.
///
/// The "never owned this run" test for the catch rule, and an exact one: the only
/// thing that can set a `wPokedexOwned` bit during a wild battle is the catch
/// itself, so a `species` payout between the battle starting and the ball keeping
/// the Pokémon *is* that Pokémon being new. It is read this way rather than from
/// `wCapturedMonSpecies` directly because that byte is the cartridge's **internal**
/// species index and the owned bitset is by **Pokédex number**; the two numberings
/// differ and nothing in WRAM converts between them
/// (`docs/design/macros-wram.md` section 2, "species numbering").
///
/// `None` for a battle restored from a checkpoint written before this existed,
/// which reads as "cannot tell" and pays the repeat amount rather than guessing
/// generously.
species_at_start: Option<u64>,
/// The internal species index `wCapturedMonSpecies` named, once a ball has kept one.
captured: Option<u8>,
/// Whether that catch was a species this run had never owned, decided on the frame
/// the capture was observed.
captured_new: bool,
}
/// Immutable per-sample byte cache. Each address requested during one sample
@ -370,6 +425,10 @@ pub struct PokemonRedReward {
tiles: OrderedSet,
tile_counts: BTreeMap<u8, u64>,
wild_wins: BTreeMap<String, u64>,
/// Catch payouts per species, by the cartridge's internal species index as a decimal
/// string. The one field `v6` adds to the checkpoint; absent in a `v5` state, which
/// reads as every species at zero.
catch_counts: BTreeMap<String, u64>,
replay_blocked: OrderedSet,
counts: Counts,
total: f64,
@ -420,6 +479,7 @@ impl PokemonRedReward {
tiles: OrderedSet::new(),
tile_counts: BTreeMap::new(),
wild_wins: BTreeMap::new(),
catch_counts: BTreeMap::new(),
replay_blocked: OrderedSet::new(),
counts: Counts::default(),
total: 0.0,
@ -567,8 +627,9 @@ impl PokemonRedReward {
}
/// Forget observations a rollback invalidates. Lifetime novelty survives,
/// and every wild-KO key paid so far is blocked from paying again, because
/// after a rollback the same battle could otherwise be replayed for reward.
/// and every wild-KO key and every caught species paid so far is blocked from
/// paying again, because after a rollback the same battle -- or the same catch --
/// could otherwise be replayed for reward.
pub fn clear_transient(&mut self) {
self.location.clear();
self.stable = 0;
@ -578,6 +639,10 @@ impl PokemonRedReward {
for key in keys {
self.replay_blocked.insert(&key);
}
let caught: Vec<String> = self.catch_counts.keys().cloned().collect();
for species in caught {
self.replay_blocked.insert(&format!("catch:{species}"));
}
}
/// Sample WRAM after one completed frame and return this frame's payouts.
@ -690,14 +755,29 @@ impl PokemonRedReward {
if in_battle == 1 || in_battle == 2 || in_battle == 255 {
self.mode = "BATTLE".to_string();
self.stable = 0;
let species_paid = self.counts.get(kind::SPECIES);
if self.battle.is_none() && in_battle != 255 {
self.battle = Some(Battle {
key: battle_key(memory, map),
wild: in_battle == 1,
saw_living: false,
ko: false,
species_at_start: Some(species_paid),
captured: None,
captured_new: false,
});
}
// The cartridge's own answer to "was one caught": `ram/wram.asm`'s comment on
// this byte is "0 if no mon was captured". `ItemUseBall` zeroes it before every
// throw and writes `wEnemyMonSpecies` into it only on the branch that keeps the
// Pokémon, and `UseBagItem`'s `.returnAfterCapturingMon` zeroes it again on the
// way out of the battle -- so it is non-zero for the hundreds of frames the
// catch's own text and Pokédex screen take, and zero everywhere else.
//
// Read rather than derived from `wPartyCount`, because a catch with a full party
// raises `wBoxCount` instead, and because `wPartyCount` also rises for a gift, a
// trade and a revive out of the PC.
let captured = memory.read8(ram::wCapturedMonSpecies);
if let Some(battle) = &mut self.battle {
let hp = word(memory, ram::wEnemyMonHP);
let max = word(memory, ram::wEnemyMonMaxHP);
@ -710,6 +790,11 @@ impl PokemonRedReward {
if battle.saw_living && hp == 0 {
battle.ko = true;
}
if battle.wild && captured != 0 && battle.captured.is_none() {
battle.captured = Some(captured);
battle.captured_new =
battle.species_at_start.is_some_and(|before| species_paid > before);
}
}
} else if in_battle == 0 {
self.mode = "OVERWORLD".to_string();
@ -728,6 +813,41 @@ impl PokemonRedReward {
}
self.wild_wins.insert(battle.key.clone(), (count + 1).min(3));
}
// The catch rule (`docs/rewards-learning.md`, the operator 2026-09-22).
//
// Paid on the way out of the battle rather than on the capture frame, so that
// it lands in the same place the wild-KO payout does and cannot fire twice for
// one battle. `wBattleResult` is 2 on exactly two paths in the game:
// `UseBagItem`'s `.returnAfterCapturingMon`, which is this one, and a link
// battle whose opponent ran (`engine/battle/core.asm`), which this cartridge
// never has. Requiring it as well as the captured species means a byte read
// out of a half-initialised battle cannot pay.
if let Some(species) = battle.captured
&& battle.wild
&& result == 2
{
let key = species.to_string();
let paid = self.catch_counts.get(&key).copied().unwrap_or(0);
if paid < MAX_CATCH_PAYOUTS
&& !self.replay_blocked.contains(&format!("catch:{key}"))
{
let value = if battle.captured_new {
catalog::rule(kind::CATCH)
.expect("the catch rule is in the catalog")
.value
} else {
catalog::CATCH_REPEAT_VALUE
};
self.emit_amount(
&mut emitted,
kind::CATCH,
format!("CAUGHT #{species}"),
value,
brain_ms,
);
}
self.catch_counts.insert(key, (paid + 1).min(MAX_CATCH_PAYOUTS));
}
}
let location = format!("{map}:{x}:{y}");
self.stable = if self.location == location { self.stable + 1 } else { 1 };
@ -825,13 +945,33 @@ impl PokemonRedReward {
label: String,
scale: f64,
brain_ms: f64,
) {
let rule = catalog::rule(kind).expect("emit is only called with catalog kinds");
self.emit_amount(emitted, kind, label, rule.value * scale, brain_ms);
}
/// [`PokemonRedReward::emit`] with the payout stated outright instead of as a multiple
/// of the catalog value.
///
/// One rule needs it. `catch` pays 0.30 for a species this run has not caught and 0.10
/// for one it has, and no binary float scales the first into exactly the second:
/// `0.3 * (1.0 / 3.0)` is `0.09999999999999999`, and that is the number that would reach
/// the ticker and the checkpoint. `boundary`'s pair, 0.05 and 0.10, *is* an exact scale
/// of two, so that rule still goes through [`PokemonRedReward::emit`].
fn emit_amount(
&mut self,
emitted: &mut Vec<RewardEvent>,
kind: &'static str,
label: String,
value: f64,
brain_ms: f64,
) {
let rule = catalog::rule(kind).expect("emit is only called with catalog kinds");
let event = RewardEvent {
kind,
label,
brain_ms,
value: rule.value * scale,
value,
stimulation_ms: rule.stimulation_ms,
};
emitted.push(event.clone());
@ -991,6 +1131,10 @@ impl PokemonRedReward {
"tiles": self.tiles.as_slice(),
"tileCounts": self.tile_counts,
"wildWins": self.wild_wins,
// The one field v6 adds. A v5 state does not carry it and restores with it
// empty, which is the documented v5 -> v6 migration and the truth about a run
// that was never paid for a catch.
"catchCounts": self.catch_counts,
"replayBlocked": self.replay_blocked.as_slice(),
"counts": self.counts,
"total": self.total,
@ -1011,6 +1155,9 @@ impl PokemonRedReward {
"wild": battle.wild,
"sawLiving": battle.saw_living,
"ko": battle.ko,
"speciesAtStart": battle.species_at_start,
"captured": battle.captured,
"capturedNew": battle.captured_new,
})),
"mode": self.mode,
})
@ -1056,6 +1203,13 @@ impl PokemonRedReward {
let counts_raw = counted_record(input.get("counts")).ok_or(BAD_CHECKPOINT)?;
let tile_counts_raw = counted_record(input.get("tileCounts")).ok_or(BAD_CHECKPOINT)?;
let wild_wins_raw = counted_record(input.get("wildWins")).ok_or(BAD_CHECKPOINT)?;
// Absent in every v5 state, and that absence is the migration: no species has been
// paid for a catch, because the rule did not exist. Present but malformed is still
// an error, the same as every other counter here.
let catch_counts = match input.get("catchCounts") {
None | Some(Value::Null) => BTreeMap::new(),
Some(value) => counted_record(Some(value)).ok_or(BAD_CHECKPOINT)?,
};
let recent = recent_raw
.iter()
@ -1078,6 +1232,19 @@ impl PokemonRedReward {
wild: value.get("wild").and_then(Value::as_bool).ok_or(BAD_HISTORY)?,
saw_living: value.get("sawLiving").and_then(Value::as_bool).ok_or(BAD_HISTORY)?,
ko: value.get("ko").and_then(Value::as_bool).ok_or(BAD_HISTORY)?,
// All three are v6's, and all three are optional for the same reason
// `catchCounts` is. A v5 battle carries no `speciesAtStart`, which reads as
// "cannot tell whether the caught species was new" and pays the repeat
// amount: the conservative half of the rule, and at most 0.20 once.
species_at_start: value.get("speciesAtStart").and_then(Value::as_u64),
captured: value
.get("captured")
.and_then(Value::as_u64)
.and_then(|species| u8::try_from(species).ok()),
captured_new: value
.get("capturedNew")
.and_then(Value::as_bool)
.unwrap_or(false),
}),
};
let replay_blocked = match input.get("replayBlocked") {
@ -1103,6 +1270,7 @@ impl PokemonRedReward {
}
self.tile_counts = tile_counts;
self.wild_wins = wild_wins_raw;
self.catch_counts = catch_counts;
self.replay_blocked = replay_blocked.iter().map(String::as_str).collect();
self.counts = Counts::default();
for (key, count) in &counts_raw {
@ -1174,6 +1342,10 @@ impl GameAdapter for PokemonRedReward {
REWARD_ADAPTER
}
fn migrates_from(&self) -> &'static [&'static str] {
MIGRATES_FROM
}
fn rom_allowed(&self, sha256: &str) -> bool {
sha256 == SUPPORTED_ROM
}

View file

@ -47,6 +47,7 @@ pub mod ram {
pub const wBattleType: u16 = 0xd05a; // 53338
pub const wTrainerNo: u16 = 0xd05d; // 53341
pub const wPartyMenuTypeOrMessageID: u16 = 0xd07d; // 53373
pub const wCapturedMonSpecies: u16 = 0xd11c; // 53532
pub const wForcePlayerToChooseMon: u16 = 0xd11f; // 53535
pub const wTextBoxID: u16 = 0xd125; // 53541
pub const wPartyCount: u16 = 0xd163; // 53603

View file

@ -83,6 +83,34 @@ impl Fixture {
self.visit(x, 0)
}
/// One wild battle that ends in a ball keeping the Pokémon, byte for byte as the
/// cartridge writes it at the pinned commit.
///
/// `InitBattleVariables` clears `wBattleResult`; `ItemUseBall`'s capture branch sets the
/// Pokédex bit (for a species the player did not already own) and writes
/// `wEnemyMonSpecies` into `wCapturedMonSpecies`; `UseBagItem`'s
/// `.returnAfterCapturingMon` then zeroes that byte, sets `wBattleResult` to 2 and leaves
/// the battle. `dex` is the Pokédex *number* minus one, i.e. the bit index, and `None` is a
/// species this run already owns.
fn catch(&mut self, species: u8, dex: Option<u16>) -> Vec<RewardEvent> {
self.memory.set(ram::wBattleResult, 0);
self.memory.set(ram::wIsInBattle, 1);
self.memory.set(ram::wEnemyMonSpecies, species);
self.memory.set(ram::wEnemyMonHP + 1, 10);
self.memory.set(ram::wEnemyMonMaxHP + 1, 10);
let mut events = self.sample();
if let Some(index) = dex {
self.memory.or(ram::wPokedexOwned + (index >> 3), 1 << (index & 7));
}
self.memory.set(ram::wCapturedMonSpecies, species);
events.extend(self.sample());
self.memory.set(ram::wCapturedMonSpecies, 0);
self.memory.set(ram::wBattleResult, 2);
self.memory.set(ram::wIsInBattle, 0);
events.extend(self.sample());
events
}
/// Write a warp table: `wNumberOfWarps` plus one four-byte `Y, X, warp id, map id` entry per
/// `(x, y)`, the layout `ram/wram.asm` documents at the pinned commit.
fn warps(&mut self, warps: &[(u8, u8)]) {
@ -320,6 +348,136 @@ fn a_wild_run_capture_or_single_faint_never_pays_a_ko_while_a_verified_ko_does()
assert_eq!(f.reward.statistics().counts[kind::BATTLE], 1);
}
#[test]
fn a_catch_pays_the_new_species_amount_once_the_repeat_amount_after_and_stops_at_three() {
let mut f = Fixture::new();
f.sample();
// A species this run has never owned: the cartridge sets the Pokédex bit on the way
// through, so the existing `species` rule pays 0.50 and the new rule pays 0.30.
let first = f.catch(0xb0, Some(3));
assert_eq!(kinds(&first), ["species", "catch"]);
assert_eq!(labels(&first), ["OWNED #4", "CAUGHT #176"]);
assert!((first[0].value - 0.5).abs() < 1e-12, "the species rule is untouched");
assert!((first[1].value - 0.30).abs() < 1e-12);
// The same species again: a repeat, twice, and then the cap.
for _ in 0..2 {
let again = f.catch(0xb0, None);
assert_eq!(kinds(&again), ["catch"]);
assert_eq!(again[0].value, 0.10, "the repeat amount is exactly 0.10, not 0.3/3");
}
assert!(f.catch(0xb0, None).is_empty(), "three payouts per species is the cap");
assert_eq!(f.reward.statistics().counts[kind::CATCH], 3);
// Another species starts its own count, and its own 0.30.
let other = f.catch(0x99, Some(0));
assert_eq!(kinds(&other), ["species", "catch"]);
assert!((other[1].value - 0.30).abs() < 1e-12);
}
#[test]
fn a_catch_of_a_species_this_run_already_owns_pays_the_repeat_amount() {
let mut f = Fixture::new();
f.sample();
// Owned before the battle -- a gift, a trade, an evolution -- so no Pokédex bit is set
// during it and the catch is not a new species.
f.memory.or(ram::wPokedexOwned, 1);
assert_eq!(kinds(&f.sample()), ["species"]);
let events = f.catch(0x99, None);
assert_eq!(kinds(&events), ["catch"]);
assert_eq!(events[0].value, 0.10);
}
#[test]
fn nothing_but_a_wild_catch_pays_the_catch_rule() {
// A trainer battle: balls cannot be thrown, and `wIsInBattle` is 2.
let mut f = Fixture::new();
f.sample();
f.memory.set(ram::wIsInBattle, 2);
f.memory.set(ram::wEnemyMonHP + 1, 10);
f.memory.set(ram::wEnemyMonMaxHP + 1, 10);
f.sample();
f.memory.set(ram::wCapturedMonSpecies, 0xb0);
f.sample();
f.memory.set(ram::wCapturedMonSpecies, 0);
f.memory.set(ram::wBattleResult, 2);
f.memory.set(ram::wIsInBattle, 0);
assert!(f.sample().is_empty(), "a trainer battle never pays the catch rule");
// The Safari Zone and the old man's tutorial are excluded a step earlier: the whole
// sample is dropped with a visible mode, so no battle is ever opened.
for battle_type in [1u8, 2] {
let mut f = Fixture::new();
f.sample();
f.memory.set(ram::wBattleType, battle_type);
f.memory.set(ram::wIsInBattle, 1);
assert!(f.sample().is_empty());
f.memory.set(ram::wCapturedMonSpecies, 0xb0);
assert!(f.sample().is_empty());
f.memory.set(ram::wCapturedMonSpecies, 0);
f.memory.set(ram::wBattleResult, 2);
f.memory.set(ram::wIsInBattle, 0);
f.memory.set(ram::wBattleType, 0);
assert!(f.sample().is_empty());
assert_eq!(f.reward.statistics().counts[kind::CATCH], 0);
}
// A ball that missed: `wCapturedMonSpecies` never leaves zero and the battle ends as a
// run or a loss.
let mut f = Fixture::new();
f.sample();
f.memory.set(ram::wIsInBattle, 1);
f.memory.set(ram::wEnemyMonHP + 1, 10);
f.memory.set(ram::wEnemyMonMaxHP + 1, 10);
f.sample();
f.memory.set(ram::wIsInBattle, 0);
assert!(f.sample().is_empty());
}
#[test]
fn a_rollback_cannot_replay_a_catch() {
let mut f = Fixture::new();
f.sample();
assert_eq!(kinds(&f.catch(0xb0, Some(3))), ["species", "catch"]);
f.reward.clear_transient();
let state = f.reward.export_state();
f.reward.import_state(&state).unwrap();
assert!(f.catch(0xb0, None).is_empty(), "an already-paid species cannot pay after rollback");
assert_eq!(f.reward.statistics().counts[kind::CATCH], 1);
}
#[test]
fn a_v5_state_restores_under_v6_with_the_catch_counter_at_zero() {
let mut f = Fixture::new();
f.sample();
f.catch(0xb0, Some(3));
let v6 = f.reward.export_state();
assert_eq!(v6["catchCounts"], json!({ "176": 1 }));
// The v5 shape is this one without the counter the rule added: same `version`, same field
// names, same meanings. That is the whole of the documented migration.
let mut v5 = v6.clone();
v5.as_object_mut().unwrap().remove("catchCounts");
assert_eq!(v5["version"], json!(STATE_VERSION), "v5 and v6 states share a schema version");
let mut restored = PokemonRedReward::new();
restored.import_state(&v5).unwrap();
let mut expected = v6.clone();
expected["catchCounts"] = json!({});
assert_eq!(restored.export_state(), expected, "the counter starts at 0, nothing else moves");
// A genuine v5 `counts` object carries eight kinds and no `catch`, which reads as zero.
let mut older = v5.clone();
older["counts"].as_object_mut().unwrap().remove("catch");
let mut restored = PokemonRedReward::new();
restored.import_state(&older).unwrap();
assert_eq!(restored.statistics().counts[kind::CATCH], 0);
assert_eq!(restored.statistics().counts[kind::SPECIES], 1);
}
#[test]
fn a_repeated_wild_ko_decays_then_stops() {
let mut f = Fixture::new();
@ -378,6 +536,7 @@ fn malformed_checkpoint_fields_are_named_in_the_error() {
("counts", json!([])),
("tileCounts", json!("not a record")),
("wildWins", json!(3)),
("catchCounts", json!(3)),
] {
let mut broken = good.clone();
broken[field] = wrong;
@ -706,7 +865,8 @@ fn the_recent_ticker_keeps_the_newest_eight_events_newest_first() {
#[test]
fn the_adapter_reports_its_identity_and_pinned_rom() {
let reward = PokemonRedReward::new();
assert_eq!(reward.id(), "pokered-unique8-v5");
assert_eq!(reward.id(), "pokered-unique8-v6");
assert_eq!(reward.migrates_from(), ["pokered-unique8-v5"]);
assert!(reward.rom_allowed(SUPPORTED_ROM));
assert!(!reward.rom_allowed(
"5ca7ba01642a3b27b0cc0b5349b52792795b62d3ed977e98a09390659af96b7b"
@ -716,6 +876,9 @@ fn the_adapter_reports_its_identity_and_pinned_rom() {
assert_eq!(symbols::ram::wNumberOfWarps, 0xd3ae);
assert_eq!(symbols::ram::wWarpEntries, 0xd3af);
assert_eq!(symbols::ram::wCurMapConnections, 0xd370);
// Resolved from ram/wram.asm by services/flysim/tools/resolve_wram.py, bracketed by
// wFontLoaded and wForcePlayerToChooseMon; never written out by hand.
assert_eq!(symbols::ram::wCapturedMonSpecies, 0xd11c);
assert_eq!(symbols::MILESTONES.len(), 17);
}

View file

@ -30,6 +30,7 @@ pub mod metrics;
pub mod pacing;
pub mod profile;
pub mod ratelimit;
pub mod reset;
pub mod sdnotify;
pub mod simloop;
pub mod snapshot;

View file

@ -42,6 +42,15 @@ struct Args {
/// the "nothing to compare" case for a fresh container.
#[arg(long, value_name = "DIR")]
print_state_compatibility: Option<std::path::PathBuf>,
/// Restart the run from the milestone archive for this ladder rung, and exit.
///
/// Run with flysim stopped: it rewrites both checkpoint stores.
/// `infra/bin/fly-reset-to-milestone` is the operator-facing wrapper and the sequence
/// around it is in `infra/docs/runbook.md`. The current state is copied to a dated
/// directory first, so this is reversible by hand.
#[arg(long, value_name = "RANK")]
reset_to_milestone: Option<u32>,
}
fn main() -> Result<()> {
@ -66,6 +75,17 @@ fn main() -> Result<()> {
return Ok(());
}
if let Some(rank) = args.reset_to_milestone {
let stamp = flysim::reset::utc_stamp(flysim::eventlog::now_wall_ms());
let durable = config.paths.save_dir.clone();
let hot = config.paths.hot_dir.clone();
let archive = flysim::reset::default_archive_dir(&durable, &stamp);
for line in flysim::reset::reset_to_milestone(&durable, &hot, rank, &archive)? {
println!("{line}");
}
return Ok(());
}
flysim::run(config)
}

View file

@ -0,0 +1,450 @@
//! Restarting a run from an earlier rung, on disk, with flysim stopped.
//!
//! The operator's decision of 2026-09-22 was "restart the live run from an early checkpoint
//! instead of from scratch". `FLY_RESET_STATE=1` cannot do that: it archives everything and the
//! next start warms up a fresh fly. What this does instead is promote one milestone archive --
//! `milestone-<N>.checkpoint`, which `store::Store::commit` writes at the first commit at a new
//! best rank and which no rotation ever unlinks -- to being the only thing either store will
//! restore.
//!
//! `infra/bin/fly-reset-to-milestone` is the operator-facing wrapper; it refuses to run while
//! flysim is up, calls `flysim --reset-to-milestone N`, and fixes ownership afterwards. The work
//! is here rather than in that script because two of the steps are inside the `FLYSIM01`
//! envelope: the ratchet's attempts and recoveries counters live in the checkpoint's manifest,
//! and a shell script has no business rewriting one.
//!
//! What it does, in order, and nothing else:
//!
//! 1. **archives** every file in the durable and hot stores into a dated directory, by copying,
//! so a step that fails later has destroyed nothing;
//! 2. **rewrites** the rung's archive with the ratchet's `attempts` and `recoveries` at zero, so
//! the recovery budget is not already spent when the restarted run begins. `best` is left
//! alone: the archive's own `best` is the rung it was taken at, which is exactly what the
//! restarted run is at, and the rank the stream shows is recomputed by the adapter from the
//! restored game state anyway;
//! 3. **installs** it as the newest generation in both stores, so the restore order
//! (`store::restore_order`: hot latest, hot previous, durable latest, ...) reaches it first;
//! 4. **clears** the milestone archives above N -- rungs the run had reached and is now below --
//! and the generation files of the run being abandoned;
//! 5. **clears the session ledgers**: the event log `events.jsonl` and its rotations. The
//! checkpoint carries `lastEventId`, so restoring an old checkpoint over a newer log would
//! re-issue ids the log already holds. The macro layer's own session ledgers (blocked,
//! talked, reached, pushed-back) are memory-only by contract
//! (`docs/design/macros.md` section 12.1: "a restored run offers every target once more"),
//! so stopping flysim is what resets those and this has nothing to do.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use anyhow::{Context, Result, bail};
use crate::store::{self, Store, StoreManifest};
/// Generations kept by the stores this tool writes. Only used for the rotation bound, which
/// this tool does not trigger; the durable store's own value.
const KEEP_GENERATIONS: usize = 8;
/// A generation number no commit ever allocates, so `Store::candidates` drops the
/// generation-file half of an archive entry and offers only `milestone-<rank>.checkpoint`.
///
/// `Sim::boot` allocates `highest_generation() + 1`, which is 1 or more, so 0 names no file.
/// The milestone archives below the rung being restored are kept exactly this way: still on
/// disk, still restorable as a deeper fallback, and with no generation file pretending to be
/// their contents.
const NO_GENERATION: u64 = 0;
/// What the reset did, one line per step, for the operator's terminal and the run record.
pub type Report = Vec<String>;
/// Promote `rank`'s milestone archive to be the restore source of both stores.
///
/// `archive` is the dated directory the current state is copied into; it must not exist.
/// Refuses if the milestone archive is missing, which is the "that rung was never reached"
/// case and the one mistake worth refusing rather than guessing at.
pub fn reset_to_milestone(
durable_dir: &Path,
hot_dir: &Path,
rank: u32,
archive: &Path,
) -> Result<Report> {
let durable = Store::new(durable_dir, KEEP_GENERATIONS);
let hot = Store::new(hot_dir, KEEP_GENERATIONS);
let source = durable.archive_path(rank);
if !source.is_file() {
bail!(
"no milestone archive for rung {rank}: {} does not exist. `ls {}` shows the rungs \
this run actually reached.",
source.display(),
durable_dir.display()
);
}
if archive.exists() {
bail!("the archive directory {} already exists", archive.display());
}
let mut report: Report = Vec::new();
// 1. Copy everything aside first.
let copied_durable = copy_tree(durable_dir, &archive.join("durable"))?;
let copied_hot = copy_tree(hot_dir, &archive.join("hot"))?;
report.push(format!(
"archived {copied_durable} durable and {copied_hot} hot files to {}",
archive.display()
));
// 2. Zero the two recovery counters inside the envelope.
let mut checkpoint = store::load(&source)
.with_context(|| format!("decoding {}", source.display()))?;
let spent = (checkpoint.runtime.ratchet.attempts, checkpoint.runtime.ratchet.recoveries);
checkpoint.runtime.ratchet.attempts = 0;
checkpoint.runtime.ratchet.recoveries = 0;
let generation = durable.highest_generation().max(hot.highest_generation()) + 1;
checkpoint.runtime.generation = generation;
let bytes = store::encode(&checkpoint.agent, &checkpoint.runtime)?;
report.push(format!(
"rung {rank} (best {}, ladder rank recomputed from the game state): ratchet attempts \
{} -> 0, recoveries {} -> 0",
checkpoint.runtime.ratchet.best, spent.0, spent.1
));
// 3/4. Clear both stores, keeping the milestone archives at or below this rung, and write
// the promoted state as the newest generation of each.
let kept = clear_store(durable_dir, Some(rank))?;
clear_store(hot_dir, None)?;
report.push(format!(
"cleared the hot store and every milestone archive above rung {rank}; kept {} at or \
below it: {kept:?}",
kept.len()
));
// The hot store lives on a tmpfs that a stopped container may not have mounted yet, so it
// is created rather than assumed; the durable one already exists or the milestone archive
// above could not have been read.
durable.create()?;
hot.create()?;
store::write_atomic(&durable.generation_path(generation), &bytes)?;
store::write_atomic(&durable.archive_path(rank), &bytes)?;
store::write_atomic(&hot.generation_path(generation), &bytes)?;
let mut archives: BTreeMap<u32, u64> =
kept.iter().map(|rung| (*rung, NO_GENERATION)).collect();
archives.insert(rank, generation);
let durable_manifest = StoreManifest {
generation,
latest: Some(generation),
previous: None,
archives,
};
write_manifest(&durable, &durable_manifest)?;
write_manifest(
&hot,
&StoreManifest {
generation,
latest: Some(generation),
previous: None,
archives: BTreeMap::new(),
},
)?;
report.push(format!(
"generation {generation} is now hot latest and durable latest in {} and {}",
hot_dir.display(),
durable_dir.display()
));
// 5. The session ledgers.
let logs = remove_matching(durable_dir, |name| {
name == "events.jsonl" || (name.starts_with("events-") && name.ends_with(".jsonl"))
})?;
report.push(format!(
"reset the session ledgers: {logs} event-log files removed (the macro layer's are \
memory-only and are reset by stopping flysim)"
));
Ok(report)
}
fn write_manifest(store: &Store, manifest: &StoreManifest) -> Result<()> {
store.create()?;
store::write_atomic(&store.manifest_path(), &serde_json::to_vec_pretty(manifest)?)
}
/// Copy every regular file of `from` into `to`, creating `to`. A missing source is zero files,
/// not an error: the hot store lives on a tmpfs that a stopped container does not have.
fn copy_tree(from: &Path, to: &Path) -> Result<usize> {
if !from.is_dir() {
return Ok(0);
}
std::fs::create_dir_all(to)
.with_context(|| format!("creating {}", to.display()))?;
let mut copied = 0;
for entry in std::fs::read_dir(from)?.flatten() {
if !entry.file_type().is_ok_and(|kind| kind.is_file()) {
continue;
}
std::fs::copy(entry.path(), to.join(entry.file_name()))
.with_context(|| format!("copying {}", entry.path().display()))?;
copied += 1;
}
Ok(copied)
}
/// Remove every checkpoint, tmp file and manifest from `dir`, keeping `milestone-<r>.checkpoint`
/// for `r <= keep_up_to`. Returns the rungs kept, ascending.
fn clear_store(dir: &Path, keep_up_to: Option<u32>) -> Result<Vec<u32>> {
let mut kept = Vec::new();
if !dir.is_dir() {
return Ok(kept);
}
for entry in std::fs::read_dir(dir)?.flatten() {
let name = entry.file_name().to_string_lossy().to_string();
let milestone = name
.strip_prefix("milestone-")
.and_then(|rest| rest.strip_suffix(".checkpoint"))
.and_then(|rung| rung.parse::<u32>().ok());
let remove = match milestone {
// The promoted rung's own archive is rewritten straight after this, so it is
// removed here like the rest and reinstated with the counters cleared.
Some(rung) => match keep_up_to {
Some(limit) if rung < limit => {
kept.push(rung);
false
}
_ => true,
},
None => {
name == "manifest.json"
|| name.ends_with(".checkpoint")
|| name.ends_with(".checkpoint.tmp")
}
};
if remove {
std::fs::remove_file(entry.path())
.with_context(|| format!("removing {}", entry.path().display()))?;
}
}
kept.sort_unstable();
Ok(kept)
}
fn remove_matching(dir: &Path, wanted: impl Fn(&str) -> bool) -> Result<usize> {
if !dir.is_dir() {
return Ok(0);
}
let mut removed = 0;
for entry in std::fs::read_dir(dir)?.flatten() {
if wanted(&entry.file_name().to_string_lossy()) {
std::fs::remove_file(entry.path())?;
removed += 1;
}
}
Ok(removed)
}
/// `YYYYMMDDTHHMMSSZ` in UTC, for the dated archive directory's name.
///
/// Built on the event log's own calendar conversion, which is the service's only one; the time of
/// day is arithmetic on the same millisecond count.
pub fn utc_stamp(wall_ms: u64) -> String {
let second_of_day = (wall_ms % 86_400_000) / 1_000;
format!(
"{}T{:02}{:02}{:02}Z",
crate::eventlog::utc_day(wall_ms),
second_of_day / 3_600,
(second_of_day / 60) % 60,
second_of_day % 60,
)
}
/// The default dated archive directory: a sibling of the durable store, which is its own
/// mountpoint and so cannot be renamed -- the same shape `infra/05-deploy.sh` uses for
/// `FLY_RESET_STATE=1`.
pub fn default_archive_dir(durable_dir: &Path, stamp: &str) -> PathBuf {
let mut name = durable_dir.as_os_str().to_os_string();
name.push(format!(".reset-{stamp}"));
PathBuf::from(name)
}
#[cfg(test)]
mod tests {
use super::*;
/// A real `FLYSIM01` envelope, small but structurally complete, so these tests decode and
/// re-encode what the running service writes rather than a stand-in.
fn envelope(generation: u64, ratchet: flybrain_gb::RatchetState) -> Vec<u8> {
use flybrain_core::decoder::DecoderState;
use flybrain_core::lif::LifState;
use flybrain_core::ordered::NumberMap;
use flybrain_core::plasticity::PlasticityState;
let agent = flybrain_core::agent::AgentState {
version: 1,
remainder: 0.75,
warmed_up: true,
network: LifState {
membrane: vec![0.5, -0.25],
refractory: vec![0, 3],
last_spike_ms: vec![-1_000_000.0, 12.0],
visual_drive: vec![0.1],
rng: -12_345,
reward_remaining: 40.0,
ms: 9_000.0,
population_rate: 1.5,
rates: NumberMap::from_pairs([("forward", 2.0)]),
plasticity: PlasticityState {
version: "fly-kc-mbon-rstdp-v2".to_string(),
topology: 42,
enabled: true,
updates: 3.0,
signal: 0.25,
gains: vec![1.0, 0.9],
traces: vec![0.0, 0.1],
touched: vec![0.0, 8_000.0],
},
},
decoder: DecoderState {
version: 4,
calibrated: true,
baseline: NumberMap::from_pairs([("forward", 1.0)]),
held_until: NumberMap::new(),
next_allowed: NumberMap::new(),
next_decision: 100.0,
current: None,
fatigue: NumberMap::new(),
macro_next_decision: 0.0,
macro_current: None,
macro_fatigue: NumberMap::new(),
},
};
let runtime = store::RuntimeState {
generation,
wall_ms: 1_700_000_000_000,
rom_sha256: "ab".repeat(32),
emulator_frame: 12_345,
compatibility: "kernel/pokered-unique8-v6/fingerprint".to_string(),
speed: 1.0,
buttons: 0,
rank_since_ms: 4_242.0,
last_event_id: 77,
reward: serde_json::json!({ "version": 4, "total": 1.25 }),
ratchet,
emulator: vec![7; 64],
framebuffer: vec![9; 32],
ratchet_game: vec![1, 2, 3],
ratchet_frame: vec![4, 5, 6],
};
store::encode(&agent, &runtime).unwrap()
}
/// A store dir holding a milestone archive for each rung in `rungs`, their generations, a
/// manifest and an event log, all written through the store's own commit path.
fn state_dir(root: &Path, rungs: &[u32], ratchet: flybrain_gb::RatchetState) -> Store {
let store = Store::new(root, KEEP_GENERATIONS);
store.create().unwrap();
for (index, rung) in rungs.iter().enumerate() {
let generation = index as u64 + 1;
store.commit(generation, &envelope(generation, ratchet), Some(*rung)).unwrap();
}
std::fs::write(root.join("events.jsonl"), b"{}\n").unwrap();
std::fs::write(root.join("events-20260921.jsonl"), b"{}\n").unwrap();
store
}
#[test]
fn a_missing_rung_is_refused_and_nothing_is_touched() {
let tmp = tempfile::tempdir().unwrap();
let durable = tmp.path().join("state");
let hot = tmp.path().join("hot");
state_dir(&durable, &[3, 5], flybrain_gb::RatchetState::default());
let before = std::fs::read_dir(&durable).unwrap().flatten().count();
let error = reset_to_milestone(&durable, &hot, 9, &tmp.path().join("archive"))
.unwrap_err()
.to_string();
assert!(error.contains("no milestone archive for rung 9"), "{error}");
assert_eq!(std::fs::read_dir(&durable).unwrap().flatten().count(), before);
assert!(!tmp.path().join("archive").exists(), "nothing was archived");
}
#[test]
fn the_rung_becomes_both_stores_latest_with_the_recovery_budget_back() {
let tmp = tempfile::tempdir().unwrap();
let durable = tmp.path().join("state");
let hot = tmp.path().join("hot");
let spent = flybrain_gb::RatchetState {
best: 5,
attempts: 3,
recoveries: 11,
..flybrain_gb::RatchetState::default()
};
state_dir(&durable, &[3, 5, 9, 11], spent);
state_dir(&hot, &[11], spent);
let archive = tmp.path().join("archive-20260922");
let report = reset_to_milestone(&durable, &hot, 5, &archive).unwrap();
assert!(report.iter().any(|line| line.contains("attempts 3 -> 0")), "{report:?}");
// Everything that was there is in the archive.
assert!(archive.join("durable/milestone-11.checkpoint").is_file());
assert!(archive.join("durable/events.jsonl").is_file());
assert!(archive.join("hot/manifest.json").is_file());
// The rungs above 5 are gone; the ones below it stay as deeper fallbacks.
assert!(!durable.join("milestone-9.checkpoint").exists());
assert!(!durable.join("milestone-11.checkpoint").exists());
assert!(durable.join("milestone-3.checkpoint").is_file());
assert!(durable.join("milestone-5.checkpoint").is_file());
assert!(!durable.join("events.jsonl").exists());
assert!(!durable.join("events-20260921.jsonl").exists());
assert!(!hot.join("milestone-11.checkpoint").exists());
// Both stores restore the rung, and the counters are back.
for store in [Store::new(&hot, KEEP_GENERATIONS), Store::new(&durable, KEEP_GENERATIONS)] {
let candidates = store.candidates("x");
let first = store::load(&candidates[0].path).unwrap();
assert_eq!(first.runtime.ratchet.best, 5);
assert_eq!((first.runtime.ratchet.attempts, first.runtime.ratchet.recoveries), (0, 0));
}
// ... including through the promoted milestone archive itself.
let archived = store::load(&durable.join("milestone-5.checkpoint")).unwrap();
assert_eq!((archived.runtime.ratchet.attempts, archived.runtime.ratchet.recoveries), (0, 0));
// The rungs below it are offered, and only as their own archive files.
let manifest = Store::new(&durable, KEEP_GENERATIONS).manifest().unwrap();
assert_eq!(manifest.archives.get(&3), Some(&NO_GENERATION));
assert_eq!(manifest.latest, manifest.archives.get(&5).copied());
assert!(
Store::new(&durable, KEEP_GENERATIONS)
.candidates("durable")
.iter()
.any(|candidate| candidate.path.ends_with("milestone-3.checkpoint"))
);
}
#[test]
fn a_second_reset_refuses_to_write_over_an_existing_archive() {
let tmp = tempfile::tempdir().unwrap();
let durable = tmp.path().join("state");
let hot = tmp.path().join("hot");
state_dir(&durable, &[4], flybrain_gb::RatchetState::default());
let archive = tmp.path().join("archive");
reset_to_milestone(&durable, &hot, 4, &archive).unwrap();
let error = reset_to_milestone(&durable, &hot, 4, &archive).unwrap_err().to_string();
assert!(error.contains("already exists"), "{error}");
}
#[test]
fn the_stamp_is_the_event_logs_calendar_plus_a_time_of_day() {
assert_eq!(utc_stamp(0), "19700101T000000Z");
// 2026-09-22T16:15:00Z
assert_eq!(utc_stamp(1_790_093_700_000), "20260922T161500Z");
}
#[test]
fn the_default_archive_is_a_dated_sibling_of_the_store() {
assert_eq!(
default_archive_dir(Path::new("/srv/fly/state"), "20260922T161500Z"),
PathBuf::from("/srv/fly/state.reset-20260922T161500Z")
);
}
}

View file

@ -718,12 +718,36 @@ impl Sim {
if runtime.rom_sha256 != self.rom_sha256 {
bail!("checkpoint is for another cartridge ({})", runtime.rom_sha256);
}
if runtime.compatibility != self.compatibility {
bail!(
"compatibility mismatch\n checkpoint: {}\n this build: {}",
// Byte-identical, or the one documented migration the operator asked for.
//
// `FLY_ACCEPT_ADAPTERS` is read here rather than carried in `Config` because it is a
// property of a *deploy*, not of a run: `infra/05-deploy.sh` writes it into
// `/etc/fly/fly.env` only for the deploy that needs it, and an operator who wants the
// migration off again deletes one line. An empty or unset variable is no migration at
// all, which is what every deploy before this one did.
let accepted = flybrain_gb::compatibility::accepted_adapters(
std::env::var(flybrain_gb::compatibility::ACCEPT_ADAPTERS_ENV).ok().as_deref(),
);
match flybrain_gb::compatibility::decide(
&runtime.compatibility,
&self.compatibility,
self.adapter.migrates_from(),
&accepted,
) {
flybrain_gb::compatibility::RestoreDecision::Exact => {}
flybrain_gb::compatibility::RestoreDecision::MigrateAdapter { from } => {
tracing::warn!(
from = %from,
to = %self.adapter.id(),
"restoring a checkpoint from an earlier adapter, by the migration \
FLY_ACCEPT_ADAPTERS opted this deploy into"
);
}
flybrain_gb::compatibility::RestoreDecision::Refuse(reason) => bail!(
"compatibility mismatch: {reason}\n checkpoint: {}\n this build: {}",
runtime.compatibility,
self.compatibility
);
),
}
if runtime.framebuffer.len() != FRAMEBUFFER_LEN {
bail!("checkpoint framebuffer is {} bytes", runtime.framebuffer.len());

View file

@ -241,8 +241,8 @@ pub struct FeedMacroOutcome {
}
/// Reward categories the feed reports counts for. The adapter's own interned kinds
/// (`milestone`, `exploration`, `map`, `species`, `trainer`, `battle`, `badge`, `boundary`) map
/// onto these.
/// (`milestone`, `exploration`, `map`, `species`, `trainer`, `battle`, `badge`, `boundary`,
/// `catch`) map onto these.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum RewardKind {
@ -282,6 +282,15 @@ impl RewardKind {
// protocol is concerned: finding a door is finding somewhere new, and the design asks
// for no new feed kind.
"boundary" => Self::Explore,
// `catch` is a wild battle the fly won by keeping the Pokémon, so it publishes on
// the same counter a wild KO does. The feed's kinds are a closed set
// (`docs/feed-protocol.md`) and this rule asked for no new one.
//
// Deliberately *not* `pokedex`: on a catch of a species this run has never owned,
// the cartridge sets the Pokédex bit and the adapter's existing `species` rule pays
// for it on the same frame, so the `pokedex` counter already moves. Mapping `catch`
// there as well would count one event twice.
"catch" => Self::Wildwin,
// The platformer.
"band" => Self::Explore,
"coin" => Self::Wildwin,
@ -739,6 +748,7 @@ mod tests {
}
assert_eq!(RewardKind::from_adapter("nonsense"), None);
assert_eq!(RewardKind::from_adapter("boundary"), Some(RewardKind::Explore));
assert_eq!(RewardKind::from_adapter("catch"), Some(RewardKind::Wildwin));
}
#[test]

View file

@ -0,0 +1,224 @@
//! A `v5` checkpoint restored under `v6`: accepted with the opt-in, refused without it.
//!
//! The unit tests in `flybrain-gb` cover the decision function and the adapter's own state
//! migration separately. This is the two of them against one artefact: a real `FLYSIM01`
//! envelope carrying a `pokered-unique8-v5` compatibility string and a `v5` reward ledger —
//! written, encoded, decoded, and then put through exactly what `Sim::try_restore` puts a
//! candidate through.
//!
//! No ROM and no dataset, deliberately. Building a `Sim` would need both, and neither is part of
//! the question: what decides a restore is the compatibility string and `import_state`.
use flybrain_gb::GameAdapter;
use flybrain_gb::compatibility::{RestoreDecision, accepted_adapters, decide};
use flybrain_gb::pokemon_red::PokemonRedReward;
use flysim::store::{self, RuntimeState};
/// The live string's shape, with the adapter left open. The dataset fingerprint is shortened —
/// nothing here parses it, and a seven-digest one would be 455 characters of noise.
fn compatibility(adapter: &str) -> String {
format!(
"lif-1ms-f64-v2/{adapter}/aabbccddeeff00/fly-kc-mbon-rstdp-v2/\
binjgb:c60e138da5a795ebb55e56b11b7e90024e41112c/\
pokered:0cd19d3b877b7dc66d12c7050bed9a7f38154d4b/statefmt:199616-x86_64-unknown-linux-gnu"
)
}
/// A `v5` reward ledger: `STATE_VERSION` 4, every field `v5` wrote, and **no** `catchCounts`.
///
/// Written out by hand rather than exported from an adapter, because an exported one would be a
/// `v6` state with the counter deleted — this is the shape the release box's checkpoints really
/// carry, field for field.
fn v5_reward() -> serde_json::Value {
serde_json::json!({
"version": 4,
"seen": ["adventure", "map:0", "early:outside", "dex:3", "boundary:0:edge:n:near"],
"tiles": ["0:5:6", "0:5:7"],
"tileCounts": { "0": 2 },
"wildWins": { "0:112:4": 2 },
"replayBlocked": [],
"counts": {
"milestone": 2, "exploration": 0, "map": 1, "species": 1,
"trainer": 0, "battle": 2, "badge": 0, "boundary": 1
},
"total": 2.05,
"recent": [{ "kind": "species", "label": "OWNED #4", "brainMs": 1234.5, "value": 0.5 }],
"last": { "species": { "kind": "species", "label": "OWNED #4", "brainMs": 1234.5, "value": 0.5 } },
"initialized": true,
"sawBoot": true,
"location": "0:5:7",
"stable": 9,
"progress": 3,
"badges": 0,
"battle": null,
"mode": "OVERWORLD"
})
}
fn v5_checkpoint() -> Vec<u8> {
use flybrain_core::decoder::DecoderState;
use flybrain_core::lif::LifState;
use flybrain_core::ordered::NumberMap;
use flybrain_core::plasticity::PlasticityState;
let agent = flybrain_core::agent::AgentState {
version: 1,
remainder: 0.25,
warmed_up: true,
network: LifState {
membrane: vec![0.1, -0.2],
refractory: vec![0, 1],
last_spike_ms: vec![-1_000_000.0, 5.0],
visual_drive: vec![0.3],
rng: 42,
reward_remaining: 0.0,
ms: 1_234.5,
population_rate: 1.0,
rates: NumberMap::from_pairs([("forward", 1.0)]),
plasticity: PlasticityState {
version: "fly-kc-mbon-rstdp-v2".to_string(),
topology: 7,
enabled: true,
updates: 1.0,
signal: 0.0,
gains: vec![1.0],
traces: vec![0.0],
touched: vec![0.0],
},
},
decoder: DecoderState {
version: 4,
calibrated: true,
baseline: NumberMap::from_pairs([("forward", 1.0)]),
held_until: NumberMap::new(),
next_allowed: NumberMap::new(),
next_decision: 0.0,
current: None,
fatigue: NumberMap::new(),
macro_next_decision: 0.0,
macro_current: None,
macro_fatigue: NumberMap::new(),
},
};
let runtime = RuntimeState {
generation: 41,
wall_ms: 1_790_000_000_000,
rom_sha256: flybrain_gb::pokemon_red::SUPPORTED_ROM.to_string(),
emulator_frame: 1_000_000,
compatibility: compatibility("pokered-unique8-v5"),
speed: 1.0,
buttons: 0,
rank_since_ms: 1_000.0,
last_event_id: 4_242,
reward: v5_reward(),
ratchet: flybrain_gb::RatchetState { best: 3, attempts: 1, recoveries: 4, ..Default::default() },
emulator: vec![3; 64],
framebuffer: vec![0; 32],
ratchet_game: vec![1],
ratchet_frame: vec![2],
};
store::encode(&agent, &runtime).expect("the fixture encodes")
}
#[test]
fn a_v5_checkpoint_is_refused_under_v6_without_the_opt_in() {
let checkpoint = store::decode(&v5_checkpoint()).expect("the fixture decodes");
let adapter = PokemonRedReward::new();
let current = compatibility(adapter.id());
assert_ne!(checkpoint.runtime.compatibility, current, "v6 is not v5");
for opt_in in [None, Some(""), Some("pokered-unique8-v4"), Some("some-other-adapter")] {
assert!(
matches!(
decide(
&checkpoint.runtime.compatibility,
&current,
adapter.migrates_from(),
&accepted_adapters(opt_in),
),
RestoreDecision::Refuse(_)
),
"FLY_ACCEPT_ADAPTERS={opt_in:?} must not migrate anything"
);
}
}
#[test]
fn a_v5_checkpoint_restores_under_v6_with_the_opt_in_and_the_counter_starts_at_zero() {
let checkpoint = store::decode(&v5_checkpoint()).expect("the fixture decodes");
let mut adapter = PokemonRedReward::new();
let current = compatibility(adapter.id());
assert_eq!(
decide(
&checkpoint.runtime.compatibility,
&current,
adapter.migrates_from(),
&accepted_adapters(Some("pokered-unique8-v5")),
),
RestoreDecision::MigrateAdapter { from: "pokered-unique8-v5".to_string() }
);
// The migration itself: `import_state`, exactly as `Sim::try_restore` calls it.
adapter.import_state(&checkpoint.runtime.reward).expect("a v5 ledger is a valid v6 ledger");
let after = adapter.export_state();
assert_eq!(after["catchCounts"], serde_json::json!({}), "the new counter starts at 0");
assert_eq!(after["counts"]["catch"], serde_json::json!(0));
// And nothing else moved: every field the v5 state carried round-trips to the same value,
// and the only key v6 adds is the counter.
//
// `counts` is the one field that is not byte-identical, and it is not a change of meaning:
// it serializes every kind in the catalog, so a v6 state lists `catch` where a v5 state had
// nothing to list. Every kind the v5 state did carry keeps its number.
let before = v5_reward();
for (key, value) in before.as_object().unwrap() {
if key == "counts" {
for (kind, count) in value.as_object().unwrap() {
assert_eq!(&after["counts"][kind], count, "counts.{kind}");
}
let added: Vec<&String> = after["counts"]
.as_object()
.unwrap()
.keys()
.filter(|kind| !value.as_object().unwrap().contains_key(*kind))
.collect();
assert_eq!(added, vec!["catch"], "v6 counts one more kind and no others");
continue;
}
assert_eq!(&after[key], value, "{key} must survive the migration byte for byte");
}
let added: Vec<&String> = after
.as_object()
.unwrap()
.keys()
.filter(|key| !before.as_object().unwrap().contains_key(*key))
.collect();
assert_eq!(added, vec!["catchCounts"], "v6 adds one field and no others");
// The rest of what a restore reads is untouched by the migration.
assert_eq!(adapter.progress().rank, 3);
assert_eq!(checkpoint.runtime.last_event_id, 4_242);
assert_eq!(checkpoint.runtime.ratchet.best, 3);
}
#[test]
fn nothing_but_the_adapter_segment_may_differ_for_the_migration_to_apply() {
let adapter = PokemonRedReward::new();
let accepted = accepted_adapters(Some("pokered-unique8-v5"));
let current = compatibility(adapter.id());
// A v5 string whose state format also moved: a different build, not a rule change.
let other_abi = compatibility("pokered-unique8-v5").replace("199616", "199617");
assert!(matches!(
decide(&other_abi, &current, adapter.migrates_from(), &accepted),
RestoreDecision::Refuse(_)
));
// And an identical string needs no opt-in at all.
assert_eq!(
decide(&current, &current, adapter.migrates_from(), &[]),
RestoreDecision::Exact
);
}

View file

@ -0,0 +1,257 @@
//! The catch reward against the real cartridge.
//!
//! Gated on `FLY_ROM` *and* on a checkpoint, the way every ROM test in this workspace is, and
//! skips cleanly without either — the cartridge never enters this repository and a checkpoint is
//! not a fixture, it is the state the release box was really in:
//!
//! ```sh
//! FLY_ROM="$HOME/roms/pokemon-red.gb" \
//! FLY_CATCH_CHECKPOINT=.local/checkpoints/<a rung-9 forest checkpoint> \
//! cargo test --release -p flysim --test rom_catch -- --nocapture
//! ```
//!
//! ## What only the cartridge can answer
//!
//! The synthetic trace in `pokemon_red/tests.rs` writes `wCapturedMonSpecies`, `wBattleResult`
//! and the Pokédex bit itself, from the disassembly. It cannot say that those are the bytes
//! *this* cartridge writes when a ball keeps a Pokémon, in that order, on frames an adapter
//! sampling once a frame actually sees. That is this test, and it is the "survey" half of
//! `docs/design/macros-wram.md`'s evidence for the row: a real battle, real button presses, and
//! the byte read out of the running game rather than written into a fake one.
//!
//! ## How the catch is produced
//!
//! No steering and no scripted button sequence: the shipping macro palette, the shipping macro
//! layer and the shipping decoder, with a stub readout that leans on one macro population at a
//! time — the same driver `tests/rom_macros_mode.rs` uses and for the same reason. The one thing
//! this harness does that the rotation does not is lean on `THROW BALL`'s channel while a wild
//! battle is up, because the question here is what the adapter reads from a catch, not whether a
//! game-blind readout finds its way to one.
//!
//! The checkpoint must hold at least one ball in the bag. The macro palette can buy one
//! (`BUY BALL`, `MB·PBALL`, inside a mart), but that is a walk across a city and back and it is a
//! different test's question; this one says out loud that it skipped.
use flybrain_core::decoder::PopulationDecoder;
use flybrain_core::decoder::gameboy::gameboy_decoder_config_with_macros;
use flybrain_core::ordered::NumberMap;
use flybrain_gb::adapter::RewardEvent;
use flybrain_gb::pokemon_red::state;
use flybrain_gb::pokemon_red::symbols::ram;
use flybrain_gb::pokemon_red::{PokemonRedReward, catalog};
use flybrain_gb::{
AdapterLedger, DEFAULT_AUDIO_FRAMES, DEFAULT_AUDIO_FREQUENCY, Emulator, GameAdapter,
};
use flysim::config::Config;
use flysim::macros::{MacroLayer, macro_layer};
use flysim::snapshot::MacroMode;
const MS_PER_FRAME: f64 = 1000.0 / 59.7275;
const SEED: u32 = 20_260_922;
/// The hot population's rate against every other one's, which is also the stub's calibration
/// rate — so a channel that is not the hot one scores exactly 1.0.
const HOT: f64 = 16.0;
const REST: f64 = 10.0;
/// `THROW BALL`'s channel (`pokemon_red::macros::palette`).
const BALL: &str = "MB·BALL";
/// Frames the stub leans on one channel before the rotation moves on, the shape of the real
/// group's hysteresis-then-fatigue rotation.
const BURST_FRAMES: u32 = 24;
fn rates(hot: Option<&str>) -> NumberMap {
let mut rates = NumberMap::new();
for channel in flybrain_gb::macro_channels("pokemon-red") {
rates.set(channel, REST);
}
for bucket in 0..8 {
rates.set(&format!("command_{bucket}"), REST);
}
if let Some(channel) = hot {
rates.set(channel, HOT);
}
rates
}
fn rom() -> Option<Vec<u8>> {
let path = std::env::var_os("FLY_ROM")?;
match std::fs::read(&path) {
Ok(bytes) => Some(bytes),
Err(error) => panic!("FLY_ROM is set to {path:?} but could not be read: {error}"),
}
}
fn checkpoint() -> Option<flysim::store::Checkpoint> {
let path = std::env::var_os("FLY_CATCH_CHECKPOINT")?;
Some(
flysim::store::load(std::path::Path::new(&path))
.expect("the checkpoint should be a FLYSIM01 envelope"),
)
}
struct Run {
gb: Emulator,
adapter: PokemonRedReward,
layer: MacroLayer,
decoder: PopulationDecoder,
channels: Vec<&'static str>,
ms: f64,
frame: u32,
/// Every payout the adapter has made since the run started.
payouts: Vec<RewardEvent>,
}
impl Run {
fn resume(rom: &[u8], checkpoint: &flysim::store::Checkpoint) -> Self {
let mut gb = Emulator::new(rom, DEFAULT_AUDIO_FREQUENCY, DEFAULT_AUDIO_FRAMES)
.expect("binjgb should accept the cartridge");
let mut adapter = PokemonRedReward::new();
gb.import_state(&checkpoint.runtime.emulator).expect("the checkpoint's emulator state");
// A checkpoint written by an earlier adapter rebaselines rather than failing, which is
// exactly the `v5` -> `v6` case this rule ships with.
adapter.import_state(&checkpoint.runtime.reward).expect("the checkpoint's reward ledger");
let channels = flybrain_gb::macro_channels("pokemon-red");
let preset = gameboy_decoder_config_with_macros(&channels);
let hold_ms = preset.macros.as_ref().expect("the preset has a macro group").hold_ms;
let mut decoder = PopulationDecoder::new(preset).expect("the preset is well formed");
decoder.calibrate(&rates(None));
let mut config = Config::default();
config.loop_.game = "pokemon-red".to_string();
config.macros.mode = MacroMode::Macros;
config.validate().expect("pokemon-red has a palette in macros mode");
let mut layer = macro_layer(&config, hold_ms, SEED).expect("a layer in macros mode");
let _ = layer.observe(&mut gb, &AdapterLedger(&adapter), 0.0);
Self {
gb,
adapter,
layer,
decoder,
channels,
ms: 0.0,
frame: 0,
payouts: Vec::new(),
}
}
fn byte(&mut self, address: u16) -> u8 {
self.gb.read_wram(address)
}
fn in_wild_battle(&mut self) -> bool {
self.byte(ram::wIsInBattle) == 1
}
/// Balls in the bag, of any kind (`constants/item_constants.asm`: MASTER_BALL 1,
/// ULTRA_BALL 2, GREAT_BALL 3, POKE_BALL 4 — the same four `THROW BALL` looks for).
fn balls(&mut self) -> usize {
state::bag(&mut self.gb)
.iter()
.filter(|item| (0x01..=0x04).contains(&item.id) && item.count > 0)
.map(|item| usize::from(item.count))
.sum()
}
fn step(&mut self) {
// Lean on `THROW BALL` while a wild battle is up; otherwise rotate, which is what gets
// the fly into the grass in the first place.
let hot = if self.in_wild_battle() {
Some(BALL)
} else {
let slot = (self.frame / BURST_FRAMES) as usize % self.channels.len();
Some(self.channels[slot])
};
let bound = self.layer.bound_channels();
let active = self.decoder.decode_bound(&rates(hot), self.ms, false, None, Some(&bound));
let mask = {
let ledger = AdapterLedger(&self.adapter);
self.layer.decide(&active, 0, self.ms, &mut self.gb, &ledger).mask
};
self.gb.set_buttons(mask as u8);
self.gb.run_frame().expect("a frame should complete");
self.ms += MS_PER_FRAME;
self.frame += 1;
let ms = self.ms;
self.payouts.extend(self.adapter.sample(&mut self.gb, ms));
let ledger = AdapterLedger(&self.adapter);
let _ = self.layer.observe(&mut self.gb, &ledger, ms);
}
fn catches(&self) -> Vec<&RewardEvent> {
self.payouts.iter().filter(|event| event.kind == catalog::kind::CATCH).collect()
}
}
#[test]
fn a_catch_on_the_cartridge_pays_the_catch_rule_once_with_the_species_in_its_label() {
let Some(rom) = rom() else {
eprintln!("skipped: FLY_ROM is not set");
return;
};
let Some(checkpoint) = checkpoint() else {
eprintln!("skipped: no FLY_CATCH_CHECKPOINT");
return;
};
let mut run = Run::resume(&rom, &checkpoint);
let balls = run.balls();
if balls == 0 {
eprintln!(
"skipped: the checkpoint's bag holds no ball (map {:#04x}). `BUY BALL` can buy one \
inside a mart; point FLY_CATCH_CHECKPOINT at a state that already has one.",
run.adapter.map_id().unwrap_or(u32::MAX)
);
return;
}
eprintln!("bag holds {balls} balls; map {:#04x}", run.adapter.map_id().unwrap_or(u32::MAX));
// Twenty brain minutes is generous for a forest checkpoint: the live run threw 28 balls in
// its first Viridian Forest session (`pokemon_red::macros::palette`).
let budget = 20 * 60 * 60;
let mut battles = 0u32;
let mut was_in_battle = false;
for _ in 0..budget {
run.step();
let now = run.in_wild_battle();
if now && !was_in_battle {
battles += 1;
}
was_in_battle = now;
if !run.catches().is_empty() {
break;
}
}
let catches = run.catches();
assert!(
!catches.is_empty(),
"no catch in {:.1} brain minutes: {battles} wild battles, {} balls left, map {:#04x}, \
macros {:?}",
run.ms / 60_000.0,
run.balls(),
run.adapter.map_id().unwrap_or(u32::MAX),
run.layer.counts()
);
let caught = catches[0];
eprintln!(
"caught after {:.1} brain minutes and {battles} wild battles: {} for {}",
run.ms / 60_000.0,
caught.label,
caught.value
);
assert!(caught.label.starts_with("CAUGHT #"), "{}", caught.label);
assert!(
(caught.value - 0.30).abs() < 1e-12 || caught.value == catalog::CATCH_REPEAT_VALUE,
"a catch pays one of the rule's two amounts, not {}",
caught.value
);
// The cartridge's own flag is clear again by the time the payout lands, which is what makes
// the payout a battle-exit event rather than a per-frame one.
assert_eq!(run.byte(ram::wCapturedMonSpecies), 0);
assert_eq!(
run.adapter.progress().counts[catalog::kind::CATCH],
1,
"one battle, one payout"
);
// And a species the run is paid for catching is a species the Pokédex knows: the same event
// sets the bit the `species` rule reads, whether or not it was new to this run.
assert!(run.balls() < balls, "a ball was spent");
}

View file

@ -200,6 +200,15 @@ EXTRA_RAM = (
'wCurMapTileset',
'wTilesetBank',
'wTilesetBlocksPtr',
# The catch reward (`docs/rewards-learning.md`, `docs/design/macros-wram.md` section 2).
# ram/wram.asm's own comment is "0 if no mon was captured": ItemUseBall zeroes it before
# every throw and writes wEnemyMonSpecies into it only on the branch that keeps the
# Pokemon, and UseBagItem zeroes it again on the way out of the battle. It is the
# cartridge's own answer to "was this one caught", and the only signal that needs no
# second rule to tell a catch apart from a gift, a trade or an evolution.
# services/flysim/tools/resolve_wram.py is the second reading of it, from ram/wram.asm at
# this commit, bracketed by wFontLoaded and wForcePlayerToChooseMon.
'wCapturedMonSpecies',
)

View file

@ -49,6 +49,33 @@ WANTED = {
# not bank 0, so this is the read the memory seam grew a bank for.
'wTilesetBank': 'the ROM bank the blockset lives in',
'wTilesetBlocksPtr': 'blocks to tiles, 16 bytes per block',
# The catch reward (`docs/rewards-learning.md`, `docs/design/macros-wram.md`
# section 10). ram/wram.asm's own comment is "0 if no mon was captured":
# ItemUseBall zeroes it before every throw and writes wEnemyMonSpecies into it
# only on the branch that keeps the caught Pokemon, and UseBagItem zeroes it
# again on the way out of the battle. It is the cartridge's own answer to "was
# this one caught", and the only signal that needs no second rule to tell a
# catch apart from a gift, a trade or an evolution.
'wCapturedMonSpecies': 'the species a ball just caught, 0 for none',
}
#: Constants the decomp defines through its `const` enumeration rather than with a
#: plain `EQU`, so `constants()` cannot evaluate their expressions. They matter here
#: because `NUM_TMS + NUM_HMS` is the size of `wMonHLearnset`, and that one
#: declaration is what kills the cursor on its way through the battle engine's
#: scratch bytes -- the region `wCapturedMonSpecies` lives in.
#:
#: Each is *counted* from the decomp rather than written out by hand, which is the
#: same rule the rest of this tool follows. `DEF NUM_HMS EQU const_value - HM01` is
#: by construction the number of `add_hm` definitions after `HM01`, and
#: `item_constants.asm`'s own `ASSERT NUM_TMS == const_value - TM01` ties `NUM_TMS`
#: to the number of `add_tm` definitions -- so `NUM_TMS` is counted *and* compared
#: against the literal the same file declares, and a decomp that moved one without
#: the other stops the run instead of producing an address.
COUNTED = {
'NUM_HMS': ('constants/item_constants.asm', r'^\s*add_hm\s+\w+'),
'NUM_TMS': ('constants/item_constants.asm', r'^\s*add_tm\s+\w+'),
}
@ -68,6 +95,10 @@ def constants(root: Path) -> dict[str, int]:
BLOCK_WIDTH`). A name whose expression never becomes evaluable is simply left
out, which kills the cursor at any declaration that uses it.
"""
counted = {
name: len(re.findall(pattern, (root / path).read_text(), re.M))
for name, (path, pattern) in COUNTED.items()
}
pending: dict[str, str] = {}
sources = sorted((root / 'constants').glob('*.asm')) + sorted(
(root / 'constants').glob('*.inc')
@ -77,7 +108,7 @@ def constants(root: Path) -> dict[str, int]:
r'^\s*(?:DEF|def)\s+(\w+)\s+(?:EQU|equ)\s+([^;\n]+)', path.read_text(), re.M
):
pending.setdefault(name, value.strip())
out: dict[str, int] = {}
out: dict[str, int] = dict(counted)
while pending:
progressed = False
for name in list(pending):
@ -89,6 +120,12 @@ def constants(root: Path) -> dict[str, int]:
progressed = True
if not progressed:
break
for name, value in counted.items():
if out.get(name, value) != value:
raise SystemExit(
f'{name}: the decomp declares {out[name]} and defines {value} of them'
)
out[name] = value
return out
@ -367,11 +404,17 @@ def main() -> None:
raise SystemExit('the walk disagrees with symbols.rs; nothing emitted')
print(f'{checked} of {len(table)} pinned addresses re-derived from wram.asm, no disagreement')
missing = [name for name in WANTED if name not in resolved]
# A name this tool has already emitted is pinned, so the walk meets it as an
# anchor rather than resolving it: it was re-derived all the same, and the
# comparison above is what says so.
missing = [name for name in WANTED if name not in resolved and name not in table]
if missing:
raise SystemExit(f'unanchored, so not resolved: {", ".join(missing)}')
for name in WANTED:
print(f'{name} = ${resolved[name]:04x} ({WANTED[name]})')
if name in resolved:
print(f'{name} = ${resolved[name]:04x} ({WANTED[name]})')
else:
print(f'{name} = ${table[name]:04x} (already pinned; {WANTED[name]})')
if not args.emit:
return