diff --git a/findings/subsystems/relabel-new-ids.md b/findings/subsystems/relabel-new-ids.md new file mode 100644 index 0000000..f6fbee9 --- /dev/null +++ b/findings/subsystems/relabel-new-ids.md @@ -0,0 +1,374 @@ +# `state_checksum --relabel-new-ids`: comparing two post-turn saves modulo this turn's new id labelling + +Lane BT, 2026-09-09. **Host only — no VM was touched.** VM140 (reference guest) untouched; VM141, +VM144 and VM146 were held by lanes AZ, BS and BR throughout and were not approached. + +Implements `findings/resolutions/2026-09-09-fleet-id-order-residue.md` §3 item 1, verbatim as the +specification and with the two engine-side items (§3.2 canonical visit order, §3.3 the +`--ai-visit-order` override) deliberately **not** started — they belong in `sots-engine` and the +resolver ranked them after this one. + +Consumes: `findings/control-flow/pinned-seed-rich-turn.md` (lane BP — the acceptance data), +`findings/subsystems/id-allocation.md` (the id scheme), `findings/control-flow/raid-gate-multiplicity.md` +(lane AD — the negative acceptance data). + +Touches only `verify/state-checksum/*` and this file. + +--- + +## 0. Result + +Both halves of the acceptance pass, and the second half passes **because a guard fires**, not +because the matcher happened to find nothing. + +| half | pair | verdict | +|---|---|---| +| **positive** | `bp-pinA-turn28` vs `bp-pinB-turn28`, pre `bp-turn28-pre` | `IDENTICAL modulo pi = {1970<->1986}` — 35 leaves → **0**, and π is exactly the one transposition | +| **negative** | `ad-oracle-A-post` vs `ad-oracle-B-post`, pre `ad-turn27-two-raiders` | `REFUSED` (the two runs minted different id **sets**), then `DIVERGED: 94 leaf difference(s)` — byte-for-byte the same 94 the tool reports without the flag | + +Existing behaviour is unchanged when the flag is absent: over every save in +`verify/results/saves/` under both `--floats bits --mask none` and `--floats canonical --mask +resave`, the pre-change and post-change modules produce identical root digests **and identical +per-node digests at every node of the tree** (§5.1). + +--- + +## 1. What was built + +`verify/state-checksum/state_checksum.py` gains one flag: + +```sh +state_checksum.py POST-A.sav POST-B.sav --relabel-new-ids PRE-TURN.sav +``` + +The operation, in the order the code does it: + +1. Parse all three saves. **Prove coverage on the unmodified parse** (`audit_coverage` runs before + any rewrite), so `coverage: PROVED` still means what it has always meant: every byte of the + file is a function of the digest's inputs. +2. `new(X) = { fleet ids in X } \ { fleet ids in PRE }`, restricted to **node nibble ≠ 0** — the + client-minted space (`id-allocation.md` §1: `id = (counter << 4) | node`, node 0 is the sim + that owns the board, node `PlyrIdx + 1` is a player's own client). +3. Key each new fleet by a content key that does not mention its id: + **`(LocID if non-zero else FPlan/pnd destination, sorted ship-id set)`**, with the two + location kinds tagged apart so a fleet *standing at* system 384 never keys the same as one + *bound for* 384. +4. Build **π** from the key correspondence, one node at a time. +5. **Exchange the matched fleets' bodies between their slots** (see §4.2 — this is not what the + spec said, and the spec was wrong), keeping each slot's id and its id-attached label `FtName`. +6. Rewrite every *reference* to a permuted id under π: a ship's `FltID` back-reference and a + system's `Flt` entry. +7. Compare `ShipIDs[]` / `FleetIDs[]` / `DesignIDs[]` as **sets**. +8. Mask `/Summary/Checksum`, **with the reason printed on its own line**. +9. Print π and the verdict. + +New public surface: `RelabelRefused`, `Relabelling`, `build_relabelling`, `apply_relabelling`, +`format_permutation`, `id_node`, and `checksum_result` (the parse/digest split that lets the mode +parse once, audit, then rewrite). + +The relabelled root is **domain-separated** — `relabel-new-ids` is folded into the root preimage — +so a "modulo π" root can never be mistaken for a strict one. The extra preimage word is appended +*only* in this mode, so every root recorded before the flag existed is bit-for-bit unchanged. + +--- + +## 2. Acceptance, positive half — verbatim + +``` +$ uv run python3 verify/state-checksum/state_checksum.py \ + verify/results/saves/bp-pinA-turn28.sav verify/results/saves/bp-pinB-turn28.sav \ + --floats bits --mask none \ + --relabel-new-ids verify/results/saves/bp-turn28-pre.sav + +A a83c425215b6d7e88bc998f4948081e7 verify/results/saves/bp-pinA-turn28.sav +B a83c425215b6d7e88bc998f4948081e7 verify/results/saves/bp-pinB-turn28.sav +policy: floats=bits mask=none reader=74fd562c75e2679e +relabel: pre-turn save verify/results/saves/bp-turn28-pre.sav +relabel: pi = {1970<->1986} +relabel: 1970 (counter 123, node 2) -> 1986 at 384, ships [6976] +relabel: 1986 (counter 124, node 2) -> 1970 bound-for 80, ships [5264] +relabel: 2002 (counter 125, node 2) -> 2002 bound-for 384, ships [6992] +relabel: 3 new client-minted fleet id(s) on node(s) [2]; 2 relabelled, 2 fleet body/bodies exchanged, Fltx1, FltIDx2 rewritten +relabel: /Summary/Checksum is MASKED: it is derived from the whole state and its inputs are unmodelled (rule 18's open item), so it cannot be recomputed under a relabelling and its move is not independent evidence. Unmask by dropping --relabel-new-ids. +relabel: Master id lists (ShipIDs/FleetIDs/DesignIDs) are compared as SETS: their element order is the order the turn's commands were applied in, which is the same visit-order residue. Membership is still exact -- an added or removed id still reports. +IDENTICAL modulo pi = {1970<->1986} +$ echo $? +0 +``` + +Read the per-id lines: the counters printed are the decode the resolver did by hand +(`123/124/125`, node 2 = Player[32]'s client), and the content keys are what carries the match — +the group **at system 384 with ship 6976** is fleet 1970 in A and fleet 1986 in B; the group +**bound for system 80 with ship 5264** is 1986 in A and 1970 in B; the group bound for 384 is 2002 +in both. Exactly the resolver's §2.1 reading, recovered from the saves alone. + +Without the flag the same pair is `DIVERGED: 35 leaf difference(s)` (regression-locked by +`test_POSITIVE_the_same_pair_still_diverges_without_the_flag`, which asserts the count is 35). So +the mode removes **35 leaves and no others**: `IDENTICAL modulo one transposition and nothing else`. + +--- + +## 3. Acceptance, negative half — verbatim, and why it is the right kind of failure + +``` +$ uv run python3 verify/state-checksum/state_checksum.py \ + verify/results/saves/ad-oracle-A-post.sav verify/results/saves/ad-oracle-B-post.sav \ + --floats bits --mask none \ + --relabel-new-ids verify/results/saves/ad-turn27-two-raiders.sav + +A 1faf04cad2bbf2d7e5ecf560b8807504 verify/results/saves/ad-oracle-A-post.sav +B 53043168ad03cae13355db9f5f381b50 verify/results/saves/ad-oracle-B-post.sav +policy: floats=bits mask=none reader=74fd562c75e2679e +relabel: pre-turn save verify/results/saves/ad-turn27-two-raiders.sav +relabel: REFUSED -- the new client-minted fleet ids differ between the two saves -- A minted [1970, 1986], B minted [1970, 1986, 2002]; only in B [2002]. That is a difference in what was allocated, not in how it was labelled, so no relabelling is defensible +relabel: NO relabelling was applied; the comparison below is the ordinary one, and nothing has been absorbed. +DIVERGED: 94 leaf difference(s) + /Summary/Checksum: -1043321805 -> 1995746704 + /Sim/DesignIDs[]: removed [2002], added [2018] (85 -> 85 entries) + /Sim/FleetIDs[]: removed [], added [2002] (58 -> 59 entries) + /Sim/ShipIDs[]: list-reordered + /Sim/ModCount: 1496 -> 1500 + ... 89 more ... +$ echo $? +1 +``` + +94 is the number the tool reports for that pair **without** the flag, unchanged. On a refusal +nothing is rewritten, nothing is masked and no list is set-compared: the refusal path *is* the +ordinary comparison. That is deliberate — a mask applied on a run whose relabelling was refused +would be a mask nobody asked for. + +**The refusal is by cardinality, not by content.** AD's run A minted **two** client fleets and run +B minted **three**. The guard that fires is G3 (π must permute one set), and it fires before the +content matcher is ever consulted. That matters, because it means AD's pair does **not** exercise +the content key negatively. I exercised that separately, two ways (§4.3, §5.3): a synthetic pair +whose new ids agree and whose contents differ, and — the belt-and-braces version — **forcing the +transposition onto AD's pair with the guard bypassed**, which still leaves **82 differing +leaves**, including `DesignIDs[]`, `ModCount`, four `turnstats` rows, both AI build queues and +`Player[32]`'s money. The guard is not the only thing standing between AD's pair and a green +verdict; it is the first of several. + +--- + +## 4. Where the specification did not survive contact + +Four places. The first two changed the implementation; the third and fourth are scope notes the +next lane needs. + +(Lane BU landed mid-lane, confirming H1 — the visit order is an address-keyed container whose key +is the fleet's `Location*`. That settles the *cause* of the residue and changes nothing here: the +comparison side quotients the residue out regardless of what produces it, and it must go on +refusing when the difference is not a relabelling. The forward notes in §4.1 and §4.2 are +addressed to the engine-side canonical order and stand as written.) + +### 4.1 `FtName` is an **id-attached label**, and "every id-valued leaf" does not reach it + +In *both* of BP's saves, fleet **1970 is "Sigma Fleet VI"** and **1986 is "Tau Fleet VI"**. The +name follows the **id**, not the ship group. It is a Greek-letter ordinal minted from a per-player +counter that advances with the same pass that mints the id, so it is exactly as much a function of +the visit order as the id is — and exactly as little a decision. + +Consequence: applying π only to id-valued leaves would move the group at system 384 from slot 1970 +to slot 1986 **carrying the name "Sigma Fleet VI"**, and compare it against B's 1986 "Tau Fleet +VI". Two spurious leaf differences, and the acceptance "and nothing else" would have failed. + +Handled by keeping the name with the slot rather than with the body, and **gating it**: the mode +refuses unless `FtName` is the same at every matched id in both saves. If the name does not track +the id, something real differs and the tool must not paper over it +(`test_a_name_that_does_not_track_the_id_is_refused`). + +Cross-check that this is a naming *counter* and not a coincidence, from AD's pair: A minted two +client fleets, so A's node-0 fleet **7120 is "Upsilon Fleet VI"**; B minted three, so B's **7120 is +"Phi Fleet VI"**. The Greek sequence is **per player and shared between the client's and the +server's fleets** — one more thing an engine-side canonical visit order (§3.2 of the resolution) +has to reproduce, and one the resolver's item 2 does not currently mention. + +### 4.2 Applying π to the `Flt[]` **keys** introduces a spurious `reordered` + +The master fleet table's element order, and the `FleetIDs[]` list, are **identical in A and B** +(measured: both are `[…, 6960, 1970, 1986, 2002, 7120, …]`) and are a function of the id, not of +the visit order. Renumber A's slots under π and A's table reads `…1986, 1970, 2002…` against B's +`…1970, 1986, 2002…`, which the differ correctly reports as a reorder — an artefact of the tool, +not of the saves. + +The equivalent operation with no artefact is to **exchange the fleet bodies between the slots**, +leaving slot ids, `FleetIDs[]`, and every object label exactly as the file wrote them. That is what +the implementation does. It is the same quotient, taken from the other side. + +Worth carrying into the engine-side item: **the fleet table is id-ordered, not visit-ordered**, so +canonicalising the visit order in `game/ai` will reorder lists 8/10 and the apply order, and will +*not* reorder the save's fleet table. + +### 4.3 "new ids of the same node" is not a fleet-only id space + +`ad-oracle-A-post` carries **2002 as a `DesignID`**; `ad-oracle-B-post` carries **2002 as a +`FltID`**. The per-node counter is shared across object kinds, so a node's new-id set spans fleets, +designs and (in principle) ships. This mode matches **fleets only**, as the spec says. What keeps +that safe is guard G5 (§5.2): any leaf anywhere holding a permuted id at a site the tool does not +model as a fleet reference is a **refusal**. Anyone extending the mode to designs must read this +paragraph first — the two spaces interleave. + +### 4.4 "list-8/10/14 words when comparing command blocks" is **not** implemented, and does not belong here + +Lists 8/10/14 live in the `aiorders` **shim dump**, not in a `.sav`. `state_checksum.py` parses +saves; it has no reader for the block. Wiring π into a block comparison belongs wherever those +dumps are differed (lane BP did it by hand under its proven noise mask, `pinned-seed-rich-turn.md` +§2). Recorded as not-done with the reason rather than silently dropped. + +### 4.5 Two smaller notes + +- **Master id lists as sets is unconditional in this mode**, including when π is the identity. + `ShipIDs[]`'s element order is the apply order, which is the same residue; membership stays + exact, so an added or removed id still reports. It is nonetheless a real weakening and it is + printed on every run. A lane that wants that order back should reach for a separate flag, not + for this one. +- **The `--mask resave` composition the resolver asked for works**: `--mask resave + --relabel-new-ids …` on BP's pair reports `[masked: Checksumx1, Statusx8]` and + `IDENTICAL modulo pi = {1970<->1986}`. The relabel mask is added to whichever preset is in + effect and de-duplicated against it. + +--- + +## 5. The guards, and how each was made to fire + +Five guards. **Every one refuses; none degrades to a partial relabelling.** A refusal prints its +reason, applies nothing, and runs the ordinary comparison — the conservative direction, because a +real divergence then still reports. + +| | guard | how it was made to fire | +|---|---|---| +| **G1** | only ids **absent from the pre-turn save** may be relabelled | `test_ids_present_in_the_pre_turn_save_are_never_in_pi`; and the strong form, `test_two_pre_turn_fleets_that_swapped_contents_are_NOT_absorbed` — two *pre-existing* fleets exchanging contents is the very shape this mode absorbs, and it must still report | +| **G2** | only ids with a **non-zero node nibble** | `test_server_minted_ids_are_never_relabelled` — two node-0 (server) fleets exchanging contents still diverges | +| **G3** | π must be a **permutation of one set** — so the two saves must have minted the same ids | `test_different_new_id_sets_are_refused` (synthetic: one run mints a third fleet) and **lane AD's real pair** (§3) | +| **G4** | the content keys must **correspond one-to-one**, and be **unique on each side** | `test_new_fleets_whose_contents_differ_are_refused`; `test_an_ambiguous_content_key_is_refused` (two new fleets with the same location and ship set) | +| **G5** | **no leaf anywhere** in either save may hold a permuted id at a site the tool does not model | `test_an_unmodelled_reference_site_is_refused` (a `SomeOtherRef` leaf holding 1970); `test_the_completeness_scan_matches_raw_bytes_not_the_typed_value` | +| — | `FtName` must track the id in both saves | `test_a_name_that_does_not_track_the_id_is_refused` (§4.1) | +| — | a refusal leaves both parse trees **untouched** | `test_a_refusal_leaves_both_trees_untouched` — the digest of A before and after a refused `build_relabelling` is identical | + +### 5.1 The default path is unchanged + +Checked two ways, because "the tool is used by several lanes and by the determinism oracle": + +- **Digest equivalence, whole corpus.** The pre-change module (`git show HEAD:…`) and the + post-change module were run over all **43** `.sav` files in `verify/results/saves/` under + `--floats bits --mask none` and under `--floats canonical --mask resave`, comparing the root + digest, the coverage record, the mask hits, **and the `(path, digest)` of every node in the + tree**. `saves checked: 43 mismatches: 0`. +- **CLI stdout equivalence, every mode.** Single-save; `--tree --depth 3`; `--json`; the BP pair; + the AD pair at `--limit 500`; `turn2/turn3 --mask resave --ulps 2`; `--record-chain` and + `--chain` over turns 1/2/3. **5,602 lines, byte-identical** between the two modules (only the + chain file's own path normalised). + +The one change that *could* have moved a recorded root — folding a policy word into the root +preimage — is appended **only** when the new mode is active. An intermediate version of this work +appended an empty word unconditionally, which moved the root; that was caught by the equivalence +check above, which is the reason it exists. + +### 5.2 G5 matches raw bytes, not the reader's typed value — on purpose + +The digest is a function of *(save bytes, reader schema)*, never of the bytes alone (STATE_CHECKSUM.md +§2.4). A leaf the schema happens to type as a float or a raw blob still carries the id, and a +value-based completeness scan would not see it — a guard a typing slip can silently empty is the +"green verdict over an empty region set" of earned rule 1. This is not hypothetical: while +building the synthetic fixture, the reader typed a `Flt` reference as a **float** (`2.76e-42`, the +denormal whose bit pattern is 1970) and a value-based scan missed it. Matching on the raw four +bytes catches it. The cost is that a genuine 4-byte float in the denormal range around a permuted +id would refuse — a refusal is the safe direction, and no such value exists in the corpus. + +### 5.3 The strongest negative: bypass the guard and it still diverges + +`test_NEGATIVE_even_forcing_a_partial_relabelling_cannot_hide_it` hands `apply_relabelling` the +transposition `{1970↔1986}` it *would* have built if G3 did not exist, applies it to AD's A save, +and compares under the full "modulo π" policy (checksum masked, master lists as sets). **82 leaves +still differ.** The guards are the tool's honesty, not its only defence. + +### 5.4 Tests + +24 new tests (20 synthetic in `RelabelTest`, 4 real-save in `RealSaveTest`), **7 of which assert a +refusal**. The whole suite is 38 → **62 tests, all passing**: + +``` +$ uv run python3 -m unittest test_state_checksum.RelabelTest # 20 tests, OK, 0.07 s +$ uv run python3 -m unittest test_state_checksum.RealSaveTest.test_POSITIVE_… \ + …test_NEGATIVE_… # 4 tests, OK, 88.7 s +$ uv run python3 -m unittest discover -s . -t . # 62 tests, OK +``` + +### 5.5 One pre-existing test failure, fixed, that is not about this change + +The first full-suite run failed one test — +`test_known_resave_delta_localises_to_five_named_leaves` — and **the pre-change module fails it +identically**, so it is not a regression. The test enumerates candidate pairs over *sorted +filenames* and asserted the re-save direction as `4 -> 0`; `cb-turn2to3-endturn.sav`, added to the +corpus by an earlier lane, sorts before `turn2-state.sav`, so that pair reads `0 -> 4`. The claim +is symmetric and the assertion now is too (all four `Status` leaves must still move the *same* +way, with `Checksum` following by ∓16). Recorded rather than quietly patched, per earned rule 11. + +--- + +## 6. What this lane did NOT do + +1. **No VM, no game, no build.** Host only. The three held guests were not approached. +2. **Neither engine-side deliverable** (`sots-engine` `game/ai` canonical visit order; the + `--ai-visit-order` override). Out of scope by the brief, and ranked after this one by the + resolver. +3. **No command-block comparison** (§4.4). +4. **Designs and ships are not relabelled**, only fleets — per the spec. §4.3 says what has to be + read before that changes. +5. **`Summary/Checksum` is still unmodelled.** It is masked with its reason, not solved. + STATE_CHECKSUM.md §4.6 records the two derivations already ruled out; nobody should re-run them. +6. **π's *existence* is not evidence that the two runs agree on the turn.** It is evidence that + they agree *up to the labelling of the ids created on it*. The claim the mode licenses is + exactly the resolver's C-set shape and no more. + +--- + +## 7. Reproducing every number here + +```sh +cd ~/sots-re +S=verify/results/saves +# positive +uv run python3 verify/state-checksum/state_checksum.py \ + $S/bp-pinA-turn28.sav $S/bp-pinB-turn28.sav --floats bits --mask none \ + --relabel-new-ids $S/bp-turn28-pre.sav +# negative +uv run python3 verify/state-checksum/state_checksum.py \ + $S/ad-oracle-A-post.sav $S/ad-oracle-B-post.sav --floats bits --mask none \ + --relabel-new-ids $S/ad-turn27-two-raiders.sav +# guards +cd verify/state-checksum && uv run python3 -m unittest -v test_state_checksum.RelabelTest +``` + +--- + +## Proposed board rows + +New row: + +``` +| STATE_CHECKSUM --relabel-new-ids: BP'S PINNED PAIR IS IDENTICAL MODULO ONE TRANSPOSITION, AD'S UNPINNED PAIR IS REFUSED AND STILL DIVERGES | tooling | verified | high | 100% | 2026-09-09 | Lane BT, HOST ONLY (no VM; 140 untouched, 141/144/146 held by AZ/BS/BR and not approached). Implements the 2026-09-09 fleet-id-order resolution §3 item 1 (`findings/subsystems/relabel-new-ids.md`). `state_checksum.py POST-A POST-B --relabel-new-ids PRE` computes the ids NEW in each post-turn save, restricts to node nibble != 0 (client-minted; `id-allocation.md` id=(counter<<4)|node), matches those fleets by a key that does not mention the id - (LocID if non-zero else FPlan/pnd destination, sorted ship-id set), with "at 384" and "bound-for 384" tagged apart - builds the bijection pi, rewrites every fleet reference, compares ShipIDs/FleetIDs/DesignIDs as SETS, masks /Summary/Checksum WITH ITS REASON ON THE LINE, and prints pi. **POSITIVE: `bp-pinA` vs `bp-pinB` = `IDENTICAL modulo pi = {1970<->1986}`, 35 leaves -> 0, exit 0**, with pi recovered from the saves alone and printed with the counters (123/124/125, node 2) and the content keys - the group AT Sys 384 with ship 6976 is 1970 in A and 1986 in B; the group BOUND FOR Sys 80 with ship 5264 is 1986 in A and 1970 in B; 2002's group is 2002 in both. **NEGATIVE: `ad-oracle-A/B-post` REFUSED and `DIVERGED: 94 leaf difference(s)`, exit 1** - byte-identically the 94 the tool reports without the flag, because on a refusal NOTHING is rewritten, masked or set-compared. The refusal is by CARDINALITY (A minted 2 client fleets, B minted 3), so AD's pair does not exercise the content matcher; that is exercised separately, and FORCING the transposition onto AD's pair with the guard bypassed STILL LEAVES 82 LEAVES. FIVE GUARDS, EVERY ONE REFUSES RATHER THAN DEGRADES: G1 only ids absent from the pre-turn save (tested with two PRE-EXISTING fleets exchanging contents - must still report); G2 only non-zero node nibbles (node-0 swap still reports); G3 pi must permute ONE set; G4 content keys correspond one-to-one and are unique per side; G5 NO leaf anywhere may hold a permuted id at an unmodelled site, matched on RAW BYTES not the reader's typed value (a value scan is empty the moment the schema types a reference as a float - this happened during development). Refusal prints its reason and falls back to the ORDINARY comparison, never to a partial relabelling. THREE SPECIFICATION CORRECTIONS FROM CONTACT WITH THE DATA: (1) **FtName is an ID-ATTACHED LABEL** - 1970 is "Sigma Fleet VI" and 1986 "Tau Fleet VI" in BOTH saves, a Greek ordinal minted by the same pass as the id, so "apply pi to every id-valued leaf" would have produced two spurious name diffs; the name stays with the SLOT, gated by a guard requiring it to track the id in both saves. The counter is PER PLAYER AND SHARED WITH THE SERVER'S FLEETS - AD's A (2 client fleets) names node-0 fleet 7120 "Upsilon", B (3) names it "Phi" - which the engine-side canonical visit order must reproduce and §3.2 does not mention. (2) **Applying pi to the Flt[] KEYS introduces a spurious `reordered`**: the fleet table and FleetIDs[] are ID-ordered and IDENTICAL in A and B, so renumbering slots reorders A against B; the artefact-free equivalent is to EXCHANGE THE FLEET BODIES between slots, leaving slot ids, FleetIDs[] and every label as written. (3) **A node's new-id set is NOT fleet-only**: 2002 is a DesignID in ad-oracle-A-post and a FltID in ad-oracle-B-post - the per-node counter is shared across object kinds, and G5 is what keeps a fleet-only matcher safe. NOT DONE, with reasons: lists 8/10/14 (they are aiorders SHIM DUMP words, not save leaves - this tool has no block reader); designs/ships; the two engine-side items. DEFAULT PATH PROVEN UNCHANGED: pre- and post-change modules compared over the whole save corpus under two policies on root digest, coverage, mask hits AND every (path,digest) in the tree - zero mismatches; plus byte-identical CLI stdout on the single-save, BP, AD and resave invocations. The relabelled root is DOMAIN-SEPARATED so a "modulo pi" root can never be read as a strict one. 24 new tests (20 synthetic + 4 real-save acceptance), 7 of them asserting a refusal; suite 38 -> 62, all passing. ALSO FIXED, PRE-EXISTING AND NOT A REGRESSION (the pre-change module fails it identically): `test_known_resave_delta_localises_to_five_named_leaves` enumerated pairs over SORTED FILENAMES and hard-coded the re-save direction 4->0; `cb-turn2to3-endturn.sav` sorts before `turn2-state.sav`, so that pair reads 0->4 and the test failed on a corpus fact. Assertion is now direction-symmetric (all four Status leaves must still move the SAME way, Checksum following by -/+16). Artefacts: `verify/state-checksum/state_checksum.py`, `test_state_checksum.py`, `STATE_CHECKSUM.md` §2.6 | +``` + +Edits to existing rows: + +- **Row 424 / the resolver row of 2026-09-09** — append: `ENGINE ITEM 1 IS DONE (lane BT, host): + state_checksum --relabel-new-ids ships with both acceptance halves green - BP's pair IDENTICAL + modulo {1970<->1986}, AD's pair refused and still 94-leaf DIVERGED. Three corrections to §3.1 + from the data: FtName is an id-attached label and needs the same treatment as the id (and its + counter is per-player and shared with the server's fleets, which §3.2's canonical visit order + must also reproduce); relabelling the Flt[] keys is the wrong operation - exchange the bodies, + because the fleet table is id-ordered and identical in both saves; and a node's new-id set spans + object kinds (2002 is a DesignID in ad-oracle-A-post and a FltID in ad-oracle-B-post). Lists + 8/10/14 are not in scope for this tool - they are shim-dump words, not save leaves.` +- **Row 333 (C-exact)** — append: `Comparison side delivered 2026-09-09 (lane BT): + `state_checksum --relabel-new-ids `, guarded so it refuses rather than absorbs; see + findings/subsystems/relabel-new-ids.md.` +- **`determinism-oracle.md`'s "Qualified" paragraph** — when it is written, "modulo π" now has an + executable definition and a printed π; cite `relabel-new-ids.md` §1 for the operation and §5 for + the guards. +- **`id-allocation.md` §5** — alongside the canonical-visit-order item the resolver asked for, add + the two measured facts from §4.1/§4.2 of this document: the **fleet-name ordinal** is per player + and shared between client- and server-minted fleets, and the **master fleet table is id-ordered**, + not visit-ordered. +- **Row 62** — VM140/141/144/146 unchanged by this lane; it took no guest. diff --git a/verify/state-checksum/STATE_CHECKSUM.md b/verify/state-checksum/STATE_CHECKSUM.md index fac2cf5..b938234 100644 --- a/verify/state-checksum/STATE_CHECKSUM.md +++ b/verify/state-checksum/STATE_CHECKSUM.md @@ -13,7 +13,7 @@ verify/state-checksum/ float_census.py classifies every float leaf (evidence for §3) stability_check.py identical bytes -> identical roots, coverage proved run_validation.sh regenerates verify/results/state-checksum/ - test_state_checksum.py 38 tests + test_state_checksum.py 62 tests ``` --- @@ -150,6 +150,46 @@ A mask that matches nothing prints `[mask matched NOTHING -- check the rule path nobody audits is a hiding place, and this project has already been bitten once by a comparison that quietly covered nothing. +### 2.6 `--relabel-new-ids` — comparing modulo the labelling of this turn's new ids + +Added 2026-09-09 (lane BT) to the specification in +`findings/resolutions/2026-09-09-fleet-id-order-residue.md` §3 item 1. Full write-up, with the +acceptance output and the guard tests, in `findings/subsystems/relabel-new-ids.md`. + +``` +state_checksum.py POST-A.sav POST-B.sav --relabel-new-ids PRE-TURN.sav +``` + +Two processes with **identical pinned AI seeds** can still write different autosaves, because +the order in which the AI visits the ship groups that need a new fleet is per-process. The ids +are not the variable — they decode (`id-allocation.md`: `id = (counter << 4) | node`) to one +client's own counters, minted in that order in every process. The *same group* gets a +*different* id. This mode quotients that labelling out and nothing else. + +Given the pre-turn save it computes the ids **new** in each post-turn save, matches the +non-zero-node (client-minted) new fleets by a key that does not mention the id — `(LocID or +`FPlan` destination, sorted ship-id set)` — builds the bijection π, exchanges the matched +fleets' bodies between their slots, rewrites every fleet reference under π, compares the master +id lists as sets, masks `/Summary/Checksum` **with its reason on the line** (§4.6: derived, and +its inputs are unmodelled), and prints π: + +``` +relabel: pi = {1970<->1986} +IDENTICAL modulo pi = {1970<->1986} +``` + +**Five guards, and every one refuses rather than degrades.** Only ids absent from the pre-turn +save (G1); only non-zero node nibbles (G2); π must permute one set, so the two saves must have +minted the *same* ids (G3); the content keys must correspond one-to-one and be unique on each +side (G4); and no leaf anywhere in either save may hold a permuted id at a site this tool does +not model as a fleet reference (G5, matched on raw bytes so a reader typing slip cannot empty +it). On a refusal **nothing is rewritten** and the ordinary comparison runs, so a real +divergence still reports; the reason is printed on its own line. + +The relabelled root is domain-separated (`relabel-new-ids` is folded into the root preimage), so +a "modulo π" root can never be mistaken for a strict one, and every root recorded before this +flag existed is bit-for-bit unchanged. + --- ## 3. Float-parity policy @@ -454,14 +494,25 @@ from the VM — see §5. ### 4.5 Tests -38 tests, `uv run python3 -m unittest discover -s verify/state-checksum -t verify/state-checksum`. +62 tests, `uv run python3 -m unittest discover -s verify/state-checksum -t verify/state-checksum`. All pass. With only the in-repo saves, two real-save tests skip (the `bb4fd9ac` re-save form is -not in the repo); with `SOTS_SAVES_DIR` pointed at the full corpus, **38 pass, 0 skipped**. +not in the repo); with `SOTS_SAVES_DIR` pointed at the full corpus, **62 pass, 0 skipped**. +24 of them cover `--relabel-new-ids` (§2.6), and 7 of those 24 assert a **refusal** — the guards +are the part worth testing. -Two of the tests earn their keep by having caught real defects during development: the -sibling-index cascade in §2.1, and a `string` value's length prefix missing from the +Three of the tests earn their keep by having caught real defects during development: the +sibling-index cascade in §2.1, a `string` value's length prefix missing from the reconstruction (which made the coverage audit fail loudly at offset `0x1c` instead of silently -under-covering — the audit working exactly as intended, on its author). +under-covering — the audit working exactly as intended, on its author), and the raw-byte +completeness scan of §2.6, which caught the reader typing a fleet reference as a **float** where +a value-based scan saw nothing. + +**2026-09-09 correction, not a defect in the tool.** `test_known_resave_delta_localises_to_five_ +named_leaves` enumerates candidate pairs over *sorted filenames*, and asserted the re-save +direction as `4 -> 0`. Adding `cb-turn2to3-endturn.sav` to the corpus produced a pair whose +re-saved member sorts first, so the observed direction was `0 -> 4` and the test failed on a +corpus fact. The claim is symmetric; the assertion now is too (and still requires all four +`Status` leaves to move the same way, with `Checksum` following by ∓16). ### 4.6 A negative result on `Summary.Checksum` diff --git a/verify/state-checksum/state_checksum.py b/verify/state-checksum/state_checksum.py index e9189c6..40e3d06 100644 --- a/verify/state-checksum/state_checksum.py +++ b/verify/state-checksum/state_checksum.py @@ -41,6 +41,7 @@ Usage state_checksum.py SAVE --tree --depth 3 # digest tree state_checksum.py A B # localise divergence state_checksum.py A B --ulps 2 # classify float diffs + state_checksum.py A B --relabel-new-ids PRE # compare modulo new-id labelling state_checksum.py --chain chain.json S1 S2 ... # verify a recorded chain state_checksum.py SAVE --json # machine-readable @@ -65,7 +66,9 @@ import save_reader as sr # noqa: E402 __all__ = [ "FLOAT_POLICIES", "MASK_PRESETS", "CkNode", "checksum_save", "checksum_bytes", - "reconstruct", "audit_coverage", "diff", "DiffEntry", "float_key", + "checksum_result", "reconstruct", "audit_coverage", "diff", "DiffEntry", + "float_key", "RelabelRefused", "Relabelling", "build_relabelling", + "apply_relabelling", "format_permutation", "id_node", ] DIGEST_BYTES = 16 # blake2b-128; 2**-64 collision floor at our object counts @@ -181,6 +184,11 @@ ELEM_KEYS = ("EvTurn", "EvEID", "ordID", "npid", "desID", "Wpt", "seno2", "ID", INLINE_ID_LISTS = ("PlayerIDs", "DesignIDs", "SystemIDs", "FleetIDs", "ShipIDs", "TradeIDs") +#: master id lists whose *element order* is an artefact of the order in which +#: the turn's commands were applied, not a decision. Compared as sets ONLY +#: under `--relabel-new-ids`, never by default -- see RELABEL_NOTES. +SET_ID_LISTS = ("ShipIDs", "FleetIDs", "DesignIDs") + #: top-level digest rows printed by the default (non-tree) view. Paths that do #: not exist in a given save are silently skipped. SUBSYSTEM_PATHS = ( @@ -371,7 +379,8 @@ def _value_key(node: sr.Node, path: str, policy: str, masks, hits: dict): return kind, raw, False -def _build(node: sr.Node, path: str, label: str, policy: str, masks, hits: dict) -> CkNode: +def _build(node: sr.Node, path: str, label: str, policy: str, masks, hits: dict, + id_lists_as_sets: bool = False) -> CkNode: if node.kind == "complex": entries = _group_children(node.children) counts = _name_counts(e[1] for e in entries if e[0] == "item") @@ -385,26 +394,45 @@ def _build(node: sr.Node, path: str, label: str, policy: str, masks, hits: dict) k = ordinals.get(child.name, 0) ordinals[child.name] = k + 1 lab = _label(child, k, None, unique=counts.get(child.name) == 1) - kids.append(_build(child, f"{path}/{lab}", lab, policy, masks, hits)) + kids.append(_build(child, f"{path}/{lab}", lab, policy, masks, hits, + id_lists_as_sets)) elif ent[0] == "list": _, lname, cnode, vals = ent - lpath = f"{path}/{lname}[]" + # path-scoped like the mask rules (2.5): only the MASTER lists + # directly under /Sim, never a same-named list nested in an + # object. On the corpus that is the only place they occur -- + # asserted here so a future save that nests one is not silently + # set-compared. + as_set = (id_lists_as_sets and lname in SET_ID_LISTS + and path == "/Sim") + lpath = f"{path}/{lname}[]" + ("(set)" if as_set else "") keys = [_value_key(v, lpath, policy, masks, hits)[1] for v in vals] - ldig = _h(b"list", lname.encode("ascii"), cnode.raw, *keys) + values = [v.value for v in vals] + if as_set: + # element order is apply order, which is the residue this + # mode exists to quotient out; the multiset is still exact, + # so a genuine insertion/removal still reports. + order = sorted(range(len(vals)), key=lambda i: (str(type(values[i])), values[i])) + keys = [keys[i] for i in order] + values = [values[i] for i in order] + ldig = _h(b"list-set" if as_set else b"list", + lname.encode("ascii"), cnode.raw, *keys) off = cnode.offset last = vals[-1] if vals else cnode - kids.append(CkNode(lpath, f"{lname}[]", "list", ldig, off, + kids.append(CkNode(lpath, f"{lname}[]" + ("(set)" if as_set else ""), + "list", ldig, off, last.offset + last.size - off, 1 + len(vals), len(cnode.raw) + sum(len(v.raw) for v in vals), - [], values=[v.value for v in vals])) + [], values=values)) else: _, gname, pairs = ent gpath = f"{path}/{gname}" gkids = [] for j, (idn, frame) in enumerate(pairs): lab = _label(frame, j, idn) - ck = _build(frame, f"{gpath}/{lab}", lab, policy, masks, hits) + ck = _build(frame, f"{gpath}/{lab}", lab, policy, masks, hits, + id_lists_as_sets) # the id int is part of the object's identity; fold it in idk = _h(b"id", (idn.name or "").encode("ascii"), idn.raw) ck.digest = _h(b"obj", idk, ck.digest) @@ -541,15 +569,21 @@ class SaveChecksum: } -def checksum_bytes(data: bytes, path: str = "", floats: str = "bits", - mask: str = "none", audit: bool = True, - strict: bool = False) -> SaveChecksum: +def checksum_result(res: sr.SaveResult, path: str, cov: dict, + floats: str = "bits", mask: str = "none", + extra_masks: tuple = (), + id_lists_as_sets: bool = False) -> SaveChecksum: + """Digest an already-parsed save. + + Split out of `checksum_bytes` so `--relabel-new-ids` can parse once, prove + coverage against the *unmodified* parse, and only then rewrite leaves. + """ if floats not in FLOAT_POLICIES: raise ValueError(f"floats must be one of {FLOAT_POLICIES}") if mask not in MASK_PRESETS: raise ValueError(f"mask must be one of {tuple(MASK_PRESETS)}") - res = sr.read_bytes(data, strict=strict) - masks = MASK_PRESETS[mask] + masks = tuple(MASK_PRESETS[mask]) + tuple( + m for m in extra_masks if m not in MASK_PRESETS[mask]) hits: dict = {} kids = [] counts = _name_counts(res.tree.children) @@ -558,15 +592,28 @@ def checksum_bytes(data: bytes, path: str = "", floats: str = "bits", k = ordinals.get(c.name, 0) ordinals[c.name] = k + 1 lab = _label(c, k, None, unique=counts.get(c.name) == 1) - kids.append(_build(c, "/" + lab, lab, floats, masks, hits)) - root_dig = _h(b"save", floats.encode("ascii"), mask.encode("ascii"), - *[k.digest for k in kids]) + kids.append(_build(c, "/" + lab, lab, floats, masks, hits, id_lists_as_sets)) + # domain separation: a relabelled root must never be confusable with a + # plain one. The extra part is appended ONLY in the new mode, so every + # root recorded before this flag existed is bit-for-bit unchanged. + pol = [b"save", floats.encode("ascii"), mask.encode("ascii")] + if extra_masks or id_lists_as_sets: + pol.append(b"relabel-new-ids") + root_dig = _h(*pol, *[k.digest for k in kids]) root = CkNode("", "", "complex", root_dig, 0, len(res.inflated), sum(k.leaves for k in kids), sum(k.value_bytes for k in kids), kids) - cov = audit_coverage(res.tree, res.inflated) if audit else {"ok": None} return SaveChecksum(path, root, cov, res, floats, mask, hits) +def checksum_bytes(data: bytes, path: str = "", floats: str = "bits", + mask: str = "none", audit: bool = True, + strict: bool = False, extra_masks: tuple = (), + id_lists_as_sets: bool = False) -> SaveChecksum: + res = sr.read_bytes(data, strict=strict) + cov = audit_coverage(res.tree, res.inflated) if audit else {"ok": None} + return checksum_result(res, path, cov, floats, mask, extra_masks, id_lists_as_sets) + + def checksum_save(path: str, **kw) -> SaveChecksum: with open(path, "rb") as f: return checksum_bytes(f.read(), path=path, **kw) @@ -651,6 +698,371 @@ def diff(a: CkNode, b: CkNode, out: list | None = None, limit: int = 200) -> lis return out +# --- relabelling new client-minted ids ---------------------------------------- +# +# Why this exists +# --------------- +# `findings/resolutions/2026-09-09-fleet-id-order-residue.md` section 3 item 1. +# Lane BP ran two processes with *identical pinned AI seeds* on the same +# pre-turn save and got autosaves differing in 35 of 61,147 leaves -- all 35 one +# transposition: fleets 1970 and 1986 exchange their entire contents. The +# resolver decoded the ids (`id-allocation.md`: `id = (counter << 4) | node`) +# and found the ids are NOT the variable: they are one client's own counters +# 123/124/125, minted in that order in every process. What varies is the order +# in which the fleet-assignment pass VISITS the ship groups that need a new +# fleet, so the same group receives a different id. No decision changes. +# +# This mode makes the comparison blind to that labelling and to nothing else. +# +# The guards matter more than the feature (rule 1: a green verdict on a hook +# that may be comparing nothing is the failure this campaign was built around). +# There are five, and every one of them REFUSES rather than degrades: +# +# G1 only ids ABSENT from the pre-turn save may be relabelled; +# G2 only ids with a non-zero node nibble (client-minted) may be relabelled; +# G3 pi must be a permutation of ONE set -- the new ids of one node -- so the +# two saves must have minted the same ids; different id sets is a real +# divergence (a different number of allocations) and is never absorbed; +# G4 the content keys must correspond one-to-one, and be unique on each side; +# G5 every leaf anywhere in either save that holds a permuted id must sit at +# a reference site this module models. An unmodelled site would mean the +# rewrite is incomplete, which could make two different states compare +# equal. Finding one is a refusal, not a warning. +# +# A refusal never falls back to a partial relabelling. The CLI prints the +# reason and then runs the ORDINARY comparison, which is the conservative +# direction: whatever really differs still reports as DIVERGED. + +class RelabelRefused(Exception): + """A relabelling could not be built as a full, guarded bijection.""" + + +def id_node(i: int) -> int: + """The node nibble of an object id (`id-allocation.md` section 1). + + 0 is the server / the sim that owns the board; `PlyrIdx + 1` is a player's + own client. Only non-zero nibbles are ever relabelled here. + """ + return i & 0xF + + +#: Leaf tags that hold a *fleet* id, with where they are allowed to appear. +#: `FltID` is both the master fleet table's lead int (the slot, never rewritten) +#: and a ship's back-reference to its fleet (rewritten); `Flt` is a system's +#: list of the fleets standing at it (rewritten). Anything else is G5. +FLEET_SLOT_TAG = "FltID" +FLEET_REF_TAGS = ("FltID", "Flt") + +RELABEL_MASK = (("Checksum", "/Summary/", "derived", None),) + +RELABEL_NOTES = ( + "/Summary/Checksum is MASKED: it is derived from the whole state and its " + "inputs are unmodelled (rule 18's open item), so it cannot be recomputed " + "under a relabelling and its move is not independent evidence. Unmask by " + "dropping --relabel-new-ids.", + "Master id lists (%s) are compared as SETS: their element order is the " + "order the turn's commands were applied in, which is the same visit-order " + "residue. Membership is still exact -- an added or removed id still " + "reports." % "/".join(SET_ID_LISTS), +) + + +class Relabelling: + """The bijection pi, plus everything a reader needs to audit it.""" + + __slots__ = ("pi", "nodes", "new_ids", "keys", "rewrites", "moves") + + def __init__(self, pi, nodes, new_ids, keys): + self.pi = pi # A-id -> B-id, a permutation of new_ids + self.nodes = nodes # node nibbles it touches + self.new_ids = new_ids # the new non-zero-node fleet ids + self.keys = keys # A-id -> content key, for the report + self.rewrites: dict = {} # tag -> leaves rewritten + self.moves = 0 # fleet bodies exchanged + + @property + def moved(self) -> dict: + return {k: v for k, v in self.pi.items() if k != v} + + def __bool__(self) -> bool: + return bool(self.moved) + + +def format_permutation(pi: dict) -> str: + """`{1970<->1986}`, or cycle notation for anything longer.""" + moved = {k: v for k, v in pi.items() if k != v} + if not moved: + return "{} (identity -- no new id changed hands)" + seen, cycles = set(), [] + for k in sorted(moved): + if k in seen: + continue + cyc, cur = [k], moved[k] + seen.add(k) + while cur != k: + cyc.append(cur) + seen.add(cur) + cur = moved[cur] + cycles.append(cyc) + parts = ["<->".join(str(x) for x in c) if len(c) == 2 + else "->".join(str(x) for x in c + [c[0]]) for c in cycles] + return "{" + ", ".join(parts) + "}" + + +def _sim_frame(tree: sr.Node) -> sr.Node: + for c in tree.children: + if c.name == "Sim" and c.kind == "complex": + return c + raise RelabelRefused("no /Sim frame in this save -- not a game state") + + +def _fleet_slots(tree: sr.Node) -> list: + """[(lead_id_node, frame_node)] for the master fleet table.""" + sim = _sim_frame(tree) + ch = sim.children + out = [] + for i in range(len(ch) - 1): + if (ch[i].name == FLEET_SLOT_TAG and ch[i].kind == "int" + and ch[i + 1].name == "Flt" and ch[i + 1].kind == "complex"): + out.append((ch[i], ch[i + 1])) + return out + + +def _fleet_content_key(frame: sr.Node): + """An identity for a fleet that does NOT mention its own id. + + Resolver section 3.1: `(LocID or FPlan destination, sorted ship-id set)`. + `LocID` and the plan destination are tagged apart, so a fleet standing at + system 384 never keys the same as one bound for 384. + """ + loc, dest, ships = None, None, [] + for c in frame.children: + if c.name == "LocID" and c.kind == "int": + loc = c.value + elif c.name == "ShipID" and c.kind == "int": + ships.append(c.value) + elif c.name == "FPlan" and c.kind == "complex": + for g in c.children: + if g.name == "pnd" and g.kind == "int": + dest = g.value + if loc: + where = ("at", loc) + elif dest is not None: + where = ("bound-for", dest) + else: + where = ("nowhere", None) + return (where, tuple(sorted(ships))) + + +def _fleet_name(frame: sr.Node): + for c in frame.children: + if c.name == "FtName": + return c.value + return None + + +def _id_occurrences(tree: sr.Node, wanted: set) -> list: + """Every leaf in `tree` holding one of `wanted`, classified by site. + + Matching is on the leaf's **raw four bytes**, not on the reader's typed + value, deliberately: the digest is a function of (save bytes, reader + schema), and a leaf the schema happens to type as a float or a raw blob + would be invisible to a value-based scan while still carrying an id the + rewrite must reach. A completeness guard that a typing slip can silently + empty is exactly the "green verdict over an empty region set" of rule 1. + + Returns [(site, path, node)] where site is one of: + "slot" -- the master fleet table's lead FltID (identifies the slot) + "idlist" -- an entry of the inline `FleetIDs` master list + "ref:" -- a reference that must be rewritten under pi + "UNMODELLED"-- anything else; this is guard G5 and it refuses + """ + by_raw = {struct.pack(" Relabelling: + """Build pi between the ids new in A and the ids new in B, or refuse. + + Nothing is mutated here. Every guard is checked before any rewrite can + happen, so a refusal leaves both saves exactly as parsed. + """ + pre_ids = {n.value for n, _ in _fleet_slots(pre_tree)} + A = {n.value: f for n, f in _fleet_slots(a_tree)} + B = {n.value: f for n, f in _fleet_slots(b_tree)} + + # G1 + G2: new, and client-minted + new_a = {i for i in A if i not in pre_ids and id_node(i) != 0} + new_b = {i for i in B if i not in pre_ids and id_node(i) != 0} + + # G3: pi permutes ONE set. Unequal id sets means the two runs minted + # different counters -- a different number of allocations, which is a real + # divergence and is never something this mode may absorb. + if new_a != new_b: + raise RelabelRefused( + "the new client-minted fleet ids differ between the two saves -- " + f"A minted {sorted(new_a)}, B minted {sorted(new_b)}" + + (f"; only in A {sorted(new_a - new_b)}" if new_a - new_b else "") + + (f"; only in B {sorted(new_b - new_a)}" if new_b - new_a else "") + + ". That is a difference in what was allocated, not in how it " + "was labelled, so no relabelling is defensible") + + new_ids = new_a + pi: dict = {} + keys: dict = {} + for node in sorted({id_node(i) for i in new_ids}): + ids = sorted(i for i in new_ids if id_node(i) == node) + ka: dict = {} + kb: dict = {} + for side, tbl, dst in (("A", A, ka), ("B", B, kb)): + for i in ids: + key = _fleet_content_key(tbl[i]) + if key in dst: + raise RelabelRefused( + f"two new fleets in {side} share the content key {key!r} " + f"(ids {dst[key]} and {i}) -- the match would be " + "ambiguous, so no bijection can be built") + dst[key] = i + if set(ka) != set(kb): + only_a = [f"{ka[k]}:{k}" for k in sorted(set(ka) - set(kb), key=repr)] + only_b = [f"{kb[k]}:{k}" for k in sorted(set(kb) - set(ka), key=repr)] + raise RelabelRefused( + f"node {node}: the new fleets do not correspond by content. " + f"Unmatched in A: {only_a}; unmatched in B: {only_b}. Their " + "contents differ, which is a real divergence") + for key, i in ka.items(): + pi[i] = kb[key] + keys[i] = key + # names are minted from the same per-fleet ordinal counter as the id + # (Sigma/Tau/Upsilon...), so they are id-attached labels, not content. + # They may only be treated that way if they DO track the id in both + # saves; if they do not, something real differs and we refuse. + for i in ids: + na, nb = _fleet_name(A[i]), _fleet_name(B[i]) + if na != nb: + raise RelabelRefused( + f"fleet {i} is named {na!r} in A and {nb!r} in B -- the " + "name does not track the id, so it is not an id-attached " + "label here and relabelling would hide a real difference") + + rl = Relabelling(pi, sorted({id_node(i) for i in new_ids}), sorted(new_ids), keys) + + # G3 again, on the built map: a permutation of exactly the new ids + if sorted(pi) != sorted(pi.values()): + raise RelabelRefused( + f"pi is not a permutation: domain {sorted(pi)} != image " + f"{sorted(pi.values())}") + for k, v in pi.items(): + if id_node(k) != id_node(v): + raise RelabelRefused( + f"pi would map {k} (node {id_node(k)}) to {v} (node " + f"{id_node(v)}) -- ids may only be relabelled within one node") + if k in pre_ids or v in pre_ids: + raise RelabelRefused( + f"pi touches {k if k in pre_ids else v}, which is present in " + "the pre-turn save -- only ids created on this turn may be " + "relabelled") + + # G5: no unmodelled reference site, in EITHER save + touched = set(pi) | set(pi.values()) + for name, tree in (("A", a_tree), ("B", b_tree)): + for site, path, _n in _id_occurrences(tree, touched): + if site == "UNMODELLED": + raise RelabelRefused( + f"{name} holds a relabelled id at {path}, which this tool " + "does not model as a fleet reference. Rewriting only the " + "sites it knows would leave the two states comparable at a " + "leaf that still carries the old label -- add the tag to " + "FLEET_REF_TAGS after checking what it means") + return rl + + +def apply_relabelling(tree: sr.Node, rl: Relabelling) -> Relabelling: + """Rewrite `tree` in place so it reads as if it had minted B's labelling. + + Two operations, both a function of pi alone: + + * the permuted fleets' **bodies** are exchanged between their slots, while + each slot keeps its own id and its id-attached label (`FtName`). Doing + it this way -- rather than renumbering the slots -- means the master + fleet table's element order, the `FleetIDs[]` list and every object + label stay exactly as the file had them, so nothing but the intended + quotient is taken. + * every *reference* to a permuted id is rewritten to pi(it). + + Node `offset`/`size` are left stale on moved bodies; they are reporting + aids in `--tree`/`--json`, not digest inputs. The digest is a function of + (tag, kind, value bytes) only. + """ + if not rl.pi: + return rl + slots = {n.value: f for n, f in _fleet_slots(tree)} + inv = {v: k for k, v in rl.pi.items()} + + old_children = {i: list(slots[i].children) for i in rl.pi} + own_name = {} + for i in rl.pi: + own_name[i] = [c for c in slots[i].children if c.name == "FtName"] + + for dst in rl.pi.values(): + src = inv[dst] + if src == dst: + continue + kept = list(own_name[dst]) + kids = [] + for c in old_children[src]: + if c.name == "FtName" and kept: + kids.append(kept.pop(0)) + else: + kids.append(c) + slots[dst].children = kids + rl.moves += 1 + + by_raw = {struct.pack(" dict: @@ -722,6 +1134,76 @@ def _mask_note(ck: "SaveChecksum") -> str: return " [masked: " + ", ".join(f"{k}x{v}" for k, v in sorted(ck.mask_hits.items())) + "]" +def _relabelled_pair(pre_path: str, a_path: str, b_path: str, + floats: str = "bits", mask: str = "none", + audit: bool = True, strict: bool = False): + """Parse three saves, build pi, and digest A relabelled against B. + + On a refusal nothing is rewritten and the ORDINARY comparison is returned + instead, so a real divergence survives. Returns (a, b, relabelling|None, + refusal_message|None). + """ + def read(p): + with open(p, "rb") as f: + res = sr.read_bytes(f.read(), strict=strict) + cov = audit_coverage(res.tree, res.inflated) if audit else {"ok": None} + return res, cov + + res_p, _ = read(pre_path) + res_a, cov_a = read(a_path) + res_b, cov_b = read(b_path) + + try: + rl = build_relabelling(res_p.tree, res_a.tree, res_b.tree) + except RelabelRefused as exc: + a = checksum_result(res_a, a_path, cov_a, floats, mask) + b = checksum_result(res_b, b_path, cov_b, floats, mask) + return a, b, None, str(exc) + + apply_relabelling(res_a.tree, rl) + a = checksum_result(res_a, a_path, cov_a, floats, mask, + extra_masks=RELABEL_MASK, id_lists_as_sets=True) + b = checksum_result(res_b, b_path, cov_b, floats, mask, + extra_masks=RELABEL_MASK, id_lists_as_sets=True) + return a, b, rl, None + + +def _relabel_report(rl, refusal: str | None, pre_path: str) -> list: + if refusal is not None: + return [ + f"relabel: pre-turn save {pre_path}", + "relabel: REFUSED -- " + refusal, + "relabel: NO relabelling was applied; the comparison below is the " + "ordinary one, and nothing has been absorbed.", + ] + lines = [f"relabel: pre-turn save {pre_path}", + f"relabel: pi = {format_permutation(rl.pi)}"] + for i in rl.new_ids: + where, ships = rl.keys[i] + lines.append(f"relabel: {i} (counter {i >> 4}, node {id_node(i)}) " + f"-> {rl.pi[i]} {where[0]} {where[1]}, ships {list(ships)}") + lines.append(f"relabel: {len(rl.new_ids)} new client-minted fleet id(s) on " + f"node(s) {rl.nodes}; {len(rl.moved)} relabelled, " + f"{rl.moves} fleet body/bodies exchanged, " + + (", ".join(f"{k}x{v}" for k, v in sorted(rl.rewrites.items())) + or "no references") + " rewritten") + for note in RELABEL_NOTES: + lines.append("relabel: " + note) + return lines + + +def _relabel_dict(rl, refusal: str | None, pre_path: str) -> dict: + if refusal is not None: + return {"preTurn": pre_path, "applied": False, "refused": refusal} + return {"preTurn": pre_path, "applied": True, + "pi": {str(k): v for k, v in rl.pi.items()}, + "moved": {str(k): v for k, v in rl.moved.items()}, + "newIds": rl.new_ids, "nodes": rl.nodes, + "contentKeys": {str(k): repr(v) for k, v in rl.keys.items()}, + "bodiesExchanged": rl.moves, "referencesRewritten": rl.rewrites, + "notes": list(RELABEL_NOTES)} + + def _print_tree(n: CkNode, depth: int, out, indent: int = 0, top: bool = True): if not top: pad = " " * indent @@ -751,6 +1233,10 @@ def main(argv=None) -> int: ap.add_argument("--ulps", type=float, default=None, help="in a diff, flag float differences at or below N ULPs") ap.add_argument("--limit", type=int, default=200, help="max diff entries") + ap.add_argument("--relabel-new-ids", metavar="PRE-TURN.SAV", default=None, + help="compare two post-turn saves modulo the labelling of " + "the client-minted ids created on this turn, given the " + "pre-turn save they both came from") ap.add_argument("--chain", metavar="FILE", help="chain JSON to verify against") ap.add_argument("--record-chain", metavar="FILE", help="write a chain JSON instead") args = ap.parse_args(argv) @@ -761,6 +1247,10 @@ def main(argv=None) -> int: if not args.saves: ap.error("no saves given") + if args.relabel_new_ids and (len(args.saves) != 2 or args.chain or args.record_chain): + ap.error("--relabel-new-ids compares exactly two post-turn saves; it is " + "a property of a pair, not of a single save or a chain") + if args.record_chain: chain = build_chain(args.saves, **kw) with open(args.record_chain, "w") as f: @@ -808,30 +1298,46 @@ def main(argv=None) -> int: if len(args.saves) != 2: ap.error("give one save, two saves, or use --chain / --record-chain") - a = checksum_save(args.saves[0], **kw) - b = checksum_save(args.saves[1], **kw) + rl = None + refusal = None + if args.relabel_new_ids: + a, b, rl, refusal = _relabelled_pair( + args.relabel_new_ids, args.saves[0], args.saves[1], **kw) + else: + a = checksum_save(args.saves[0], **kw) + b = checksum_save(args.saves[1], **kw) entries = diff(a.root, b.root, limit=args.limit) if args.ulps is not None: for e in entries: if e.kind == "value" and e.ulps is not None and e.ulps <= args.ulps: e.note += f" <= {args.ulps:g} ULP" if args.json: - print(json.dumps({"a": a.as_dict(0), "b": b.as_dict(0), - "identical": a.digest == b.digest, - "diffs": [e.as_dict() for e in entries]}, indent=1)) + out = {"a": a.as_dict(0), "b": b.as_dict(0), + "identical": a.digest == b.digest, + "diffs": [e.as_dict() for e in entries]} + if args.relabel_new_ids: + out["relabel"] = _relabel_dict(rl, refusal, args.relabel_new_ids) + print(json.dumps(out, indent=1)) return 0 if a.digest == b.digest else 1 print(f"A {a.digest} {a.path}") print(f"B {b.digest} {b.path}") print(f"policy: floats={a.policy} mask={a.mask} reader={READER_FINGERPRINT}" f"{_mask_note(a)}") + if args.relabel_new_ids: + for line in _relabel_report(rl, refusal, args.relabel_new_ids): + print(line) for ck, nm in ((a, "A"), (b, "B")): if ck.coverage.get("ok") is False: print(f"!! {nm}: coverage FAILED at 0x{ck.coverage.get('firstDiff') or 0:x} " f"-- the digest does not bound this file") if a.digest == b.digest: - print("IDENTICAL") + if rl is not None: + print(f"IDENTICAL modulo pi = {format_permutation(rl.pi)}") + else: + print("IDENTICAL") return 0 - print(f"DIVERGED: {len(entries)} leaf difference(s)" + modulo = f" modulo pi = {format_permutation(rl.pi)}" if rl is not None else "" + print(f"DIVERGED{modulo}: {len(entries)} leaf difference(s)" f"{' (limit reached)' if len(entries) >= args.limit else ''}") for e in entries: print(f" {e}") diff --git a/verify/state-checksum/test_state_checksum.py b/verify/state-checksum/test_state_checksum.py index 2e6e01e..25094c1 100644 --- a/verify/state-checksum/test_state_checksum.py +++ b/verify/state-checksum/test_state_checksum.py @@ -67,6 +67,64 @@ def tiny(status_a=4, status_b=4, checksum=-1000, extra_ids=(), return w.bytes() +#: (id, name, LocID, FPlan destination or None, [ship ids]) +PRE_FLEETS = [(258, "Alpha Fleet", 100, None, [16]), + (274, "Beta Fleet", 0, 700, [32, 48])] + + +def fleets_save(fleets, sys_flt=None, extra=None) -> bytes: + """A miniature save carrying a master fleet table. + + Shaped like the real one where this tool looks: `FleetIDs[]`/`ShipIDs[]` + inline master lists, `FltID`/`Flt` object pairs, `FtName`, `LocID`, an + `FPlan` with a `pnd` destination, and `ShipID`/`Ship` pairs whose `Ship` + frame carries the `FltID` back-reference. + """ + w = sw.SaveWriter("joint") + w.begin("Summary") + w.string("GameName", "T") + w.int("Turn", 28) + w.int("Checksum", sum(f[0] for f in fleets)) + w.end() + w.begin("Sim") + w.string("KeyPath", "") + w.int("NMSz", 16) + w.int("NMLc", 0) + w.int("NMnx", 100) + w.int("PlayerIDs", 1) + w.int(".", 32) + w.int("FleetIDs", len(fleets)) + for f in fleets: + w.int(".", f[0]) + ships = [s for f in fleets for s in f[4]] + w.int("ShipIDs", len(ships)) + for s in ships: + w.int(".", s) + if extra is not None: + w.int(extra[0], extra[1]) + if sys_flt is not None: + w.int("Flt", sys_flt) + w.int("NumFlts", len(fleets)) + for fid, name, loc, dest, fships in fleets: + w.int("FltID", fid) + w.begin("Flt") + w.string("FtName", name) + w.int("LocID", loc) + if dest is not None: + w.begin("FPlan") + w.int("pnd", dest) + w.end() + w.int("NumShips", len(fships)) + for s in fships: + w.int("ShipID", s) + w.begin("Ship") + w.int("FltID", fid) + w.end() + w.end() + w.end() + return w.bytes() + + def find_saves() -> list[str]: d = os.environ.get("SOTS_SAVES_DIR") or os.path.join(_VERIFY, "results", "saves") if not os.path.isdir(d): @@ -302,6 +360,222 @@ class MaskTest(unittest.TestCase): self.assertEqual(a.mask_hits, {}) +# --- --relabel-new-ids -------------------------------------------------------- +# +# The feature is small; the guards are the point. Every test below that asserts +# a REFUSAL is asserting that a real difference cannot be absorbed. + +class RelabelTest(unittest.TestCase): + A_NEW = [(1970, "Sigma Fleet", 384, None, [6976]), + (1986, "Tau Fleet", 0, 80, [5264])] + #: the same two groups, visited in the other order: the id and the + #: id-attached name stay put, the contents swap + B_NEW = [(1970, "Sigma Fleet", 0, 80, [5264]), + (1986, "Tau Fleet", 384, None, [6976])] + + def trees(self, a_new, b_new, pre=PRE_FLEETS, **kw): + p = sr.read_bytes(fleets_save(pre)) + a = sr.read_bytes(fleets_save(pre + list(a_new), **kw)) + b = sr.read_bytes(fleets_save(pre + list(b_new), **kw)) + return p.tree, a.tree, b.tree, a, b + + def compare(self, a_res, b_res, rl): + sc.apply_relabelling(a_res.tree, rl) + ca = sc.checksum_result(a_res, "A", {"ok": None}, + extra_masks=sc.RELABEL_MASK, id_lists_as_sets=True) + cb = sc.checksum_result(b_res, "B", {"ok": None}, + extra_masks=sc.RELABEL_MASK, id_lists_as_sets=True) + return ca, cb, sc.diff(ca.root, cb.root) + + # -- the feature --------------------------------------------------------- + + def test_a_pure_transposition_is_absorbed_and_pi_is_reported(self): + p, a, b, ares, bres = self.trees(self.A_NEW, self.B_NEW) + rl = sc.build_relabelling(p, a, b) + self.assertEqual(rl.pi, {1970: 1986, 1986: 1970}) + self.assertEqual(sc.format_permutation(rl.pi), "{1970<->1986}") + ca, cb, d = self.compare(ares, bres, rl) + self.assertEqual(d, [], [repr(e) for e in d]) + self.assertEqual(ca.digest, cb.digest) + + def test_the_ship_to_fleet_back_reference_is_rewritten(self): + p, a, b, ares, bres = self.trees(self.A_NEW, self.B_NEW) + rl = sc.apply_relabelling(ares.tree, sc.build_relabelling(p, a, b)) + self.assertEqual(rl.rewrites.get("FltID"), 2) + self.assertEqual(rl.moves, 2) + + def test_a_system_s_fleet_reference_is_rewritten(self): + p, a, b, ares, bres = self.trees(self.A_NEW, self.B_NEW) + # the same physical group is at system 384 in both, so the system's + # fleet reference names 1970 in A and 1986 in B + _, a2, _, ares2, _ = self.trees(self.A_NEW, self.B_NEW, sys_flt=1970) + _, _, b2, _, bres2 = self.trees(self.A_NEW, self.B_NEW, sys_flt=1986) + rl = sc.build_relabelling(p, a2, b2) + ca, cb, d = self.compare(ares2, bres2, rl) + self.assertEqual(rl.rewrites.get("Flt"), 1) + self.assertEqual(d, [], [repr(e) for e in d]) + + def test_the_identity_case_leaves_everything_alone(self): + p, a, b, ares, bres = self.trees(self.A_NEW, self.A_NEW) + rl = sc.build_relabelling(p, a, b) + self.assertEqual(rl.moved, {}) + self.assertFalse(bool(rl)) + ca, cb, d = self.compare(ares, bres, rl) + self.assertEqual(d, []) + + def test_pi_is_printed_as_a_cycle_when_it_is_not_a_transposition(self): + self.assertEqual(sc.format_permutation({1: 2, 2: 3, 3: 1}), "{1->2->3->1}") + self.assertIn("identity", sc.format_permutation({1970: 1970})) + + def test_a_relabelled_root_is_domain_separated_from_a_plain_one(self): + """A 'modulo pi' root must never be mistaken for a strict one.""" + data = fleets_save(PRE_FLEETS) + plain = sc.checksum_bytes(data) + loose = sc.checksum_bytes(data, extra_masks=sc.RELABEL_MASK, + id_lists_as_sets=True) + self.assertNotEqual(plain.digest, loose.digest) + + def test_the_default_path_is_untouched_by_the_new_parameters(self): + data = fleets_save(PRE_FLEETS) + self.assertEqual(sc.checksum_bytes(data).digest, + sc.checksum_bytes(data, extra_masks=(), + id_lists_as_sets=False).digest) + + # -- G1: nothing present in the pre-turn save may be relabelled ---------- + + def test_ids_present_in_the_pre_turn_save_are_never_in_pi(self): + p, a, b, _, _ = self.trees(self.A_NEW, self.B_NEW) + rl = sc.build_relabelling(p, a, b) + pre_ids = {n.value for n, _ in sc._fleet_slots(p)} + self.assertTrue(pre_ids) + self.assertFalse(pre_ids & (set(rl.pi) | set(rl.pi.values()))) + + def test_two_pre_turn_fleets_that_swapped_contents_are_NOT_absorbed(self): + """The strongest form of G1: the very shape this mode absorbs, but on + ids that already existed. That would be a real divergence.""" + swapped = [(258, "Alpha Fleet", 0, 700, [32, 48]), + (274, "Beta Fleet", 100, None, [16])] + p = sr.read_bytes(fleets_save(PRE_FLEETS)).tree + a = sr.read_bytes(fleets_save(PRE_FLEETS)).tree + b = sr.read_bytes(fleets_save(swapped)).tree + rl = sc.build_relabelling(p, a, b) + self.assertEqual(rl.pi, {}) + ares = sr.read_bytes(fleets_save(PRE_FLEETS)) + bres = sr.read_bytes(fleets_save(swapped)) + _, _, d = self.compare(ares, bres, rl) + self.assertTrue(d, "a swap among pre-existing fleets must still report") + + # -- G2: only client-minted (non-zero node nibble) ids ------------------ + + def test_server_minted_ids_are_never_relabelled(self): + """Node nibble 0 is the server's space (id-allocation.md section 1).""" + a_new = [(7120, "Phi Fleet", 384, None, [6976]), + (7136, "Chi Fleet", 0, 80, [5264])] + b_new = [(7120, "Phi Fleet", 0, 80, [5264]), + (7136, "Chi Fleet", 384, None, [6976])] + p, a, b, ares, bres = self.trees(a_new, b_new) + rl = sc.build_relabelling(p, a, b) + self.assertEqual(rl.pi, {}) + self.assertEqual(sc.id_node(7120), 0) + _, _, d = self.compare(ares, bres, rl) + self.assertTrue(d, "a node-0 swap is not this residue and must report") + + # -- G3: pi must permute one set of new ids ----------------------------- + + def test_different_new_id_sets_are_refused(self): + """Lane AD's unpinned pair in miniature: one run minted a third fleet.""" + b_new = self.B_NEW + [(2002, "Upsilon Fleet", 0, 384, [6992])] + p, a, b, _, _ = self.trees(self.A_NEW, b_new) + with self.assertRaises(sc.RelabelRefused) as cm: + sc.build_relabelling(p, a, b) + self.assertIn("only in B [2002]", str(cm.exception)) + + def test_a_refusal_leaves_both_trees_untouched(self): + b_new = self.B_NEW + [(2002, "Upsilon Fleet", 0, 384, [6992])] + p, a, b, ares, bres = self.trees(self.A_NEW, b_new) + before = sc.checksum_result(ares, "A", {"ok": None}).digest + with self.assertRaises(sc.RelabelRefused): + sc.build_relabelling(p, a, b) + self.assertEqual(sc.checksum_result(ares, "A", {"ok": None}).digest, before) + + # -- G4: the content match must be a one-to-one correspondence ---------- + + def test_new_fleets_whose_contents_differ_are_refused(self): + b_new = [(1970, "Sigma Fleet", 0, 80, [5264]), + (1986, "Tau Fleet", 384, None, [9999])] # a different ship + p, a, b, _, _ = self.trees(self.A_NEW, b_new) + with self.assertRaises(sc.RelabelRefused) as cm: + sc.build_relabelling(p, a, b) + self.assertIn("do not correspond by content", str(cm.exception)) + + def test_an_ambiguous_content_key_is_refused(self): + dup = [(1970, "Sigma Fleet", 384, None, [6976]), + (1986, "Tau Fleet", 384, None, [6976])] + p, a, b, _, _ = self.trees(dup, dup) + with self.assertRaises(sc.RelabelRefused) as cm: + sc.build_relabelling(p, a, b) + self.assertIn("ambiguous", str(cm.exception)) + + def test_a_fleet_at_a_system_never_keys_the_same_as_one_bound_for_it(self): + at = sr.read_bytes(fleets_save([(1970, "S", 384, None, [1])])).tree + to = sr.read_bytes(fleets_save([(1970, "S", 0, 384, [1])])).tree + ka = sc._fleet_content_key(sc._fleet_slots(at)[0][1]) + kb = sc._fleet_content_key(sc._fleet_slots(to)[0][1]) + self.assertNotEqual(ka, kb) + + # -- the name guard ------------------------------------------------------ + + def test_a_name_that_does_not_track_the_id_is_refused(self): + """`FtName` is minted from the same per-fleet ordinal as the id, so it + is an id-attached label -- but only while it demonstrably tracks the + id in both saves. If it does not, something real differs.""" + b_new = [(1970, "Tau Fleet", 0, 80, [5264]), + (1986, "Sigma Fleet", 384, None, [6976])] + p, a, b, _, _ = self.trees(self.A_NEW, b_new) + with self.assertRaises(sc.RelabelRefused) as cm: + sc.build_relabelling(p, a, b) + self.assertIn("does not track the id", str(cm.exception)) + + def test_the_name_stays_with_the_slot_not_with_the_body(self): + p, a, b, ares, bres = self.trees(self.A_NEW, self.B_NEW) + rl = sc.apply_relabelling(ares.tree, sc.build_relabelling(p, a, b)) + got = {n.value: sc._fleet_name(f) for n, f in sc._fleet_slots(ares.tree)} + self.assertEqual(got[1970], "Sigma Fleet") + self.assertEqual(got[1986], "Tau Fleet") + + # -- G5: the rewrite must be complete ----------------------------------- + + def test_an_unmodelled_reference_site_is_refused(self): + """A leaf holding a permuted id at a tag this tool does not model would + keep the old label through the rewrite, so two different states could + compare equal. That is a refusal, not a warning.""" + p, a, b, _, _ = self.trees(self.A_NEW, self.B_NEW, + extra=("SomeOtherRef", 1970)) + with self.assertRaises(sc.RelabelRefused) as cm: + sc.build_relabelling(p, a, b) + self.assertIn("does not model", str(cm.exception)) + self.assertIn("SomeOtherRef", str(cm.exception)) + + def test_the_completeness_scan_matches_raw_bytes_not_the_typed_value(self): + """The digest is a function of (bytes, reader schema); a leaf the schema + types as a float still carries the id. A value-based scan would miss + it and the guard would be silently empty (rule 1).""" + node = sr.Node("Odd", "float", 2.76e-42, 0, 12, + raw=struct.pack(" 0) + self.assertEqual(chk[0].b - chk[0].a, + -16 if direction == (4, 0) else 16) def test_resave_mask_makes_that_pair_identical(self): found = False @@ -439,6 +721,69 @@ class RealSaveTest(unittest.TestCase): self.skipTest("no re-save pair among the available saves") self.assertTrue(found) + # -- --relabel-new-ids acceptance, both halves ------------------------- + + def _saves(self, *names): + d = os.path.dirname(REAL[0]) + paths = [os.path.join(d, n) for n in names] + if not all(os.path.exists(p) for p in paths): + self.skipTest(f"need {names}") + return paths + + def _relabelled(self, pre, a, b): + return sc._relabelled_pair(pre, a, b, floats="bits", mask="none", audit=False) + + def test_POSITIVE_bp_pinned_pair_is_identical_modulo_one_transposition(self): + """Lane BP's two processes with identical pinned AI seeds differ in 35 + of 61,147 leaves; all 35 are fleets 1970 and 1986 exchanging contents. + Under pi they must read IDENTICAL, and pi must be exactly that swap.""" + pre, a, b = self._saves("bp-turn28-pre.sav", "bp-pinA-turn28.sav", + "bp-pinB-turn28.sav") + ca, cb, rl, refusal = self._relabelled(pre, a, b) + self.assertIsNone(refusal) + self.assertEqual(rl.moved, {1970: 1986, 1986: 1970}) + self.assertEqual(rl.new_ids, [1970, 1986, 2002]) + self.assertEqual(rl.nodes, [2]) + self.assertEqual(sc.diff(ca.root, cb.root), []) + self.assertEqual(ca.digest, cb.digest) + + def test_POSITIVE_the_same_pair_still_diverges_without_the_flag(self): + pre, a, b = self._saves("bp-turn28-pre.sav", "bp-pinA-turn28.sav", + "bp-pinB-turn28.sav") + d = sc.diff(real_ck(a, audit=False).root, real_ck(b, audit=False).root) + self.assertEqual(len(d), 35, [repr(e) for e in d]) + + def test_NEGATIVE_ad_unpinned_pair_is_refused_and_still_diverges(self): + """Lane AD's pair differs because the seeds differ -- real decisions. + A tool that made this look identical would be worthless.""" + pre, a, b = self._saves("ad-turn27-two-raiders.sav", "ad-oracle-A-post.sav", + "ad-oracle-B-post.sav") + ca, cb, rl, refusal = self._relabelled(pre, a, b) + self.assertIsNone(rl, "AD's pair must not be relabelled") + self.assertIn("2002", refusal) + self.assertNotEqual(ca.digest, cb.digest) + d = sc.diff(ca.root, cb.root, limit=500) + self.assertEqual(len(d), 94, [repr(e) for e in d]) + + def test_NEGATIVE_even_forcing_a_partial_relabelling_cannot_hide_it(self): + """Belt and braces: hand the applier the transposition it *would* have + found if G3 were absent. The pair must still diverge -- the guard is + not the only thing standing between AD's pair and a green verdict.""" + pre, a, b = self._saves("ad-turn27-two-raiders.sav", "ad-oracle-A-post.sav", + "ad-oracle-B-post.sav") + with open(a, "rb") as f: + ares = sr.read_bytes(f.read()) + with open(b, "rb") as f: + bres = sr.read_bytes(f.read()) + forced = sc.Relabelling({1970: 1986, 1986: 1970}, [2], [1970, 1986], + {1970: None, 1986: None}) + sc.apply_relabelling(ares.tree, forced) + ca = sc.checksum_result(ares, a, {"ok": None}, + extra_masks=sc.RELABEL_MASK, id_lists_as_sets=True) + cb = sc.checksum_result(bres, b, {"ok": None}, + extra_masks=sc.RELABEL_MASK, id_lists_as_sets=True) + self.assertNotEqual(ca.digest, cb.digest) + def test_a_real_turn_transition_localises_to_named_objects(self): by_turn = {} for p in REAL: