Compare commits
No commits in common. "f1306713d15596aadc83b2bc7e174e2058aba06a" and "2338ad40dded8d8ce0708db5abc0e64b4bda9388" have entirely different histories.
f1306713d1
...
2338ad40dd
6 changed files with 44 additions and 1321 deletions
|
|
@ -1,16 +1,16 @@
|
||||||
# SotS RE campaign — coverage dashboard
|
# SotS RE campaign — coverage dashboard
|
||||||
|
|
||||||
Generated 2026-09-09 08:13 UTC · `sots-re` @ d46b3f8,2026-09-09 · `sots-engine` @ e7e2bd6,2026-09-09 (245 commits) · regenerate with `tools/dashboard.py`
|
Generated 2026-09-09 07:41 UTC · `sots-re` @ d6f725a,2026-09-09 · `sots-engine` @ 45b16ef,2026-09-09 (244 commits) · regenerate with `tools/dashboard.py`
|
||||||
|
|
||||||
> **North star:** A functional reimplementation of the engine — behavior-equivalent, NOT byte-for-byte
|
> **North star:** A functional reimplementation of the engine — behavior-equivalent, NOT byte-for-byte
|
||||||
|
|
||||||
## 1. Map coverage (campaign/board.md)
|
## 1. Map coverage (campaign/board.md)
|
||||||
|
|
||||||
444 targets · mapped-or-better **396/444** `[█████████░] 89%` · verified **354/444** `[████████░░] 80%`
|
443 targets · mapped-or-better **395/443** `[█████████░] 89%` · verified **353/443** `[████████░░] 80%`
|
||||||
|
|
||||||
| Status | Count | % |
|
| Status | Count | % |
|
||||||
|---|---:|---:|
|
|---|---:|---:|
|
||||||
| verified | 354 | 80% |
|
| verified | 353 | 80% |
|
||||||
| mapped | 42 | 9% |
|
| mapped | 42 | 9% |
|
||||||
| in-progress | 5 | 1% |
|
| in-progress | 5 | 1% |
|
||||||
| backlog | 41 | 9% |
|
| backlog | 41 | 9% |
|
||||||
|
|
@ -25,7 +25,7 @@ Generated 2026-09-09 08:13 UTC · `sots-re` @ d46b3f8,2026-09-09 · `sots-engine
|
||||||
| verify | 105 | 15 | 3 | 35 | 0 | 158 |
|
| verify | 105 | 15 | 3 | 35 | 0 | 158 |
|
||||||
| phase2 | 13 | 3 | 1 | 0 | 0 | 17 |
|
| phase2 | 13 | 3 | 1 | 0 | 0 | 17 |
|
||||||
| meta | 93 | 6 | 0 | 1 | 0 | 100 |
|
| meta | 93 | 6 | 0 | 1 | 0 | 100 |
|
||||||
| other | 20 | 2 | 0 | 0 | 0 | 22 |
|
| other | 19 | 2 | 0 | 0 | 0 | 21 |
|
||||||
|
|
||||||
## 2. Binary understanding
|
## 2. Binary understanding
|
||||||
|
|
||||||
|
|
@ -47,7 +47,7 @@ Generated 2026-09-09 08:13 UTC · `sots-re` @ d46b3f8,2026-09-09 · `sots-engine
|
||||||
| Module | LOC | Test files | Checks | Wired | Doc |
|
| Module | LOC | Test files | Checks | Wired | Doc |
|
||||||
|---|---:|---:|---:|---|---|
|
|---|---:|---:|---:|---|---|
|
||||||
| `app` | 4,838 | 9 | 207 | yes | N-output-term.md |
|
| `app` | 4,838 | 9 | 207 | yes | N-output-term.md |
|
||||||
| `game/ai` | 2,138 | 8 | 322 | yes | L4-ai-orders.md |
|
| `game/ai` | 2,128 | 8 | 322 | yes | L4-ai-orders.md |
|
||||||
| `game/combat` | 582 | 2 | 34 | yes | L2-predictions.md |
|
| `game/combat` | 582 | 2 | 34 | yes | L2-predictions.md |
|
||||||
| `game/config` | 558 | 4 | 131 | yes | M1.md |
|
| `game/config` | 558 | 4 | 131 | yes | M1.md |
|
||||||
| `game/data` | 2,077 | 12 | 462 | yes | game-data.md |
|
| `game/data` | 2,077 | 12 | 462 | yes | game-data.md |
|
||||||
|
|
@ -64,7 +64,7 @@ Generated 2026-09-09 08:13 UTC · `sots-re` @ d46b3f8,2026-09-09 · `sots-engine
|
||||||
| `shim` | 16,709 | 0 | 0 | direct (WIN32) | H-probes.md |
|
| `shim` | 16,709 | 0 | 0 | direct (WIN32) | H-probes.md |
|
||||||
| `shim/hooks` | 13,593 | 0 | 0 | direct (WIN32) | L1-predictions.md |
|
| `shim/hooks` | 13,593 | 0 | 0 | direct (WIN32) | L1-predictions.md |
|
||||||
| `shim/trace` | 2,258 | 9 | 273 | direct (WIN32) | shim-trace.md |
|
| `shim/trace` | 2,258 | 9 | 273 | direct (WIN32) | shim-trace.md |
|
||||||
| **total** | **59,514** | **122** | **4257** | | |
|
| **total** | **59,504** | **122** | **4257** | | |
|
||||||
|
|
||||||
Board `engine:` rows: verified **32**, mapped 0, in flight 0 (of 32) — verified & merged `[██████████] 100%`
|
Board `engine:` rows: verified **32**, mapped 0, in flight 0 (of 32) — verified & merged `[██████████] 100%`
|
||||||
|
|
||||||
|
|
@ -133,11 +133,11 @@ Most recent open:
|
||||||
|
|
||||||
## 9. Delta since previous dashboard
|
## 9. Delta since previous dashboard
|
||||||
|
|
||||||
- verified targets: 353 → 354 (+1) · mapped-or-better: 395 → 396 (+1)
|
- verified targets: 352 → 353 (+1) · mapped-or-better: 394 → 395 (+1)
|
||||||
- engine LOC: 59,504 → 59,514 (+10) · test files: 122 → 122 (+0) · checks: 4,257 → 4,257 (+0)
|
- engine LOC: 58,647 → 59,504 (+857) · test files: 122 → 122 (+0) · checks: 4,257 → 4,257 (+0)
|
||||||
- addresses verified: 1,163 → 1,163 (+0) · recovered layouts: 384 → 384 (+0) · open questions: 26 → 26 (+0)
|
- addresses verified: 1,163 → 1,163 (+0) · recovered layouts: 384 → 384 (+0) · open questions: 26 → 26 (+0)
|
||||||
- standalone leaves closed: 45 → 45 (+0) · leaves still diverging: 63 → 63 (+0)
|
- standalone leaves closed: 45 → 45 (+0) · leaves still diverging: 63 → 63 (+0)
|
||||||
|
|
||||||
---
|
---
|
||||||
warnings: board.md: unknown types subsystems, tooling; mars-rng.md: no oracle total row parsed; mars-stream.md: no oracle total row parsed; mars-vfs.md: no oracle total row parsed
|
warnings: board.md: unknown types subsystems; mars-rng.md: no oracle total row parsed; mars-stream.md: no oracle total row parsed; mars-vfs.md: no oracle total row parsed
|
||||||
<!-- dashboard-metrics {"verified": 354, "mapped_plus": 396, "targets": 444, "loc": 59514, "tests": 122, "checks": 4257, "addr_verified": 1163, "addr_total": 1294, "layouts": 384, "open_q": 26, "sa_closed": 45, "sa_left": 63} -->
|
<!-- dashboard-metrics {"verified": 353, "mapped_plus": 395, "targets": 443, "loc": 59504, "tests": 122, "checks": 4257, "addr_verified": 1163, "addr_total": 1294, "layouts": 384, "open_q": 26, "sa_closed": 45, "sa_left": 63} -->
|
||||||
|
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -1,374 +0,0 @@
|
||||||
# `state_checksum --relabel-new-ids`: comparing two post-turn saves modulo this turn's new id labelling
|
|
||||||
|
|
||||||
Lane BT, 2026-09-09. **Host only — no VM was touched.** VM140 (reference guest) untouched; VM141,
|
|
||||||
VM144 and VM146 were held by lanes AZ, BS and BR throughout and were not approached.
|
|
||||||
|
|
||||||
Implements `findings/resolutions/2026-09-09-fleet-id-order-residue.md` §3 item 1, verbatim as the
|
|
||||||
specification and with the two engine-side items (§3.2 canonical visit order, §3.3 the
|
|
||||||
`--ai-visit-order` override) deliberately **not** started — they belong in `sots-engine` and the
|
|
||||||
resolver ranked them after this one.
|
|
||||||
|
|
||||||
Consumes: `findings/control-flow/pinned-seed-rich-turn.md` (lane BP — the acceptance data),
|
|
||||||
`findings/subsystems/id-allocation.md` (the id scheme), `findings/control-flow/raid-gate-multiplicity.md`
|
|
||||||
(lane AD — the negative acceptance data).
|
|
||||||
|
|
||||||
Touches only `verify/state-checksum/*` and this file.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 0. Result
|
|
||||||
|
|
||||||
Both halves of the acceptance pass, and the second half passes **because a guard fires**, not
|
|
||||||
because the matcher happened to find nothing.
|
|
||||||
|
|
||||||
| half | pair | verdict |
|
|
||||||
|---|---|---|
|
|
||||||
| **positive** | `bp-pinA-turn28` vs `bp-pinB-turn28`, pre `bp-turn28-pre` | `IDENTICAL modulo pi = {1970<->1986}` — 35 leaves → **0**, and π is exactly the one transposition |
|
|
||||||
| **negative** | `ad-oracle-A-post` vs `ad-oracle-B-post`, pre `ad-turn27-two-raiders` | `REFUSED` (the two runs minted different id **sets**), then `DIVERGED: 94 leaf difference(s)` — byte-for-byte the same 94 the tool reports without the flag |
|
|
||||||
|
|
||||||
Existing behaviour is unchanged when the flag is absent: over every save in
|
|
||||||
`verify/results/saves/` under both `--floats bits --mask none` and `--floats canonical --mask
|
|
||||||
resave`, the pre-change and post-change modules produce identical root digests **and identical
|
|
||||||
per-node digests at every node of the tree** (§5.1).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. What was built
|
|
||||||
|
|
||||||
`verify/state-checksum/state_checksum.py` gains one flag:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
state_checksum.py POST-A.sav POST-B.sav --relabel-new-ids PRE-TURN.sav
|
|
||||||
```
|
|
||||||
|
|
||||||
The operation, in the order the code does it:
|
|
||||||
|
|
||||||
1. Parse all three saves. **Prove coverage on the unmodified parse** (`audit_coverage` runs before
|
|
||||||
any rewrite), so `coverage: PROVED` still means what it has always meant: every byte of the
|
|
||||||
file is a function of the digest's inputs.
|
|
||||||
2. `new(X) = { fleet ids in X } \ { fleet ids in PRE }`, restricted to **node nibble ≠ 0** — the
|
|
||||||
client-minted space (`id-allocation.md` §1: `id = (counter << 4) | node`, node 0 is the sim
|
|
||||||
that owns the board, node `PlyrIdx + 1` is a player's own client).
|
|
||||||
3. Key each new fleet by a content key that does not mention its id:
|
|
||||||
**`(LocID if non-zero else FPlan/pnd destination, sorted ship-id set)`**, with the two
|
|
||||||
location kinds tagged apart so a fleet *standing at* system 384 never keys the same as one
|
|
||||||
*bound for* 384.
|
|
||||||
4. Build **π** from the key correspondence, one node at a time.
|
|
||||||
5. **Exchange the matched fleets' bodies between their slots** (see §4.2 — this is not what the
|
|
||||||
spec said, and the spec was wrong), keeping each slot's id and its id-attached label `FtName`.
|
|
||||||
6. Rewrite every *reference* to a permuted id under π: a ship's `FltID` back-reference and a
|
|
||||||
system's `Flt` entry.
|
|
||||||
7. Compare `ShipIDs[]` / `FleetIDs[]` / `DesignIDs[]` as **sets**.
|
|
||||||
8. Mask `/Summary/Checksum`, **with the reason printed on its own line**.
|
|
||||||
9. Print π and the verdict.
|
|
||||||
|
|
||||||
New public surface: `RelabelRefused`, `Relabelling`, `build_relabelling`, `apply_relabelling`,
|
|
||||||
`format_permutation`, `id_node`, and `checksum_result` (the parse/digest split that lets the mode
|
|
||||||
parse once, audit, then rewrite).
|
|
||||||
|
|
||||||
The relabelled root is **domain-separated** — `relabel-new-ids` is folded into the root preimage —
|
|
||||||
so a "modulo π" root can never be mistaken for a strict one. The extra preimage word is appended
|
|
||||||
*only* in this mode, so every root recorded before the flag existed is bit-for-bit unchanged.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Acceptance, positive half — verbatim
|
|
||||||
|
|
||||||
```
|
|
||||||
$ uv run python3 verify/state-checksum/state_checksum.py \
|
|
||||||
verify/results/saves/bp-pinA-turn28.sav verify/results/saves/bp-pinB-turn28.sav \
|
|
||||||
--floats bits --mask none \
|
|
||||||
--relabel-new-ids verify/results/saves/bp-turn28-pre.sav
|
|
||||||
|
|
||||||
A a83c425215b6d7e88bc998f4948081e7 verify/results/saves/bp-pinA-turn28.sav
|
|
||||||
B a83c425215b6d7e88bc998f4948081e7 verify/results/saves/bp-pinB-turn28.sav
|
|
||||||
policy: floats=bits mask=none reader=74fd562c75e2679e
|
|
||||||
relabel: pre-turn save verify/results/saves/bp-turn28-pre.sav
|
|
||||||
relabel: pi = {1970<->1986}
|
|
||||||
relabel: 1970 (counter 123, node 2) -> 1986 at 384, ships [6976]
|
|
||||||
relabel: 1986 (counter 124, node 2) -> 1970 bound-for 80, ships [5264]
|
|
||||||
relabel: 2002 (counter 125, node 2) -> 2002 bound-for 384, ships [6992]
|
|
||||||
relabel: 3 new client-minted fleet id(s) on node(s) [2]; 2 relabelled, 2 fleet body/bodies exchanged, Fltx1, FltIDx2 rewritten
|
|
||||||
relabel: /Summary/Checksum is MASKED: it is derived from the whole state and its inputs are unmodelled (rule 18's open item), so it cannot be recomputed under a relabelling and its move is not independent evidence. Unmask by dropping --relabel-new-ids.
|
|
||||||
relabel: Master id lists (ShipIDs/FleetIDs/DesignIDs) are compared as SETS: their element order is the order the turn's commands were applied in, which is the same visit-order residue. Membership is still exact -- an added or removed id still reports.
|
|
||||||
IDENTICAL modulo pi = {1970<->1986}
|
|
||||||
$ echo $?
|
|
||||||
0
|
|
||||||
```
|
|
||||||
|
|
||||||
Read the per-id lines: the counters printed are the decode the resolver did by hand
|
|
||||||
(`123/124/125`, node 2 = Player[32]'s client), and the content keys are what carries the match —
|
|
||||||
the group **at system 384 with ship 6976** is fleet 1970 in A and fleet 1986 in B; the group
|
|
||||||
**bound for system 80 with ship 5264** is 1986 in A and 1970 in B; the group bound for 384 is 2002
|
|
||||||
in both. Exactly the resolver's §2.1 reading, recovered from the saves alone.
|
|
||||||
|
|
||||||
Without the flag the same pair is `DIVERGED: 35 leaf difference(s)` (regression-locked by
|
|
||||||
`test_POSITIVE_the_same_pair_still_diverges_without_the_flag`, which asserts the count is 35). So
|
|
||||||
the mode removes **35 leaves and no others**: `IDENTICAL modulo one transposition and nothing else`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Acceptance, negative half — verbatim, and why it is the right kind of failure
|
|
||||||
|
|
||||||
```
|
|
||||||
$ uv run python3 verify/state-checksum/state_checksum.py \
|
|
||||||
verify/results/saves/ad-oracle-A-post.sav verify/results/saves/ad-oracle-B-post.sav \
|
|
||||||
--floats bits --mask none \
|
|
||||||
--relabel-new-ids verify/results/saves/ad-turn27-two-raiders.sav
|
|
||||||
|
|
||||||
A 1faf04cad2bbf2d7e5ecf560b8807504 verify/results/saves/ad-oracle-A-post.sav
|
|
||||||
B 53043168ad03cae13355db9f5f381b50 verify/results/saves/ad-oracle-B-post.sav
|
|
||||||
policy: floats=bits mask=none reader=74fd562c75e2679e
|
|
||||||
relabel: pre-turn save verify/results/saves/ad-turn27-two-raiders.sav
|
|
||||||
relabel: REFUSED -- the new client-minted fleet ids differ between the two saves -- A minted [1970, 1986], B minted [1970, 1986, 2002]; only in B [2002]. That is a difference in what was allocated, not in how it was labelled, so no relabelling is defensible
|
|
||||||
relabel: NO relabelling was applied; the comparison below is the ordinary one, and nothing has been absorbed.
|
|
||||||
DIVERGED: 94 leaf difference(s)
|
|
||||||
/Summary/Checksum: -1043321805 -> 1995746704
|
|
||||||
/Sim/DesignIDs[]: removed [2002], added [2018] (85 -> 85 entries)
|
|
||||||
/Sim/FleetIDs[]: removed [], added [2002] (58 -> 59 entries)
|
|
||||||
/Sim/ShipIDs[]: list-reordered
|
|
||||||
/Sim/ModCount: 1496 -> 1500
|
|
||||||
... 89 more ...
|
|
||||||
$ echo $?
|
|
||||||
1
|
|
||||||
```
|
|
||||||
|
|
||||||
94 is the number the tool reports for that pair **without** the flag, unchanged. On a refusal
|
|
||||||
nothing is rewritten, nothing is masked and no list is set-compared: the refusal path *is* the
|
|
||||||
ordinary comparison. That is deliberate — a mask applied on a run whose relabelling was refused
|
|
||||||
would be a mask nobody asked for.
|
|
||||||
|
|
||||||
**The refusal is by cardinality, not by content.** AD's run A minted **two** client fleets and run
|
|
||||||
B minted **three**. The guard that fires is G3 (π must permute one set), and it fires before the
|
|
||||||
content matcher is ever consulted. That matters, because it means AD's pair does **not** exercise
|
|
||||||
the content key negatively. I exercised that separately, two ways (§4.3, §5.3): a synthetic pair
|
|
||||||
whose new ids agree and whose contents differ, and — the belt-and-braces version — **forcing the
|
|
||||||
transposition onto AD's pair with the guard bypassed**, which still leaves **82 differing
|
|
||||||
leaves**, including `DesignIDs[]`, `ModCount`, four `turnstats` rows, both AI build queues and
|
|
||||||
`Player[32]`'s money. The guard is not the only thing standing between AD's pair and a green
|
|
||||||
verdict; it is the first of several.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Where the specification did not survive contact
|
|
||||||
|
|
||||||
Four places. The first two changed the implementation; the third and fourth are scope notes the
|
|
||||||
next lane needs.
|
|
||||||
|
|
||||||
(Lane BU landed mid-lane, confirming H1 — the visit order is an address-keyed container whose key
|
|
||||||
is the fleet's `Location*`. That settles the *cause* of the residue and changes nothing here: the
|
|
||||||
comparison side quotients the residue out regardless of what produces it, and it must go on
|
|
||||||
refusing when the difference is not a relabelling. The forward notes in §4.1 and §4.2 are
|
|
||||||
addressed to the engine-side canonical order and stand as written.)
|
|
||||||
|
|
||||||
### 4.1 `FtName` is an **id-attached label**, and "every id-valued leaf" does not reach it
|
|
||||||
|
|
||||||
In *both* of BP's saves, fleet **1970 is "Sigma Fleet VI"** and **1986 is "Tau Fleet VI"**. The
|
|
||||||
name follows the **id**, not the ship group. It is a Greek-letter ordinal minted from a per-player
|
|
||||||
counter that advances with the same pass that mints the id, so it is exactly as much a function of
|
|
||||||
the visit order as the id is — and exactly as little a decision.
|
|
||||||
|
|
||||||
Consequence: applying π only to id-valued leaves would move the group at system 384 from slot 1970
|
|
||||||
to slot 1986 **carrying the name "Sigma Fleet VI"**, and compare it against B's 1986 "Tau Fleet
|
|
||||||
VI". Two spurious leaf differences, and the acceptance "and nothing else" would have failed.
|
|
||||||
|
|
||||||
Handled by keeping the name with the slot rather than with the body, and **gating it**: the mode
|
|
||||||
refuses unless `FtName` is the same at every matched id in both saves. If the name does not track
|
|
||||||
the id, something real differs and the tool must not paper over it
|
|
||||||
(`test_a_name_that_does_not_track_the_id_is_refused`).
|
|
||||||
|
|
||||||
Cross-check that this is a naming *counter* and not a coincidence, from AD's pair: A minted two
|
|
||||||
client fleets, so A's node-0 fleet **7120 is "Upsilon Fleet VI"**; B minted three, so B's **7120 is
|
|
||||||
"Phi Fleet VI"**. The Greek sequence is **per player and shared between the client's and the
|
|
||||||
server's fleets** — one more thing an engine-side canonical visit order (§3.2 of the resolution)
|
|
||||||
has to reproduce, and one the resolver's item 2 does not currently mention.
|
|
||||||
|
|
||||||
### 4.2 Applying π to the `Flt[]` **keys** introduces a spurious `reordered`
|
|
||||||
|
|
||||||
The master fleet table's element order, and the `FleetIDs[]` list, are **identical in A and B**
|
|
||||||
(measured: both are `[…, 6960, 1970, 1986, 2002, 7120, …]`) and are a function of the id, not of
|
|
||||||
the visit order. Renumber A's slots under π and A's table reads `…1986, 1970, 2002…` against B's
|
|
||||||
`…1970, 1986, 2002…`, which the differ correctly reports as a reorder — an artefact of the tool,
|
|
||||||
not of the saves.
|
|
||||||
|
|
||||||
The equivalent operation with no artefact is to **exchange the fleet bodies between the slots**,
|
|
||||||
leaving slot ids, `FleetIDs[]`, and every object label exactly as the file wrote them. That is what
|
|
||||||
the implementation does. It is the same quotient, taken from the other side.
|
|
||||||
|
|
||||||
Worth carrying into the engine-side item: **the fleet table is id-ordered, not visit-ordered**, so
|
|
||||||
canonicalising the visit order in `game/ai` will reorder lists 8/10 and the apply order, and will
|
|
||||||
*not* reorder the save's fleet table.
|
|
||||||
|
|
||||||
### 4.3 "new ids of the same node" is not a fleet-only id space
|
|
||||||
|
|
||||||
`ad-oracle-A-post` carries **2002 as a `DesignID`**; `ad-oracle-B-post` carries **2002 as a
|
|
||||||
`FltID`**. The per-node counter is shared across object kinds, so a node's new-id set spans fleets,
|
|
||||||
designs and (in principle) ships. This mode matches **fleets only**, as the spec says. What keeps
|
|
||||||
that safe is guard G5 (§5.2): any leaf anywhere holding a permuted id at a site the tool does not
|
|
||||||
model as a fleet reference is a **refusal**. Anyone extending the mode to designs must read this
|
|
||||||
paragraph first — the two spaces interleave.
|
|
||||||
|
|
||||||
### 4.4 "list-8/10/14 words when comparing command blocks" is **not** implemented, and does not belong here
|
|
||||||
|
|
||||||
Lists 8/10/14 live in the `aiorders` **shim dump**, not in a `.sav`. `state_checksum.py` parses
|
|
||||||
saves; it has no reader for the block. Wiring π into a block comparison belongs wherever those
|
|
||||||
dumps are differed (lane BP did it by hand under its proven noise mask, `pinned-seed-rich-turn.md`
|
|
||||||
§2). Recorded as not-done with the reason rather than silently dropped.
|
|
||||||
|
|
||||||
### 4.5 Two smaller notes
|
|
||||||
|
|
||||||
- **Master id lists as sets is unconditional in this mode**, including when π is the identity.
|
|
||||||
`ShipIDs[]`'s element order is the apply order, which is the same residue; membership stays
|
|
||||||
exact, so an added or removed id still reports. It is nonetheless a real weakening and it is
|
|
||||||
printed on every run. A lane that wants that order back should reach for a separate flag, not
|
|
||||||
for this one.
|
|
||||||
- **The `--mask resave` composition the resolver asked for works**: `--mask resave
|
|
||||||
--relabel-new-ids …` on BP's pair reports `[masked: Checksumx1, Statusx8]` and
|
|
||||||
`IDENTICAL modulo pi = {1970<->1986}`. The relabel mask is added to whichever preset is in
|
|
||||||
effect and de-duplicated against it.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. The guards, and how each was made to fire
|
|
||||||
|
|
||||||
Five guards. **Every one refuses; none degrades to a partial relabelling.** A refusal prints its
|
|
||||||
reason, applies nothing, and runs the ordinary comparison — the conservative direction, because a
|
|
||||||
real divergence then still reports.
|
|
||||||
|
|
||||||
| | guard | how it was made to fire |
|
|
||||||
|---|---|---|
|
|
||||||
| **G1** | only ids **absent from the pre-turn save** may be relabelled | `test_ids_present_in_the_pre_turn_save_are_never_in_pi`; and the strong form, `test_two_pre_turn_fleets_that_swapped_contents_are_NOT_absorbed` — two *pre-existing* fleets exchanging contents is the very shape this mode absorbs, and it must still report |
|
|
||||||
| **G2** | only ids with a **non-zero node nibble** | `test_server_minted_ids_are_never_relabelled` — two node-0 (server) fleets exchanging contents still diverges |
|
|
||||||
| **G3** | π must be a **permutation of one set** — so the two saves must have minted the same ids | `test_different_new_id_sets_are_refused` (synthetic: one run mints a third fleet) and **lane AD's real pair** (§3) |
|
|
||||||
| **G4** | the content keys must **correspond one-to-one**, and be **unique on each side** | `test_new_fleets_whose_contents_differ_are_refused`; `test_an_ambiguous_content_key_is_refused` (two new fleets with the same location and ship set) |
|
|
||||||
| **G5** | **no leaf anywhere** in either save may hold a permuted id at a site the tool does not model | `test_an_unmodelled_reference_site_is_refused` (a `SomeOtherRef` leaf holding 1970); `test_the_completeness_scan_matches_raw_bytes_not_the_typed_value` |
|
|
||||||
| — | `FtName` must track the id in both saves | `test_a_name_that_does_not_track_the_id_is_refused` (§4.1) |
|
|
||||||
| — | a refusal leaves both parse trees **untouched** | `test_a_refusal_leaves_both_trees_untouched` — the digest of A before and after a refused `build_relabelling` is identical |
|
|
||||||
|
|
||||||
### 5.1 The default path is unchanged
|
|
||||||
|
|
||||||
Checked two ways, because "the tool is used by several lanes and by the determinism oracle":
|
|
||||||
|
|
||||||
- **Digest equivalence, whole corpus.** The pre-change module (`git show HEAD:…`) and the
|
|
||||||
post-change module were run over all **43** `.sav` files in `verify/results/saves/` under
|
|
||||||
`--floats bits --mask none` and under `--floats canonical --mask resave`, comparing the root
|
|
||||||
digest, the coverage record, the mask hits, **and the `(path, digest)` of every node in the
|
|
||||||
tree**. `saves checked: 43 mismatches: 0`.
|
|
||||||
- **CLI stdout equivalence, every mode.** Single-save; `--tree --depth 3`; `--json`; the BP pair;
|
|
||||||
the AD pair at `--limit 500`; `turn2/turn3 --mask resave --ulps 2`; `--record-chain` and
|
|
||||||
`--chain` over turns 1/2/3. **5,602 lines, byte-identical** between the two modules (only the
|
|
||||||
chain file's own path normalised).
|
|
||||||
|
|
||||||
The one change that *could* have moved a recorded root — folding a policy word into the root
|
|
||||||
preimage — is appended **only** when the new mode is active. An intermediate version of this work
|
|
||||||
appended an empty word unconditionally, which moved the root; that was caught by the equivalence
|
|
||||||
check above, which is the reason it exists.
|
|
||||||
|
|
||||||
### 5.2 G5 matches raw bytes, not the reader's typed value — on purpose
|
|
||||||
|
|
||||||
The digest is a function of *(save bytes, reader schema)*, never of the bytes alone (STATE_CHECKSUM.md
|
|
||||||
§2.4). A leaf the schema happens to type as a float or a raw blob still carries the id, and a
|
|
||||||
value-based completeness scan would not see it — a guard a typing slip can silently empty is the
|
|
||||||
"green verdict over an empty region set" of earned rule 1. This is not hypothetical: while
|
|
||||||
building the synthetic fixture, the reader typed a `Flt` reference as a **float** (`2.76e-42`, the
|
|
||||||
denormal whose bit pattern is 1970) and a value-based scan missed it. Matching on the raw four
|
|
||||||
bytes catches it. The cost is that a genuine 4-byte float in the denormal range around a permuted
|
|
||||||
id would refuse — a refusal is the safe direction, and no such value exists in the corpus.
|
|
||||||
|
|
||||||
### 5.3 The strongest negative: bypass the guard and it still diverges
|
|
||||||
|
|
||||||
`test_NEGATIVE_even_forcing_a_partial_relabelling_cannot_hide_it` hands `apply_relabelling` the
|
|
||||||
transposition `{1970↔1986}` it *would* have built if G3 did not exist, applies it to AD's A save,
|
|
||||||
and compares under the full "modulo π" policy (checksum masked, master lists as sets). **82 leaves
|
|
||||||
still differ.** The guards are the tool's honesty, not its only defence.
|
|
||||||
|
|
||||||
### 5.4 Tests
|
|
||||||
|
|
||||||
24 new tests (20 synthetic in `RelabelTest`, 4 real-save in `RealSaveTest`), **7 of which assert a
|
|
||||||
refusal**. The whole suite is 38 → **62 tests, all passing**:
|
|
||||||
|
|
||||||
```
|
|
||||||
$ uv run python3 -m unittest test_state_checksum.RelabelTest # 20 tests, OK, 0.07 s
|
|
||||||
$ uv run python3 -m unittest test_state_checksum.RealSaveTest.test_POSITIVE_… \
|
|
||||||
…test_NEGATIVE_… # 4 tests, OK, 88.7 s
|
|
||||||
$ uv run python3 -m unittest discover -s . -t . # 62 tests, OK
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5.5 One pre-existing test failure, fixed, that is not about this change
|
|
||||||
|
|
||||||
The first full-suite run failed one test —
|
|
||||||
`test_known_resave_delta_localises_to_five_named_leaves` — and **the pre-change module fails it
|
|
||||||
identically**, so it is not a regression. The test enumerates candidate pairs over *sorted
|
|
||||||
filenames* and asserted the re-save direction as `4 -> 0`; `cb-turn2to3-endturn.sav`, added to the
|
|
||||||
corpus by an earlier lane, sorts before `turn2-state.sav`, so that pair reads `0 -> 4`. The claim
|
|
||||||
is symmetric and the assertion now is too (all four `Status` leaves must still move the *same*
|
|
||||||
way, with `Checksum` following by ∓16). Recorded rather than quietly patched, per earned rule 11.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. What this lane did NOT do
|
|
||||||
|
|
||||||
1. **No VM, no game, no build.** Host only. The three held guests were not approached.
|
|
||||||
2. **Neither engine-side deliverable** (`sots-engine` `game/ai` canonical visit order; the
|
|
||||||
`--ai-visit-order` override). Out of scope by the brief, and ranked after this one by the
|
|
||||||
resolver.
|
|
||||||
3. **No command-block comparison** (§4.4).
|
|
||||||
4. **Designs and ships are not relabelled**, only fleets — per the spec. §4.3 says what has to be
|
|
||||||
read before that changes.
|
|
||||||
5. **`Summary/Checksum` is still unmodelled.** It is masked with its reason, not solved.
|
|
||||||
STATE_CHECKSUM.md §4.6 records the two derivations already ruled out; nobody should re-run them.
|
|
||||||
6. **π's *existence* is not evidence that the two runs agree on the turn.** It is evidence that
|
|
||||||
they agree *up to the labelling of the ids created on it*. The claim the mode licenses is
|
|
||||||
exactly the resolver's C-set shape and no more.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. Reproducing every number here
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cd ~/sots-re
|
|
||||||
S=verify/results/saves
|
|
||||||
# positive
|
|
||||||
uv run python3 verify/state-checksum/state_checksum.py \
|
|
||||||
$S/bp-pinA-turn28.sav $S/bp-pinB-turn28.sav --floats bits --mask none \
|
|
||||||
--relabel-new-ids $S/bp-turn28-pre.sav
|
|
||||||
# negative
|
|
||||||
uv run python3 verify/state-checksum/state_checksum.py \
|
|
||||||
$S/ad-oracle-A-post.sav $S/ad-oracle-B-post.sav --floats bits --mask none \
|
|
||||||
--relabel-new-ids $S/ad-turn27-two-raiders.sav
|
|
||||||
# guards
|
|
||||||
cd verify/state-checksum && uv run python3 -m unittest -v test_state_checksum.RelabelTest
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Proposed board rows
|
|
||||||
|
|
||||||
New row:
|
|
||||||
|
|
||||||
```
|
|
||||||
| STATE_CHECKSUM --relabel-new-ids: BP'S PINNED PAIR IS IDENTICAL MODULO ONE TRANSPOSITION, AD'S UNPINNED PAIR IS REFUSED AND STILL DIVERGES | tooling | verified | high | 100% | 2026-09-09 | Lane BT, HOST ONLY (no VM; 140 untouched, 141/144/146 held by AZ/BS/BR and not approached). Implements the 2026-09-09 fleet-id-order resolution §3 item 1 (`findings/subsystems/relabel-new-ids.md`). `state_checksum.py POST-A POST-B --relabel-new-ids PRE` computes the ids NEW in each post-turn save, restricts to node nibble != 0 (client-minted; `id-allocation.md` id=(counter<<4)|node), matches those fleets by a key that does not mention the id - (LocID if non-zero else FPlan/pnd destination, sorted ship-id set), with "at 384" and "bound-for 384" tagged apart - builds the bijection pi, rewrites every fleet reference, compares ShipIDs/FleetIDs/DesignIDs as SETS, masks /Summary/Checksum WITH ITS REASON ON THE LINE, and prints pi. **POSITIVE: `bp-pinA` vs `bp-pinB` = `IDENTICAL modulo pi = {1970<->1986}`, 35 leaves -> 0, exit 0**, with pi recovered from the saves alone and printed with the counters (123/124/125, node 2) and the content keys - the group AT Sys 384 with ship 6976 is 1970 in A and 1986 in B; the group BOUND FOR Sys 80 with ship 5264 is 1986 in A and 1970 in B; 2002's group is 2002 in both. **NEGATIVE: `ad-oracle-A/B-post` REFUSED and `DIVERGED: 94 leaf difference(s)`, exit 1** - byte-identically the 94 the tool reports without the flag, because on a refusal NOTHING is rewritten, masked or set-compared. The refusal is by CARDINALITY (A minted 2 client fleets, B minted 3), so AD's pair does not exercise the content matcher; that is exercised separately, and FORCING the transposition onto AD's pair with the guard bypassed STILL LEAVES 82 LEAVES. FIVE GUARDS, EVERY ONE REFUSES RATHER THAN DEGRADES: G1 only ids absent from the pre-turn save (tested with two PRE-EXISTING fleets exchanging contents - must still report); G2 only non-zero node nibbles (node-0 swap still reports); G3 pi must permute ONE set; G4 content keys correspond one-to-one and are unique per side; G5 NO leaf anywhere may hold a permuted id at an unmodelled site, matched on RAW BYTES not the reader's typed value (a value scan is empty the moment the schema types a reference as a float - this happened during development). Refusal prints its reason and falls back to the ORDINARY comparison, never to a partial relabelling. THREE SPECIFICATION CORRECTIONS FROM CONTACT WITH THE DATA: (1) **FtName is an ID-ATTACHED LABEL** - 1970 is "Sigma Fleet VI" and 1986 "Tau Fleet VI" in BOTH saves, a Greek ordinal minted by the same pass as the id, so "apply pi to every id-valued leaf" would have produced two spurious name diffs; the name stays with the SLOT, gated by a guard requiring it to track the id in both saves. The counter is PER PLAYER AND SHARED WITH THE SERVER'S FLEETS - AD's A (2 client fleets) names node-0 fleet 7120 "Upsilon", B (3) names it "Phi" - which the engine-side canonical visit order must reproduce and §3.2 does not mention. (2) **Applying pi to the Flt[] KEYS introduces a spurious `reordered`**: the fleet table and FleetIDs[] are ID-ordered and IDENTICAL in A and B, so renumbering slots reorders A against B; the artefact-free equivalent is to EXCHANGE THE FLEET BODIES between slots, leaving slot ids, FleetIDs[] and every label as written. (3) **A node's new-id set is NOT fleet-only**: 2002 is a DesignID in ad-oracle-A-post and a FltID in ad-oracle-B-post - the per-node counter is shared across object kinds, and G5 is what keeps a fleet-only matcher safe. NOT DONE, with reasons: lists 8/10/14 (they are aiorders SHIM DUMP words, not save leaves - this tool has no block reader); designs/ships; the two engine-side items. DEFAULT PATH PROVEN UNCHANGED: pre- and post-change modules compared over the whole save corpus under two policies on root digest, coverage, mask hits AND every (path,digest) in the tree - zero mismatches; plus byte-identical CLI stdout on the single-save, BP, AD and resave invocations. The relabelled root is DOMAIN-SEPARATED so a "modulo pi" root can never be read as a strict one. 24 new tests (20 synthetic + 4 real-save acceptance), 7 of them asserting a refusal; suite 38 -> 62, all passing. ALSO FIXED, PRE-EXISTING AND NOT A REGRESSION (the pre-change module fails it identically): `test_known_resave_delta_localises_to_five_named_leaves` enumerated pairs over SORTED FILENAMES and hard-coded the re-save direction 4->0; `cb-turn2to3-endturn.sav` sorts before `turn2-state.sav`, so that pair reads 0->4 and the test failed on a corpus fact. Assertion is now direction-symmetric (all four Status leaves must still move the SAME way, Checksum following by -/+16). Artefacts: `verify/state-checksum/state_checksum.py`, `test_state_checksum.py`, `STATE_CHECKSUM.md` §2.6 |
|
|
||||||
```
|
|
||||||
|
|
||||||
Edits to existing rows:
|
|
||||||
|
|
||||||
- **Row 424 / the resolver row of 2026-09-09** — append: `ENGINE ITEM 1 IS DONE (lane BT, host):
|
|
||||||
state_checksum --relabel-new-ids ships with both acceptance halves green - BP's pair IDENTICAL
|
|
||||||
modulo {1970<->1986}, AD's pair refused and still 94-leaf DIVERGED. Three corrections to §3.1
|
|
||||||
from the data: FtName is an id-attached label and needs the same treatment as the id (and its
|
|
||||||
counter is per-player and shared with the server's fleets, which §3.2's canonical visit order
|
|
||||||
must also reproduce); relabelling the Flt[] keys is the wrong operation - exchange the bodies,
|
|
||||||
because the fleet table is id-ordered and identical in both saves; and a node's new-id set spans
|
|
||||||
object kinds (2002 is a DesignID in ad-oracle-A-post and a FltID in ad-oracle-B-post). Lists
|
|
||||||
8/10/14 are not in scope for this tool - they are shim-dump words, not save leaves.`
|
|
||||||
- **Row 333 (C-exact)** — append: `Comparison side delivered 2026-09-09 (lane BT):
|
|
||||||
`state_checksum --relabel-new-ids <pre.sav>`, guarded so it refuses rather than absorbs; see
|
|
||||||
findings/subsystems/relabel-new-ids.md.`
|
|
||||||
- **`determinism-oracle.md`'s "Qualified" paragraph** — when it is written, "modulo π" now has an
|
|
||||||
executable definition and a printed π; cite `relabel-new-ids.md` §1 for the operation and §5 for
|
|
||||||
the guards.
|
|
||||||
- **`id-allocation.md` §5** — alongside the canonical-visit-order item the resolver asked for, add
|
|
||||||
the two measured facts from §4.1/§4.2 of this document: the **fleet-name ordinal** is per player
|
|
||||||
and shared between client- and server-minted fleets, and the **master fleet table is id-ordered**,
|
|
||||||
not visit-ordered.
|
|
||||||
- **Row 62** — VM140/141/144/146 unchanged by this lane; it took no guest.
|
|
||||||
|
|
@ -13,7 +13,7 @@ verify/state-checksum/
|
||||||
float_census.py classifies every float leaf (evidence for §3)
|
float_census.py classifies every float leaf (evidence for §3)
|
||||||
stability_check.py identical bytes -> identical roots, coverage proved
|
stability_check.py identical bytes -> identical roots, coverage proved
|
||||||
run_validation.sh regenerates verify/results/state-checksum/
|
run_validation.sh regenerates verify/results/state-checksum/
|
||||||
test_state_checksum.py 62 tests
|
test_state_checksum.py 38 tests
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
@ -150,46 +150,6 @@ A mask that matches nothing prints `[mask matched NOTHING -- check the rule path
|
||||||
nobody audits is a hiding place, and this project has already been bitten once by a comparison
|
nobody audits is a hiding place, and this project has already been bitten once by a comparison
|
||||||
that quietly covered nothing.
|
that quietly covered nothing.
|
||||||
|
|
||||||
### 2.6 `--relabel-new-ids` — comparing modulo the labelling of this turn's new ids
|
|
||||||
|
|
||||||
Added 2026-09-09 (lane BT) to the specification in
|
|
||||||
`findings/resolutions/2026-09-09-fleet-id-order-residue.md` §3 item 1. Full write-up, with the
|
|
||||||
acceptance output and the guard tests, in `findings/subsystems/relabel-new-ids.md`.
|
|
||||||
|
|
||||||
```
|
|
||||||
state_checksum.py POST-A.sav POST-B.sav --relabel-new-ids PRE-TURN.sav
|
|
||||||
```
|
|
||||||
|
|
||||||
Two processes with **identical pinned AI seeds** can still write different autosaves, because
|
|
||||||
the order in which the AI visits the ship groups that need a new fleet is per-process. The ids
|
|
||||||
are not the variable — they decode (`id-allocation.md`: `id = (counter << 4) | node`) to one
|
|
||||||
client's own counters, minted in that order in every process. The *same group* gets a
|
|
||||||
*different* id. This mode quotients that labelling out and nothing else.
|
|
||||||
|
|
||||||
Given the pre-turn save it computes the ids **new** in each post-turn save, matches the
|
|
||||||
non-zero-node (client-minted) new fleets by a key that does not mention the id — `(LocID or
|
|
||||||
`FPlan` destination, sorted ship-id set)` — builds the bijection π, exchanges the matched
|
|
||||||
fleets' bodies between their slots, rewrites every fleet reference under π, compares the master
|
|
||||||
id lists as sets, masks `/Summary/Checksum` **with its reason on the line** (§4.6: derived, and
|
|
||||||
its inputs are unmodelled), and prints π:
|
|
||||||
|
|
||||||
```
|
|
||||||
relabel: pi = {1970<->1986}
|
|
||||||
IDENTICAL modulo pi = {1970<->1986}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Five guards, and every one refuses rather than degrades.** Only ids absent from the pre-turn
|
|
||||||
save (G1); only non-zero node nibbles (G2); π must permute one set, so the two saves must have
|
|
||||||
minted the *same* ids (G3); the content keys must correspond one-to-one and be unique on each
|
|
||||||
side (G4); and no leaf anywhere in either save may hold a permuted id at a site this tool does
|
|
||||||
not model as a fleet reference (G5, matched on raw bytes so a reader typing slip cannot empty
|
|
||||||
it). On a refusal **nothing is rewritten** and the ordinary comparison runs, so a real
|
|
||||||
divergence still reports; the reason is printed on its own line.
|
|
||||||
|
|
||||||
The relabelled root is domain-separated (`relabel-new-ids` is folded into the root preimage), so
|
|
||||||
a "modulo π" root can never be mistaken for a strict one, and every root recorded before this
|
|
||||||
flag existed is bit-for-bit unchanged.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 3. Float-parity policy
|
## 3. Float-parity policy
|
||||||
|
|
@ -494,25 +454,14 @@ from the VM — see §5.
|
||||||
|
|
||||||
### 4.5 Tests
|
### 4.5 Tests
|
||||||
|
|
||||||
62 tests, `uv run python3 -m unittest discover -s verify/state-checksum -t verify/state-checksum`.
|
38 tests, `uv run python3 -m unittest discover -s verify/state-checksum -t verify/state-checksum`.
|
||||||
All pass. With only the in-repo saves, two real-save tests skip (the `bb4fd9ac` re-save form is
|
All pass. With only the in-repo saves, two real-save tests skip (the `bb4fd9ac` re-save form is
|
||||||
not in the repo); with `SOTS_SAVES_DIR` pointed at the full corpus, **62 pass, 0 skipped**.
|
not in the repo); with `SOTS_SAVES_DIR` pointed at the full corpus, **38 pass, 0 skipped**.
|
||||||
24 of them cover `--relabel-new-ids` (§2.6), and 7 of those 24 assert a **refusal** — the guards
|
|
||||||
are the part worth testing.
|
|
||||||
|
|
||||||
Three of the tests earn their keep by having caught real defects during development: the
|
Two of the tests earn their keep by having caught real defects during development: the
|
||||||
sibling-index cascade in §2.1, a `string` value's length prefix missing from the
|
sibling-index cascade in §2.1, and a `string` value's length prefix missing from the
|
||||||
reconstruction (which made the coverage audit fail loudly at offset `0x1c` instead of silently
|
reconstruction (which made the coverage audit fail loudly at offset `0x1c` instead of silently
|
||||||
under-covering — the audit working exactly as intended, on its author), and the raw-byte
|
under-covering — the audit working exactly as intended, on its author).
|
||||||
completeness scan of §2.6, which caught the reader typing a fleet reference as a **float** where
|
|
||||||
a value-based scan saw nothing.
|
|
||||||
|
|
||||||
**2026-09-09 correction, not a defect in the tool.** `test_known_resave_delta_localises_to_five_
|
|
||||||
named_leaves` enumerates candidate pairs over *sorted filenames*, and asserted the re-save
|
|
||||||
direction as `4 -> 0`. Adding `cb-turn2to3-endturn.sav` to the corpus produced a pair whose
|
|
||||||
re-saved member sorts first, so the observed direction was `0 -> 4` and the test failed on a
|
|
||||||
corpus fact. The claim is symmetric; the assertion now is too (and still requires all four
|
|
||||||
`Status` leaves to move the same way, with `Checksum` following by ∓16).
|
|
||||||
|
|
||||||
### 4.6 A negative result on `Summary.Checksum`
|
### 4.6 A negative result on `Summary.Checksum`
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -41,7 +41,6 @@ Usage
|
||||||
state_checksum.py SAVE --tree --depth 3 # digest tree
|
state_checksum.py SAVE --tree --depth 3 # digest tree
|
||||||
state_checksum.py A B # localise divergence
|
state_checksum.py A B # localise divergence
|
||||||
state_checksum.py A B --ulps 2 # classify float diffs
|
state_checksum.py A B --ulps 2 # classify float diffs
|
||||||
state_checksum.py A B --relabel-new-ids PRE # compare modulo new-id labelling
|
|
||||||
state_checksum.py --chain chain.json S1 S2 ... # verify a recorded chain
|
state_checksum.py --chain chain.json S1 S2 ... # verify a recorded chain
|
||||||
state_checksum.py SAVE --json # machine-readable
|
state_checksum.py SAVE --json # machine-readable
|
||||||
|
|
||||||
|
|
@ -66,9 +65,7 @@ import save_reader as sr # noqa: E402
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"FLOAT_POLICIES", "MASK_PRESETS", "CkNode", "checksum_save", "checksum_bytes",
|
"FLOAT_POLICIES", "MASK_PRESETS", "CkNode", "checksum_save", "checksum_bytes",
|
||||||
"checksum_result", "reconstruct", "audit_coverage", "diff", "DiffEntry",
|
"reconstruct", "audit_coverage", "diff", "DiffEntry", "float_key",
|
||||||
"float_key", "RelabelRefused", "Relabelling", "build_relabelling",
|
|
||||||
"apply_relabelling", "format_permutation", "id_node",
|
|
||||||
]
|
]
|
||||||
|
|
||||||
DIGEST_BYTES = 16 # blake2b-128; 2**-64 collision floor at our object counts
|
DIGEST_BYTES = 16 # blake2b-128; 2**-64 collision floor at our object counts
|
||||||
|
|
@ -184,11 +181,6 @@ ELEM_KEYS = ("EvTurn", "EvEID", "ordID", "npid", "desID", "Wpt", "seno2", "ID",
|
||||||
INLINE_ID_LISTS = ("PlayerIDs", "DesignIDs", "SystemIDs", "FleetIDs",
|
INLINE_ID_LISTS = ("PlayerIDs", "DesignIDs", "SystemIDs", "FleetIDs",
|
||||||
"ShipIDs", "TradeIDs")
|
"ShipIDs", "TradeIDs")
|
||||||
|
|
||||||
#: master id lists whose *element order* is an artefact of the order in which
|
|
||||||
#: the turn's commands were applied, not a decision. Compared as sets ONLY
|
|
||||||
#: under `--relabel-new-ids`, never by default -- see RELABEL_NOTES.
|
|
||||||
SET_ID_LISTS = ("ShipIDs", "FleetIDs", "DesignIDs")
|
|
||||||
|
|
||||||
#: top-level digest rows printed by the default (non-tree) view. Paths that do
|
#: top-level digest rows printed by the default (non-tree) view. Paths that do
|
||||||
#: not exist in a given save are silently skipped.
|
#: not exist in a given save are silently skipped.
|
||||||
SUBSYSTEM_PATHS = (
|
SUBSYSTEM_PATHS = (
|
||||||
|
|
@ -379,8 +371,7 @@ def _value_key(node: sr.Node, path: str, policy: str, masks, hits: dict):
|
||||||
return kind, raw, False
|
return kind, raw, False
|
||||||
|
|
||||||
|
|
||||||
def _build(node: sr.Node, path: str, label: str, policy: str, masks, hits: dict,
|
def _build(node: sr.Node, path: str, label: str, policy: str, masks, hits: dict) -> CkNode:
|
||||||
id_lists_as_sets: bool = False) -> CkNode:
|
|
||||||
if node.kind == "complex":
|
if node.kind == "complex":
|
||||||
entries = _group_children(node.children)
|
entries = _group_children(node.children)
|
||||||
counts = _name_counts(e[1] for e in entries if e[0] == "item")
|
counts = _name_counts(e[1] for e in entries if e[0] == "item")
|
||||||
|
|
@ -394,45 +385,26 @@ def _build(node: sr.Node, path: str, label: str, policy: str, masks, hits: dict,
|
||||||
k = ordinals.get(child.name, 0)
|
k = ordinals.get(child.name, 0)
|
||||||
ordinals[child.name] = k + 1
|
ordinals[child.name] = k + 1
|
||||||
lab = _label(child, k, None, unique=counts.get(child.name) == 1)
|
lab = _label(child, k, None, unique=counts.get(child.name) == 1)
|
||||||
kids.append(_build(child, f"{path}/{lab}", lab, policy, masks, hits,
|
kids.append(_build(child, f"{path}/{lab}", lab, policy, masks, hits))
|
||||||
id_lists_as_sets))
|
|
||||||
elif ent[0] == "list":
|
elif ent[0] == "list":
|
||||||
_, lname, cnode, vals = ent
|
_, lname, cnode, vals = ent
|
||||||
# path-scoped like the mask rules (2.5): only the MASTER lists
|
lpath = f"{path}/{lname}[]"
|
||||||
# directly under /Sim, never a same-named list nested in an
|
|
||||||
# object. On the corpus that is the only place they occur --
|
|
||||||
# asserted here so a future save that nests one is not silently
|
|
||||||
# set-compared.
|
|
||||||
as_set = (id_lists_as_sets and lname in SET_ID_LISTS
|
|
||||||
and path == "/Sim")
|
|
||||||
lpath = f"{path}/{lname}[]" + ("(set)" if as_set else "")
|
|
||||||
keys = [_value_key(v, lpath, policy, masks, hits)[1] for v in vals]
|
keys = [_value_key(v, lpath, policy, masks, hits)[1] for v in vals]
|
||||||
values = [v.value for v in vals]
|
ldig = _h(b"list", lname.encode("ascii"), cnode.raw, *keys)
|
||||||
if as_set:
|
|
||||||
# element order is apply order, which is the residue this
|
|
||||||
# mode exists to quotient out; the multiset is still exact,
|
|
||||||
# so a genuine insertion/removal still reports.
|
|
||||||
order = sorted(range(len(vals)), key=lambda i: (str(type(values[i])), values[i]))
|
|
||||||
keys = [keys[i] for i in order]
|
|
||||||
values = [values[i] for i in order]
|
|
||||||
ldig = _h(b"list-set" if as_set else b"list",
|
|
||||||
lname.encode("ascii"), cnode.raw, *keys)
|
|
||||||
off = cnode.offset
|
off = cnode.offset
|
||||||
last = vals[-1] if vals else cnode
|
last = vals[-1] if vals else cnode
|
||||||
kids.append(CkNode(lpath, f"{lname}[]" + ("(set)" if as_set else ""),
|
kids.append(CkNode(lpath, f"{lname}[]", "list", ldig, off,
|
||||||
"list", ldig, off,
|
|
||||||
last.offset + last.size - off,
|
last.offset + last.size - off,
|
||||||
1 + len(vals),
|
1 + len(vals),
|
||||||
len(cnode.raw) + sum(len(v.raw) for v in vals),
|
len(cnode.raw) + sum(len(v.raw) for v in vals),
|
||||||
[], values=values))
|
[], values=[v.value for v in vals]))
|
||||||
else:
|
else:
|
||||||
_, gname, pairs = ent
|
_, gname, pairs = ent
|
||||||
gpath = f"{path}/{gname}"
|
gpath = f"{path}/{gname}"
|
||||||
gkids = []
|
gkids = []
|
||||||
for j, (idn, frame) in enumerate(pairs):
|
for j, (idn, frame) in enumerate(pairs):
|
||||||
lab = _label(frame, j, idn)
|
lab = _label(frame, j, idn)
|
||||||
ck = _build(frame, f"{gpath}/{lab}", lab, policy, masks, hits,
|
ck = _build(frame, f"{gpath}/{lab}", lab, policy, masks, hits)
|
||||||
id_lists_as_sets)
|
|
||||||
# the id int is part of the object's identity; fold it in
|
# the id int is part of the object's identity; fold it in
|
||||||
idk = _h(b"id", (idn.name or "").encode("ascii"), idn.raw)
|
idk = _h(b"id", (idn.name or "").encode("ascii"), idn.raw)
|
||||||
ck.digest = _h(b"obj", idk, ck.digest)
|
ck.digest = _h(b"obj", idk, ck.digest)
|
||||||
|
|
@ -569,21 +541,15 @@ class SaveChecksum:
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def checksum_result(res: sr.SaveResult, path: str, cov: dict,
|
def checksum_bytes(data: bytes, path: str = "<bytes>", floats: str = "bits",
|
||||||
floats: str = "bits", mask: str = "none",
|
mask: str = "none", audit: bool = True,
|
||||||
extra_masks: tuple = (),
|
strict: bool = False) -> SaveChecksum:
|
||||||
id_lists_as_sets: bool = False) -> SaveChecksum:
|
|
||||||
"""Digest an already-parsed save.
|
|
||||||
|
|
||||||
Split out of `checksum_bytes` so `--relabel-new-ids` can parse once, prove
|
|
||||||
coverage against the *unmodified* parse, and only then rewrite leaves.
|
|
||||||
"""
|
|
||||||
if floats not in FLOAT_POLICIES:
|
if floats not in FLOAT_POLICIES:
|
||||||
raise ValueError(f"floats must be one of {FLOAT_POLICIES}")
|
raise ValueError(f"floats must be one of {FLOAT_POLICIES}")
|
||||||
if mask not in MASK_PRESETS:
|
if mask not in MASK_PRESETS:
|
||||||
raise ValueError(f"mask must be one of {tuple(MASK_PRESETS)}")
|
raise ValueError(f"mask must be one of {tuple(MASK_PRESETS)}")
|
||||||
masks = tuple(MASK_PRESETS[mask]) + tuple(
|
res = sr.read_bytes(data, strict=strict)
|
||||||
m for m in extra_masks if m not in MASK_PRESETS[mask])
|
masks = MASK_PRESETS[mask]
|
||||||
hits: dict = {}
|
hits: dict = {}
|
||||||
kids = []
|
kids = []
|
||||||
counts = _name_counts(res.tree.children)
|
counts = _name_counts(res.tree.children)
|
||||||
|
|
@ -592,26 +558,13 @@ def checksum_result(res: sr.SaveResult, path: str, cov: dict,
|
||||||
k = ordinals.get(c.name, 0)
|
k = ordinals.get(c.name, 0)
|
||||||
ordinals[c.name] = k + 1
|
ordinals[c.name] = k + 1
|
||||||
lab = _label(c, k, None, unique=counts.get(c.name) == 1)
|
lab = _label(c, k, None, unique=counts.get(c.name) == 1)
|
||||||
kids.append(_build(c, "/" + lab, lab, floats, masks, hits, id_lists_as_sets))
|
kids.append(_build(c, "/" + lab, lab, floats, masks, hits))
|
||||||
# domain separation: a relabelled root must never be confusable with a
|
root_dig = _h(b"save", floats.encode("ascii"), mask.encode("ascii"),
|
||||||
# plain one. The extra part is appended ONLY in the new mode, so every
|
*[k.digest for k in kids])
|
||||||
# root recorded before this flag existed is bit-for-bit unchanged.
|
|
||||||
pol = [b"save", floats.encode("ascii"), mask.encode("ascii")]
|
|
||||||
if extra_masks or id_lists_as_sets:
|
|
||||||
pol.append(b"relabel-new-ids")
|
|
||||||
root_dig = _h(*pol, *[k.digest for k in kids])
|
|
||||||
root = CkNode("", "<save>", "complex", root_dig, 0, len(res.inflated),
|
root = CkNode("", "<save>", "complex", root_dig, 0, len(res.inflated),
|
||||||
sum(k.leaves for k in kids), sum(k.value_bytes for k in kids), kids)
|
sum(k.leaves for k in kids), sum(k.value_bytes for k in kids), kids)
|
||||||
return SaveChecksum(path, root, cov, res, floats, mask, hits)
|
|
||||||
|
|
||||||
|
|
||||||
def checksum_bytes(data: bytes, path: str = "<bytes>", floats: str = "bits",
|
|
||||||
mask: str = "none", audit: bool = True,
|
|
||||||
strict: bool = False, extra_masks: tuple = (),
|
|
||||||
id_lists_as_sets: bool = False) -> SaveChecksum:
|
|
||||||
res = sr.read_bytes(data, strict=strict)
|
|
||||||
cov = audit_coverage(res.tree, res.inflated) if audit else {"ok": None}
|
cov = audit_coverage(res.tree, res.inflated) if audit else {"ok": None}
|
||||||
return checksum_result(res, path, cov, floats, mask, extra_masks, id_lists_as_sets)
|
return SaveChecksum(path, root, cov, res, floats, mask, hits)
|
||||||
|
|
||||||
|
|
||||||
def checksum_save(path: str, **kw) -> SaveChecksum:
|
def checksum_save(path: str, **kw) -> SaveChecksum:
|
||||||
|
|
@ -698,371 +651,6 @@ def diff(a: CkNode, b: CkNode, out: list | None = None, limit: int = 200) -> lis
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
# --- relabelling new client-minted ids ----------------------------------------
|
|
||||||
#
|
|
||||||
# Why this exists
|
|
||||||
# ---------------
|
|
||||||
# `findings/resolutions/2026-09-09-fleet-id-order-residue.md` section 3 item 1.
|
|
||||||
# Lane BP ran two processes with *identical pinned AI seeds* on the same
|
|
||||||
# pre-turn save and got autosaves differing in 35 of 61,147 leaves -- all 35 one
|
|
||||||
# transposition: fleets 1970 and 1986 exchange their entire contents. The
|
|
||||||
# resolver decoded the ids (`id-allocation.md`: `id = (counter << 4) | node`)
|
|
||||||
# and found the ids are NOT the variable: they are one client's own counters
|
|
||||||
# 123/124/125, minted in that order in every process. What varies is the order
|
|
||||||
# in which the fleet-assignment pass VISITS the ship groups that need a new
|
|
||||||
# fleet, so the same group receives a different id. No decision changes.
|
|
||||||
#
|
|
||||||
# This mode makes the comparison blind to that labelling and to nothing else.
|
|
||||||
#
|
|
||||||
# The guards matter more than the feature (rule 1: a green verdict on a hook
|
|
||||||
# that may be comparing nothing is the failure this campaign was built around).
|
|
||||||
# There are five, and every one of them REFUSES rather than degrades:
|
|
||||||
#
|
|
||||||
# G1 only ids ABSENT from the pre-turn save may be relabelled;
|
|
||||||
# G2 only ids with a non-zero node nibble (client-minted) may be relabelled;
|
|
||||||
# G3 pi must be a permutation of ONE set -- the new ids of one node -- so the
|
|
||||||
# two saves must have minted the same ids; different id sets is a real
|
|
||||||
# divergence (a different number of allocations) and is never absorbed;
|
|
||||||
# G4 the content keys must correspond one-to-one, and be unique on each side;
|
|
||||||
# G5 every leaf anywhere in either save that holds a permuted id must sit at
|
|
||||||
# a reference site this module models. An unmodelled site would mean the
|
|
||||||
# rewrite is incomplete, which could make two different states compare
|
|
||||||
# equal. Finding one is a refusal, not a warning.
|
|
||||||
#
|
|
||||||
# A refusal never falls back to a partial relabelling. The CLI prints the
|
|
||||||
# reason and then runs the ORDINARY comparison, which is the conservative
|
|
||||||
# direction: whatever really differs still reports as DIVERGED.
|
|
||||||
|
|
||||||
class RelabelRefused(Exception):
|
|
||||||
"""A relabelling could not be built as a full, guarded bijection."""
|
|
||||||
|
|
||||||
|
|
||||||
def id_node(i: int) -> int:
|
|
||||||
"""The node nibble of an object id (`id-allocation.md` section 1).
|
|
||||||
|
|
||||||
0 is the server / the sim that owns the board; `PlyrIdx + 1` is a player's
|
|
||||||
own client. Only non-zero nibbles are ever relabelled here.
|
|
||||||
"""
|
|
||||||
return i & 0xF
|
|
||||||
|
|
||||||
|
|
||||||
#: Leaf tags that hold a *fleet* id, with where they are allowed to appear.
|
|
||||||
#: `FltID` is both the master fleet table's lead int (the slot, never rewritten)
|
|
||||||
#: and a ship's back-reference to its fleet (rewritten); `Flt` is a system's
|
|
||||||
#: list of the fleets standing at it (rewritten). Anything else is G5.
|
|
||||||
FLEET_SLOT_TAG = "FltID"
|
|
||||||
FLEET_REF_TAGS = ("FltID", "Flt")
|
|
||||||
|
|
||||||
RELABEL_MASK = (("Checksum", "/Summary/", "derived", None),)
|
|
||||||
|
|
||||||
RELABEL_NOTES = (
|
|
||||||
"/Summary/Checksum is MASKED: it is derived from the whole state and its "
|
|
||||||
"inputs are unmodelled (rule 18's open item), so it cannot be recomputed "
|
|
||||||
"under a relabelling and its move is not independent evidence. Unmask by "
|
|
||||||
"dropping --relabel-new-ids.",
|
|
||||||
"Master id lists (%s) are compared as SETS: their element order is the "
|
|
||||||
"order the turn's commands were applied in, which is the same visit-order "
|
|
||||||
"residue. Membership is still exact -- an added or removed id still "
|
|
||||||
"reports." % "/".join(SET_ID_LISTS),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class Relabelling:
|
|
||||||
"""The bijection pi, plus everything a reader needs to audit it."""
|
|
||||||
|
|
||||||
__slots__ = ("pi", "nodes", "new_ids", "keys", "rewrites", "moves")
|
|
||||||
|
|
||||||
def __init__(self, pi, nodes, new_ids, keys):
|
|
||||||
self.pi = pi # A-id -> B-id, a permutation of new_ids
|
|
||||||
self.nodes = nodes # node nibbles it touches
|
|
||||||
self.new_ids = new_ids # the new non-zero-node fleet ids
|
|
||||||
self.keys = keys # A-id -> content key, for the report
|
|
||||||
self.rewrites: dict = {} # tag -> leaves rewritten
|
|
||||||
self.moves = 0 # fleet bodies exchanged
|
|
||||||
|
|
||||||
@property
|
|
||||||
def moved(self) -> dict:
|
|
||||||
return {k: v for k, v in self.pi.items() if k != v}
|
|
||||||
|
|
||||||
def __bool__(self) -> bool:
|
|
||||||
return bool(self.moved)
|
|
||||||
|
|
||||||
|
|
||||||
def format_permutation(pi: dict) -> str:
|
|
||||||
"""`{1970<->1986}`, or cycle notation for anything longer."""
|
|
||||||
moved = {k: v for k, v in pi.items() if k != v}
|
|
||||||
if not moved:
|
|
||||||
return "{} (identity -- no new id changed hands)"
|
|
||||||
seen, cycles = set(), []
|
|
||||||
for k in sorted(moved):
|
|
||||||
if k in seen:
|
|
||||||
continue
|
|
||||||
cyc, cur = [k], moved[k]
|
|
||||||
seen.add(k)
|
|
||||||
while cur != k:
|
|
||||||
cyc.append(cur)
|
|
||||||
seen.add(cur)
|
|
||||||
cur = moved[cur]
|
|
||||||
cycles.append(cyc)
|
|
||||||
parts = ["<->".join(str(x) for x in c) if len(c) == 2
|
|
||||||
else "->".join(str(x) for x in c + [c[0]]) for c in cycles]
|
|
||||||
return "{" + ", ".join(parts) + "}"
|
|
||||||
|
|
||||||
|
|
||||||
def _sim_frame(tree: sr.Node) -> sr.Node:
|
|
||||||
for c in tree.children:
|
|
||||||
if c.name == "Sim" and c.kind == "complex":
|
|
||||||
return c
|
|
||||||
raise RelabelRefused("no /Sim frame in this save -- not a game state")
|
|
||||||
|
|
||||||
|
|
||||||
def _fleet_slots(tree: sr.Node) -> list:
|
|
||||||
"""[(lead_id_node, frame_node)] for the master fleet table."""
|
|
||||||
sim = _sim_frame(tree)
|
|
||||||
ch = sim.children
|
|
||||||
out = []
|
|
||||||
for i in range(len(ch) - 1):
|
|
||||||
if (ch[i].name == FLEET_SLOT_TAG and ch[i].kind == "int"
|
|
||||||
and ch[i + 1].name == "Flt" and ch[i + 1].kind == "complex"):
|
|
||||||
out.append((ch[i], ch[i + 1]))
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _fleet_content_key(frame: sr.Node):
|
|
||||||
"""An identity for a fleet that does NOT mention its own id.
|
|
||||||
|
|
||||||
Resolver section 3.1: `(LocID or FPlan destination, sorted ship-id set)`.
|
|
||||||
`LocID` and the plan destination are tagged apart, so a fleet standing at
|
|
||||||
system 384 never keys the same as one bound for 384.
|
|
||||||
"""
|
|
||||||
loc, dest, ships = None, None, []
|
|
||||||
for c in frame.children:
|
|
||||||
if c.name == "LocID" and c.kind == "int":
|
|
||||||
loc = c.value
|
|
||||||
elif c.name == "ShipID" and c.kind == "int":
|
|
||||||
ships.append(c.value)
|
|
||||||
elif c.name == "FPlan" and c.kind == "complex":
|
|
||||||
for g in c.children:
|
|
||||||
if g.name == "pnd" and g.kind == "int":
|
|
||||||
dest = g.value
|
|
||||||
if loc:
|
|
||||||
where = ("at", loc)
|
|
||||||
elif dest is not None:
|
|
||||||
where = ("bound-for", dest)
|
|
||||||
else:
|
|
||||||
where = ("nowhere", None)
|
|
||||||
return (where, tuple(sorted(ships)))
|
|
||||||
|
|
||||||
|
|
||||||
def _fleet_name(frame: sr.Node):
|
|
||||||
for c in frame.children:
|
|
||||||
if c.name == "FtName":
|
|
||||||
return c.value
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _id_occurrences(tree: sr.Node, wanted: set) -> list:
|
|
||||||
"""Every leaf in `tree` holding one of `wanted`, classified by site.
|
|
||||||
|
|
||||||
Matching is on the leaf's **raw four bytes**, not on the reader's typed
|
|
||||||
value, deliberately: the digest is a function of (save bytes, reader
|
|
||||||
schema), and a leaf the schema happens to type as a float or a raw blob
|
|
||||||
would be invisible to a value-based scan while still carrying an id the
|
|
||||||
rewrite must reach. A completeness guard that a typing slip can silently
|
|
||||||
empty is exactly the "green verdict over an empty region set" of rule 1.
|
|
||||||
|
|
||||||
Returns [(site, path, node)] where site is one of:
|
|
||||||
"slot" -- the master fleet table's lead FltID (identifies the slot)
|
|
||||||
"idlist" -- an entry of the inline `FleetIDs` master list
|
|
||||||
"ref:<tag>" -- a reference that must be rewritten under pi
|
|
||||||
"UNMODELLED"-- anything else; this is guard G5 and it refuses
|
|
||||||
"""
|
|
||||||
by_raw = {struct.pack("<i", v): v for v in wanted}
|
|
||||||
sim = _sim_frame(tree)
|
|
||||||
slot_leads = {id(n) for n, _ in _fleet_slots(tree)}
|
|
||||||
idlist_entries = set()
|
|
||||||
ch = sim.children
|
|
||||||
for i, c in enumerate(ch):
|
|
||||||
if c.name in INLINE_ID_LISTS and c.kind == "int" and isinstance(c.value, int):
|
|
||||||
for k in range(min(c.value, len(ch) - i - 1)):
|
|
||||||
idlist_entries.add(id(ch[i + 1 + k]))
|
|
||||||
|
|
||||||
out = []
|
|
||||||
|
|
||||||
def walk(n, path):
|
|
||||||
for c in n.children:
|
|
||||||
p = f"{path}/{c.name if c.name is not None else '.'}"
|
|
||||||
if c.kind == "complex":
|
|
||||||
walk(c, p)
|
|
||||||
continue
|
|
||||||
if c.raw not in by_raw:
|
|
||||||
continue
|
|
||||||
if id(c) in slot_leads:
|
|
||||||
site = "slot"
|
|
||||||
elif id(c) in idlist_entries:
|
|
||||||
site = "idlist"
|
|
||||||
elif c.name in FLEET_REF_TAGS:
|
|
||||||
site = f"ref:{c.name}"
|
|
||||||
else:
|
|
||||||
site = "UNMODELLED"
|
|
||||||
out.append((site, p, c))
|
|
||||||
|
|
||||||
walk(tree, "")
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def build_relabelling(pre_tree: sr.Node, a_tree: sr.Node, b_tree: sr.Node) -> Relabelling:
|
|
||||||
"""Build pi between the ids new in A and the ids new in B, or refuse.
|
|
||||||
|
|
||||||
Nothing is mutated here. Every guard is checked before any rewrite can
|
|
||||||
happen, so a refusal leaves both saves exactly as parsed.
|
|
||||||
"""
|
|
||||||
pre_ids = {n.value for n, _ in _fleet_slots(pre_tree)}
|
|
||||||
A = {n.value: f for n, f in _fleet_slots(a_tree)}
|
|
||||||
B = {n.value: f for n, f in _fleet_slots(b_tree)}
|
|
||||||
|
|
||||||
# G1 + G2: new, and client-minted
|
|
||||||
new_a = {i for i in A if i not in pre_ids and id_node(i) != 0}
|
|
||||||
new_b = {i for i in B if i not in pre_ids and id_node(i) != 0}
|
|
||||||
|
|
||||||
# G3: pi permutes ONE set. Unequal id sets means the two runs minted
|
|
||||||
# different counters -- a different number of allocations, which is a real
|
|
||||||
# divergence and is never something this mode may absorb.
|
|
||||||
if new_a != new_b:
|
|
||||||
raise RelabelRefused(
|
|
||||||
"the new client-minted fleet ids differ between the two saves -- "
|
|
||||||
f"A minted {sorted(new_a)}, B minted {sorted(new_b)}"
|
|
||||||
+ (f"; only in A {sorted(new_a - new_b)}" if new_a - new_b else "")
|
|
||||||
+ (f"; only in B {sorted(new_b - new_a)}" if new_b - new_a else "")
|
|
||||||
+ ". That is a difference in what was allocated, not in how it "
|
|
||||||
"was labelled, so no relabelling is defensible")
|
|
||||||
|
|
||||||
new_ids = new_a
|
|
||||||
pi: dict = {}
|
|
||||||
keys: dict = {}
|
|
||||||
for node in sorted({id_node(i) for i in new_ids}):
|
|
||||||
ids = sorted(i for i in new_ids if id_node(i) == node)
|
|
||||||
ka: dict = {}
|
|
||||||
kb: dict = {}
|
|
||||||
for side, tbl, dst in (("A", A, ka), ("B", B, kb)):
|
|
||||||
for i in ids:
|
|
||||||
key = _fleet_content_key(tbl[i])
|
|
||||||
if key in dst:
|
|
||||||
raise RelabelRefused(
|
|
||||||
f"two new fleets in {side} share the content key {key!r} "
|
|
||||||
f"(ids {dst[key]} and {i}) -- the match would be "
|
|
||||||
"ambiguous, so no bijection can be built")
|
|
||||||
dst[key] = i
|
|
||||||
if set(ka) != set(kb):
|
|
||||||
only_a = [f"{ka[k]}:{k}" for k in sorted(set(ka) - set(kb), key=repr)]
|
|
||||||
only_b = [f"{kb[k]}:{k}" for k in sorted(set(kb) - set(ka), key=repr)]
|
|
||||||
raise RelabelRefused(
|
|
||||||
f"node {node}: the new fleets do not correspond by content. "
|
|
||||||
f"Unmatched in A: {only_a}; unmatched in B: {only_b}. Their "
|
|
||||||
"contents differ, which is a real divergence")
|
|
||||||
for key, i in ka.items():
|
|
||||||
pi[i] = kb[key]
|
|
||||||
keys[i] = key
|
|
||||||
# names are minted from the same per-fleet ordinal counter as the id
|
|
||||||
# (Sigma/Tau/Upsilon...), so they are id-attached labels, not content.
|
|
||||||
# They may only be treated that way if they DO track the id in both
|
|
||||||
# saves; if they do not, something real differs and we refuse.
|
|
||||||
for i in ids:
|
|
||||||
na, nb = _fleet_name(A[i]), _fleet_name(B[i])
|
|
||||||
if na != nb:
|
|
||||||
raise RelabelRefused(
|
|
||||||
f"fleet {i} is named {na!r} in A and {nb!r} in B -- the "
|
|
||||||
"name does not track the id, so it is not an id-attached "
|
|
||||||
"label here and relabelling would hide a real difference")
|
|
||||||
|
|
||||||
rl = Relabelling(pi, sorted({id_node(i) for i in new_ids}), sorted(new_ids), keys)
|
|
||||||
|
|
||||||
# G3 again, on the built map: a permutation of exactly the new ids
|
|
||||||
if sorted(pi) != sorted(pi.values()):
|
|
||||||
raise RelabelRefused(
|
|
||||||
f"pi is not a permutation: domain {sorted(pi)} != image "
|
|
||||||
f"{sorted(pi.values())}")
|
|
||||||
for k, v in pi.items():
|
|
||||||
if id_node(k) != id_node(v):
|
|
||||||
raise RelabelRefused(
|
|
||||||
f"pi would map {k} (node {id_node(k)}) to {v} (node "
|
|
||||||
f"{id_node(v)}) -- ids may only be relabelled within one node")
|
|
||||||
if k in pre_ids or v in pre_ids:
|
|
||||||
raise RelabelRefused(
|
|
||||||
f"pi touches {k if k in pre_ids else v}, which is present in "
|
|
||||||
"the pre-turn save -- only ids created on this turn may be "
|
|
||||||
"relabelled")
|
|
||||||
|
|
||||||
# G5: no unmodelled reference site, in EITHER save
|
|
||||||
touched = set(pi) | set(pi.values())
|
|
||||||
for name, tree in (("A", a_tree), ("B", b_tree)):
|
|
||||||
for site, path, _n in _id_occurrences(tree, touched):
|
|
||||||
if site == "UNMODELLED":
|
|
||||||
raise RelabelRefused(
|
|
||||||
f"{name} holds a relabelled id at {path}, which this tool "
|
|
||||||
"does not model as a fleet reference. Rewriting only the "
|
|
||||||
"sites it knows would leave the two states comparable at a "
|
|
||||||
"leaf that still carries the old label -- add the tag to "
|
|
||||||
"FLEET_REF_TAGS after checking what it means")
|
|
||||||
return rl
|
|
||||||
|
|
||||||
|
|
||||||
def apply_relabelling(tree: sr.Node, rl: Relabelling) -> Relabelling:
|
|
||||||
"""Rewrite `tree` in place so it reads as if it had minted B's labelling.
|
|
||||||
|
|
||||||
Two operations, both a function of pi alone:
|
|
||||||
|
|
||||||
* the permuted fleets' **bodies** are exchanged between their slots, while
|
|
||||||
each slot keeps its own id and its id-attached label (`FtName`). Doing
|
|
||||||
it this way -- rather than renumbering the slots -- means the master
|
|
||||||
fleet table's element order, the `FleetIDs[]` list and every object
|
|
||||||
label stay exactly as the file had them, so nothing but the intended
|
|
||||||
quotient is taken.
|
|
||||||
* every *reference* to a permuted id is rewritten to pi(it).
|
|
||||||
|
|
||||||
Node `offset`/`size` are left stale on moved bodies; they are reporting
|
|
||||||
aids in `--tree`/`--json`, not digest inputs. The digest is a function of
|
|
||||||
(tag, kind, value bytes) only.
|
|
||||||
"""
|
|
||||||
if not rl.pi:
|
|
||||||
return rl
|
|
||||||
slots = {n.value: f for n, f in _fleet_slots(tree)}
|
|
||||||
inv = {v: k for k, v in rl.pi.items()}
|
|
||||||
|
|
||||||
old_children = {i: list(slots[i].children) for i in rl.pi}
|
|
||||||
own_name = {}
|
|
||||||
for i in rl.pi:
|
|
||||||
own_name[i] = [c for c in slots[i].children if c.name == "FtName"]
|
|
||||||
|
|
||||||
for dst in rl.pi.values():
|
|
||||||
src = inv[dst]
|
|
||||||
if src == dst:
|
|
||||||
continue
|
|
||||||
kept = list(own_name[dst])
|
|
||||||
kids = []
|
|
||||||
for c in old_children[src]:
|
|
||||||
if c.name == "FtName" and kept:
|
|
||||||
kids.append(kept.pop(0))
|
|
||||||
else:
|
|
||||||
kids.append(c)
|
|
||||||
slots[dst].children = kids
|
|
||||||
rl.moves += 1
|
|
||||||
|
|
||||||
by_raw = {struct.pack("<i", v): v for v in rl.pi}
|
|
||||||
for site, _path, node in _id_occurrences(tree, set(rl.pi)):
|
|
||||||
if not site.startswith("ref:"):
|
|
||||||
continue
|
|
||||||
old = by_raw[node.raw]
|
|
||||||
new = rl.pi[old]
|
|
||||||
if new == old: # a fixed point of pi rewrites nothing
|
|
||||||
continue
|
|
||||||
node.raw = struct.pack("<i", new)
|
|
||||||
if isinstance(node.value, int):
|
|
||||||
node.value = new
|
|
||||||
rl.rewrites[site[4:]] = rl.rewrites.get(site[4:], 0) + 1
|
|
||||||
return rl
|
|
||||||
|
|
||||||
|
|
||||||
# --- chain --------------------------------------------------------------------
|
# --- chain --------------------------------------------------------------------
|
||||||
|
|
||||||
def build_chain(files: list[str], **kw) -> dict:
|
def build_chain(files: list[str], **kw) -> dict:
|
||||||
|
|
@ -1134,76 +722,6 @@ def _mask_note(ck: "SaveChecksum") -> str:
|
||||||
return " [masked: " + ", ".join(f"{k}x{v}" for k, v in sorted(ck.mask_hits.items())) + "]"
|
return " [masked: " + ", ".join(f"{k}x{v}" for k, v in sorted(ck.mask_hits.items())) + "]"
|
||||||
|
|
||||||
|
|
||||||
def _relabelled_pair(pre_path: str, a_path: str, b_path: str,
|
|
||||||
floats: str = "bits", mask: str = "none",
|
|
||||||
audit: bool = True, strict: bool = False):
|
|
||||||
"""Parse three saves, build pi, and digest A relabelled against B.
|
|
||||||
|
|
||||||
On a refusal nothing is rewritten and the ORDINARY comparison is returned
|
|
||||||
instead, so a real divergence survives. Returns (a, b, relabelling|None,
|
|
||||||
refusal_message|None).
|
|
||||||
"""
|
|
||||||
def read(p):
|
|
||||||
with open(p, "rb") as f:
|
|
||||||
res = sr.read_bytes(f.read(), strict=strict)
|
|
||||||
cov = audit_coverage(res.tree, res.inflated) if audit else {"ok": None}
|
|
||||||
return res, cov
|
|
||||||
|
|
||||||
res_p, _ = read(pre_path)
|
|
||||||
res_a, cov_a = read(a_path)
|
|
||||||
res_b, cov_b = read(b_path)
|
|
||||||
|
|
||||||
try:
|
|
||||||
rl = build_relabelling(res_p.tree, res_a.tree, res_b.tree)
|
|
||||||
except RelabelRefused as exc:
|
|
||||||
a = checksum_result(res_a, a_path, cov_a, floats, mask)
|
|
||||||
b = checksum_result(res_b, b_path, cov_b, floats, mask)
|
|
||||||
return a, b, None, str(exc)
|
|
||||||
|
|
||||||
apply_relabelling(res_a.tree, rl)
|
|
||||||
a = checksum_result(res_a, a_path, cov_a, floats, mask,
|
|
||||||
extra_masks=RELABEL_MASK, id_lists_as_sets=True)
|
|
||||||
b = checksum_result(res_b, b_path, cov_b, floats, mask,
|
|
||||||
extra_masks=RELABEL_MASK, id_lists_as_sets=True)
|
|
||||||
return a, b, rl, None
|
|
||||||
|
|
||||||
|
|
||||||
def _relabel_report(rl, refusal: str | None, pre_path: str) -> list:
|
|
||||||
if refusal is not None:
|
|
||||||
return [
|
|
||||||
f"relabel: pre-turn save {pre_path}",
|
|
||||||
"relabel: REFUSED -- " + refusal,
|
|
||||||
"relabel: NO relabelling was applied; the comparison below is the "
|
|
||||||
"ordinary one, and nothing has been absorbed.",
|
|
||||||
]
|
|
||||||
lines = [f"relabel: pre-turn save {pre_path}",
|
|
||||||
f"relabel: pi = {format_permutation(rl.pi)}"]
|
|
||||||
for i in rl.new_ids:
|
|
||||||
where, ships = rl.keys[i]
|
|
||||||
lines.append(f"relabel: {i} (counter {i >> 4}, node {id_node(i)}) "
|
|
||||||
f"-> {rl.pi[i]} {where[0]} {where[1]}, ships {list(ships)}")
|
|
||||||
lines.append(f"relabel: {len(rl.new_ids)} new client-minted fleet id(s) on "
|
|
||||||
f"node(s) {rl.nodes}; {len(rl.moved)} relabelled, "
|
|
||||||
f"{rl.moves} fleet body/bodies exchanged, "
|
|
||||||
+ (", ".join(f"{k}x{v}" for k, v in sorted(rl.rewrites.items()))
|
|
||||||
or "no references") + " rewritten")
|
|
||||||
for note in RELABEL_NOTES:
|
|
||||||
lines.append("relabel: " + note)
|
|
||||||
return lines
|
|
||||||
|
|
||||||
|
|
||||||
def _relabel_dict(rl, refusal: str | None, pre_path: str) -> dict:
|
|
||||||
if refusal is not None:
|
|
||||||
return {"preTurn": pre_path, "applied": False, "refused": refusal}
|
|
||||||
return {"preTurn": pre_path, "applied": True,
|
|
||||||
"pi": {str(k): v for k, v in rl.pi.items()},
|
|
||||||
"moved": {str(k): v for k, v in rl.moved.items()},
|
|
||||||
"newIds": rl.new_ids, "nodes": rl.nodes,
|
|
||||||
"contentKeys": {str(k): repr(v) for k, v in rl.keys.items()},
|
|
||||||
"bodiesExchanged": rl.moves, "referencesRewritten": rl.rewrites,
|
|
||||||
"notes": list(RELABEL_NOTES)}
|
|
||||||
|
|
||||||
|
|
||||||
def _print_tree(n: CkNode, depth: int, out, indent: int = 0, top: bool = True):
|
def _print_tree(n: CkNode, depth: int, out, indent: int = 0, top: bool = True):
|
||||||
if not top:
|
if not top:
|
||||||
pad = " " * indent
|
pad = " " * indent
|
||||||
|
|
@ -1233,10 +751,6 @@ def main(argv=None) -> int:
|
||||||
ap.add_argument("--ulps", type=float, default=None,
|
ap.add_argument("--ulps", type=float, default=None,
|
||||||
help="in a diff, flag float differences at or below N ULPs")
|
help="in a diff, flag float differences at or below N ULPs")
|
||||||
ap.add_argument("--limit", type=int, default=200, help="max diff entries")
|
ap.add_argument("--limit", type=int, default=200, help="max diff entries")
|
||||||
ap.add_argument("--relabel-new-ids", metavar="PRE-TURN.SAV", default=None,
|
|
||||||
help="compare two post-turn saves modulo the labelling of "
|
|
||||||
"the client-minted ids created on this turn, given the "
|
|
||||||
"pre-turn save they both came from")
|
|
||||||
ap.add_argument("--chain", metavar="FILE", help="chain JSON to verify against")
|
ap.add_argument("--chain", metavar="FILE", help="chain JSON to verify against")
|
||||||
ap.add_argument("--record-chain", metavar="FILE", help="write a chain JSON instead")
|
ap.add_argument("--record-chain", metavar="FILE", help="write a chain JSON instead")
|
||||||
args = ap.parse_args(argv)
|
args = ap.parse_args(argv)
|
||||||
|
|
@ -1247,10 +761,6 @@ def main(argv=None) -> int:
|
||||||
if not args.saves:
|
if not args.saves:
|
||||||
ap.error("no saves given")
|
ap.error("no saves given")
|
||||||
|
|
||||||
if args.relabel_new_ids and (len(args.saves) != 2 or args.chain or args.record_chain):
|
|
||||||
ap.error("--relabel-new-ids compares exactly two post-turn saves; it is "
|
|
||||||
"a property of a pair, not of a single save or a chain")
|
|
||||||
|
|
||||||
if args.record_chain:
|
if args.record_chain:
|
||||||
chain = build_chain(args.saves, **kw)
|
chain = build_chain(args.saves, **kw)
|
||||||
with open(args.record_chain, "w") as f:
|
with open(args.record_chain, "w") as f:
|
||||||
|
|
@ -1298,46 +808,30 @@ def main(argv=None) -> int:
|
||||||
if len(args.saves) != 2:
|
if len(args.saves) != 2:
|
||||||
ap.error("give one save, two saves, or use --chain / --record-chain")
|
ap.error("give one save, two saves, or use --chain / --record-chain")
|
||||||
|
|
||||||
rl = None
|
a = checksum_save(args.saves[0], **kw)
|
||||||
refusal = None
|
b = checksum_save(args.saves[1], **kw)
|
||||||
if args.relabel_new_ids:
|
|
||||||
a, b, rl, refusal = _relabelled_pair(
|
|
||||||
args.relabel_new_ids, args.saves[0], args.saves[1], **kw)
|
|
||||||
else:
|
|
||||||
a = checksum_save(args.saves[0], **kw)
|
|
||||||
b = checksum_save(args.saves[1], **kw)
|
|
||||||
entries = diff(a.root, b.root, limit=args.limit)
|
entries = diff(a.root, b.root, limit=args.limit)
|
||||||
if args.ulps is not None:
|
if args.ulps is not None:
|
||||||
for e in entries:
|
for e in entries:
|
||||||
if e.kind == "value" and e.ulps is not None and e.ulps <= args.ulps:
|
if e.kind == "value" and e.ulps is not None and e.ulps <= args.ulps:
|
||||||
e.note += f" <= {args.ulps:g} ULP"
|
e.note += f" <= {args.ulps:g} ULP"
|
||||||
if args.json:
|
if args.json:
|
||||||
out = {"a": a.as_dict(0), "b": b.as_dict(0),
|
print(json.dumps({"a": a.as_dict(0), "b": b.as_dict(0),
|
||||||
"identical": a.digest == b.digest,
|
"identical": a.digest == b.digest,
|
||||||
"diffs": [e.as_dict() for e in entries]}
|
"diffs": [e.as_dict() for e in entries]}, indent=1))
|
||||||
if args.relabel_new_ids:
|
|
||||||
out["relabel"] = _relabel_dict(rl, refusal, args.relabel_new_ids)
|
|
||||||
print(json.dumps(out, indent=1))
|
|
||||||
return 0 if a.digest == b.digest else 1
|
return 0 if a.digest == b.digest else 1
|
||||||
print(f"A {a.digest} {a.path}")
|
print(f"A {a.digest} {a.path}")
|
||||||
print(f"B {b.digest} {b.path}")
|
print(f"B {b.digest} {b.path}")
|
||||||
print(f"policy: floats={a.policy} mask={a.mask} reader={READER_FINGERPRINT}"
|
print(f"policy: floats={a.policy} mask={a.mask} reader={READER_FINGERPRINT}"
|
||||||
f"{_mask_note(a)}")
|
f"{_mask_note(a)}")
|
||||||
if args.relabel_new_ids:
|
|
||||||
for line in _relabel_report(rl, refusal, args.relabel_new_ids):
|
|
||||||
print(line)
|
|
||||||
for ck, nm in ((a, "A"), (b, "B")):
|
for ck, nm in ((a, "A"), (b, "B")):
|
||||||
if ck.coverage.get("ok") is False:
|
if ck.coverage.get("ok") is False:
|
||||||
print(f"!! {nm}: coverage FAILED at 0x{ck.coverage.get('firstDiff') or 0:x} "
|
print(f"!! {nm}: coverage FAILED at 0x{ck.coverage.get('firstDiff') or 0:x} "
|
||||||
f"-- the digest does not bound this file")
|
f"-- the digest does not bound this file")
|
||||||
if a.digest == b.digest:
|
if a.digest == b.digest:
|
||||||
if rl is not None:
|
print("IDENTICAL")
|
||||||
print(f"IDENTICAL modulo pi = {format_permutation(rl.pi)}")
|
|
||||||
else:
|
|
||||||
print("IDENTICAL")
|
|
||||||
return 0
|
return 0
|
||||||
modulo = f" modulo pi = {format_permutation(rl.pi)}" if rl is not None else ""
|
print(f"DIVERGED: {len(entries)} leaf difference(s)"
|
||||||
print(f"DIVERGED{modulo}: {len(entries)} leaf difference(s)"
|
|
||||||
f"{' (limit reached)' if len(entries) >= args.limit else ''}")
|
f"{' (limit reached)' if len(entries) >= args.limit else ''}")
|
||||||
for e in entries:
|
for e in entries:
|
||||||
print(f" {e}")
|
print(f" {e}")
|
||||||
|
|
|
||||||
|
|
@ -67,64 +67,6 @@ def tiny(status_a=4, status_b=4, checksum=-1000, extra_ids=(),
|
||||||
return w.bytes()
|
return w.bytes()
|
||||||
|
|
||||||
|
|
||||||
#: (id, name, LocID, FPlan destination or None, [ship ids])
|
|
||||||
PRE_FLEETS = [(258, "Alpha Fleet", 100, None, [16]),
|
|
||||||
(274, "Beta Fleet", 0, 700, [32, 48])]
|
|
||||||
|
|
||||||
|
|
||||||
def fleets_save(fleets, sys_flt=None, extra=None) -> bytes:
|
|
||||||
"""A miniature save carrying a master fleet table.
|
|
||||||
|
|
||||||
Shaped like the real one where this tool looks: `FleetIDs[]`/`ShipIDs[]`
|
|
||||||
inline master lists, `FltID`/`Flt` object pairs, `FtName`, `LocID`, an
|
|
||||||
`FPlan` with a `pnd` destination, and `ShipID`/`Ship` pairs whose `Ship`
|
|
||||||
frame carries the `FltID` back-reference.
|
|
||||||
"""
|
|
||||||
w = sw.SaveWriter("joint")
|
|
||||||
w.begin("Summary")
|
|
||||||
w.string("GameName", "T")
|
|
||||||
w.int("Turn", 28)
|
|
||||||
w.int("Checksum", sum(f[0] for f in fleets))
|
|
||||||
w.end()
|
|
||||||
w.begin("Sim")
|
|
||||||
w.string("KeyPath", "")
|
|
||||||
w.int("NMSz", 16)
|
|
||||||
w.int("NMLc", 0)
|
|
||||||
w.int("NMnx", 100)
|
|
||||||
w.int("PlayerIDs", 1)
|
|
||||||
w.int(".", 32)
|
|
||||||
w.int("FleetIDs", len(fleets))
|
|
||||||
for f in fleets:
|
|
||||||
w.int(".", f[0])
|
|
||||||
ships = [s for f in fleets for s in f[4]]
|
|
||||||
w.int("ShipIDs", len(ships))
|
|
||||||
for s in ships:
|
|
||||||
w.int(".", s)
|
|
||||||
if extra is not None:
|
|
||||||
w.int(extra[0], extra[1])
|
|
||||||
if sys_flt is not None:
|
|
||||||
w.int("Flt", sys_flt)
|
|
||||||
w.int("NumFlts", len(fleets))
|
|
||||||
for fid, name, loc, dest, fships in fleets:
|
|
||||||
w.int("FltID", fid)
|
|
||||||
w.begin("Flt")
|
|
||||||
w.string("FtName", name)
|
|
||||||
w.int("LocID", loc)
|
|
||||||
if dest is not None:
|
|
||||||
w.begin("FPlan")
|
|
||||||
w.int("pnd", dest)
|
|
||||||
w.end()
|
|
||||||
w.int("NumShips", len(fships))
|
|
||||||
for s in fships:
|
|
||||||
w.int("ShipID", s)
|
|
||||||
w.begin("Ship")
|
|
||||||
w.int("FltID", fid)
|
|
||||||
w.end()
|
|
||||||
w.end()
|
|
||||||
w.end()
|
|
||||||
return w.bytes()
|
|
||||||
|
|
||||||
|
|
||||||
def find_saves() -> list[str]:
|
def find_saves() -> list[str]:
|
||||||
d = os.environ.get("SOTS_SAVES_DIR") or os.path.join(_VERIFY, "results", "saves")
|
d = os.environ.get("SOTS_SAVES_DIR") or os.path.join(_VERIFY, "results", "saves")
|
||||||
if not os.path.isdir(d):
|
if not os.path.isdir(d):
|
||||||
|
|
@ -360,222 +302,6 @@ class MaskTest(unittest.TestCase):
|
||||||
self.assertEqual(a.mask_hits, {})
|
self.assertEqual(a.mask_hits, {})
|
||||||
|
|
||||||
|
|
||||||
# --- --relabel-new-ids --------------------------------------------------------
|
|
||||||
#
|
|
||||||
# The feature is small; the guards are the point. Every test below that asserts
|
|
||||||
# a REFUSAL is asserting that a real difference cannot be absorbed.
|
|
||||||
|
|
||||||
class RelabelTest(unittest.TestCase):
|
|
||||||
A_NEW = [(1970, "Sigma Fleet", 384, None, [6976]),
|
|
||||||
(1986, "Tau Fleet", 0, 80, [5264])]
|
|
||||||
#: the same two groups, visited in the other order: the id and the
|
|
||||||
#: id-attached name stay put, the contents swap
|
|
||||||
B_NEW = [(1970, "Sigma Fleet", 0, 80, [5264]),
|
|
||||||
(1986, "Tau Fleet", 384, None, [6976])]
|
|
||||||
|
|
||||||
def trees(self, a_new, b_new, pre=PRE_FLEETS, **kw):
|
|
||||||
p = sr.read_bytes(fleets_save(pre))
|
|
||||||
a = sr.read_bytes(fleets_save(pre + list(a_new), **kw))
|
|
||||||
b = sr.read_bytes(fleets_save(pre + list(b_new), **kw))
|
|
||||||
return p.tree, a.tree, b.tree, a, b
|
|
||||||
|
|
||||||
def compare(self, a_res, b_res, rl):
|
|
||||||
sc.apply_relabelling(a_res.tree, rl)
|
|
||||||
ca = sc.checksum_result(a_res, "A", {"ok": None},
|
|
||||||
extra_masks=sc.RELABEL_MASK, id_lists_as_sets=True)
|
|
||||||
cb = sc.checksum_result(b_res, "B", {"ok": None},
|
|
||||||
extra_masks=sc.RELABEL_MASK, id_lists_as_sets=True)
|
|
||||||
return ca, cb, sc.diff(ca.root, cb.root)
|
|
||||||
|
|
||||||
# -- the feature ---------------------------------------------------------
|
|
||||||
|
|
||||||
def test_a_pure_transposition_is_absorbed_and_pi_is_reported(self):
|
|
||||||
p, a, b, ares, bres = self.trees(self.A_NEW, self.B_NEW)
|
|
||||||
rl = sc.build_relabelling(p, a, b)
|
|
||||||
self.assertEqual(rl.pi, {1970: 1986, 1986: 1970})
|
|
||||||
self.assertEqual(sc.format_permutation(rl.pi), "{1970<->1986}")
|
|
||||||
ca, cb, d = self.compare(ares, bres, rl)
|
|
||||||
self.assertEqual(d, [], [repr(e) for e in d])
|
|
||||||
self.assertEqual(ca.digest, cb.digest)
|
|
||||||
|
|
||||||
def test_the_ship_to_fleet_back_reference_is_rewritten(self):
|
|
||||||
p, a, b, ares, bres = self.trees(self.A_NEW, self.B_NEW)
|
|
||||||
rl = sc.apply_relabelling(ares.tree, sc.build_relabelling(p, a, b))
|
|
||||||
self.assertEqual(rl.rewrites.get("FltID"), 2)
|
|
||||||
self.assertEqual(rl.moves, 2)
|
|
||||||
|
|
||||||
def test_a_system_s_fleet_reference_is_rewritten(self):
|
|
||||||
p, a, b, ares, bres = self.trees(self.A_NEW, self.B_NEW)
|
|
||||||
# the same physical group is at system 384 in both, so the system's
|
|
||||||
# fleet reference names 1970 in A and 1986 in B
|
|
||||||
_, a2, _, ares2, _ = self.trees(self.A_NEW, self.B_NEW, sys_flt=1970)
|
|
||||||
_, _, b2, _, bres2 = self.trees(self.A_NEW, self.B_NEW, sys_flt=1986)
|
|
||||||
rl = sc.build_relabelling(p, a2, b2)
|
|
||||||
ca, cb, d = self.compare(ares2, bres2, rl)
|
|
||||||
self.assertEqual(rl.rewrites.get("Flt"), 1)
|
|
||||||
self.assertEqual(d, [], [repr(e) for e in d])
|
|
||||||
|
|
||||||
def test_the_identity_case_leaves_everything_alone(self):
|
|
||||||
p, a, b, ares, bres = self.trees(self.A_NEW, self.A_NEW)
|
|
||||||
rl = sc.build_relabelling(p, a, b)
|
|
||||||
self.assertEqual(rl.moved, {})
|
|
||||||
self.assertFalse(bool(rl))
|
|
||||||
ca, cb, d = self.compare(ares, bres, rl)
|
|
||||||
self.assertEqual(d, [])
|
|
||||||
|
|
||||||
def test_pi_is_printed_as_a_cycle_when_it_is_not_a_transposition(self):
|
|
||||||
self.assertEqual(sc.format_permutation({1: 2, 2: 3, 3: 1}), "{1->2->3->1}")
|
|
||||||
self.assertIn("identity", sc.format_permutation({1970: 1970}))
|
|
||||||
|
|
||||||
def test_a_relabelled_root_is_domain_separated_from_a_plain_one(self):
|
|
||||||
"""A 'modulo pi' root must never be mistaken for a strict one."""
|
|
||||||
data = fleets_save(PRE_FLEETS)
|
|
||||||
plain = sc.checksum_bytes(data)
|
|
||||||
loose = sc.checksum_bytes(data, extra_masks=sc.RELABEL_MASK,
|
|
||||||
id_lists_as_sets=True)
|
|
||||||
self.assertNotEqual(plain.digest, loose.digest)
|
|
||||||
|
|
||||||
def test_the_default_path_is_untouched_by_the_new_parameters(self):
|
|
||||||
data = fleets_save(PRE_FLEETS)
|
|
||||||
self.assertEqual(sc.checksum_bytes(data).digest,
|
|
||||||
sc.checksum_bytes(data, extra_masks=(),
|
|
||||||
id_lists_as_sets=False).digest)
|
|
||||||
|
|
||||||
# -- G1: nothing present in the pre-turn save may be relabelled ----------
|
|
||||||
|
|
||||||
def test_ids_present_in_the_pre_turn_save_are_never_in_pi(self):
|
|
||||||
p, a, b, _, _ = self.trees(self.A_NEW, self.B_NEW)
|
|
||||||
rl = sc.build_relabelling(p, a, b)
|
|
||||||
pre_ids = {n.value for n, _ in sc._fleet_slots(p)}
|
|
||||||
self.assertTrue(pre_ids)
|
|
||||||
self.assertFalse(pre_ids & (set(rl.pi) | set(rl.pi.values())))
|
|
||||||
|
|
||||||
def test_two_pre_turn_fleets_that_swapped_contents_are_NOT_absorbed(self):
|
|
||||||
"""The strongest form of G1: the very shape this mode absorbs, but on
|
|
||||||
ids that already existed. That would be a real divergence."""
|
|
||||||
swapped = [(258, "Alpha Fleet", 0, 700, [32, 48]),
|
|
||||||
(274, "Beta Fleet", 100, None, [16])]
|
|
||||||
p = sr.read_bytes(fleets_save(PRE_FLEETS)).tree
|
|
||||||
a = sr.read_bytes(fleets_save(PRE_FLEETS)).tree
|
|
||||||
b = sr.read_bytes(fleets_save(swapped)).tree
|
|
||||||
rl = sc.build_relabelling(p, a, b)
|
|
||||||
self.assertEqual(rl.pi, {})
|
|
||||||
ares = sr.read_bytes(fleets_save(PRE_FLEETS))
|
|
||||||
bres = sr.read_bytes(fleets_save(swapped))
|
|
||||||
_, _, d = self.compare(ares, bres, rl)
|
|
||||||
self.assertTrue(d, "a swap among pre-existing fleets must still report")
|
|
||||||
|
|
||||||
# -- G2: only client-minted (non-zero node nibble) ids ------------------
|
|
||||||
|
|
||||||
def test_server_minted_ids_are_never_relabelled(self):
|
|
||||||
"""Node nibble 0 is the server's space (id-allocation.md section 1)."""
|
|
||||||
a_new = [(7120, "Phi Fleet", 384, None, [6976]),
|
|
||||||
(7136, "Chi Fleet", 0, 80, [5264])]
|
|
||||||
b_new = [(7120, "Phi Fleet", 0, 80, [5264]),
|
|
||||||
(7136, "Chi Fleet", 384, None, [6976])]
|
|
||||||
p, a, b, ares, bres = self.trees(a_new, b_new)
|
|
||||||
rl = sc.build_relabelling(p, a, b)
|
|
||||||
self.assertEqual(rl.pi, {})
|
|
||||||
self.assertEqual(sc.id_node(7120), 0)
|
|
||||||
_, _, d = self.compare(ares, bres, rl)
|
|
||||||
self.assertTrue(d, "a node-0 swap is not this residue and must report")
|
|
||||||
|
|
||||||
# -- G3: pi must permute one set of new ids -----------------------------
|
|
||||||
|
|
||||||
def test_different_new_id_sets_are_refused(self):
|
|
||||||
"""Lane AD's unpinned pair in miniature: one run minted a third fleet."""
|
|
||||||
b_new = self.B_NEW + [(2002, "Upsilon Fleet", 0, 384, [6992])]
|
|
||||||
p, a, b, _, _ = self.trees(self.A_NEW, b_new)
|
|
||||||
with self.assertRaises(sc.RelabelRefused) as cm:
|
|
||||||
sc.build_relabelling(p, a, b)
|
|
||||||
self.assertIn("only in B [2002]", str(cm.exception))
|
|
||||||
|
|
||||||
def test_a_refusal_leaves_both_trees_untouched(self):
|
|
||||||
b_new = self.B_NEW + [(2002, "Upsilon Fleet", 0, 384, [6992])]
|
|
||||||
p, a, b, ares, bres = self.trees(self.A_NEW, b_new)
|
|
||||||
before = sc.checksum_result(ares, "A", {"ok": None}).digest
|
|
||||||
with self.assertRaises(sc.RelabelRefused):
|
|
||||||
sc.build_relabelling(p, a, b)
|
|
||||||
self.assertEqual(sc.checksum_result(ares, "A", {"ok": None}).digest, before)
|
|
||||||
|
|
||||||
# -- G4: the content match must be a one-to-one correspondence ----------
|
|
||||||
|
|
||||||
def test_new_fleets_whose_contents_differ_are_refused(self):
|
|
||||||
b_new = [(1970, "Sigma Fleet", 0, 80, [5264]),
|
|
||||||
(1986, "Tau Fleet", 384, None, [9999])] # a different ship
|
|
||||||
p, a, b, _, _ = self.trees(self.A_NEW, b_new)
|
|
||||||
with self.assertRaises(sc.RelabelRefused) as cm:
|
|
||||||
sc.build_relabelling(p, a, b)
|
|
||||||
self.assertIn("do not correspond by content", str(cm.exception))
|
|
||||||
|
|
||||||
def test_an_ambiguous_content_key_is_refused(self):
|
|
||||||
dup = [(1970, "Sigma Fleet", 384, None, [6976]),
|
|
||||||
(1986, "Tau Fleet", 384, None, [6976])]
|
|
||||||
p, a, b, _, _ = self.trees(dup, dup)
|
|
||||||
with self.assertRaises(sc.RelabelRefused) as cm:
|
|
||||||
sc.build_relabelling(p, a, b)
|
|
||||||
self.assertIn("ambiguous", str(cm.exception))
|
|
||||||
|
|
||||||
def test_a_fleet_at_a_system_never_keys_the_same_as_one_bound_for_it(self):
|
|
||||||
at = sr.read_bytes(fleets_save([(1970, "S", 384, None, [1])])).tree
|
|
||||||
to = sr.read_bytes(fleets_save([(1970, "S", 0, 384, [1])])).tree
|
|
||||||
ka = sc._fleet_content_key(sc._fleet_slots(at)[0][1])
|
|
||||||
kb = sc._fleet_content_key(sc._fleet_slots(to)[0][1])
|
|
||||||
self.assertNotEqual(ka, kb)
|
|
||||||
|
|
||||||
# -- the name guard ------------------------------------------------------
|
|
||||||
|
|
||||||
def test_a_name_that_does_not_track_the_id_is_refused(self):
|
|
||||||
"""`FtName` is minted from the same per-fleet ordinal as the id, so it
|
|
||||||
is an id-attached label -- but only while it demonstrably tracks the
|
|
||||||
id in both saves. If it does not, something real differs."""
|
|
||||||
b_new = [(1970, "Tau Fleet", 0, 80, [5264]),
|
|
||||||
(1986, "Sigma Fleet", 384, None, [6976])]
|
|
||||||
p, a, b, _, _ = self.trees(self.A_NEW, b_new)
|
|
||||||
with self.assertRaises(sc.RelabelRefused) as cm:
|
|
||||||
sc.build_relabelling(p, a, b)
|
|
||||||
self.assertIn("does not track the id", str(cm.exception))
|
|
||||||
|
|
||||||
def test_the_name_stays_with_the_slot_not_with_the_body(self):
|
|
||||||
p, a, b, ares, bres = self.trees(self.A_NEW, self.B_NEW)
|
|
||||||
rl = sc.apply_relabelling(ares.tree, sc.build_relabelling(p, a, b))
|
|
||||||
got = {n.value: sc._fleet_name(f) for n, f in sc._fleet_slots(ares.tree)}
|
|
||||||
self.assertEqual(got[1970], "Sigma Fleet")
|
|
||||||
self.assertEqual(got[1986], "Tau Fleet")
|
|
||||||
|
|
||||||
# -- G5: the rewrite must be complete -----------------------------------
|
|
||||||
|
|
||||||
def test_an_unmodelled_reference_site_is_refused(self):
|
|
||||||
"""A leaf holding a permuted id at a tag this tool does not model would
|
|
||||||
keep the old label through the rewrite, so two different states could
|
|
||||||
compare equal. That is a refusal, not a warning."""
|
|
||||||
p, a, b, _, _ = self.trees(self.A_NEW, self.B_NEW,
|
|
||||||
extra=("SomeOtherRef", 1970))
|
|
||||||
with self.assertRaises(sc.RelabelRefused) as cm:
|
|
||||||
sc.build_relabelling(p, a, b)
|
|
||||||
self.assertIn("does not model", str(cm.exception))
|
|
||||||
self.assertIn("SomeOtherRef", str(cm.exception))
|
|
||||||
|
|
||||||
def test_the_completeness_scan_matches_raw_bytes_not_the_typed_value(self):
|
|
||||||
"""The digest is a function of (bytes, reader schema); a leaf the schema
|
|
||||||
types as a float still carries the id. A value-based scan would miss
|
|
||||||
it and the guard would be silently empty (rule 1)."""
|
|
||||||
node = sr.Node("Odd", "float", 2.76e-42, 0, 12,
|
|
||||||
raw=struct.pack("<i", 1970))
|
|
||||||
frame = sr.Node("Sim", "complex", None, 0, 0)
|
|
||||||
frame.children = [node]
|
|
||||||
root = sr.Node(None, "complex", None, 0, 0)
|
|
||||||
root.children = [frame]
|
|
||||||
found = sc._id_occurrences(root, {1970})
|
|
||||||
self.assertEqual([s for s, _, _ in found], ["UNMODELLED"])
|
|
||||||
|
|
||||||
# -- CLI -----------------------------------------------------------------
|
|
||||||
|
|
||||||
def test_the_flag_is_rejected_outside_a_two_save_comparison(self):
|
|
||||||
with self.assertRaises(SystemExit):
|
|
||||||
sc.main(["one.sav", "--relabel-new-ids", "pre.sav"])
|
|
||||||
|
|
||||||
|
|
||||||
# --- chain --------------------------------------------------------------------
|
# --- chain --------------------------------------------------------------------
|
||||||
|
|
||||||
class ChainTest(unittest.TestCase):
|
class ChainTest(unittest.TestCase):
|
||||||
|
|
@ -691,20 +417,12 @@ class RealSaveTest(unittest.TestCase):
|
||||||
self.assertEqual(len(d), 5, [repr(e) for e in d])
|
self.assertEqual(len(d), 5, [repr(e) for e in d])
|
||||||
statuses = [e for e in d if e.path.endswith("/Status")]
|
statuses = [e for e in d if e.path.endswith("/Status")]
|
||||||
self.assertEqual(len(statuses), 4)
|
self.assertEqual(len(statuses), 4)
|
||||||
# `pairs` is enumerated over sorted filenames, so the re-saved file
|
|
||||||
# can be either side; the claim is symmetric, so assert it that way
|
|
||||||
# rather than assuming which name sorts first (2026-09-09: adding
|
|
||||||
# `cb-turn2to3-endturn.sav` to the corpus put one pair the other
|
|
||||||
# way round and this test failed on a corpus fact, not a defect).
|
|
||||||
direction = (statuses[0].a, statuses[0].b)
|
|
||||||
self.assertIn(direction, ((4, 0), (0, 4)))
|
|
||||||
for e in statuses:
|
for e in statuses:
|
||||||
self.assertIn("/Sim/players/Player[", e.path)
|
self.assertIn("/Sim/players/Player[", e.path)
|
||||||
self.assertEqual((e.a, e.b), direction)
|
self.assertEqual((e.a, e.b), (4, 0))
|
||||||
chk = [e for e in d if e.path == "/Summary/Checksum"]
|
chk = [e for e in d if e.path == "/Summary/Checksum"]
|
||||||
self.assertEqual(len(chk), 1)
|
self.assertEqual(len(chk), 1)
|
||||||
self.assertEqual(chk[0].b - chk[0].a,
|
self.assertEqual(chk[0].b - chk[0].a, -16) # 4 x (4 -> 0)
|
||||||
-16 if direction == (4, 0) else 16)
|
|
||||||
|
|
||||||
def test_resave_mask_makes_that_pair_identical(self):
|
def test_resave_mask_makes_that_pair_identical(self):
|
||||||
found = False
|
found = False
|
||||||
|
|
@ -721,69 +439,6 @@ class RealSaveTest(unittest.TestCase):
|
||||||
self.skipTest("no re-save pair among the available saves")
|
self.skipTest("no re-save pair among the available saves")
|
||||||
self.assertTrue(found)
|
self.assertTrue(found)
|
||||||
|
|
||||||
# -- --relabel-new-ids acceptance, both halves -------------------------
|
|
||||||
|
|
||||||
def _saves(self, *names):
|
|
||||||
d = os.path.dirname(REAL[0])
|
|
||||||
paths = [os.path.join(d, n) for n in names]
|
|
||||||
if not all(os.path.exists(p) for p in paths):
|
|
||||||
self.skipTest(f"need {names}")
|
|
||||||
return paths
|
|
||||||
|
|
||||||
def _relabelled(self, pre, a, b):
|
|
||||||
return sc._relabelled_pair(pre, a, b, floats="bits", mask="none", audit=False)
|
|
||||||
|
|
||||||
def test_POSITIVE_bp_pinned_pair_is_identical_modulo_one_transposition(self):
|
|
||||||
"""Lane BP's two processes with identical pinned AI seeds differ in 35
|
|
||||||
of 61,147 leaves; all 35 are fleets 1970 and 1986 exchanging contents.
|
|
||||||
Under pi they must read IDENTICAL, and pi must be exactly that swap."""
|
|
||||||
pre, a, b = self._saves("bp-turn28-pre.sav", "bp-pinA-turn28.sav",
|
|
||||||
"bp-pinB-turn28.sav")
|
|
||||||
ca, cb, rl, refusal = self._relabelled(pre, a, b)
|
|
||||||
self.assertIsNone(refusal)
|
|
||||||
self.assertEqual(rl.moved, {1970: 1986, 1986: 1970})
|
|
||||||
self.assertEqual(rl.new_ids, [1970, 1986, 2002])
|
|
||||||
self.assertEqual(rl.nodes, [2])
|
|
||||||
self.assertEqual(sc.diff(ca.root, cb.root), [])
|
|
||||||
self.assertEqual(ca.digest, cb.digest)
|
|
||||||
|
|
||||||
def test_POSITIVE_the_same_pair_still_diverges_without_the_flag(self):
|
|
||||||
pre, a, b = self._saves("bp-turn28-pre.sav", "bp-pinA-turn28.sav",
|
|
||||||
"bp-pinB-turn28.sav")
|
|
||||||
d = sc.diff(real_ck(a, audit=False).root, real_ck(b, audit=False).root)
|
|
||||||
self.assertEqual(len(d), 35, [repr(e) for e in d])
|
|
||||||
|
|
||||||
def test_NEGATIVE_ad_unpinned_pair_is_refused_and_still_diverges(self):
|
|
||||||
"""Lane AD's pair differs because the seeds differ -- real decisions.
|
|
||||||
A tool that made this look identical would be worthless."""
|
|
||||||
pre, a, b = self._saves("ad-turn27-two-raiders.sav", "ad-oracle-A-post.sav",
|
|
||||||
"ad-oracle-B-post.sav")
|
|
||||||
ca, cb, rl, refusal = self._relabelled(pre, a, b)
|
|
||||||
self.assertIsNone(rl, "AD's pair must not be relabelled")
|
|
||||||
self.assertIn("2002", refusal)
|
|
||||||
self.assertNotEqual(ca.digest, cb.digest)
|
|
||||||
d = sc.diff(ca.root, cb.root, limit=500)
|
|
||||||
self.assertEqual(len(d), 94, [repr(e) for e in d])
|
|
||||||
|
|
||||||
def test_NEGATIVE_even_forcing_a_partial_relabelling_cannot_hide_it(self):
|
|
||||||
"""Belt and braces: hand the applier the transposition it *would* have
|
|
||||||
found if G3 were absent. The pair must still diverge -- the guard is
|
|
||||||
not the only thing standing between AD's pair and a green verdict."""
|
|
||||||
pre, a, b = self._saves("ad-turn27-two-raiders.sav", "ad-oracle-A-post.sav",
|
|
||||||
"ad-oracle-B-post.sav")
|
|
||||||
with open(a, "rb") as f:
|
|
||||||
ares = sr.read_bytes(f.read())
|
|
||||||
with open(b, "rb") as f:
|
|
||||||
bres = sr.read_bytes(f.read())
|
|
||||||
forced = sc.Relabelling({1970: 1986, 1986: 1970}, [2], [1970, 1986],
|
|
||||||
{1970: None, 1986: None})
|
|
||||||
sc.apply_relabelling(ares.tree, forced)
|
|
||||||
ca = sc.checksum_result(ares, a, {"ok": None},
|
|
||||||
extra_masks=sc.RELABEL_MASK, id_lists_as_sets=True)
|
|
||||||
cb = sc.checksum_result(bres, b, {"ok": None},
|
|
||||||
extra_masks=sc.RELABEL_MASK, id_lists_as_sets=True)
|
|
||||||
self.assertNotEqual(ca.digest, cb.digest)
|
|
||||||
|
|
||||||
def test_a_real_turn_transition_localises_to_named_objects(self):
|
def test_a_real_turn_transition_localises_to_named_objects(self):
|
||||||
by_turn = {}
|
by_turn = {}
|
||||||
for p in REAL:
|
for p in REAL:
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue