From b6018a0f19593b5e55eb9f919554d2f03ec3dd8f Mon Sep 17 00:00:00 2001 From: alex Date: Wed, 9 Sep 2026 09:40:29 -0400 Subject: [PATCH] plan: pivot to displacement; gate.sh replaces the hollow gate; Rung B worklist from lane CV --- campaign/DASHBOARD.md | 26 ++++++------ campaign/backlog.md | 93 ++++++++++++++++++++++++++++++++++++++++++- campaign/board.md | 3 ++ tools/gate.sh | 63 +++++++++++++++++++++++++++++ 4 files changed, 170 insertions(+), 15 deletions(-) create mode 100755 tools/gate.sh diff --git a/campaign/DASHBOARD.md b/campaign/DASHBOARD.md index cc6c47c..cc49b48 100644 --- a/campaign/DASHBOARD.md +++ b/campaign/DASHBOARD.md @@ -1,17 +1,17 @@ # SotS RE campaign — coverage dashboard -Generated 2026-09-09 08:13 UTC · `sots-re` @ d46b3f8,2026-09-09 · `sots-engine` @ e7e2bd6,2026-09-09 (245 commits) · regenerate with `tools/dashboard.py` +Generated 2026-09-09 13:40 UTC · `sots-re` @ c4ea347,2026-09-09 · `sots-engine` @ e7e2bd6,2026-09-09 (245 commits) · regenerate with `tools/dashboard.py` > **North star:** A functional reimplementation of the engine — behavior-equivalent, NOT byte-for-byte ## 1. Map coverage (campaign/board.md) -444 targets · mapped-or-better **396/444** `[█████████░] 89%` · verified **354/444** `[████████░░] 80%` +447 targets · mapped-or-better **399/447** `[█████████░] 89%` · verified **356/447** `[████████░░] 80%` | Status | Count | % | |---|---:|---:| -| verified | 354 | 80% | -| mapped | 42 | 9% | +| verified | 356 | 80% | +| mapped | 43 | 10% | | in-progress | 5 | 1% | | backlog | 41 | 9% | | blocked | 2 | 0% | @@ -21,17 +21,17 @@ Generated 2026-09-09 08:13 UTC · `sots-re` @ d46b3f8,2026-09-09 · `sots-engine | objects | 46 | 6 | 0 | 3 | 1 | 56 | | control-flow | 41 | 2 | 1 | 0 | 0 | 44 | | subsystems | 4 | 8 | 0 | 2 | 1 | 15 | -| engine | 32 | 0 | 0 | 0 | 0 | 32 | +| engine | 33 | 1 | 0 | 0 | 0 | 34 | | verify | 105 | 15 | 3 | 35 | 0 | 158 | | phase2 | 13 | 3 | 1 | 0 | 0 | 17 | -| meta | 93 | 6 | 0 | 1 | 0 | 100 | +| meta | 94 | 6 | 0 | 1 | 0 | 101 | | other | 20 | 2 | 0 | 0 | 0 | 22 | ## 2. Binary understanding - RTTI type descriptors: **1,924** (`Game::` 1,404, `Mars::` 194; serializable types 179) - Classes with recovered member layouts: **384** / 1,598 named classes `[██░░░░░░░░] 24%` — `objects/layouts.json` (serializer recovery) plus classes recovered by hand in `struct-recovery.md` + `schema-gaps-resolved.md`. Note 179 types are *serializable*; the recovery also reaches non-serializable ones, so this is not a subset of that -- Functions: **41,411** (parsed from `01-fingerprint.md`); named/annotated in the **address contract** (`ghidra/addresses.json`, not Ghidra's full rename count): **1294**, verified **1163** `[█████████░] 90%` +- Functions: **41,411** (parsed from `01-fingerprint.md`); named/annotated in the **address contract** (`ghidra/addresses.json`, not Ghidra's full rename count): **1298**, verified **1167** `[█████████░] 90%` ## 3. Data layer @@ -66,7 +66,7 @@ Generated 2026-09-09 08:13 UTC · `sots-re` @ d46b3f8,2026-09-09 · `sots-engine | `shim/trace` | 2,258 | 9 | 273 | direct (WIN32) | shim-trace.md | | **total** | **59,514** | **122** | **4257** | | | -Board `engine:` rows: verified **32**, mapped 0, in flight 0 (of 32) — verified & merged `[██████████] 100%` +Board `engine:` rows: verified **33**, mapped 1, in flight 0 (of 34) — verified & merged `[██████████] 97%` ## 5. Phase 2 milestones @@ -121,7 +121,7 @@ Detail: `verify/results/standalone/report.txt`. ## 8. Open questions -Open **26** · resolved/parked 11 · backlog items: 1. The engine today 0, 2. The milestone, restated 5, 3. Engine work to Rung A 0, 4. Future engine modules 5, 5. Workloads to manufacture 7, 6. Multiplayer revival 4, 7. Method and infrastructure 5, Parked 2 +Open **26** · resolved/parked 11 · backlog items: 1. The engine today 0, 2. The milestone, restated — 2026-09-09 7, 2-old. The milestone as restated 2026-09-08 5, 3. The plan — three tracks, in priority order 0, 4. Future engine modules 5, 5. Workloads to manufacture 7, 6. Multiplayer revival 4, 7. Method and infrastructure 5, Parked 2 Most recent open: @@ -133,11 +133,11 @@ Most recent open: ## 9. Delta since previous dashboard -- verified targets: 353 → 354 (+1) · mapped-or-better: 395 → 396 (+1) -- engine LOC: 59,504 → 59,514 (+10) · test files: 122 → 122 (+0) · checks: 4,257 → 4,257 (+0) -- addresses verified: 1,163 → 1,163 (+0) · recovered layouts: 384 → 384 (+0) · open questions: 26 → 26 (+0) +- verified targets: 354 → 356 (+2) · mapped-or-better: 396 → 399 (+3) +- engine LOC: 59,514 → 59,514 (+0) · test files: 122 → 122 (+0) · checks: 4,257 → 4,257 (+0) +- addresses verified: 1,163 → 1,167 (+4) · recovered layouts: 384 → 384 (+0) · open questions: 26 → 26 (+0) - standalone leaves closed: 45 → 45 (+0) · leaves still diverging: 63 → 63 (+0) --- warnings: board.md: unknown types subsystems, tooling; mars-rng.md: no oracle total row parsed; mars-stream.md: no oracle total row parsed; mars-vfs.md: no oracle total row parsed - + diff --git a/campaign/backlog.md b/campaign/backlog.md index 09b03cd..49f3dac 100644 --- a/campaign/backlog.md +++ b/campaign/backlog.md @@ -25,7 +25,51 @@ run the economy/research/movement/output pieces to the original's numbers, and s **16 of 44** driver phases and **5 of 37** tail phases modelled (10 committed). It cannot yet reproduce a turn. -## 2. The milestone, restated (again — 2026-09-08 evening) +## 2. The milestone, restated — 2026-09-09 (after the determinism arc) + +> `sots_turn` loads a save, runs one strategic turn, and writes an autosave that byte-matches +> the original's — **for a stated procedure and route**. + +Two days of lanes went into determinism and the foundation is now laid. What is *measured*: + +- **The sim is deterministic given the command stream.** Every draw of a turn attributed by + return address; the strategic generator's residual is 0 on every measured turn. +- **The AI is deterministic given (seeds, visit order).** Its per-process seed is one word per + client, minted from the global generator — which does *nothing else* on an encounter-free turn. + Pinning the seeds collapses a rich turn's divergence from 94 leaves to **one transposition**, and + that transposition is the **order of two heap pointers**: the fleet-assignment pass walks a + container keyed on `fleet->Location`. There is **no original order to match**; we choose one + member of its outcome set and say so (`orders.h`). The visit order is *written into the block* + (list 10), so a captured run carries its own. +- **The route is part of the procedure.** The same turn reached by continuation and by load gives + different, individually reproducible results, because a `Colonize` task alive in the AI agent by + continuation is *absent* after a load. Certified pairs live in `verify/results/saves/certified-pairs.md` + as `(save, procedure, route, hashes, processes, masks)`; a hash without its route is not a claim. +- **Exposure is a screen, not a decision procedure** (rule 26(c)): it says a control is *likely* to + vary; it cannot certify one will agree. + +**So the rungs stand as written below, with one scope line each:** + +- **Rung A** — byte-match modulo the command stream, **on a load→turn procedure**. Canonical pair + `ref-turn2 → turn3`. 204 leaves open; the RE for them is done (§3-old items 1b/1c/7). +- **Rung B** — byte-match given the recorded stream. **Climbed to outcome 3 on the first rich turn + (lane CV):** the replay runs and consumes the whole stream; the result is a **ranked leaf list of + thirteen named subsystems, zero ungrouped** — 1,092 leaves. Two blockers sit *upstream* of the + turn (§3, track 0). The tail's sim draw fires on this workload (`tscr = 253`), so Rung B needs + the same generator model as Rung A — that correction is load-bearing here. +- **Rung C** — `game/ai` deterministic given (save, seeds, visit order). **C-exact survives + literally** on that basis; the comparison side (`--relabel-new-ids`) ships with both acceptance + halves. The engine side is not built: `game/ai` mints no fleet ids yet. + +**What stops now.** Further RNG/determinism lanes, unless a rung is actually blocked by one. Board +row count as a success signal — 89% of 444 rows measures campaign activity, not the product. The +value-domain census as a work generator — it is a screen; use it when a lane needs a corpus count. + +**What the headline is.** Displacement (`tools/displacement.py`): **3 replaced / 7 compared / +7 modelled / 3 mapped**, unmoved for two days. And the phase catalogue: 14 of 44 driver phases, +2 of 37 tail. Those two numbers are the product. + +## 2-old. The milestone as restated 2026-09-08 (kept for the record) > `sots_turn` loads a save, runs one strategic turn, and writes an autosave that byte-matches > the original's. @@ -87,7 +131,52 @@ candidate walk (lane L4) — the other two are stream-insensitive **by code path Rung B is unaffected: `--turn-commands` replay stays the no-AI path, and the canonical pair stays `turn2-state → turn3-state` until the pin probe promotes the creation turn. -## 3. Engine work to Rung A (ranked by leaves closed per lane) +## 3. The plan — three tracks, in priority order + +### Track 0 — make the gate honest, then clear the two upstream blockers + +The gate reported 59/59 for a day while the corpus tests **skipped** (`SOTS_SAVES_DIR` unset on +CT111). With the corpus it is **57/59**: the coverage ratchet broke when the corpus grew 22 → 43 +(rule 27, as designed), and a writer defect appeared. `tools/gate.sh --fresh` is now the gate and +it fails if a corpus test skips. + +| # | item | module | evidence | why first | +|---|---|---|---|---| +| 0a | **`ProjectName` reads `usnc` as one item; the wire is a `usp`/`usc` pair.** 12 of 43 saves round-trip 12 bytes short. | `mars/stream` | lane CV; `ad-turn27` has `usnc == 1` | no byte-match claim is possible on the `ad-*`/`ar-*` family until the writer is exact; re-prove byte-neutrality after | +| 0b | **Re-type what the ratchet caught** — the spy saves (`ap-*`) and whatever else dropped `pct` below 99.99. | `mars/stream` | `tools/gate.sh` output | the ratchet did its job; honour it | +| 0c | **List-5 applier cannot fire** because the memory capture cannot type 7 of 8 fields. One UI run (two sliders, two values) settles it. | shim capture | lane CV | the only list with an applier, at zero | + +### Track 1 — displacement: `compared → replaced`, then `modelled → compared` + +Only `replaced` means the original's code did not run. The bar, written once: an un-instrumented +oracle first (`certified-pairs.md` format), then the same procedure with the hook in `replace` +mode and the **same hashes**, with a count from the hook proving the path executed (rule 1). + +| # | module | today | evidence for the attempt | target | +|---|---|---|---|---| +| 1a | `TechTree::ProcessResearch` + unlock cascade | compared | 35 calls / 3 workloads / 0 div; advance prediction held on a changed workload | **replaced** — lane CR running | +| 1b | `ServerSystem::ComputeTotalOutput` + `GroupOutput` | compared | 24,357 calls, 0 undeclared writes — but **13 distinct states** (rule 23) | replaced, after widening the state set | +| 1c | `ServerSystem::ProcessTurn` (colony) | compared | 36 calls 0 div; 3 owned systems, gates all zero | replaced, on a save with gate traffic | +| 1d | `game/nav`, `game/design`, `game/combat` retreat, `mars/vfs` | modelled, **never hooked** | host-tested only | compared — one lane each | + +### Track 2 — Rung B worklist, ranked against a real rich turn (lane CV, residual 0) + +| leaves | subsystem | note | +|---:|---|---| +| 539 | morale event ring `cme2` | **not a module** — a fixed 11-slot ring shifted by 27 appended events of three kinds (colonisation, savings band, population). It is the reporting surface of the next three rows and closes when they do | +| 154 | colony growth / repair / bonuses | §3-old 1b/1c — `Pv*` is **mixed**: `PvPop`/`PvInfra` hold the uncapped projection, `PvSuit`/`PvRes` the pre-turn current | +| 150 | fleet objects | needs the gather/mint (Rung C engine side) | +| **79** | **`T34 RecordObservedDesigns`** | **largest independently closable stub, no upstream dependency — do this one first** | +| 30 | player money + counters | §3-old item 1 chain | +| 23 | `TShn` | watchpoint specified (§3-old 4) | +| 23 | system↔fleet membership | with fleet objects | +| 20 / 20 / 15 / 13 / 10 / 8 / 6 | ShipRecs · build queues · other system · crep/CD · id lists+generator · turnstats · trade manager | | +| 5 of 6 regressions | one `ntdev` predicate | cheapest fix in the list | + +`ModCount` residual 2 is localised to lists 1/7/12; the only uniform reading is list 7 at 2 bumps — +a prediction fitted to one observation, falsified by any capture with a colonize count ≠ 2. + +### §3-old — Rung A leaf ranking (still valid for the items it names) | # | engine deliverable | module | RE input (done unless noted) | closes | |---|---|---|---|---| diff --git a/campaign/board.md b/campaign/board.md index 91f2ec5..265cf63 100644 --- a/campaign/board.md +++ b/campaign/board.md @@ -449,3 +449,6 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`). | THE AI'S FLEET-VISIT ORDER IS AN ADDRESS-KEYED CONTAINER AND THE KEY IS THE FLEET'S `Location*` - H1 CONFIRMED, H1b DEAD, and the campaign's two-file outcome set IS the two orderings of two heap pointers | control-flow | verified | high | 100% | 2026-09-09 | Lane BU, VM146, stage 2 of the chain lane BR unblocked (`findings/control-flow/fleet-visit-order-mechanism.md`). Build `bu-55408a9-20260909T071101Z`, fresh CT111 tree (rule 24), lane worktree (rule 21), `shim.cfg.bupin` = `shim.cfg.bppin` with EXACTLY THREE non-comment lines added (`aivisit=on/.out/.ships`), diffed against the real bppin PULLED OFF THE GUEST and quoted in the finding, marked `# exhaustive` so `tools/check_shim_configs.py` ENFORCES that all 27 registered hooks are named. THREE fresh processes, route LOAD, seeds pinned to lane L's observed values; 28 of 28 hooks off, zero unknown keys, zero COVERAGE, 3/3 `pinned=1` in every run, three `aivisit ... MH_OK` in every run, `exe base=0x005e0000` for the 8th process on this guest. THE HOOK: `AssignFleetsAndIssueOrders 0x006c16c0` walks a `std::vector` by index (cursor `[ebp-0x38]`, `_Mylast` REFETCHED every iteration, back-edge `jne 0x6c17e0` at 0x006c21be), and that walk is BRACKETED BY TWO LOOPS OVER THE SAME VECTOR that call one function each - **`StrategyAIAgent::ClaimShipsOfFleet 0x006a4290` and `ReleaseShipsOfFleet 0x0069da10`, which have EXACTLY ONE CALLER EACH IN THE WHOLE IMAGE and it is that function**. So the instrument is two ordinary function-entry detours plus a bracket on 0x006c16c0, with NO return-address filter and NO mid-function patch - the loop body could not be patched anyway, its first instruction 0x006c17e6 IS a branch target. **RESULT: the visit order is NOT ascending element address (0 of 6 multi-element sequences over three processes), NOT ascending fleet id (0/6), NOT ascending systemId (0/6), NOT ascending minimum ship id (0/6). It is ASCENDING `fleet->Location` (StarFleet+0xa0): 6 OF 6**, with the fleet pointer breaking the one tie (two fleets at system 384 share one interned Location object and are ordered by element address within it). So the container is address-keyed and the comparator is lexicographic in (Location*, StarFleet*), BOTH heap addresses. **P3 - the committed prediction, "the visit order is ascending element address" - IS FALSIFIED, and the hypothesis class it tested is CONFIRMED with a sharper key.** THE CROSS-PROCESS PROOF, and it does not rest on one sample: three processes visited THE SAME SIX FLEETS IN THREE DIFFERENT ORDERS (run1 1970/1602/1986/354/1570/1346; run2 1602/1970/1986/354/1346/1570; run3 1970/1986/1602/354/1570/1346) and EVERY difference is a pair whose Location addresses exchanged relative order. Runs 1 and 2 wrote BYTE-IDENTICAL autosaves and still visited in different orders - the reordered pairs were pre-existing fleets, which is why the save did not move. **AND THE BRANCH FLIP IS MEASURED: run 3's `loc(system-80 group) = 0x333ed4c8 < loc(system-384 group) = 0x333ed710`, the OPPOSITE of runs 1 and 2, it visited system 80 first, and its autosave is `724528ff...` BYTE-IDENTICAL (cmp) to lane BP's run B - while runs 1 and 2 have `loc(384) < loc(80)`, visit 384 first, and are BYTE-IDENTICAL to BP's runs A and L.** So the campaign's two-file outcome set on `ad-turn27` IS the two orderings of two heap pointers. **H1b (an uninitialised-word comparator) is EXCLUDED for this path**: an uninitialised key cannot reproduce a strict total order by a specific live pointer field in 6 of 6 sequences and cannot track that field across a branch flip. **A SECOND FINDING THE STATIC READ HAD MISSED: the newly minted fleet ids are ALREADY ON THE FLEET OBJECTS when the head loop first touches them** (`fid=1970` on the element at idx=0 of the first burst, before the pass gate and before any emission), so `AssignFleetsAndIssueOrders` INHERITS the id-to-group pairing rather than deciding it; the mint is upstream in the gather (`AcquireFleetsForTask 0x006ceef0` + 0x006abf80/0x006b7c90/0x006cb310), and since the delivered vector is Location-ordered and the ids follow that order, the natural reading is that the SAME container orders the mint (INFERENCE - the gather was not hooked). ENGINE CONSEQUENCE, and it MOVES deliverable 2 by one level: the canonical order must be imposed WHERE THE GROUP SET IS GATHERED AND THE IDS ARE MINTED, not at the assignment step, or the engine canonicalises the emission order while leaving the minting order to its own container. On the choice of canonical order this lane has evidence rather than convention: the original's order is NOT ascending systemId and NOT ascending minimum ship id in ANY of the three processes, so THERE IS NO ORIGINAL ORDER TO MATCH and `(systemId, min ship id)` is as good as any - keep it, and keep the tie-break, because the original's own tie-break is a second pointer. RULE 26(d) AND RULE 19, BOTH MET BY MEASUREMENT: runs 1 and 2 are BYTE-IDENTICAL to lane BP's UNPINNED run L, so a process carrying a seed pin PLUS three new detours inside the AI's fleet-assignment path reproduced a natural run byte for byte, twice; run 3 reproduced BP's other branch byte for byte. `aivisit` is behaviourally neutral on this state, measured, not argued. P1/P2/P4/P5 all hold; under BP's noise mask every run's block differs from BP's same-branch run in NOTHING outside the noise set and from the other branch in EXACTLY the three transposition words. Rule 17 discharged: 0x006c16c0's body is 0x006c16c0-0x006c247a `ret` with a FOUR-ENTRY JUMP TABLE living PAST the ret at 0x006c247c (target of `jmp [eax*4+0x6c247c]` at 0x006c22fb). Six more per-process AI seeds, none shared - L1 reproduced a 6th and 7th time. Artefacts: `verify/results/shim/bu/`; the three output saves are byte-identical to `bp-pinA-turn28.sav` x2 and `bp-pinB-turn28.sav` x1 and are deliberately not duplicated | | the fleet-gather ordering contract is BOUND IN THE ENGINE, not just recorded | engine | verified | high | 100% | 2026-09-09 | Lane BU's finding turned into a thing the engine cannot get wrong later. `game/ai` takes `fleetId` as a **parameter** - it emits orders and does not form fleets or mint ids - so **the gather/mint subsystem does not exist yet**, and building it now to satisfy a contract would be speculative. Instead the requirement sits in `src/game/ai/orders.h` at the emission layer where the gather will attach, with the three consequences that bind: **(1) order the groups by a key that is a function of the SAVE and mint in that order**, `(systemId, min ship id)` with a total tie-break - **and say plainly that there is NO original order to match** (the original's is neither, in any of three processes), so we are choosing one member of its outcome set, not recovering a hidden rule; **(2) impose it AT THE GATHER, not at assignment** - the fleets already carry their ids when the assignment walk first touches them, so canonicalising there is one level too low and fixes nothing; **(3) a byte-exact replay needs the original's order and the block already carries it** - list 10 records `{systemId, fleetId, ships[]}` per group in visit order, so the optional override is three words per new fleet, the same cost class as pinning a seed. Lane BU's `aivisit` instrument merged; engine `45b16ef`, host 59/59 and shim both green **EXTENDED 2026-09-09 by lane BT, from contact with the data**: the contract needs a **fourth** consequence. **`FtName` is an ID-ATTACHED LABEL** - a Greek ordinal minted by the same pass as the id - and **its counter is PER PLAYER and SHARED WITH THE SERVER'S FLEETS**: in AD's pair, the run that minted two client fleets names a **node-0** fleet "Upsilon" and the run that minted three names the same fleet "Phi". **So the canonical gather order determines fleet NAMES too, across a boundary that has nothing to do with the AI.** Added to `orders.h`. | | STATE_CHECKSUM --relabel-new-ids: BP'S PINNED PAIR IS IDENTICAL MODULO ONE TRANSPOSITION, AD'S UNPINNED PAIR IS REFUSED AND STILL DIVERGES | tooling | verified | high | 100% | 2026-09-09 | Lane BT, HOST ONLY (no VM; 140 untouched, 141/144/146 held by AZ/BS/BR and not approached). Implements the 2026-09-09 fleet-id-order resolution §3 item 1 (`findings/subsystems/relabel-new-ids.md`). `state_checksum.py POST-A POST-B --relabel-new-ids PRE` computes the ids NEW in each post-turn save, restricts to node nibble != 0 (client-minted; `id-allocation.md` id=(counter<<4)|node), matches those fleets by a key that does not mention the id - (LocID if non-zero else FPlan/pnd destination, sorted ship-id set), with "at 384" and "bound-for 384" tagged apart - builds the bijection pi, rewrites every fleet reference, compares ShipIDs/FleetIDs/DesignIDs as SETS, masks /Summary/Checksum WITH ITS REASON ON THE LINE, and prints pi. **POSITIVE: `bp-pinA` vs `bp-pinB` = `IDENTICAL modulo pi = {1970<->1986}`, 35 leaves -> 0, exit 0**, with pi recovered from the saves alone and printed with the counters (123/124/125, node 2) and the content keys - the group AT Sys 384 with ship 6976 is 1970 in A and 1986 in B; the group BOUND FOR Sys 80 with ship 5264 is 1986 in A and 1970 in B; 2002's group is 2002 in both. **NEGATIVE: `ad-oracle-A/B-post` REFUSED and `DIVERGED: 94 leaf difference(s)`, exit 1** - byte-identically the 94 the tool reports without the flag, because on a refusal NOTHING is rewritten, masked or set-compared. The refusal is by CARDINALITY (A minted 2 client fleets, B minted 3), so AD's pair does not exercise the content matcher; that is exercised separately, and FORCING the transposition onto AD's pair with the guard bypassed STILL LEAVES 82 LEAVES. FIVE GUARDS, EVERY ONE REFUSES RATHER THAN DEGRADES: G1 only ids absent from the pre-turn save (tested with two PRE-EXISTING fleets exchanging contents - must still report); G2 only non-zero node nibbles (node-0 swap still reports); G3 pi must permute ONE set; G4 content keys correspond one-to-one and are unique per side; G5 NO leaf anywhere may hold a permuted id at an unmodelled site, matched on RAW BYTES not the reader's typed value (a value scan is empty the moment the schema types a reference as a float - this happened during development). Refusal prints its reason and falls back to the ORDINARY comparison, never to a partial relabelling. THREE SPECIFICATION CORRECTIONS FROM CONTACT WITH THE DATA: (1) **FtName is an ID-ATTACHED LABEL** - 1970 is "Sigma Fleet VI" and 1986 "Tau Fleet VI" in BOTH saves, a Greek ordinal minted by the same pass as the id, so "apply pi to every id-valued leaf" would have produced two spurious name diffs; the name stays with the SLOT, gated by a guard requiring it to track the id in both saves. The counter is PER PLAYER AND SHARED WITH THE SERVER'S FLEETS - AD's A (2 client fleets) names node-0 fleet 7120 "Upsilon", B (3) names it "Phi" - which the engine-side canonical visit order must reproduce and §3.2 does not mention. (2) **Applying pi to the Flt[] KEYS introduces a spurious `reordered`**: the fleet table and FleetIDs[] are ID-ordered and IDENTICAL in A and B, so renumbering slots reorders A against B; the artefact-free equivalent is to EXCHANGE THE FLEET BODIES between slots, leaving slot ids, FleetIDs[] and every label as written. (3) **A node's new-id set is NOT fleet-only**: 2002 is a DesignID in ad-oracle-A-post and a FltID in ad-oracle-B-post - the per-node counter is shared across object kinds, and G5 is what keeps a fleet-only matcher safe. NOT DONE, with reasons: lists 8/10/14 (they are aiorders SHIM DUMP words, not save leaves - this tool has no block reader); designs/ships; the two engine-side items. DEFAULT PATH PROVEN UNCHANGED: pre- and post-change modules compared over the whole save corpus under two policies on root digest, coverage, mask hits AND every (path,digest) in the tree - zero mismatches; plus byte-identical CLI stdout on the single-save, BP, AD and resave invocations. The relabelled root is DOMAIN-SEPARATED so a "modulo pi" root can never be read as a strict one. 24 new tests (20 synthetic + 4 real-save acceptance), 7 of them asserting a refusal; suite 38 -> 62, all passing. ALSO FIXED, PRE-EXISTING AND NOT A REGRESSION (the pre-change module fails it identically): `test_known_resave_delta_localises_to_five_named_leaves` enumerated pairs over SORTED FILENAMES and hard-coded the re-save direction 4->0; `cb-turn2to3-endturn.sav` sorts before `turn2-state.sav`, so that pair reads 0->4 and the test failed on a corpus fact. Assertion is now direction-symmetric (all four Status leaves must still move the SAME way, Checksum following by -/+16). Artefacts: `verify/state-checksum/state_checksum.py`, `test_state_checksum.py`, `STATE_CHECKSUM.md` §2.6 | +| RUNG B ON A RICH TURN: THE REPLAY RUNS AND CONSUMES THE WHOLE STREAM; 1092 LEAVES, THIRTEEN NAMED SUBSYSTEMS, ZERO UNGROUPED | engine | verified | high | 100% | 2026-09-09 | Lane CV, HOST ONLY (no VM; 140/141/145/146 untouched). First replay of a RICH turn's command stream: lane BR's deep block (`BR2-deep-aiorders.txt`) converted to `.tcb` and replayed by `sots_turn --turn-commands` against `ad-turn27-two-raiders.sav` (`1c8baa27...`), compared with `bp-pinB-turn28.sav` (`724528ff...` = BR's own output, cmp-verified). **VERDICT: OUTCOME 3 -- a named leaf list, not a byte match.** `state_checksum --floats bits --mask none` = **DIVERGED: 1092 leaf difference(s)**; the do-nothing baseline (input vs target) is **1166**, so the turn+replay CLOSES 80 and REGRESSES 6 (never netted). The stream's entire contribution to the state is ONE leaf: `/Sim/ModCount` 1430 -> 1500 against a target of 1502 (85 commands = 81 elements + 4 gates, 70 bumps charged = 85 - the 15 list-23 population commands the cost table charges 0; 2 driver bumps; **residual 2**). **THE RESIDUAL IS LOCALISED:** the same binary on the CANONICAL pair (`turn2-state.sav` + lane CB's tcb) puts `ModCount` on **12 -> 24 EXACTLY, zero residual**, matching `turn3-state.sav` and confirming CB's own prediction -- so the cost table is right on lists 3/5/8/10/14/23, and the missing 2 must sit in lists **1 (3 elements), 7 (2), 12 (12)**, the only three the rich turn adds. The only UNIFORM per-element explanation is **list 7 (colonisation) at 2 bumps per command** (3 does not divide 2, nor does 12) -- a constant fitted to ONE observation (rule 23), so it is a PREDICTION with a named cheap falsifier: any capture with a colonize count other than 2. GUARD SET, ALL SIX RUN (rule 1): (a) 85/81 matches BR's block element-for-element and per-list; (b) the DEEP payload provably reached the replayer -- shallow vs deep captures of the same run move exactly 7 elements from `incomplete` to `declined` (list 8's 3 routes, list 10's 4 ship vectors), i.e. on those seven the gap is now the ENGINE not the record; (c) `--replay-count-only` produces a BYTE-IDENTICAL save, confirming `leaves written by commands 0`; (d) deleting one `elem` line makes the reader REFUSE with exit 2; (e) replaying against another board warns; (f) output != input and != the no-stream output. TWO BLOCKERS PUT OUTCOMES 1 AND 2 OUT OF REACH BEFORE THE TURN RUNS: **(1) the engine's typed writer is not byte-exact on this save** -- `--roundtrip` DIFFERS by exactly 12 bytes = one item, at 0x3f02: `struct ProjectName` reads `usnc` as a count of ONE item per element (`a.any("usc")`, commented "usnc is 0 in every save available") but the wire pair is `usp` THEN `usc`, and this save has one record with `usnc == 1`. The generated schema already lists both. 12 of 43 corpus saves fail the same way; `mars_stream_save` and `app_turn` fail on it in ctest (57/59, pre-existing, denominator intact). Surfaces as `/Sim/sprjs/usp: only-in-B`. NAMED, NOT FIXED (schema-verified module, byte-neutrality must be re-proved). **(2) `--relabel-new-ids` REFUSES by guard G3** -- "A minted [], B minted [1970, 1986, 2002]" -- because this engine mints NO client fleet ids at all; the refusal applies nothing and the 1092 is byte-identically the no-flag number. Outcome 2 needs list 10's applier first. **THE TAIL'S SIM DRAW FIRES HERE:** every trade sector in this save reads `tscr = 253`, not the corpus's 252 -- this IS lane AD's two-raider state, measured at 2 entries / 2 strategic words at 0x00820e18 -- so the backlog's Rung B correction is load-bearing on this workload and `S02`/`T23` are stubs; `--commit-rng` leaves the count at 1092 because the 16 words are the wrong words. RANKED WORKLIST (full list in the finding, residual 0): morale event ring `cme2` **539**; colony growth/repair/bonuses **154**; fleet objects **150**; observed designs/techs **79**; player money+counters **30**; `TShn` **23**; system<->fleet membership **23**; ShipRecs **20**; build queues **20**; other system **15**; crep/CD **13**; master id lists+generator **10**; turnstats **8**; trade manager **6**; writer defect **1**; Checksum **1**. **THE 539 MUST BE DECODED BEFORE IT IS RANKED:** `cme2` is a FIXED 11-SLOT RING (158 events in input, output and target alike), so the turn's **27 appended events across 10 systems** shift every slot. The 27 are three kinds only: 10x `mtp=12` "Colony established on Avalon." (+2), 10x `mtp=20` "Imperial savings of 5,000,000 or more." (+2, down a band from the pre-turn 15,000,000), 7x `mtp=16` "Population limits inhibit freedom." (-1). So the ring is the REPORTING SURFACE of colonisation, the savings chain and the population term -- it owns none of them. **THE LARGEST INDEPENDENTLY CLOSABLE MODULE IS `T34 RecordObservedDesigns` (stub): 79 leaves, no upstream dependency.** TWO MEASURED CORRECTIONS: **`Pv*` is NOT a previous-turn snapshot** -- target `PvInfra` 1.03 and `PvPop` 617,999,994 where both pre- and post-turn `Infra`/`Pop` are 1.0 / 600,000,000 (= current x 1.03), so `Pv*` holds the UNCAPPED GROWTH PROJECTION on a capped colony, while `PvSuit` (5/5) and `PvRes` (4/4) DO equal the pre-turn current: the family is mixed and must not get one rule. And **five of the six regressions are `ntdev`** on Sys[160/400/416/432/448], where the engine writes 1 and both input and target hold 0, while the SAME rule closed 26->27 on thirteen other systems -- a one-predicate bug, the cheapest fix here. Also: the capture already holds the three design NAMES the target creates (`Egg Thief Mk 4/5`, `Bravestar Mk 3` = Des[2018/2034/2050], node-2 counters 126/127/128 continuing the fleets' 123/124/125 -- one counter across object kinds, lane BT §4.3), and list 1 is declined for want of a design-registration path. AND: **the one list `command_replay.cpp` CAN apply -- list 5, the planetary sliders -- fired ZERO times on all 15 elements**, because the shim's memory window cannot type 7 of its 8 fields; the settling experiment is one UI run (two different sliders to two different values), not engine work. TOOL CHANGE: `tools/aiorders_to_tcb.py` now reads the deep dump's `aivec`/`aistr` rows, consulting them ONLY at a word the existing field map already types as a vector, cross-checking the begin/end length against the followed count, and recording anything it cannot place as a `#` comment plus a stderr line (12 list-12 vectors, 15 list-23 vectors, 3 list-1 design names, 1 list-3 vector) rather than inventing a field mapping. REGRESSION-CHECKED against every aiorders log in the repo: 8 shallow logs byte-identical, and **the 4 lane-CB logs were deep dumps whose payloads this converter had been silently discarding** -- the only field changes are list 8's route and list 10's ship vector going `v1` -> `v1:272`/`v1:1728`, which is EXACTLY what `tools/tcb_from_json.py` already emits for the same capture, so the two converters now agree everywhere but list 23 (where the committed `cb-turn2to3.tcb` predates the current `tcb_from_json.py`'s own `UNMAPPED_HEAD[23]`; no behavioural difference, recorded not touched). The canonical pair's replay is byte-identical under the committed tcb and a regenerated one. Artefacts: `findings/subsystems/rung-b-rich-turn.md`, `verify/results/turncommands/br2-turn27-{deep,shallow}.tcb`, `verify/results/standalone/cv/*` | +| MY GATE WAS HOLLOW FOR A DAY: the corpus tests were SKIPPING and I reported 59/59 | meta | verified | high | 100% | 2026-09-09 | `SOTS_SAVES_DIR` was never set on CT111, so `mars_stream_save` and `app_turn` printed "unset, skipped" and **passed**. Every "59/59" I reported today did not run them. Lane CV, running on the host with the corpus, got **57/59** - and locally against all 43 saves the picture is worse: **39 failures**, two classes. (1) **The coverage ratchet BROKE when the corpus grew 22 -> 43** - `pct >= 99.99` fails on the spy saves and others; rule 27 working exactly as written, unseen. (2) **A writer defect**: `ProjectName` reads `usnc` as one item and the wire is a `usp`/`usc` pair; 12 saves round-trip 12 bytes short. Rule 1's shape - a green verdict on a harness comparing nothing - delivered by the person whose job was to notice. FIX: `tools/gate.sh` is now the gate; it syncs the corpus, sets the variable, runs every check as its own command, and **fails RED if any corpus test skips**. First fresh run: 57/59 and RED on one remaining skip, correctly | +| `mars/stream`: `SpecialProjectNameGen` is a `usp`/`usc` PAIR, not a single `usc` | engine | mapped | high | 40% | 2026-09-09 | Lane CV. `struct ProjectName` reads `usnc` as a count of ONE item per element, commented "usnc is 0 in every save available" - **falsified by `ad-turn27-two-raiders.sav`**, which has one record with `usnc == 1`. The generated schema already lists both tags. **12 of 43 corpus saves round-trip 12 bytes short**, all in the `sprjs` band; it is why `mars_stream_save` and `app_turn` fail. Surfaces in the replay as `/Sim/sprjs/usp: only-in-B`. **Named, not fixed** - it is a schema-verified module and byte-neutrality must be re-proved after the change. Track 0a in the backlog | diff --git a/tools/gate.sh b/tools/gate.sh new file mode 100755 index 0000000..adff927 --- /dev/null +++ b/tools/gate.sh @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +# The engine gate, as a script instead of as the integrator's memory. +# +# Why this exists: for a whole day the gate reported 59/59 while the two tests that read +# the save corpus were SKIPPING, because SOTS_SAVES_DIR was never set on CT111. The corpus +# had grown from 22 to 43 saves and the coverage ratchet had broken exactly as rule 27 says +# it should -- and a writer defect had appeared on 12 of them -- and the gate could not see +# either. A green verdict on a harness that is comparing nothing (rule 1), delivered by the +# person whose job was to notice it. +# +# Each check is its own command and its own exit code. Nothing here is &&-chained across +# checks, because a chain that short-circuits reports a check that never ran (rule 22's +# cousin, paid for once already). +# +# Usage: tools/gate.sh [--fresh] --fresh: rm -rf the remote build dirs first (rule 24) +set -u + +ENGINE="${ENGINE:-$HOME/sots-engine}" +RE="${RE:-$HOME/sots-re}" +CT=111 +REMOTE_TREE=/srv/re-lab/build/sots-engine +REMOTE_CORPUS=/srv/re-lab/saves-corpus +FRESH=0 +[ "${1:-}" = "--fresh" ] && FRESH=1 + +ct() { ssh spicy "pct exec $CT -- bash -lc '$*'"; } + +status=0 +report() { # name rc + if [ "$2" -eq 0 ]; then echo "GATE ok $1"; else echo "GATE FAIL $1 (exit $2)"; status=1; fi +} + +echo "== gate: $ENGINE @ $(git -C "$ENGINE" rev-parse --short HEAD), corpus $(ls "$RE"/verify/results/saves/*.sav | wc -l) saves, fresh=$FRESH" + +# 1. clean-room, host +( cd "$ENGINE" && bash tools/clean_room_check.sh ); report clean-room $? + +# 2. shim config structure, host +( cd "$ENGINE" && uv run --quiet python3 tools/check_shim_configs.py ); report shim-configs $? + +# 3. sync the tree -- BOTH halves of rule 24: exclude local build dirs, and clear the remote ones +rsync -a --delete --exclude '.git' --exclude 'build*' -e ssh "$ENGINE"/ spicy:/tmp/eng-gate/; report sync-to-host $? +if [ $FRESH -eq 1 ]; then ct "rm -rf $REMOTE_TREE/build-host $REMOTE_TREE/build-shim"; report clear-remote-builds $?; fi +ssh spicy "tar -C /tmp/eng-gate -cf - . | pct exec $CT -- tar -C $REMOTE_TREE -xf -"; report sync-to-ct $? + +# 4. sync the corpus -- the tests are only a gate if they run against it +rsync -a --delete --include='*.sav' --exclude='*' -e ssh "$RE"/verify/results/saves/ spicy:/tmp/sots-corpus/; report corpus-to-host $? +ssh spicy "pct exec $CT -- mkdir -p $REMOTE_CORPUS && tar -C /tmp/sots-corpus -cf - . | pct exec $CT -- tar -C $REMOTE_CORPUS -xf -"; report corpus-to-ct $? +NCORPUS=$(ct "ls $REMOTE_CORPUS | grep -c .sav") +echo "GATE info corpus on CT111: $NCORPUS saves" + +# 5. host build + tests, WITH the corpus +ct "cd $REMOTE_TREE && cmake -S . -B build-host -DCMAKE_BUILD_TYPE=Release >/dev/null 2>&1 && cmake --build build-host -j8 2>&1 | tail -1"; report host-build $? +ct "cd $REMOTE_TREE/build-host && SOTS_SAVES_DIR=$REMOTE_CORPUS ctest --output-on-failure 2>&1 | tail -15"; report host-ctest $? +# and prove the corpus tests RAN rather than skipped +SKIPPED=$(ct "cd $REMOTE_TREE/build-host && SOTS_SAVES_DIR=$REMOTE_CORPUS ctest -V 2>&1 | grep -c 'unset, skipped'") +if [ "${SKIPPED:-1}" -ne 0 ]; then echo "GATE FAIL corpus tests skipped ($SKIPPED) -- the gate is hollow"; status=1; else echo "GATE ok corpus tests ran (0 skipped)"; fi + +# 6. shim cross-build +ct "cd $REMOTE_TREE && cmake -S . -B build-shim -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-mingw-i686.cmake -DCMAKE_BUILD_TYPE=Release >/dev/null 2>&1 && cmake --build build-shim -j8 2>&1 | tail -1 && ls -la build-shim/binkw32.dll"; report shim-cross-build $? + +echo "== gate: $([ $status -eq 0 ] && echo GREEN || echo RED)" +exit $status