c504729341
tools/standalone_report.py drives sots-engine's sots_turn over each consecutive-turn save pair and diffs the result against the game's own post-turn save with state_checksum.py, which localises to named leaves and proves its own coverage by re-serialisation. turn1-state -> turn2-state baseline 209 diverging, after 204, closed 5 turn2-state -> turn3-state baseline 108 diverging, after 103, closed 5 regressed 0 on both `regressed` is reported next to `closed` and never netted off. It earned its place immediately: committing the phase-31 player-status restore turned two agreeing leaves into disagreeing ones, because the phase writes 1 and the file carries 4. The stable-system stand-in feeding the colony pass is a labelled hypothesis and it survived a changed workload -- the same 3 ntdev leaves closed on both pairs, six agreements, zero disagreements. Two things deliberately NOT implemented: the TShn/ltis counters (18 leaves, a `+1` would close them, but "+1 across one observed turn" is a hypothesis, not a reading), and the RNG state write-back (an advanced-but-incomplete generator is wrong in a different way from an untouched one). dashboard.py gains section 6, reading verify/results/standalone/status.json: phases modelled/committed per driver, baseline vs after, closed vs regressed, the subsystem breakdown of what still differs, and the RNG gap. Sections 6-8 renumbered to 7-9; the delta footer tracks the two new counts. DASHBOARD_README.md documents every number. findings/control-flow/standalone-scaffold.md has the ranked blocker list.
124 lines
7 KiB
Markdown
124 lines
7 KiB
Markdown
# The standalone, and the measured distance to the byte-match
|
||
|
||
Lane S2, 2026-09-08. Engine branch `wip/standalone`; full documentation in
|
||
`sots-engine/docs/S-standalone.md`. This note records what the lane measured, what it declined
|
||
to implement, and the two numbers the campaign should track from here.
|
||
|
||
---
|
||
|
||
## 0. The headline
|
||
|
||
`sots_turn` loads a real save through the engine's own reader, walks the **published phase
|
||
order of all three turn drivers**, runs what we hold, prints what we do not, and writes a save
|
||
through the engine's own writer.
|
||
|
||
```
|
||
turn1-state.sav -> turn2-state.sav (a real End Turn)
|
||
baseline (a standalone that does nothing) 209 leaves diverge
|
||
after one standalone turn 204 leaves diverge
|
||
closed 5, regressed 0
|
||
|
||
turn2-state.sav -> turn3-state.sav
|
||
baseline 108 -> 103, closed 5, regressed 0
|
||
```
|
||
|
||
Phases: **14 of 44** turn-driver phases modelled, **7** committing anything; **2 of 37** of the
|
||
post-combat tail. Generator words modelled per turn: **0** of the 18–20 consumed.
|
||
|
||
Leaves are `verify/state-checksum/state_checksum.py`'s named leaves; every one of the six
|
||
saves involved reported `coverage: PROVED` on the same run, so the diff cannot be hiding
|
||
anything.
|
||
|
||
Regenerate with `tools/standalone_report.py`; outputs land in `verify/results/standalone/`
|
||
and the dashboard's new section 6 reads `status.json`.
|
||
|
||
## 1. What the scaffold is for
|
||
|
||
Everything the milestone needs already existed in pieces — a save reader with 100 % named
|
||
coverage, a verified budget roll-up, a verified research slice, a mechanism-verified movement
|
||
model, and byte-for-byte maps of both turn drivers. What did not exist was **a place to put
|
||
them and a number that says how far they get**. That is the whole of this lane.
|
||
|
||
The phase catalog (`src/app/phase_catalog.cpp`) is the roadmap: all 32 + 12 + 37 phases, each
|
||
carrying a status and a note. Running `sots_turn --phases` prints the turn, and every
|
||
unimplemented phase prints itself. There is no way for a phase to be silently absent, and
|
||
`app_catalog` fails the build if a table develops a gap or if anything claims to be `verified`
|
||
(which in that table means "compared against the live game" — lane S2 held no VM).
|
||
|
||
## 2. The finding that justifies the design: a committed phase can make things worse
|
||
|
||
The first version of this lane implemented `StrategyServer::ProcessTurn` phase 31's
|
||
player-status restore — `Status = 1` — and committed it. The comparison tool immediately
|
||
reported **two regressed leaves** on the `turn2 → turn3` pair: two `Player.Status` words that
|
||
**agreed** with the oracle before the turn and disagreed after it.
|
||
|
||
The phase writes 1. The post-turn file carries 4. A load resets it to 0. So a writer between
|
||
phase 31 and the autosave is unaccounted, and the input save happened to already carry the
|
||
right answer.
|
||
|
||
This is the same shape as the campaign's oldest lesson in a new place: running more code is
|
||
not the same as knowing more. The standalone therefore separates **modelled** from
|
||
**committed**, and a phase whose inputs are not modelled is evaluated, reported and *not
|
||
written* unless `--commit-blocked` is passed. `regressed` is reported next to `closed` in every
|
||
run, never netted off.
|
||
|
||
Five phases are blocked behind one unresolved formula (below); `S31` and `T31` are blocked
|
||
behind their own; `P11` is blocked behind the event-text table.
|
||
|
||
## 3. Two things this lane declined to implement
|
||
|
||
**The `TShn` / `ltis` counters.** 18 leaves of the remaining 204 are `TShn` and `ltis` moving
|
||
`1 -> 2` on 8–10 systems, on both turn pairs. They look exactly like per-turn counters and a
|
||
`+1` would close 18 leaves in ten lines of code. Nothing in the campaign names their writer, so
|
||
"+1 per turn across one observed turn" is a hypothesis, not a reading, and rule 6 says label it
|
||
as one. They are the **cheapest measured target on the board** and they are named here so the
|
||
next lane can close them properly rather than plausibly.
|
||
|
||
**The RNG state.** The generator advances during a real turn; the standalone leaves the blob
|
||
byte-identical by default. An advanced-but-incomplete state is wrong in a different way from an
|
||
untouched one, and the untouched one at least reports the truth. `--commit-rng` is there for
|
||
the day lane Z's ledger closes.
|
||
|
||
## 4. One hypothesis under test, and it survived a changed workload
|
||
|
||
`ProcessColonyTurn` takes `stable` as an input; in the original it is a callee's verdict. The
|
||
standalone stands in `owned && !abandoned && !destroyed`, labelled a hypothesis in the code and
|
||
printed as one in the run log.
|
||
|
||
It drives `ntdev`, which is a named leaf, so it is falsifiable. On `turn1 → turn2` it judged 3
|
||
of 28 systems stable and closed exactly the 3 `ntdev` leaves the oracle moved. On
|
||
`turn2 → turn3` — a different turn, a different set of orders — it closed the same 3 again.
|
||
**Six agreements, zero disagreements, across two workloads.** Not proof; recorded as such.
|
||
|
||
## 5. What is now measurably in the way
|
||
|
||
Ordered by what must be solved, not by size.
|
||
|
||
| # | blocker | cost in leaves on the reference pair | who can close it |
|
||
|---|---|---:|---|
|
||
| 1 | **the RNG ledger** — 18–20 words/turn, none modelled, and the generator is saved state | 1 leaf, and it makes a byte-match *arithmetically impossible* | lane Z (in flight); nothing in `src/app` |
|
||
| 2 | **the population → base-output term** — one unresolved formula that blocks `P01 P02 P05 P06 T31` | ~10 directly (`Sav`, `BnkPr`, `BnkEl` on 4 players), and it gates 5 of the 44 phases | a formula lane against the live game |
|
||
| 3 | **the post-combat tail**, 37 phases, none implemented, and the driver the autosave is written from | 24 (`turnstats`) + the bankruptcy limits + observed designs + player reports | its own milestone |
|
||
| 4 | **the per-player system-visibility record** (`nve`) | 32 — one mechanism, eight repetitions | spine phase 24 or tail phase 21 |
|
||
| 5 | **the event pipeline** — buckets, ids and localised text | ~20 across the players | needs the string table |
|
||
| 6 | **`Summary.Checksum`** — algorithm unknown | 1, and it is the last leaf to fall | — |
|
||
| 7 | **the `Player.Status` writer** — phase writes 1, file carries 4 | 4 | small, self-contained |
|
||
| 8 | **`ModCount`** — advances 12–44 times a turn from writers across both drivers; we model 2 | 1 | falls out of implementing the other phases |
|
||
|
||
## 6. Gates run, separately
|
||
|
||
```
|
||
tools/clean_room_check.sh -> clean-room check: OK (with src/app + tests/app staged)
|
||
ctest --preset host -> 100% tests passed out of 38 (was 36; +app_catalog, +app_turn)
|
||
app_turn with SOTS_SAVES_DIR set -> 11 saves driven, 0 failures
|
||
```
|
||
|
||
`src/shim/` was not touched, so no CT111 cross-build was required.
|
||
|
||
## 7. What the corpus cannot answer
|
||
|
||
The two pairs above are the only true End-Turn transitions we hold. The other nine saves are
|
||
single states: `app_turn` drives a turn over each of them and asserts the file survives, but
|
||
there is no oracle to diff against. A third and fourth consecutive-turn pair — especially on a
|
||
Zuul game, where the research and node-travel paths differ — would make every number in §0
|
||
sturdier for the cost of two End Turns on VM140.
|