alex/sots-re
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

standalone report from merged main: default 209 -> 131 closed 78; note the with-inputs 127 distinction

Browse source
This commit is contained in:
alex 2026-09-08 15:49:04 -04:00
parent 1b5553b0aa
commit c810c552e5
4 changed files with 47 additions and 43 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

44
campaign/DASHBOARD.md
View file

@ -1,37 +1,37 @@
# SotS RE campaign — coverage dashboard
Generated 2026-09-08 19:10 UTC · `sots-re` @ 449daeb,2026-09-08 · `sots-engine` @ eafbc5f,2026-09-08 (171 commits) · regenerate with `tools/dashboard.py`
Generated 2026-09-08 19:49 UTC · `sots-re` @ 1b5553b,2026-09-08 · `sots-engine` @ 5d01c7c,2026-09-08 (176 commits) · regenerate with `tools/dashboard.py`
> **North star:** A functional reimplementation of the engine — behavior-equivalent, NOT byte-for-byte
## 1. Map coverage (campaign/board.md)
279 targets · mapped-or-better **251/279** `[█████████░] 90%` · verified **211/279** `[████████░░] 76%`
292 targets · mapped-or-better **263/292** `[█████████░] 90%` · verified **223/292** `[████████░░] 76%`
| Status | Count | % |
|---|---:|---:|
| verified | 211 | 76% |
| verified | 223 | 76% |
| mapped | 40 | 14% |
| in-progress | 2 | 1% |
| backlog | 24 | 9% |
| backlog | 25 | 9% |
| blocked | 2 | 1% |
| Type | verified | mapped | in-progress | backlog | blocked | total |
|---|---:|---:|---:|---:|---:|---:|
| objects | 35 | 6 | 0 | 2 | 1 | 44 |
| control-flow | 20 | 2 | 0 | 0 | 0 | 22 |
| objects | 37 | 6 | 0 | 2 | 1 | 46 |
| control-flow | 21 | 2 | 0 | 0 | 0 | 23 |
| subsystems | 4 | 8 | 0 | 2 | 1 | 15 |
| engine | 23 | 0 | 0 | 0 | 0 | 23 |
| verify | 58 | 14 | 2 | 19 | 0 | 93 |
| engine | 24 | 0 | 0 | 0 | 0 | 24 |
| verify | 61 | 14 | 2 | 20 | 0 | 97 |
| phase2 | 13 | 3 | 0 | 0 | 0 | 16 |
| meta | 46 | 6 | 0 | 1 | 0 | 53 |
| other | 12 | 1 | 0 | 0 | 0 | 13 |
| meta | 48 | 6 | 0 | 1 | 0 | 55 |
| other | 15 | 1 | 0 | 0 | 0 | 16 |
## 2. Binary understanding
- RTTI type descriptors: **1,924** (`Game::` 1,404, `Mars::` 194; serializable types 179)
- Classes with recovered member layouts: **384** / 1,598 named classes `[██░░░░░░░░] 24%` — `objects/layouts.json` (serializer recovery) plus classes recovered by hand in `struct-recovery.md` + `schema-gaps-resolved.md`. Note 179 types are *serializable*; the recovery also reaches non-serializable ones, so this is not a subset of that
- Functions: **41,411** (parsed from `01-fingerprint.md`); named/annotated in the **address contract** (`ghidra/addresses.json`, not Ghidra's full rename count): **1095**, verified **1020** `[█████████░] 93%`
- Functions: **41,411** (parsed from `01-fingerprint.md`); named/annotated in the **address contract** (`ghidra/addresses.json`, not Ghidra's full rename count): **1138**, verified **1057** `[█████████░] 93%`
## 3. Data layer
@ -45,8 +45,8 @@ Generated 2026-09-08 19:10 UTC · `sots-re` @ 449daeb,2026-09-08 · `sots-engine
| Module | LOC | Test files | Checks | Wired | Doc |
|---|---:|---:|---:|---|---|
| `app` | 3,116 | 7 | 122 | yes | N-output-term.md |
| `game/ai` | 304 | 2 | 66 | yes | — |
| `app` | 3,409 | 8 | 169 | yes | N-output-term.md |
| `game/ai` | 451 | 3 | 106 | yes | — |
| `game/combat` | 582 | 2 | 34 | yes | mars-parse.md |
| `game/config` | 558 | 4 | 131 | yes | M1.md |
| `game/data` | 2,077 | 12 | 462 | yes | game-data.md |
@ -60,12 +60,12 @@ Generated 2026-09-08 19:10 UTC · `sots-re` @ 449daeb,2026-09-08 · `sots-engine
| `mars/stream` | 5,407 | 7 | 310 | yes | mars-stream.md |
| `mars/text` | 899 | 8 | 245 | yes | mars-text.md |
| `mars/vfs` | 788 | 9 | 140 | yes | mars-vfs.md |
| `shim` | 12,606 | 0 | 0 | direct (WIN32) | H-probes.md |
| `shim/hooks` | 9,561 | 0 | 0 | direct (WIN32) | M0.md |
| `shim` | 13,026 | 0 | 0 | direct (WIN32) | H-probes.md |
| `shim/hooks` | 9,973 | 0 | 0 | direct (WIN32) | M0.md |
| `shim/trace` | 2,258 | 9 | 273 | direct (WIN32) | shim-trace.md |
| **total** | **46,804** | **111** | **3796** | | |
| **total** | **48,076** | **113** | **3883** | | |
Board `engine:` rows: verified **23**, mapped 0, in flight 0 (of 23) — verified & merged `[██████████] 100%`
Board `engine:` rows: verified **24**, mapped 0, in flight 0 (of 24) — verified & merged `[██████████] 100%`
## 5. Phase 2 milestones
@ -132,11 +132,11 @@ Most recent open:
## 9. Delta since previous dashboard
- verified targets: 206 → 211 (+5) · mapped-or-better: 246 → 251 (+5)
- engine LOC: 46,608 → 46,804 (+196) · test files: 110 → 111 (+1) · checks: 3,756 → 3,796 (+40)
- addresses verified: 1,020 → 1,020 (+0) · recovered layouts: 384 → 384 (+0) · open questions: 26 → 26 (+0)
- standalone leaves closed: 52 → 78 (+26) · leaves still diverging: 157 → 131 (-26)
- verified targets: 211 → 223 (+12) · mapped-or-better: 251 → 263 (+12)
- engine LOC: 46,804 → 48,076 (+1,272) · test files: 111 → 113 (+2) · checks: 3,796 → 3,883 (+87)
- addresses verified: 1,020 → 1,057 (+37) · recovered layouts: 384 → 384 (+0) · open questions: 26 → 26 (+0)
- standalone leaves closed: 78 → 78 (+0) · leaves still diverging: 131 → 131 (+0)
---
warnings: board.md: unknown types subsystems; mars-rng.md: no oracle total row parsed; mars-stream.md: no oracle total row parsed; mars-vfs.md: no oracle total row parsed
<!-- dashboard-metrics {"verified": 211, "mapped_plus": 251, "targets": 279, "loc": 46804, "tests": 111, "checks": 3796, "addr_verified": 1020, "addr_total": 1095, "layouts": 384, "open_q": 26, "sa_closed": 78, "sa_left": 131} -->
<!-- dashboard-metrics {"verified": 223, "mapped_plus": 263, "targets": 292, "loc": 48076, "tests": 113, "checks": 3883, "addr_verified": 1057, "addr_total": 1138, "layouts": 384, "open_q": 26, "sa_closed": 78, "sa_left": 131} -->

1
campaign/board.md
View file

@ -297,3 +297,4 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`).
| a CONTROL measurement worth copying | verify | verified | high | 100% | 2026-09-08 | Lane EV did not just report that its phase needs the AI roster - it MEASURED the cost of withholding it: with the roster withheld the phase over-fires and **REGRESSES 16 leaves on pair 1** - and ONLY on pair 1, because pair 2's input already carries the AI targets. That is why the phase stays Blocked and phase-granular. The AI research-selection gate itself is labelled a hypothesis with the workload named: across ~90 AI player-turns no AI ever posts NO_RESEARCH while the human posts it on every target-less turn |
| tail phases post NO events (measured negative) | control-flow | verified | high | 100% | 2026-09-08 | Confirming lane K from the other side: T18/T21/T32 post nothing on either reference pair - and this is a MEASURED negative, not an absence of evidence, since other corpus saves DO exercise them. No tail phase manufactures a turn's events |
| standalone_report gains --engine-arg | meta | verified | high | 100% | 2026-09-08 | Lane EV added a repeatable `--engine-arg` pass-through so a lane can feed the standalone the OPERATOR INPUTS A SAVE DOES NOT CARRY (e.g. --ai-player N, --data DIR) instead of hard-coding them; every run now records what it was given in the header AND in status.json. That is the right shape - the provenance travels with the number |
| headline number: default 131, with operator inputs 127 | meta | verified | high | 100% | 2026-09-08 | Worth keeping distinct rather than quoting the better one. `standalone_report.py` with NO flags reports **209 -> 131, closed 78** - that is the number for "a save and nothing else". Lane EV's P11 closes 4 more (127) but ONLY when handed `--data DIR` and `--ai-player N`, operator inputs the save does not carry, because a save does not record which players are AI-controlled (lane AI3: ServerPlayer+0xf9/+0xfa sit in a HOLE in the serialised layout). Both numbers are real; the dashboard tracks the default because it is the one that needs no human. Lane EV's --engine-arg pass-through records in status.json exactly what a run was given |

2
verify/results/standalone/report.txt
View file

@ -1,6 +1,6 @@
# standalone vs the oracle
generated 2026-09-08T19:10:01Z binary /srv/re-lab/build/eng-t2/build-host/src/app/sots_turn
generated 2026-09-08T19:48:40Z binary /srv/re-lab/build/eng-rep2/build-host/src/app/sots_turn
phases: 16/44 of the two turn drivers modelled, 12 committed (implemented 4, partial 8, blocked 4, stub 28)
5/37 of the post-combat tail modelled

43
verify/results/standalone/status.json
View file

@ -1,7 +1,8 @@
{
"schema": "sots-standalone-status/1",
"generated": "2026-09-08T19:10:01Z",
"binary": "/srv/re-lab/build/eng-t2/build-host/src/app/sots_turn",
"generated": "2026-09-08T19:48:40Z",
"binary": "/srv/re-lab/build/eng-rep2/build-host/src/app/sots_turn",
"engineArgs": [],
"reference": {
"input": "turn1-state.sav",
"oracle": "turn2-state.sav",
@ -56,9 +57,10 @@
"input": "turn1-state.sav",
"oracle": "turn2-state.sav",
"note": "real End Turn",
"engineArgs": [],
"exit": 0,
"stdout": [
"load: /srv/re-lab/build/sre2/verify/results/saves/turn1-state.sav",
"load: /srv/re-lab/build/sre3/verify/results/saves/turn1-state.sav",
" 591376 inflated bytes, 0 error(s), 0 warning(s)",
" turn 1, frame 1, modCount 0, 8 player(s), 28 system(s), 6 fleet(s)",
"roundtrip (untouched): byte-identical (591376 bytes)",
@ -72,7 +74,7 @@
"",
"this run",
" leaves written 87",
" leaves NOT written by a blocked phase 73",
" leaves NOT written by a blocked phase 97",
" generator words consumed 16 (state loaded, left untouched)",
" generator words NOT accounted (never netted off the above):",
" - encounter detection draws one unit value and one bounded integer per turn on every turn measured (2 words), with no derived rule behind the count -- its bound is the product of the contact and detector counts, so it is left unmodelled",
@ -81,8 +83,8 @@
" ! the generator advanced during this run but the save keeps its original state (--commit-rng to write it)",
" ! the modelled words are a LOWER BOUND on the turn's cost, so a committed generator is short by the unaccounted sites below and its drawn VALUES are not the game's",
"",
"wrote /tmp/tmpf9f4k25u/post-turn1-state.sav (65970 bytes gzipped, 595192 inflated)",
"metric -> /tmp/tmpf9f4k25u/metric-turn1-state.sav.json"
"wrote /tmp/tmpkd8jx34g/post-turn1-state.sav (65970 bytes gzipped, 595192 inflated)",
"metric -> /tmp/tmpkd8jx34g/metric-turn1-state.sav.json"
],
"coverage": {
"input": true,
@ -224,8 +226,8 @@
"byteMatch": false,
"standalone": {
"schema": "sots-standalone-metric/1",
"input": "/srv/re-lab/build/sre2/verify/results/saves/turn1-state.sav",
"output": "/tmp/tmpf9f4k25u/post-turn1-state.sav",
"input": "/srv/re-lab/build/sre3/verify/results/saves/turn1-state.sav",
"output": "/tmp/tmpkd8jx34g/post-turn1-state.sav",
"spine": {
"total": 44,
"verified": 0,
@ -248,7 +250,7 @@
},
"run": {
"leafWrites": 87,
"blockedLeafWrites": 73,
"blockedLeafWrites": 97,
"rngWords": 16,
"rngLoaded": true,
"rngCommitted": false,
@ -416,7 +418,7 @@
"status": "partial",
"ran": 8,
"writes": 3,
"blockedWrites": 7,
"blockedWrites": 19,
"rng": 0
},
{
@ -526,7 +528,7 @@
"status": "blocked",
"ran": 4,
"writes": 0,
"blockedWrites": 4,
"blockedWrites": 16,
"rng": 0
},
{
@ -1106,9 +1108,10 @@
"input": "turn2-state.sav",
"oracle": "turn3-state.sav",
"note": "real End Turn",
"engineArgs": [],
"exit": 0,
"stdout": [
"load: /srv/re-lab/build/sre2/verify/results/saves/turn2-state.sav",
"load: /srv/re-lab/build/sre3/verify/results/saves/turn2-state.sav",
" 603360 inflated bytes, 0 error(s), 0 warning(s)",
" turn 2, frame 2, modCount 12, 8 player(s), 28 system(s), 7 fleet(s)",
"roundtrip (untouched): byte-identical (603360 bytes)",
@ -1122,7 +1125,7 @@
"",
"this run",
" leaves written 43",
" leaves NOT written by a blocked phase 67",
" leaves NOT written by a blocked phase 73",
" generator words consumed 16 (state loaded, left untouched)",
" generator words NOT accounted (never netted off the above):",
" - encounter detection draws one unit value and one bounded integer per turn on every turn measured (2 words), with no derived rule behind the count -- its bound is the product of the contact and detector counts, so it is left unmodelled",
@ -1131,8 +1134,8 @@
" ! the generator advanced during this run but the save keeps its original state (--commit-rng to write it)",
" ! the modelled words are a LOWER BOUND on the turn's cost, so a committed generator is short by the unaccounted sites below and its drawn VALUES are not the game's",
"",
"wrote /tmp/tmpf9f4k25u/post-turn2-state.sav (66879 bytes gzipped, 603360 inflated)",
"metric -> /tmp/tmpf9f4k25u/metric-turn2-state.sav.json"
"wrote /tmp/tmpkd8jx34g/post-turn2-state.sav (66879 bytes gzipped, 603360 inflated)",
"metric -> /tmp/tmpkd8jx34g/metric-turn2-state.sav.json"
],
"coverage": {
"input": true,
@ -1232,8 +1235,8 @@
"byteMatch": false,
"standalone": {
"schema": "sots-standalone-metric/1",
"input": "/srv/re-lab/build/sre2/verify/results/saves/turn2-state.sav",
"output": "/tmp/tmpf9f4k25u/post-turn2-state.sav",
"input": "/srv/re-lab/build/sre3/verify/results/saves/turn2-state.sav",
"output": "/tmp/tmpkd8jx34g/post-turn2-state.sav",
"spine": {
"total": 44,
"verified": 0,
@ -1256,7 +1259,7 @@
},
"run": {
"leafWrites": 43,
"blockedLeafWrites": 67,
"blockedLeafWrites": 73,
"rngWords": 16,
"rngLoaded": true,
"rngCommitted": false,
@ -1424,7 +1427,7 @@
"status": "partial",
"ran": 8,
"writes": 3,
"blockedWrites": 4,
"blockedWrites": 7,
"rng": 0
},
{
@ -1534,7 +1537,7 @@
"status": "blocked",
"ran": 1,
"writes": 0,
"blockedWrites": 1,
"blockedWrites": 4,
"rng": 0
},
{