alex/sots-re
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

harness audit merged; qualify B1/B4 claims with the blind spots it found

Browse source
This commit is contained in:
alex 2026-09-08 01:30:33 -04:00
parent cdada28d78
commit b878c4c48f
2 changed files with 19 additions and 17 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

28
campaign/DASHBOARD.md
View file

@ -1,19 +1,19 @@
# SotS RE campaign — coverage dashboard
Generated 2026-09-08 05:19 UTC · `sots-re` @ ee43e96,2026-09-08 · `sots-engine` @ b925944,2026-09-08 (62 commits) · regenerate with `tools/dashboard.py`
Generated 2026-09-08 05:30 UTC · `sots-re` @ cdada28,2026-09-08 · `sots-engine` @ e1cbb81,2026-09-08 (65 commits) · regenerate with `tools/dashboard.py`
> **North star:** A functional reimplementation of the engine — behavior-equivalent, NOT byte-for-byte
## 1. Map coverage (campaign/board.md)
61 targets · mapped-or-better **47/61** `[████████░░] 77%` · verified **30/61** `[█████░░░░░] 49%`
63 targets · mapped-or-better **48/63** `[████████░░] 76%` · verified **31/63** `[█████░░░░░] 49%`
| Status | Count | % |
|---|---:|---:|
| verified | 30 | 49% |
| mapped | 17 | 28% |
| in-progress | 2 | 3% |
| backlog | 10 | 16% |
| verified | 31 | 49% |
| mapped | 17 | 27% |
| in-progress | 1 | 2% |
| backlog | 12 | 19% |
| blocked | 1 | 2% |
| Type | verified | mapped | in-progress | backlog | blocked | total |
@ -22,7 +22,7 @@ Generated 2026-09-08 05:19 UTC · `sots-re` @ ee43e96,2026-09-08 · `sots-engine
| control-flow | 0 | 2 | 0 | 0 | 0 | 2 |
| subsystems | 2 | 6 | 1 | 3 | 1 | 13 |
| engine | 10 | 0 | 0 | 0 | 0 | 10 |
| verify | 4 | 2 | 1 | 4 | 0 | 11 |
| verify | 5 | 2 | 0 | 6 | 0 | 13 |
| phase2 | 5 | 1 | 0 | 2 | 0 | 9 |
| meta | 4 | 4 | 0 | 1 | 0 | 9 |
@ -54,10 +54,10 @@ Generated 2026-09-08 05:19 UTC · `sots-re` @ ee43e96,2026-09-08 · `sots-engine
| `mars/stream` | 3,703 | 6 | 222 | yes | mars-stream.md |
| `mars/text` | 899 | 8 | 245 | yes | mars-text.md |
| `mars/vfs` | 788 | 9 | 140 | yes | mars-vfs.md |
| `shim` | 6,737 | 0 | 0 | direct (WIN32) | M0.md |
| `shim/hooks` | 4,594 | 0 | 0 | direct (WIN32) | M0.md |
| `shim/trace` | 1,836 | 8 | 248 | direct (WIN32) | shim-trace.md |
| **total** | **26,883** | **88** | **2805** | | |
| `shim` | 7,551 | 0 | 0 | direct (WIN32) | M0.md |
| `shim/hooks` | 4,981 | 0 | 0 | direct (WIN32) | M0.md |
| `shim/trace` | 2,258 | 9 | 273 | direct (WIN32) | shim-trace.md |
| **total** | **28,506** | **89** | **2830** | | |
Board `engine:` rows: verified **10**, mapped 0, in flight 0 (of 10) — verified & merged `[██████████] 100%`
@ -94,10 +94,10 @@ Most recent open:
## 8. Delta since previous dashboard
- verified targets: 29 → 30 (+1) · mapped-or-better: 46 → 47 (+1)
- engine LOC: 26,735 → 26,883 (+148) · test files: 88 → 88 (+0) · checks: 2,805 → 2,805 (+0)
- verified targets: 30 → 31 (+1) · mapped-or-better: 47 → 48 (+1)
- engine LOC: 26,883 → 28,506 (+1,623) · test files: 88 → 89 (+1) · checks: 2,805 → 2,830 (+25)
- addresses verified: 317 → 317 (+0) · recovered layouts: 21 → 21 (+0) · open questions: 27 → 27 (+0)
---
warnings: board.md line 66: unknown status 'code-complete' for 'P2-B4 colony + movement (behavioral)'; mars-rng.md: no oracle total row parsed; mars-stream.md: no oracle total row parsed; mars-vfs.md: no oracle total row parsed
<!-- dashboard-metrics {"verified": 30, "mapped_plus": 47, "targets": 61, "loc": 26883, "tests": 88, "checks": 2805, "addr_verified": 317, "addr_total": 324, "layouts": 21, "open_q": 27} -->
<!-- dashboard-metrics {"verified": 31, "mapped_plus": 48, "targets": 63, "loc": 28506, "tests": 89, "checks": 2830, "addr_verified": 317, "addr_total": 324, "layouts": 21, "open_q": 27} -->

8
campaign/board.md
View file

@ -53,7 +53,7 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`).
| tech effects (code-defined) | subsystem | mapped | high | 90% | 2026-09-07 | findings/subsystems/tech-effects.md: g_TechIdNames 196 @0x009ff9e4 (TechId=10000+i), effects in ServerPlayer::OnTechResearched 0x00891790; SpeciesDef table 0x00b10a00; 36 strategic / ~45 gates / ~115 none. Feeds game/sim + game/data |
| strategic formula gaps | subsystem | verified | high | 100% | 2026-09-07 | findings/subsystems/formula-gaps.md: all 8 answered with code (bankruptcy 3.3 on BnkPr, linear hazard, money tail, pbon, slider, [6] net, no speed clamp, decay hits current) |
| engine: game/design | engine | verified | high | 100% | 2026-09-07 | merged; see ShipDesignDef row |
| P2-B1 ComputeBudget (behavioral) | phase2 | verified | high | 90% | 2026-09-08 | LIVE + GREEN: 4,623 calls traced, 4,437 compared 0 divergences (all 8 End-Turn calls identical on 22 slots), REPLACE fed the game its budget for a full turn and the End-Turn oracle reproduced byte-for-byte. Corrections: out-param is int[22] not int[25] (next 3 words are a std::vector); researchMoneyKept only charged with a research target (real bug). Coverage: 8 slots verified on varied values, 8 slots always 0 in this save (expenses/aid/debt tail UNTESTED), 6 are inputs by construction |
| P2-B1 ComputeBudget (behavioral) | phase2 | verified | med | 75% | 2026-09-08 | LIVE + GREEN on declared regions (4,437 compares, 0 divergences; replace-mode oracle byte-identical). Corrections: out-param int[22]; researchMoneyKept gated on a research target. QUALIFIED BY THE HARNESS AUDIT: replace mode runs the ORIGINAL A SECOND TIME to harvest slots -> a real per-turn double effect (ComputeOutput repairs ships in orbit) that no region reaches - UNRESOLVED; the over-budget int at Budget+0x64 was captured only as an argument and args are never compared. Golden trace needs recapture with guards. Coverage gaps: 8 slots always 0 (expenses/aid/debt) |
| P2-B2 OnTechResearched (behavioral) | phase2 | verified | high | 80% | 2026-09-08 | LIVE: 3 completions compared, 0 divergences (incl. one tech absent from the effects table = tail-only, the case an early return would have broken). float32 CONFIRMED ON THE GAME bit-for-bit (con_mod 1.0->0.899999976 = (float)(x+(double)0.1f)). Found B3's rolling effect: ServerPlayer::RollResearchEvent draws one NextFloat at the top of the callback - modelled + RNG region declared, but the branch never fired here so NOT behaviourally proven. KNOWN GAP: ours posts no events and the player event list is still undeclared, so the clean compare bounds the economy fields only and the replace/oracle pass was deliberately run on a no-completion turn (weak check) |
| P2-B3 ProcessResearch (behavioral, RNG) | phase2 | mapped | high | 85% | 2026-09-08 | LIVE, PARTIAL PASS: 15 calls compared, 13 zero-divergence; RNG post-state matched 14/15 incl. every roll (validates MT19937 + draw mapping + odds together). 2 divergences are the declared SetResearched boundary. ORACLE FAILS by exactly one item across 40,300: an unposted EVENT_RESEARCH_OVERBUDGET - compare was blind because the event list was never a declared region. fpu_cw=0x127f => 53-bit double, x87 question SETTLED. No Zuul in the save: double roll still disassembly-only |
| RNG signatures (Ghidra) | meta | verified | high | 100% | 2026-09-08 | Seed/Twist/NextFloat/NextInt verified; draw = y/(2^32-1); NextInt [0,n] inclusive; lazy twist; left@+0x9c4. RUNTIME CONFIRMED: fpu_cw=0x127f (53-bit double, round-nearest) - our next_float model is right, float_from_pc24 is an unused contingency |
@ -63,6 +63,8 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`).
| budget tail coverage (expenses/aid/debt) | verify | backlog | — | 0% | 2026-09-08 | 8 ComputeBudget slots were always 0 in ref-turn2 (no sliders, no aid, no debt, no handicap). Need a save with expense sliders, a debtor and a research-aid treaty to exercise ExpenseTotal + the aid/bonus tail |
| hook GetDifficultyMods | meta | backlog | — | 0% | 2026-09-08 | B1 derived the two difficulty rows from trace values (AI maintenance divisor 3, research x1.5) instead of snapshotting them; hook it properly so they stop being constants |
| section-loader compare crash | verify | backlog | — | 0% | 2026-09-08 | SectionDictionary compare crashes the engine while the identical weapon path succeeds -> fault is in re-running LoadSection, not the manifest reader. Next boundary: hook LoadSection itself. docs/M2.md has 3 ranked leads |
| P2-B4 colony + movement (behavioral) | phase2 | code-complete | med | 70% | 2026-09-08 | CODE-COMPLETE (engine 89c0013), dist-b4 staged, ctest 30/30. Verified 3 signatures (ProcessTurn takes NO stack arg - Ghidra's 2nd param was fiction, would have crashed). 22 corrections incl: population growth has NO pop/capacity term (base is a suitability ratio through pow()) - our model was wrong in SHAPE; range margin is +0.05f not -0.05 (fleets stopped 0.1 short); failed jump SCATTERS around the destination (2nd draw); AccrueSystemBonus gate is ntdev not rbtn; rounding is fistp ties-to-even; only ProcessRebellion draws RNG in a colony turn. NO replace/oracle this milestone (input boundary, deliberate). UNTESTED sub-paths: plague, rebellion, slaves, terraform, addiction, unowned decay, jumps, gate teleports, node-line travel, stranded fleets |
| harness gap: undeclared side-effect lists | verify | in-progress | — | 0% | 2026-09-08 | AUDIT LANE RUNNING: a clean compare can be silently incomplete (B3's unposted event). Auditing every descriptor for undeclared writes (event/message lists, vector/map appends, writes through pointers, globals) and making unmodelled coverage structurally visible in the trace + tracecmp report |
| P2-B4 colony + movement (behavioral) | phase2 | code-complete | med | 70% | 2026-09-08 | CODE-COMPLETE (engine 89c0013), dist-b4 staged, ctest 30/30. Verified 3 signatures (ProcessTurn takes NO stack arg - Ghidra's 2nd param was fiction, would have crashed). 22 corrections incl: population growth has NO pop/capacity term (base is a suitability ratio through pow()) - our model was wrong in SHAPE; range margin is +0.05f not -0.05 (fleets stopped 0.1 short); failed jump SCATTERS around the destination (2nd draw); AccrueSystemBonus gate is ntdev not rbtn; rounding is fistp ties-to-even; only ProcessRebellion draws RNG in a colony turn. NO replace/oracle this milestone (input boundary, deliberate). UNTESTED sub-paths: plague, rebellion, slaves, terraform, addiction, unowned decay, jumps, gate teleports, node-line travel, stranded fleets | AUDIT: `ours` re-reads LIVE post-original fleet state (breaks the compare invariant, verdict partly self-fulfilling); morale events computed then DROPPED (DescribeMoraleEvents never called); PassSchedule/DescribeGateTraffic/DescribeFleetStep are DEAD CODE so the 'predicts call order' claim is unimplemented. Fix before trusting its compare |
| harness gap: undeclared side-effect lists | verify | verified | high | 100% | 2026-09-08 | FIXED STRUCTURALLY (engine 3f0721f+): compile-time-required Coverage on every descriptor (a hook without one does not compile); Guard regions that catch AND localise undeclared writes (names player+0x2b0, not 'the hash moved'); replace mode now emits records; tracecmp prints coverage on every report + --strict-coverage. A hook claiming 'complete' while a guard caught an undeclared write now counts as a DIVERGENCE. Audit found 23 undeclared side effects: docs/harness-audit.md |
| RollResearchEvent draw (behavioural) | verify | backlog | — | 0% | 2026-09-08 | ServerPlayer::RollResearchEvent draws one NextFloat when the completing def is the current target and +0x3b4 is set. Region declared, ours reproduces it, but the branch never fired on ref-turn2 - needs a workload that sets research_roll_pending |
| golden-trace recapture (post-guards) | verify | backlog | — | 0% | 2026-09-08 | region sets changed, so goldens must be recaptured: ProcessResearch (expect NEW divergences - the event defect becoming visible), ComputeBudget (verdict should stay 0), and first captures for OnTechResearched / ServerSystem::ProcessTurn / MoveFleet |
| B1 replace double-run | verify | backlog | — | 0% | 2026-09-08 | ComputeBudget replace mode runs the original a second time to harvest budget slots; ComputeOutput repairs ships in orbit as a side effect, so this is a real per-turn double effect on objects no region covers. Needs a design fix (harvest without re-running, or declare+revert) |