sots-re/findings/subsystems/golden-trace-recapture.md
alex 05095e93da lane R: golden-trace recapture with guards on the live game
Five hooks recaptured, build recap-7584bad-20260908T0615Z, no source change.

- ProcessResearch: the over-budget event is now a compare divergence
  (side.events.after.v.next_id orig=4 ours=3) instead of a 609 KB save diff.
  15 calls over 5 turns, 3 diverged, RNG matched 15/15.
- ComputeBudget: 4284 compared, 0 diverged, guard quiet - but only 20 distinct
  states and 13 of 22 slots always zero.
- MoveFleet: 8 of 45 calls diverge by 1 ULP of position. B4's clean verdict was
  a one-sample verdict.
- First guarded captures for OnTechResearched (RollResearchEvent draw fired and
  matched), the colony turn and MoveFleet.
- Guards mapped SetResearched live and found an undeclared ObservedTech append.

VM140 released at the main menu, hooks=trace.
2026-09-08 03:06:43 -04:00

291 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Golden-trace recapture with guards (lane R, 2026-09-08)
Every golden trace went stale when the harness grew `Coverage` + guard regions
(`sots-engine` 3f0721f+, `docs/harness-audit.md`). This is the recapture on the live game.
**Read the coverage, not the verdict.** Two of the five hooks now report divergences that
their previous "verified" rows did not, and both are real.
* Build: `recap-7584bad-20260908T0615Z` (sots-engine `7584bad` + four `shim.cfg.recap*`
variants; no source change). Staged `C:\SOTS\shimdist-recap`, VM140.
* Workload: `ref-turn2.sav` → Launch → **End Turn**. The B3/misc runs continue to turn 7
(5 End Turns) to reach a research completion, which turn 1 does not contain.
* Oracle held on every run's first End Turn: `(Autosave EndTurn).sav` = `bb4fd9ac89f41e3b…`,
`(Autosave).sav` = `978041acd168b56e…` — trace and compare mode do not perturb the game.
* Artefacts: `verify/traces/recap-*.jsonl[.gz]`, reports in `verify/results/compare/recap-*`,
shim logs + screenshots in `verify/results/shim/recap-*`.
## 1. Headline table (coverage first)
| hook | run | calls | compared | diverged | exit | coverage line |
|---|---|---|---|---|---|---|
| `TechTree::ProcessResearch` | `recap-b3-compare` (t2→t7) | 15 | 15 | **3** | **1** | 15 guarded calls, **13 undeclared writes in 2 calls**, 0 unstated, 0 contradicted |
| `TechTree::ProcessResearch` | first End Turn only | 3 | 3 | **1** | **1** | 3 guarded calls, 0 undeclared writes |
| `ServerPlayer::ComputeBudget` | `recap-b1-compare` | 4284 | 4284 | 0 | 0 | 4284 guarded calls, **0 undeclared writes** |
| `ServerPlayer::OnTechResearched` | `recap-misc-compare` | 2 | 2 | 0 | (1)¹ | 2 guarded calls, **6 undeclared writes in 2 calls** |
| `ServerSystem::ProcessTurn` | `recap-misc-compare` | 140 | 140 | 0 | (1)¹ | 140 guarded calls, **15 undeclared writes in 6 calls** |
| `StrategyServer::MoveFleet` | `recap-misc-compare` | 45 | 45 | **8** | (1)¹ | 45 guarded calls, **42 undeclared writes in 15 calls** |
| all five | `recap-golden-trace` (trace mode) | 3630 | — | — | 0 | 3630 guarded calls, 9 undeclared writes in 4 calls |
¹ the three misc hooks share one log; its exit code 1 comes from `MoveFleet` alone.
## 2. `TechTree::ProcessResearch` — the over-budget event is now visible
**Yes. The defect became a divergence, on exactly the call the audit predicted.**
On the first End Turn, one call of three diverges, and in exactly one field:
```
### Game::TechTree::ProcessResearch: first 1 of 1 divergent call(s)
- call_id 0 (recap-b3-compare.jsonl:2)
side.events.after.v.next_id [exact] orig={"t":"i32","v":4} ours={"t":"i32","v":3}
```
That is bit-for-bit the same signal B3's save-file oracle produced (`docs/B3.md`:
`EvNxID 4 → 3`, one `EVENT_RESEARCH_OVERBUDGET` missing across 40,300 items). The 609 KB
save diff has been replaced by one line in the compare report.
Corroboration that this is the right branch (lane E: the post site is 0x00587b97, reachable
only from the completion-roll-**failed** path, gated `!wasDone && nowDone && owner`, and it
sets `node->flag = 2` in the same branch):
| region | before | after (orig) | after (ours) |
|---|---|---|---|
| `node[144].progress` | 2879 | 5768 | 5768 ✓ |
| `node[144].flag` | 1 | **2** | **2** ✓ |
| `events.next_id` (= `ServerPlayer+0x2b0`, `EvNxID`) | 3 | **4** | **3** ✗ |
| `rng` (`left` 432→431, `mt[624]` hash, `next_index` 192→193) | | | identical ✓ |
| `overbudget` (the caller's `int*`) | 0 | 0 | 0 ✓ |
So the node arithmetic, the flag and the single RNG draw are all reproduced; the *only*
thing missing is the event. Alloc for that call was `{tech 144 (IND_Waldo), 2889 points}`,
species 2 — B3's call 0 exactly.
### Across five turns: 15 calls, 3 divergent
| call | alloc | draws | events `next_id` orig → ours | guard hits |
|---|---|---|---|---|
| 0 | {144, 2889} | 1 | **4 → 3** | none |
| 1, 2 | {90,0}, {9,0} | 0 | 0 → 0 | none |
| **3** | {144, 2898} | 0 (ceiling ⇒ guaranteed) | **7 → 5** (two events) | 9 |
| 4, 5 | zero-spend | 0 | 0 → 0 | none |
| 6 | {9, 3064} | 1 (rolled, failed) | 8 → 8 | none |
| 7, 8 | zero-spend | 0 | 0 → 0 | none |
| **9** | {9, 3074} | 0 | **12 → 10** (two events) | 4 |
| 10, 11 | zero-spend | 0 | 0 → 0 | none |
| 12 | {18, 3087} | 1 (rolled, failed) | 14 → 14 | none |
| 13, 14 | zero-spend | 0 | 0 → 0 | none |
* **RNG matched on 15 of 15** (`mt[624]` hash, `left` and `next_index` identical on every
call) — stronger than B3's 14/15, because in this session no tech-effect callback drew.
* Calls 3 and 9 are the completions. Their non-`events` divergences are exactly the
`SetResearched`-owned fields B3 declared out of scope (`cost_rp`/`state`/`turn_available`
on unlocked children; `ours` leaves them at `INT_MAX`/0/−1). Nothing new there.
* Note the dedup rule (lane E): a duplicate post inside a turn bucket returns the existing
id **without** advancing `EvNxID`, so `next_id` not moving is not proof nothing was posted.
Calls 6 and 12 rolled and failed *without* completing by cost, so no post is expected there.
### What the guards mapped: `SetResearched`, measured
This is the "cheapest map of what `SetResearched` actually touches" the audit asked for.
Undeclared byte runs inside `guard:player` / `guard:tree_header` on the two completion calls,
resolved against the recovered `ServerPlayer` table:
| offset | len | field | call 3 | call 9 |
|---|---|---|---|---|
| `player+0x10c/0x110/0x114` | 3 each | `ConMod[0..2]` (construction cost per hull class) | ✓ | |
| `player+0x124` | 3 | `OutMod` (industrial output multiplier) | ✓ | |
| `player+0x130` | 3 | `PopMod` | | ✓ |
| `player+0x274 / 0x278 / 0x27c` | 3 each | **`vector<ObservedTech> otch`** — all three vector words move (a realloc) | ✓ | ✓ (+0x278 only, 1 byte) |
| `player+0x294` | 4 | `ResTNm` — the current research target pointer is cleared | ✓ | ✓ |
| `tree_header+0x20` | 1 | **the tree's completion-order counter** | ✓ | ✓ |
Two things worth carrying forward:
1. `tree_header+0x20` is harness-audit row 9, confirmed on the game: the per-node `order`
word is compared but the counter it is read from moves outside every Result region.
2. **`otch` (`vector<ObservedTech>`) grows on a tech completion, and no coverage note in
either B2 or B3 mentions it.** It is a third list append in the same neighbourhood as the
event list, and it *is* save state (`ServerPlayer` serializer). B3's replace-mode oracle
never saw it because turn 1 contains no completion.
3. The `player+0x10c…0x124` hits are precisely `OnTechResearched`'s own declared regions
(§4) — a clean cross-check that the guard is reporting the callee, not noise.
## 3. `ServerPlayer::ComputeBudget` — verdict held, and the guard is quiet
**4284 calls, 4284 compared, 0 divergences, `tracecmp` exit 0.** The `budget_object` guard
(Budget+0x00..0x68) caught **0 undeclared writes in 4284 calls**. In particular the
over-budget int at **`Budget+0x64`** (harness-audit row 11 — the word B1 captured only as an
argument, and arguments are never compared) **never changed value on any call**. That does
not prove the original never writes it, only that it never writes a *different* value here.
Coverage, honestly:
* **Only 20 distinct `(player, budget-output)` states exist across the 4284 calls.** 4278 of
the calls are the strategy-map UI polling the human player (one pointer accounts for 4115
of them); the End-Turn pass itself is ~8 calls, one per player in server order. The
headline number is repetition, not evidence.
* **13 of the 22 named slots are 0 on every single call**: `tradeIncome`, `shipCarriedPop`,
`secondaryManager`, `bonusIncome`, `systemIncomeNeg`, `debtInterest`, `construction`,
`expenses`, `researchMoneyGiven`, `savingsGiven`, `tra`, `researchPointsGiven`, `trp`.
The board's "budget tail coverage" backlog row is confirmed, and it is worse than stated:
five of the six *input-boundary* slots (1,2,3,4,7,11 = `systemIncomePos`, `tradeIncome`,
`shipCarriedPop`, `secondaryManager`, `systemIncomeNeg`, `construction`) are also always
zero, so they prove nothing twice over. Only `systemIncomePos` carries a value.
* Nine slots ever moved: `savings`, `systemIncomePos` (input), `savingsInterest`,
`maintenance`, `researchMoneyKept`, `available`, `researchMoney`, `researchPoints`,
`totalResearchPoints`. **Eight of those are genuinely modelled.** That is the real width
of the B1 result.
* Cross-check with B3: the End-Turn call for the AI Tarkas player yields
`researchPoints = 2889`, which is exactly the allocation `ProcessResearch` received.
* The B1 replace-mode double-run defect was **not** exercised: this lane ran compare only.
## 4. `ServerPlayer::OnTechResearched` — first guarded capture; the roll branch finally fired
**2 calls, 2 compared, 0 divergences.** The workload had to run to turn 7; the reference
turn contains no completion at all, which is why B2 never captured this hook on it.
| call | tech | branch reached | declared regions the original moved |
|---|---|---|---|
| 71 | 10001 `IND_Waldo`, node 144, `silent=false` | economy modifiers | `modifiers` (`con_mod[0..2]` 1.0 → 0.899999976), `research_target` (ptr → null) |
| 147 | 10094 `IND_TRKSTL`, node 142 | **design-option mask + the pending research roll** | `design_masks` (`b` 64 → 2097216), `research_target`, `roll`, **`rng`** |
**The `RollResearchEvent` draw is now behaviourally verified** — the board's backlog row.
On call 147 `research_roll_pending` was `true` going in; the original drew exactly one word
(`left` 375 → 374, `next_index` 249 → 250) and cleared the flag, and `ours` reproduced both
on the scratch generator bit-for-bit. That closes "region declared, ours reproduces it, but
the branch never fired".
Guard `player` (0x3e0 span) findings — **B2's known gap is now measured, not assumed**:
| offset | len | field | calls |
|---|---|---|---|
| `player+0x2b0` | 1 | **`EvNxID`** (`EventStorage+0x14`, `EventStorage` embedded at `ServerPlayer+0x29c`) | both |
| `player+0x274/0x278/0x27c` | 3 | `vector<ObservedTech> otch` — the vector grew | 71 |
| `player+0x278` | 1 | same vector, end pointer only | 147 |
So the completion events (`EVENT_RESEARCH_COMPLETE` / `_UNDERBUDGET` / `_TEMPERANCE`) are
confirmed to post on **every** completion, and `ours` posts none. The guard names the byte
rather than leaving it to a save hash that `docs/B2.md` gotcha 4 explicitly told readers to
ignore.
Not exercised at all: `silent = true`, a Zuul boarding-pod grant, a plague cure, a
node-bore allocation, the temperance sweep, the arcology re-evaluation, or any of the
writes to other objects. **Two calls, two branches. This is a thin capture.**
## 5. `ServerSystem::ProcessTurn` — first guarded capture
**140 calls (28 systems × 5 turns), 140 compared, 0 divergences.**
The coverage is the finding:
* **Only two declared regions ever moved in 140 calls**: `ntdev` (15 times) and `rcex`
(7 times). `infra`, `ibon`, `pbon`, `tres`, `haltv`, `bats2`, `bats_mask`, `rcex_mask`
and `rng` were byte-identical before and after on **every one of the 140 calls**. A
0-divergence verdict here means "two integer counters agreed and nothing else moved" —
it does not exercise the decay, the bonus applies, or the battle countdown.
* `rng` unchanged on all 140 confirms B4's headline over five turns: no colony drew a word,
i.e. no rebellion fired. `fpu_cw = 0x127f` on every call.
* Owners: 3 owned systems (`owner` 0, 1, 7) × 5 turns = 15 calls; the other 125 are unowned.
Guard `system` (0x2d8 span) hits, 15 writes on 6 calls:
| offset | len | plausible field | when |
|---|---|---|---|
| `system+0x1a4` | 12 | inside `Pop2` (civilian `Population`) at 0x1a0 — the population-group vector's three words | sys 4, 15, 16 on **turn 1 only** |
| `system+0x170` | 1 | inside `Fleets` (`vector<StarFleet*>` at 0x16c) — the end pointer | sys 15, every turn |
| `system+0xd8` | 1 | **unnamed in the contract** | sys 15, every turn |
| `system+0x238` | 4 | **unnamed in the contract** | sys 15, every turn |
System 15 is the AI's home world and it emits one new fleet per turn (see §6: fleets 50, 66,
98, 114 appear one per turn) — so `system+0x170` is the build queue pushing the new fleet
into the system's fleet vector, harness-audit row 14 observed live. `+0xd8` and `+0x238` are
unidentified and worth an addresses.json entry.
**The morale-event append (audit row 3) never fired** — no addiction on this save — so that
risk is still unmeasured, not cleared.
## 6. `StrategyServer::MoveFleet` — 8 of 45 diverge; B4's clean verdict was a one-sample verdict
**45 calls, 45 compared, 8 divergent, `tracecmp` exit 1.** Every divergence is a 1-ULP
position component.
```
- call_id 42 side.pos.after.v.y [exact] orig=3.1515913 ours=3.15159106
- call_id 79 side.pos.after.v.y [exact] orig=1.58417809 ours=1.58417821
- call_id 116 side.pos.after.v.y [exact] orig=0.0167649984 ours=0.0167651176
```
Structure of the run: 30 of the 45 calls are the six waypointless fleets
(`wpt_type = -1`, early-out, no guard hit). **15 calls do real work and every one of them is
`wpt_type = 1`** (a straight run at `speed 2 × dt 1.0`). Types 2/3/4/5 never occurred, so
node-line travel, node routes, gate teleports and the probabilistic jump remain untested,
and the generator never moved in this hook.
Of the 15 moving calls, ULP deltas (orig − ours) on the position:
| call | fleet | Δx | Δy | Δz |
|---|---|---|---|---|
| 6, 41, 78, 115, 154 | 34 | 0 | 0 | 0 / 0 / 0 / 0 / **+4** |
| 42, 79, 116 | 50 | 0 | **+1 / −1 / −64** | 0 |
| 80, 117 | 66 | 0 | **+1** / 0 | 0 |
| 118, 157 | 98 | 0 | **+1 / −1** | 0 / **+1** |
| 155, 156, 158 | 50/66/114 | 0 / **−1** / 0 | 0 / **−1** / 0 | 0 |
The 64-ULP case is a near-zero result (0.0167…): the **absolute** error is ~1.2e-7 in every
case, i.e. half an ULP at the magnitude of the *inputs* (y ≈ 1.6–4.7), not of the result.
That is the signature of one rounding too many (or too few) in the position update —
`sim::AdvanceAlongDirection` narrowing an intermediate to float32 where the original keeps
the x87 53-bit double, or the reverse. It is not a formula error: x, y and z all diverge in
some call, the step length is right, and every ship-range region matched.
**Why B4 missed it:** B4 had exactly one moving call, fleet 34 — and fleet 34 matches
bit-for-bit on four of its five moves here too. The bug only shows once other fleets start
moving. A reader of B4's trace would have seen "1 straight move, verified"; the verdict said
"MoveFleet verified".
Guard `fleet` (0x120 span), 42 writes on the 15 moving calls, resolved against the
`StarFleet` table:
| offset | len | field | count |
|---|---|---|---|
| `fleet+0x10c` | 1 | `Flags` (bit 0x100 = held this turn) | 15 — every moving call |
| `fleet+0xdc` | 1 | unnamed, just past `Speed` (`FPsp2` @0xd8) | 13 |
| `fleet+0xa0` | 4 | **`Location*`** — the fleet's Location object pointer is replaced | 8 |
| `fleet+0xcc` | 1 | `Waypoints` vector `last` pointer — **a waypoint was popped** | 2 |
| `fleet+0xdb` | 2 | straddles `Speed`/0xdc | 2 |
| `fleet+0xe0` | 26 | `FPogn2` @0xe0 … `DestPos` @0xec — the destination position rewrite | 2 |
The two calls carrying `+0xcc` and `+0xe0` (115 and 155) are the **two arrivals** in the
whole run — and both compared clean, because none of the arrival machinery is declared. B4
said "an arriving call was simply clean"; that is still true and still means nothing.
## 7. What a reader of the trace would have noticed that no verdict did
* `ProcessResearch` fires **3 times per End Turn**, not 15 — B3's 15 was five turns. Turn 1
of `ref-turn2` contains **no tech completion at all**, which is why `OnTechResearched`
produced zero records on the reference workload and B2's three completions must have come
from a longer run. A lane that captures `OnTechResearched` on the documented one-turn
recipe will get an empty log and a green report.
* **The multi-turn continuation is not reproducible across sessions.** Turn 1 is
byte-identical to the oracle every time, and calls 0–3 reproduce `docs/B3.md` exactly
(`{144,2889}`, `{90,0}`, `{9,0}`, `{144,2898}`), but from turn 4 the AI picks a different
research target than B3 recorded (`{9,3064}` here vs `{142,3064}` there, `{18,3087}` vs
`{9,3086}`) while the point totals stay nearly identical. Anything past the first End Turn
should be treated as *a* run, not *the* run.
* `ComputeBudget` is called ~4280 times per session by the UI and ~8 times by the turn pass.
Any "N compares" headline for that hook is dominated by UI polling of one player.
## 8. Still owed
* The count-only `events` region for `ProcessResearch` recommended by lane E's
`docs/E-events.md` (converts audit row 1 from "known defect" to "checked"). Not wired —
this lane's captures predate that module and re-capturing a good trace was not worth it.
* `MoveFleet`'s 1-ULP position divergence needs the rounding in `AdvanceAlongDirection`
pinned against the instruction stream.
* `ServerSystem+0xd8` and `ServerSystem+0x238` are unnamed and are written every turn by the
AI home system. `StarFleet+0xdc` likewise, on every move.
* `vector<ObservedTech> otch` (`ServerPlayer+0x274`) is undeclared everywhere and is save
state.
* Nothing here exercises: replace mode (any hook), a Zuul double roll, a rebellion, an
addiction sweep, a plague, gate traffic, waypoint types 2–5, `silent = true` completions,
the budget expense/aid/debt tail, or the `Budget+0x64` over-budget write.