sots-re/findings/subsystems/eventlive-verification.md
alex 7eabfefff2 lane V: live verification of the event-posting model against an advance prediction
Lane P wrote the expected numbers into sots-engine docs/P-events-wiring.md §4 before
this run existed. Every one held.

First End Turn (ref-turn2 -> Launch -> End Turn, shim.cfg.recapb3 unchanged, build
eventlive-dd38117-20260908T0916Z, main dd38117 with no source change): 3 calls, 3
compared, 0 divergent, tracecmp exit 0 — where lane R's run exited 1 on
side.events.after.v.next_id orig=4 ours=3. Call 0 read turn=3,
events_turn_bucket_exists=true, events_next_id_in=3, events_in_turn_bucket=1,
events_dedup_risk=0, no events_scan_truncated, next_id 3->4 on both sides,
turns/turns_bytes 2/48 unchanged, node[144].progress 2879->5768, flag 1->2, rng
identical — the whole predicted list.

Five End Turns: exit 1 with 2 divergent calls instead of 3, each short by exactly 1
(next_id orig 7 ours 6, orig 12 ours 11) = the deliberately unmodelled
EVENT_TECHS_UNLOCKED. A 0-divergence result there would have been suspicious.

sizeof(Game::ObservedTech) = 44 measured live: observed_techs.bytes grew by exactly
44 on both completion calls (440->484, 484->528), confirming lane X's static pin
behaviourally. Non-researching players measured 880 = 20 x 44 and never moved.

One deviation from lane R, and it is a workload effect, not a defect: rng diverged on
call 9 because the completed tech had research_roll_pending set and the
OnTechResearched callback drew one word ours does not (left 374 vs 375). Lane R's
"RNG matched 15 of 15" was workload luck; the honest statement is that the rng region
matches on every call that does not complete a roll-triggering tech.

Zuul double roll CLOSED. No species-5 save existed, so this lane made one: a custom
game with only Zuul in the Available Species pool, 4 End Turns under the same compare
config. 8 calls, 8 compared, 0 divergent, exit 0; all four researching calls have
species=5 and advance the generator by two, not one (left 540->538, 522->520,
504->502, 485->483), and ours reproduced every post-state bit-for-bit. Save added as
verify/results/saves/zuul-turn5-species5.sav.

End-Turn oracle byte-identical to lane R's on the first turn, so none of this
perturbs the game. clean_room_check OK and host ctest 33/33, run as separate commands.
VM140 restored to the recap build at the main menu and released.
2026-09-08 05:59:22 -04:00

11 KiB
Raw Blame History

Live verification of the event-posting model, and two workloads (lane V, 2026-09-08)

Lane P wrote a numeric prediction of this run into sots-engine docs/P-events-wiring.md §4 before the run existed. This is the check. The prediction held field for field on both runs — the strongest evidence this project produces, because it was written down first.

  • Build eventlive-dd38117-20260908T0916Z (sots-engine main dd38117, no source change), cross-built on CT111 /srv/re-lab/build/sots-engine-v, staged /srv/re-lab/shim/dist-v → C:\SOTS\shimdist-v on VM140. Exports byte-identical to the real binkw32.dll (66 names).
  • Recipe: shim.cfg.recapb3 unchanged, ref-turn2.sav → Launch → End Turn, then four more End Turns in the same session (turn 2 → turn 7). One rui.ps1 call per click, screenshot between every one.
  • Oracle held: (Autosave EndTurn).sav = bb4fd9ac89f41e3b…, (Autosave).sav = 978041acd168b56e… — identical to lane R's, so the events wiring does not perturb the game.
  • Artefacts: verify/traces/eventlive-b3-*.jsonl[.gz], verify/results/compare/eventlive-b3-*.{json,md}, shim logs and screenshots in verify/results/shim/eventlive/.

1. First End Turn — 3 calls, 3 compared, 0 divergent, exit 0

Lane R's run exited 1 here with one divergent field (side.events.after.v.next_id orig=4 ours=3). That field is now clean, and the fields lane P named as "the model is right for the right reason" all read as forecast:

field, call 0 predicted observed
turn 3 3
events_turn_bucket_exists true true
events_next_id_in 3 3
events_in_turn_bucket 1 1
events_dedup_risk 0 0
events_scan_truncated absent absent
events.next_id 3 → 4 both sides 3 → 4, no divergence
events.turns / turns_bytes 2 / 0x30 unchanged 2 / 48 unchanged
node[144].progress 2879 → 5768 2879 → 5768
node[144].flag 1 → 2 1 → 2
rng identical left 432→431, next_index 192→193, mt hash identical
observed_techs.bytes unchanged, all 3 calls 440 / 880 / 880, unchanged

Alloc {tech 144, 2889}, species 2, fpu_cw = 0x127f. Calls 1 and 2 are the other players with empty event lists (EvNxID = 0) and their scratch headers came back byte-identical.

2. Five End Turns — exit 1, 2 divergent calls, each short by exactly 1

call alloc draws next_id orig → ours predicted
0 {144, 2889} 1 4 → 4 4 / 4 ✓
3 {144, 2898} 0 7 → 6 7 / 6 ✓
6 {142, 3064} 1 8 → 8 8 / 8 ✓
9 {142, 3074} 0 (+1 in the callback, §4) 12 → 11 12 / 11 ✓
12 {9, 3086} 1 14 → 14 14 / 14 ✓
1,2,4,5,7,8,10,11,13,14 zero-spend 0 0 → 0 0 / 0 ✓

Lane R's run had 3 divergent calls and the two completions were short by two ids each; both are now short by one, and that one is EVENT_TECHS_UNLOCKED, which lane P deliberately did not model because evaluating its trigger needs the child-unlock cascade ours does not run. A 0-divergence result here would have been suspicious, not good.

Note the run is not lane R's run: from turn 4 the AI picks a different research target (lane R's trap #2), and calls 6, 9 and 12 have different techs from lane R's capture. The event id totals landed on the predicted values anyway, which is a stronger result than a replay would have been.

Guard conversion (P-events-wiring.md §4.3) — held

observed_techs is a Result region now, so player+0x274/0x278/0x27c no longer appear as undeclared guard spans; the vector shows up in the diff as observed_techs.bytes instead. Undeclared spans: 13 in 2 calls (lane R) → 10 in 2 calls.

Guard spans this run, resolved against the recovered ServerPlayer table:

offset call 3 call 9 field
player+0x10c/0x110/0x114 3 each ConMod[0..2]
player+0x124 3 OutMod
player+0x196 1 byte 2 of design-option mask B at +0x194 (TechMaskA + 4) — lane R saw it from inside OnTechResearched as b 64 → 2097216 = 0x40 → 0x200040
player+0x294 4 4 ResTNm, the research-target pointer cleared
player+0x3b4 1 ResErrRoll = research_roll_pending, cleared by the callback
tree_header+0x20 1 1 the tree's completion-order counter

So the two spans that are "new" relative to lane R are both already-named fields, seen from the ProcessResearch caller instead of from inside OnTechResearched. Nothing here is unnamed.

3. sizeof(Game::ObservedTech) = 44 — measured on the running game

completion call observed_techs.bytes (orig side) delta
3 440 → 484 +44
9 484 → 528 +44

An independent behavioural confirmation of a purely static pin: lane X pinned 0x2c three ways off the instruction stream (magic divide by 44, imul reg,reg,0x2c, add edi,0x2c) and lane S mapped every field from the serializer. The running game agrees to the byte. Both non-researching players' vectors measured 880 = 20 × 44 on all 15 calls and never moved. Recorded on ObservedTech_sizeof and ServerPlayer_off_ObservedTechs in ghidra/addresses.json.

Caveat for the lane that wires the append: RecordObservedTech (0x007ba1a0) de-duplicates by tech name, so a re-observation appends nothing. "No delta" is not a contradiction until you have checked whether the tech was already observed.

4. One deviation from lane R, and it is a workload effect

rng diverged on call 9: original left 374 / next_index 250, ours 375 / 249 — the original drew one word ours did not.

That is the RollResearchEvent draw inside OnTechResearched, at exactly the two generator positions lane R measured from inside that hook (left 375 → 374, next_index 249 → 250) when IND_TRKSTL completed with research_roll_pending set. B3 declared SetResearched and its owner callback out of scope, and P-events-wiring.md §3 says the callback "consumes one extra RNG word", so this is a declared boundary showing itself.

What it corrects: lane R's "RNG matched 15 of 15" was workload luck, not a property. Lane R's five-turn session happened not to complete a roll-triggering tech on the B3 hook; mine did. The honest statement is: ProcessResearch's rng region reproduces bit-for-bit on every call that does not complete a roll-triggering tech, and is short by exactly one draw on every call that does — until whichever lane takes SetResearched models the callback.

5. Workload unblocked: Zuul double roll, closed

The board row asked for one live compare from a species-5 save where left drops by 2, not 1. No save in verify/ had a Zuul player, so the double roll was disassembly + host tests only.

Made on the VM: a custom game with only Zuul left enabled in the Available Species pool (so both players are Zuul), same 28-star / 2-player / 7 ly settings, then four End Turns under the same shim.cfg.recapb3 compare config.

8 calls, 8 compared, 0 divergent, tracecmp exit 0. The four researching calls all have species = 5 and all advance the generator by two:

call turn alloc left next_index mt hash
0 2 {144, 982} 540 → 538 84 → 86 unchanged
2 3 {144, 960} 522 → 520 102 → 104 unchanged
4 4 {144, 1417} 504 → 502 120 → 122 unchanged
6 5 {144, 1372} 485 → 483 139 → 141 unchanged

ours reproduced every one bit-for-bit (the rng region is compared, and no call diverged). Calls 1, 3, 5, 7 are the second Zuul player with a zero allocation — no draw, and clean.

Save: verify/results/saves/zuul-turn5-species5.sav (sha256 48559ab5b719b332e47b4bf43af85a616bff1508125d62edb211b80637234af6, 59,131 B, turn 5, 2 Zuul players, homeworld Gallandro, Rip Bore DE in Alpha Fleet, Slaves: 0 on the planet panel). Left on the VM as C:\SOTS\SavedGames\zuul-turn5.sav.

No tech completed in five turns (node 144 reached ~4,700 of 5,000), so this save does not yet exercise a Zuul completion, SetResearched, or the Zuul boarding-pod grant. Load it and run one more End Turn to get there.

6. Workloads NOT achieved

  • Expense sliders / a debtor / a research-aid treaty. Not attempted. The eight always-zero ComputeBudget slots stay unexercised. Getting there needs deliberate overspending across several turns and a second empire willing to sign an aid treaty — a multi-turn play session, not a click path.
  • A save that sets research_roll_pending at load time. Not made. The flag was observed set and cleared inside a turn (§4), which is what the RollResearchEvent row already covers, but there is still no save that starts with it true.
  • A tech completion in the Zuul game. One more End Turn from zuul-turn5.sav should do it.

7. VM140 click-path notes (additions to running-the-game.md)

  • The >60 s startup is real, a third time. After schtasks /Run /TN SOTS the Kerberos intro is still on screen at t+30 s and the main menu appeared at ≈ t+70 s. Screenshot and verify; never sleep-and-click.
  • One click per rui.ps1 call, confirmed again. A five-click cmd.txt on the Available Species checkboxes lost sync and left three of five toggles in the wrong state. Single clicks with a screenshot between worked every time.
  • type into the game's text fields did not register in this session — neither the Create Game name field nor the Save Game file-name field took SendKeys text (the field simply kept its default). Workaround used: save under the default name and rename the .sav on disk afterwards; the Load dialog lists files by filename, so a renamed save appears correctly.
  • Load-dialog geometry with the 8-file set is unchanged for existing lanes. zuul-turn5 sorts after ref-turn2, so ref-turn2 is still row 7 at (400, 436) and zuul-turn5 is the new row 8 at (400, 465). Rows are 29 px apart from y = 262. Verified by screenshot (verify/results/shim/eventlive/load-dialog-8files.png).
  • Custom Game Setup coordinates (1024×768 windowed): species checkboxes at y = 449, x = 178 (Human) / 322 (Hiver) / 466 (Tarkas) / 611 (Liir) / 901 (Morrigi) — all five clicked and verified; Zuul's sits at x ≈ 756 by the same 144-px pitch but was never clicked. Create Game (836, 707); game-name OK (600, 393); lobby Launch (511, 663). The setup screen remembers the previous game's settings, so 28 stars / 2 players carried over from the reference game.
  • Quit path used at the end: menu (1000, 714) → Quit to Main Menu (938, 699) → OK (537, 377).

8. VM state left behind

Restored: build recap-7584bad-20260908T0615Z from C:\SOTS\shimdist-recap, hooks=trace, main menu verified by screenshot. C:\SOTS\shimdist-v and C:\SOTS\ui\v{deploy,release,click}.ps1 left in place as a template. SavedGames now holds 8 files: the pre-existing 7 plus zuul-turn5.sav; the three autosaves belong to the Zuul game (they were the reference game's before, and are transient either way).