sots-engine/docs/CB-predictions.md

CB — predictions for the command-stream capture, written before the module exists

Lane CB, 2026-09-08. Worktree wip/cb off main 4f25f1e (the brief said 7444c3b; the integrator merged L1 in the meantime, so this branch is based on the newer main and says so). Guest VM146 (192.168.10.146). Committed before ai_orders.cpp is touched and before anything is built.

Lane RB is building sots_turn --turn-commands <blocks>. This lane supplies its input. A replay with no real stream is untested and a stream with no autosave to check it against is a log file, so every prediction below is either about the stream or about the binding between the stream and the autosave of the run that produced it.

---

0. What is being changed, and why it is small

Lane L4 already dumps the block set at StrategySim::ApplyTurnCommandBatch — one register-transparent entry stub, read-only, measured behaviour-neutral on the canonical pair. This lane does not rebuild that. It adds three things and nothing else:

1. aiorders.words=<N> — the element window, today a fixed 12 words. List 1 carries a polymorphic ShipDesignDef whose id sits past 48 bytes, so the turn-1 workload cannot be captured completely at 12.
2. aiorders.deep=on — follow the heap. Three payloads in the canonical block are behind pointers and are therefore absent from L4's capture: list 8's route vector, list 10's counted vector, and list 23's Population body. A replayer cannot reconstruct a fleet move without the route. The follower is deliberately generic — it scans the element window for anything shaped like {_Myfirst,_Mylast,_Myend} and for anything shaped like an MSVC std::string, and dumps the bytes. It does not know which list it is looking at.
3. Nothing in the instrument types an element. L4's design point — the dump reads element bytes, not element types; decoding happens offline so a wrong record shows up as a wrong value instead of being baked into the instrument — is kept, and the typed decode moves to a host-side Python step that can be corrected without another VM run.

So aiorders=on aiprobes=off remains exactly one MinHook detour, and the added work inside it is guarded reads of memory the game already owns.

---

1. The capture format (defined here because RB may not have landed first)

One JSON file per run under verify/results/turncommands/. Mechanical, and the raw words are the ground truth:

• binding — the input save and both output autosaves, each with size and sha256, plus whether they match the published determinism oracle. A capture whose binding.outputs is absent or whose hashes do not match the run is worthless for a byte-match and must be treated as such.
• batches[].blocks[] — one entry per block slot, playerId, the six gates with their payloads, and all 27 lists.
• Each element carries raw_words (the window, verbatim), vectors (each followed heap vector, with its element words), strings (each decoded std::string), and decoded — a named record only where the campaign has one, with a record field naming it and a wire array giving the values in the order the writer emits them.
• decoded is regenerable from raw_words by tools/turncommands_capture.py. RB should prefer decoded.wire and fall back to raw_words; if the two ever disagree, raw_words wins.

The one thing the decoder must get right and that is easy to get wrong: list 3's in-memory element is in the opposite order from its wire record. Lane L4 read that off the writer (0x00822870 emits +0x14, +0x10, +0x0c, +0x08, descending); lists 5, 8, 10, 14 and 23 all write ascending. That is per-list, not a rule, and the decoder encodes it as such.

---

2. Runs, and what each is for

run	save	hooks	aiorders	watch	purpose
C0	ref-turn2.sav	off	—	off	rule-19 control: the oracle bytes with no instrument at all
C1	ref-turn2.sav	trace	on, words=32, deep	off	the deliverable — canonical pair capture
C2	ref-turn2.sav	trace	on, words=32, deep	on (modcount)	the trap multiset, as RB's second check
C3	turn1-state.sav	trace	on, words=32, deep	off	the non-deterministic pair, block + its own autosave
C4	turn1-state.sav	trace	on, words=32, deep	off	the same turn twice

---

3. Predictions

P1 — C0 reproduces the oracle. (control)

(Autosave EndTurn).sav = 66,732 B, bb4fd9ac89f41e3b; (Autosave).sav = 67,219 B, 978041acd168b56e. If this fails, the guest is not the reference guest and nothing else in this lane means anything; lane L4 already reproduced these on VM145 and lane L5 held VM146, so a failure here is a lab fact, not a finding.

P2 — C1's autosaves are byte-identical to C0's, so the capture binds.

The deliverable stands or falls on this. Lane L4 ran seventeen detours on this same workload and got the oracle bytes; this run has one, doing strictly more reads inside it. If C1's autosaves differ from C0's, the capture is a stream that changed the turn it recorded, it is unusable for a byte-match, and the report must lead with that (rule 19).

P3 — the block set on the canonical pair is exactly lane L4's, plus the three payloads.

Two batches: seq=1 at load (n=1, pid 16, all 27 lists empty) and seq=2 at End Turn (n=8). Slots 0–3 are pids 16, 32, 496, 512; slots 4–7 are pid 0 with uninitialised gate payloads. Only pid 32 is non-empty, with list sizes 3:1, 5:1, 8:1, 10:1, 14:2, 23:1. Every rate gate on the four real players is set; no research-target gate is set on any player this turn.

New, and the reason for the run:

• P3a — list 8's route vector has count 1. Its single value is the destination system id. L4 could not read it and noted that turn3-state.sav's waypoint says 272. Predict 272. Falsifier: any other id, in which case the waypoint in the output save is not the route in the command and the replayer has to carry both.
• P3b — list 10's counted vector has count 1. L4's reading is "at system 288, fleet 34, [one object]". Predict the value is a ship id that already exists in ref-turn2.sav — the pre-existing ship being folded into the newly allocated fleet 34. Falsifier: a small client-allocated id in the 18/34 space, which would mean the client allocates the ship too and the id-allocation hole L4 left open is wider than one counter.
• P3c — list 23's Population vector is 24 bytes, i.e. six words, at a stride the dump does not name. Predict the six words are not all zero and that they differ between the turn-2 and turn-1 captures (L4 saw the element's trailing int go +1 on turn 1 and −1 on turn 2, so the body is turn-dependent and not a constant the replayer can ignore).

P4 — C2's trap multiset is exactly ten bumps, and they attribute.

The paying half of pid 32's block is 3:1 + 5:1 + 8:1 + 10:1 + 14:2 = 6 elements, list 23 being free; plus 4 research-rate gates, one per submitted block, set unconditionally. Total 10 — which is the number lane W2 measured on this turn from the other end, before anyone had seen the block. Predicted per-EIP:

trap EIP	count	what
0x00821a87	4	prologue gate +0x0c, research rate, one per submitted block
0x0084946e	1	list 3, build
0x0086c3e9	1	list 5, system rates
0x0088bf01	1	list 10
inlined at ApplyTurnCommandBatch+0xabc	2	list 14, the AI's two (fleet, mode) elements
list 8's site	1	fleet move

Falsifier: any total other than 10, or any bump attributed to a list this capture reports empty. That would mean the counter is not the length of the command stream and the whole ModCount-from-the-block claim in src/game/ai/orders.h is wrong.

Rule 19 for C2 specifically: the watchpoint module has never been run in the same process as the block dump. If C2's autosaves move off the oracle, C2 is reported as perturbing and C1 remains the deliverable — the trap set is a bonus check, not the capture.

P5 — C3 and C4 differ in exactly one word of the stream.

Both from turn1-state.sav. Predicted block: pid 32 with lists 1:1, 3:1, 5:1, 23:1, three research-target gates set (pids 32, 496, 512) and the human's clear, four rate gates set.

The prediction that matters: C3's and C4's block sets are byte-identical except for player 512's research-target payload at block+0x10, and each run's autosave differs from the other's only in p512.ResTNm and the derived checksum. That is Rung B's claim in its sharpest form — if the two streams differ only where the two autosaves differ, then the sim is a function of the stream and the decision layer is the only thing that is not.

Falsifiers, and each is worth more than the prediction:

• The two blocks are identical and the two autosaves differ. Then the stream does not determine the save, Rung B is not reachable by replay alone, and the divergence is downstream of the command — which contradicts L4 §3.1, which saw it in the block.
• The two blocks differ somewhere else as well (an id, an ordinal, a route). Then the client-allocated id space is also per-process and a replay has to carry ids it cannot derive.
• Player 512's target lands on a tech outside the six-member XNC set L4 named. Then k is larger than six and Rung C's tie-set claim has to start one level up. BIO_GnMod has already been observed once, so this is not unlikely; a second BIO value would settle that the resolver leaves the family.

P6 — list 1's design element yields its id and its name at words=32.

"Honor Lance" is a short-string-optimised std::string inside the element, and design id 18 is named by list 3 in the same block. Predict the widened window shows the string and a word equal to 18 inside the list-1 element. Falsifier: no 18 anywhere in 128 bytes, in which case L4 §1 P3 stands as it is — the design command does not carry its own id and the id is inferred.

P7 — what this lane will not be able to say.

Stated in advance so a green capture is not read as more than it is (rule 15):

• Two turns, one AI empire with anything to do, a board with no contact. Lists 2, 4, 6, 7, 9, 11–22, 24–27 will be empty in every block of every run, exactly as they have been in all eleven corpus saves and both of L4's runs. The capture exercises six of twenty-seven lists and one of six gates on the canonical pair, two of six on turn 1. Everything the replayer does with the other twenty-one lists is untested by this lane and must be labelled as such.
• The generic vector follower can produce a false positive — three consecutive words that happen to look like a vector. The count bound and the readability probe make that unlikely, not impossible. Any aivec line on a list whose record has no vector is to be read as noise until a second run reproduces it.
• Nothing here says what list 10 or list 23 mean. It says what bytes they carry.

---

Addendum — the seeds are part of the stream (written before C3/C4/C5, after C0/C1 were driven)

The coordinator relayed lane L1's seed probe mid-lane: every AI client's generator is seeded with a fresh per-process 32-bit word, three per process, none shared between processes. So the AI is not non-deterministic — it is MT19937 from one word per client, and MT19937 is a generator this campaign owns bit for bit. game/ai is a deterministic function of (save, per-client seed).

That changes what a capture is. A block set without its seeds records the AI's answer without its input: replayable, but not re-derivable and not re-runnable. Three words turn a log file into a reproducible pair. The capture format therefore grows one field, and the instrument grows one detour:

"ai_seeds": [ {"call": 1, "netId": 32, "observed": "0x…", "used": "0x…", "pinned": false}, … ]

observed is what the game chose; used is what the client actually got; pinned says the run forced it. Every call logs all three, so a pinned run can never be mistaken for a natural one — which matters because pinning is an intervention, not an instrument.

aiseed=log reads the fourth stack argument of Game::StrategyApp::RunAI where the callee will read it and changes nothing. aiseed=pin overwrites it before the callee sees it.

P8 — the new rule-26 control, stated as a prediction about method

Three lanes ran hooks=off on turn1-state and got three different files. So no single un-instrumented run of that workload is a control, and an instrumented run agreeing with one of them is a ~1/k coincidence rather than evidence — with k = 6 by lane L4's naming of the XNC set, that is a 17% chance of a false clean bill of health. C0 is a valid control for ref-turn2 → turn3 and for nothing else. Any rule-19 claim this lane makes about the turn-1 workload must come from a pinned pair, or it must not be made.

P9 — C5: pinned, the same seeds, twice.

C5 is C3 re-run with aiseed=pin set to the seeds C3 itself observed. Predicted: C5a and C5b produce byte-identical autosaves, and both are byte-identical to C3's, because the seed is the only per-process input and it has been removed. If that holds, it is the first fully reproducible creation-turn record this campaign has: block set, seeds and autosave, all from runs that agree.

Falsifiers, each worth more than the prediction:

• C5a and C5b still differ. Then the seed is not the only per-process input, L1's mechanism is incomplete, and Rung B needs whatever else varies captured too. This is the single most valuable outcome available in this lane.
• C5 is reproducible but differs from C3. Then pinning perturbs beyond the seed value — most likely because the detour changes when the word is consumed, not only what it is.
• RunAI is never entered. Then the seed does not reach the client by this path on this workload and the pin is a no-op that would look like a successful control. The instrument prints its own entry count precisely so this is visible; a silent empty seed list would be exactly the rule-1 failure of a confident zero.

P10 — the target gate's resolved name (lane RB's request)

RB's §163 asks the format to carry the observed tech name next to the target id, because the target-gate handler cannot be modelled from the id alone. airesearch=on resolves it — phase 18 pushes the tech's std::string as a char*, which is why it is loggable at all. Predict the three turn-1 target gates (ids 144, 90, 288) resolve to three names, of which player 512's is the one that moves between runs. Adding those three detours is a rule-19 risk on a workload that has no valid control, so it goes on the turn-1 runs only, never on the canonical pair.

---

Outcomes recorded against the predictions above

P1 held. C0, hooks=off, reproduced the oracle exactly

bb4fd9ac89f41e3b… / 66,732 B and 978041acd168b56e… / 67,219 B. VM146 is a faithful reference guest and the control is real.

P2 held. C1's autosaves are byte-identical to C0's, so the capture binds

One detour (the block dump) plus one (the seed reader), both read-only, and both oracle hashes came back. The stream and the autosave are from the same run.

P3 held, with P3b half-falsified

272 for the route (P3a, exactly as predicted from the output save's waypoint). 1728 for list 10's counted vector — a master-counter id, not a client-allocated one (P3b's falsifier). The Population body is 24 bytes and turn-dependent (P3c).

P7's false-positive warning was right, and it cost nothing

At words=32 the generic vector follower fires on neighbouring heap: 18 aivec lines on the canonical block, of which three sit at a word index a record names. The decoder reads only those three, so the noise is visible and inert — which is the outcome the warning was written for.

A defect found by the consumer, not by me — list 5

Lane RB consumed this capture and reported that my list-5 record was wrong: the element is dumped in memory order and its memory field order is not its wire order. Memory member 1 is wire member SRsc; six members are unread. RB predicted zero regressions on its first replay and got two, the AI's one non-zero slider landing in SRt where the oracle holds SRsc = 1.0.

The record is now dropped, per RB's recommendation and against my own first instinct to relabel it positionally: the values ship as rates_memory_order_{u32,f32} with wire: null, and the .tcb emits the seven ? tokens lane RB's own converter emits. A replayer that needs them must refuse rather than guess. A missing field is honest; a mislabelled one propagates — and it would have propagated inside the artefact that is supposed to outlive the adapter that read it.

What the capture bought, reported by the consumer

sots_turn --turn-commands puts /Sim/ModCount on 24 exactly, zero residual: 2 drivers + 4 research-rate gates + list 3 + list 5 + list 10 + 2 × list 14 + list 8, with the list-23 population element free. That leaf has been "not derivable from a save" since lane A2. The 1–16 / 17–27 cost boundary in src/game/ai/orders.h held on the first workload that ever populated the free half.