diff --git a/findings/control-flow/combat-done-tail.md b/findings/control-flow/combat-done-tail.md index 397dc48..8c93c98 100644 --- a/findings/control-flow/combat-done-tail.md +++ b/findings/control-flow/combat-done-tail.md @@ -276,8 +276,18 @@ will still diverge on the first turn a node line expires or a node cannon fires. Downstream of a successful roll (`FUN_007a92e0` 690 B then `FUN_007a4700` 2244 B) fleets are destroyed or halted and three events are posted: `EVENT_NODEDECAY_FLEET_DESTROYED_VIANODE`, `EVENT_NODEDECAY_FLEET_HALTED`, -`EVENT_NODEDECAY_FLEET_HALTED_VIANODE`. The roll is skipped for a line if any fleet with flag `0x20000` is -targeting it. +`EVENT_NODEDECAY_FLEET_HALTED_VIANODE`. ~~The roll is skipped for a line if any fleet with flag `0x20000` is +targeting it.~~ + +> **CORRECTED by lane Z, 2026-09-08 — the fleet check does not gate the roll.** The `0x20000`-fleet scan +> begins at 0x007ae0b2, which is 0x1d bytes **after** the `Chance(0.5f)` call at 0x007ae0a5 and is reached +> only when the roll *succeeded* (`test al,al; je 0x007ae1e2` at 0x007ae0aa). The straight-line order in the +> loop is **expiry test → draw → roll gate → fleet gate → collect**, so the fleet scan suppresses only the +> collapse (`FUN_007a92e0` / `FUN_007a4700`), never the draw. The headline claim above — one `NextFloat` per +> expired node line per turn — is unaffected, and the expiry test is now a read formula: +> `NodePath::RemainingLife` 0x006e2130 returns `npdtn − nptf/npdtf − (turn − npctm)` clamped at 0, with two +> never-expire early-outs (`npt == 0`, `npdtn == INT_MAX`). See `findings/control-flow/tail-rng-ledger.md` +> §6 and §6.1. *Source: the `Chance(0.5f)` call site and its operand are instruction-verified; the surrounding record layout and the downstream event names are from a ReVa sweep.* @@ -499,8 +509,15 @@ tail returns. The handler, read from the instruction stream (0x00784cf8..0x00784 ### 6.1 `StrategyHost::Autosave` 0x00895210 -`void __thiscall (StrategyHost* this /* the global at 0x00b29f98 */, std::string* outName, bool endTurn)`. -2364 B. Two call sites in the whole image: +`std::string* __thiscall (StrategyHost* this /* the global at 0x00b29f98 */, std::string* outName, +bool endTurn)`, **`ret 8`**. 2364 B. Two call sites in the whole image: + +> **CORRECTED by lane Z, 2026-09-08 — the return type.** The epilogue is `c2 08 00` and 0x00895b5c is +> `mov eax,esi`, which puts `outName` back in EAX: this is MSVC's named-return-value slot, not a `void`. +> A hook or reimplementation declaring it `void` drops EAX at both call sites. Neither site passes `this` +> as an argument — both hardcode `mov ecx,0xb29f98`. Lane Z also observed live that `[global+0x54]` is +> **not** the `StrategyServer`, so the global this function runs on is a different object from the +> `StrategyHost` whose `+0x54` `OnMessage` reads. | caller | `endTurn` | file | |---|---|---| @@ -579,6 +596,19 @@ this handler is the **only** reachable caller of `StrategyHost::Autosave(…, 0) but it is an inference from the oracle, not from the instruction stream. **A path you cannot exercise is a hypothesis** — this one is exercisable by lane O and worth one check. +> **CHECKED by lane Z, 2026-09-08 — half settled, and the other half narrowed.** A live hook on +> `OnAllCombatDone_Tail` recorded **exactly one call per End Turn on 8 of 8 End Turns**, at depth 0, between +> the two autosaves, with the post-turn autosave following it. So the handler **is** delivered every turn +> and the inference above was right. What is *not* settled is the stronger reading: on both saves played, +> the encounter vector is empty at `ProcessTurn` entry and holds **exactly one** encounter by the time the +> tail runs (detection creates it, phase 7 clears it), so what was observed is "the tail runs on a turn with +> **no battle**", not "on a turn with no encounter at all". Every encounter seen had `res->+0x4 != 0`, the +> flag that makes `ApplyEncounterResult` a whole-function no-op — measured cost 0 RNG words, exactly as the +> gate predicts. `findings/control-flow/tail-rng-ledger.md` §4. +> +> The same records confirm §2 behaviourally: `encounters` reads 1 at the phase-6 call and 0 at the phase-11 +> call, on every turn. Phase 7 really is a wholesale `clear()`. + --- ## 7. Corrections @@ -593,6 +623,13 @@ same word. Given §6's inference that the message is delivered every turn, `S+0x per turn while `S+0xc` (`ModCount`) advances once. They are not in lockstep, and a reimplementation must not treat `S+0x8` as a turn number. It reads as a **server-phase / driver-invocation counter**. +> **MEASURED by lane Z, 2026-09-08 — "at least twice" is the right phrasing; it is 12 to 14.** Read live at +> hook entry over eight turns on two saves, `S+0x8` advances **12–14 times per turn**, of which the two +> drivers account for 2. Both drivers were hooked, so the other 10–12 increments come from a writer nobody +> has identified, and they happen between the post-turn autosave and the next `ProcessTurn`. `S+0xc` +> advances by exactly 1 per turn over the same records. The conclusion here is strengthened: `S+0x8` is not +> a turn number and is not even a driver-invocation counter. `findings/control-flow/tail-rng-ledger.md` §5. + ### 7.2 Turn results and turn events are not where lane T said `turn-driver.md` §5 says: *"no bankruptcy, no turn-results build, no turn-events build, no autosave. Those diff --git a/findings/control-flow/tail-rng-ledger.md b/findings/control-flow/tail-rng-ledger.md new file mode 100644 index 0000000..fd776a8 --- /dev/null +++ b/findings/control-flow/tail-rng-ledger.md @@ -0,0 +1,324 @@ +# The RNG ledger for one strategic turn — measured, not inferred + +Lane Z, 2026-09-08. Program `sots` / "Sword of the Stars.exe", ImageBase 0x00400000, all addresses VAs. +Engine worktree `wip/tailrng`, `sots-engine/docs/Z-tail-rng.md` (the prediction, committed before the build). + +**The question.** The milestone is *a standalone that loads a save, runs one strategic turn, and writes an +autosave that byte-matches the original's*. Generator state is part of the saved state. Lane K +(`combat-done-tail.md` §3) found two draw sites in `StrategyServer::OnAllCombatDone_Tail` that nothing models +and that both run **before** the autosave, and concluded that a reimplementation reproducing both +`ProcessTurn` functions exactly would still diverge. Nobody had measured what a turn actually costs. + +**The answer, up front.** On the reference save, a strategic turn advances the strategic generator by +**18–20 words**, *all* of it inside `StrategyServer::ProcessTurn`, and the residual outside the two turn +drivers is **exactly zero**. The tail's cost on these turns is **0**. The defect lane K found is real and +**latent**: it will bite the first turn a node line expires or a real battle resolves, and our saves reach +neither. + +--- + +## 0. Where to start if you are building the standalone + +Three sentences, then the evidence. + +1. A turn on the reference save costs **18–20 generator words**, all of it inside + `StrategyServer::ProcessTurn`; the two autosave files bracket exactly that interval and nothing draws + between them outside the two turn drivers. +2. `verify/results/shim/tailrng/z-t6-endturn.sav` → `z-t6-autosave.sav` is a **byte-identical oracle pair + with a known RNG cost of 18 words**, verified from the file bytes independently of the live hook. Test + against that pair before any other. +3. Lane K's warning stands and is now quantified: the tail's cost is 0 **today** because no save is within + ~40 turns of a node-line expiry (§9) and no encounter has ever produced a battle (§8). Both terms are + real; both are latent. + +## 1. The instrument, and why it is not an RNG hook + +Counting draws by hooking the primitives would have undercounted, and the static work this lane +commissioned says by how much. The image has **four** draw entry points, not three: + +| VA | primitive | words per call | +|---|---|---| +| 0x0047d830 | `Mars::RNG::NextFloat(&mt)` — `__thiscall`, no stack args, plain `ret` | **exactly 1** | +| 0x004271c0 | `Mars::RNG::NextInt(&mt, uint* pMax)` — `ret 4`, rejection loop, bound re-read each iteration | **1 or more** | +| 0x008e6dd0 | `Mars::RNG::Chance(RNG*, float p)` — `ret 4`; takes the **object** base and does the `add ecx,4` itself | **0 or 1** — see §6.1 | +| **0x004f7670** | **`Mars::RNG::NextUInt(RNG*)`** — plain `ret`, no stack args. **In no previous lane's primitive set**, and called from 0x007b6700 inside `ProcessTurn`'s closure | 1 | + +plus **inlined draws in twelve functions**, two of them reachable from the turn roots (0x007aa240 under +`ProcessTurn`, 0x007a7f30 under the combat resolver). A primitive-counting hook would have missed all of +those silently. + +So the instrument reads **state**, not calls. `Mars::RNG` is +`{void* vptr; uint32 mt[624]; uint32* next; int32 left}`, `sizeof 0x9cc` — `next` at +0x9c4 is a **pointer** +into the block and `left` at +0x9c8 is the counter, and the three inner primitives are entered with +`ECX = &mt[0] = RNG+4`, which is what lane T's "the generator is entered at `rng+4`" note was seeing. The +draw is `if (left == 0) Twist(); y = *next++; --left;` — a pre-check against **0**, never −1. + +The block transform is a pure function, so the blocks a generator visits form a forward-only chain. +`RngLedger` (`sots-engine/src/shim/hooks/rng_ledger.{h,cpp}`) indexes that chain from the first block it +sees and positions any state exactly: + +``` +words(block, left) = block * 624 + (624 - left) +``` + +Differences between positions are then exact **across twists, across rejection loops, and across draws +nobody hooked**. Nine host tests pin the arithmetic, including the block boundary (`left == 0` is a real +state and must not be off by 624), a rejection loop counted against a shadow generator, and the +out-of-order case below. + +**One ordering subtlety, and it is load-bearing.** `Hook<>` takes the `before` snapshot at entry but renders +it (calls the region's `describe`) only *after* the original returns — so for nested calls the inner +snapshots are rendered first, and by the time an outer `before` is rendered the chain has moved past it. +Walking a Mersenne Twister backwards is not possible. Every hook therefore **observes at entry from +`describe_args`**, which indexes the entry block while it is still current; the render then resolves it from +the memo. Without that, every outer-call `before` would read `words: null`. + +Six nested trace hooks, all watching the object at `S+0x16c`: + +`StrategyHost::Autosave` (both markers) › `StrategyServer::ProcessTurn` › `OnAllCombatDone_Tail` › +`ApplyEncounterResult` (phase 6) › node-line decay (phase 11) › `ProcessNodeSpaceTravel` (runs twice a turn). + +--- + +## 2. The ledger + +Save `ref-turn2` (2-player Morrigi vs AI), four consecutive End Turns, build `z-tailrng-20260908T1314Z`, +`hooks=trace`. Words are 32-bit MT outputs consumed by the generator at `S+0x16c`. + +| turn | `ProcessTurn` | tail | `ApplyEncounterResult` | node-line decay | `ProcessNodeSpaceTravel` ×2 | **bracket total** | **residual** | +|---|---|---|---|---|---|---|---| +| 3 | **19** | 0 | 0 | 0 | 0, 0 | (incomplete — see below) | — | +| 4 | **18** | 0 | 0 | 0 | 0, 0 | **18** | **0** | +| 5 | **20** | 0 | 0 | 0 | 0, 0 | **20** | **0** | +| 6 | **18** | 0 | 0 | 0 | 0, 0 | **18** | **0** | + +"Bracket" is `Autosave(endTurn=1)` → `Autosave(endTurn=0)`: the pre-turn save file to the post-turn save +file, which is exactly the interval a standalone has to reproduce. The turn-3 bracket is incomplete **by +construction** and is reported rather than dropped: the pre-turn autosave of the first End Turn after a load +runs before any turn driver, so the hook has no server pointer yet and its record carries `words: null`. + +**So: we consume 18–20 words per turn and model 0 of them.** + +Not 0 because the modelling is bad — because *nothing in the repo models any part of a turn's RNG +consumption as a count*. B1/B3/B4 compare the generator's state around three specific functions and get it +right; no lane has ever stated a turn's total. This table is that statement. + +### 2.1 The generator does not move outside the turn pipeline + +Every turn's `ProcessTurn` entry position equals the previous turn's post-turn autosave position, exactly: +211 → 211, 229 → 229, 249 → 249. The UI, the renderer and the per-frame tick draw **nothing** from the +strategic generator between turns. For the standalone this is worth as much as the total: the interval it +must reproduce is closed. + +--- + +## 3. An independent check, from the files rather than from memory + +The two autosaves of the turn-6 bracket were pulled off the VM and their `Sim.RNG` blobs parsed by +`verify/save-reader/save_reader.py` (the frame's payload is 2503 bytes: `mt[624]`, then `left` as int32 at ++2496). Twisting the pre-turn block forward until it matches the post-turn block, and applying the same +position formula: + +``` +z-t6-endturn.sav (pre-turn) left = 375 +z-t6-autosave.sav (post-turn) left = 357 +twists = 0 -> words consumed between the two files = 18 +``` + +The live ledger recorded 18 for that turn, `left` 375 → 357. **Two instruments that share no code path — one +reading process memory through a hook, one reading gzip-compressed file bytes through the save reader — +agree exactly.** Files and the checker in `verify/results/shim/tailrng/`. + +**And the two do not share a hidden assumption** (the trap of method rule 8, which is live here because both +sides know how to twist an MT block). `twists = 0`: the block is byte-identical in the two files, so the +file-side number is `left_before − left_after` and involves the twist implementation **not at all**. The +agreement is therefore about the game's behaviour, not about two copies of the same algorithm agreeing with +each other. + +This is the pair a standalone should be tested against first: it is a byte-identical oracle *with a known +RNG cost attached*, which none of the eleven corpus saves has. + +--- + +## 4. Lane K's inference, settled + +> **§6, labelled hypothesis:** "I did not prove that `SNMAllCombatDone` is delivered on turns with no +> combat." + +**The handler runs on every End Turn.** Four out of four, `OnAllCombatDone_Tail` recorded exactly one call +per End Turn, at depth 0, between the two autosaves, with the post-turn autosave following it. The +determinism-note inference was right. + +The stronger claim — that it runs with an *empty encounter vector* — is **not** settled by this workload and +must not be reported as though it were. On `ref-turn2` the encounter vector is **empty at `ProcessTurn` +entry and holds exactly one encounter by the time the tail runs** on every one of the four turns: detection +(`ProcessTurn` phase 31) creates it, and the tail's phase 7 clears it. So what is proved is "the tail runs +on a turn with **no battle**", not "on a turn with no encounter at all". See §8 for what closing the +remaining gap needs. + +Two things fall out of the same records and are worth more than the phrasing: + +* **Phase 7 really is a wholesale `clear()`.** `encounters` reads **1** at the phase-6 + `ApplyEncounterResult` call and **0** at the phase-11 node-line-decay call, on every turn. Lane K read + that off the instruction stream against a decompile that reads as a conditional prune; it is now also a + behavioural fact. +* **Every encounter on these turns has `res->+0x4 != 0`** (`res_no_battle = 1`), the flag that makes + `ApplyEncounterResult` a whole-function no-op. Its measured cost is 0 words, which is what that gate + predicts, and which is why the combat resolver has never run under any instrument this campaign has built. + +--- + +## 5. Correction to `combat-done-tail.md` §7.1 — and to this lane's own prediction + +Lane K wrote that `S+0x8` "advances **at least twice** per turn". That is correct and this lane's prediction +that it advances **exactly** twice is **wrong**. Observed `S+0x8` at hook entry: + +| turn | `ProcessTurn` entry | tail entry | tail's callees | next turn's `ProcessTurn` entry | +|---|---|---|---|---| +| 3 | 22 | 23 | 24 | 34 | +| 4 | 34 | 35 | 36 | 48 | +| 5 | 48 | 49 | 50 | 60 | +| 6 | 60 | 61 | 62 | — | + +The two drivers account for 2 of the **12 to 14** increments per turn. Ten to twelve more happen between the +post-turn autosave and the next `ProcessTurn`, from a writer this lane did not identify (both drivers are +hooked, so it is neither of them). Lane K's operational conclusion is strengthened, not weakened: **`S+0x8` +must never be treated as a turn number.** `S+0xc` (`ModCount`) reads 3, 4, 5, 6 across the same four turns +and is the turn counter. + +## 6. Corrections to `combat-done-tail.md` §3 and §6.1 + +Both from the instruction stream, both load-bearing for anyone reimplementing these functions. + +**§3 — the node-line fleet check does not gate the roll.** Lane K: *"The roll is skipped for a line if any +fleet with flag `0x20000` is targeting it."* The straight-line order in node-line decay's first loop is + +``` +0x007ae088 call NodePath::RemainingLife ; expiry test +0x007ae08f jg 0x007ae1e2 ; not expired -> next record, NO DRAW +0x007ae095 mov ecx,[esi+0x16c] ; THE DRAW +0x007ae0a5 call 0x008e6dd0 ; Mars::RNG::Chance(0.5f) +0x007ae0aa test al,al ; je 0x007ae1e2 ; roll failed -> next record +0x007ae0b2 ... ; THE 0x20000-FLEET SCAN STARTS HERE +``` + +The scan begins 0x1d bytes **after** the `Chance` call and is reached only when the roll *succeeded*. It +suppresses the collapse (`0x007a92e0` / `0x007a4700`), never the draw. Lane K's headline — one `NextFloat` +per expired node line per turn — survives intact and is now pinned to a formula. + +**§6.1 — `StrategyHost::Autosave` is `ret 8` and returns a value.** Its epilogue is `c2 08 00`, and +`0x00895b5c mov eax,esi` puts the `std::string*` (the MSVC named-return slot) in EAX. A hook declaring it +`void` drops EAX at both call sites. Neither call site passes `this`: both hardcode `mov ecx,0xb29f98`. +(The `+0x54` candidate on that global was recorded live and is **not** the `StrategyServer` — `server_agrees` +is false on all eight autosave records, so the global that the autosave uses is a different object from the +`StrategyHost` whose `+0x54` `OnMessage` reads.) + +### 6.1 The expiry predicate, now concrete + +`NodePath::RemainingLife` 0x006e2130, `__thiscall(NodePath*, int turn)`, `ret 4`, whole 122-byte body read: + +```c +if (npt(+0x04) == 0) return INT_MAX; // permanent line, never expires +if (npdtn(+0x1c) == INT_MAX) return INT_MAX; // immortal line +aged = (npctm(+0x14) >= 0 && turn >= npctm) ? turn - npctm : 0; +wear = (npdtf(+0x20) != INT_MAX && npdtf > 0) ? nptf(+0x24) / npdtf : 0; // SIGNED idiv +rem = npdtn - wear - aged; +return rem > 0 ? rem : 0; // callee-side clamp +``` + +A line is expired exactly when this returns 0. **The lifetime is derived, never ticked** — the function +writes nothing, and neither does the loop around it — so there is no decrement-ordering question and a +snapshot at function entry is a valid prediction basis. `nptf` is never sign-checked, so the division's +signed truncation must be reproduced literally. + +`Chance` 0x008e6dd0 returns **false with no draw** when `p <= 0` and **true with no draw** when `p >= 1`; +`0.5f` takes neither, so it is exactly one word, and the comparison is `p > r` (equality returns false). +A NaN `p` falls through both early-outs and *does* draw — irrelevant here, noted because it is the kind of +edge a reimplementation gets wrong. + +--- + +## 7. One generator, confirmed twice + +Static: the image has one persistent strategic `Mars::RNG`, at `S+0x16c`, constructed by the +`StrategyServer` ctor 0x007d78d0 (`push 0x9cc` + `Seed(0)` at 0x007d7d25/0x007d7d3d) and reseeded only from +`Read` and `LoadGame`. `StrategyClient+0x134` and `Mars::CombatSim+0x108` exist but are unreachable from the +turn roots; three more are stack temporaries in map generation. **The combat resolver draws from the same +`S+0x16c` object** — all three RNG entry points in its 750-node direct-call closure load `[reg+0x16c]`. + +Behavioural: across 32 ledger observations in four turns, **every** state resolved on a single forward chain +— no `words: null`, no second chain. If a second generator had been in play, the ledger would have said so +by construction. + +--- + +## 8. What is not settled, listed as loudly as the results + +* **The combat resolver 0x007d5af0 (7499 B) has still never run under an instrument.** Every encounter this + workload produced had the no-battle flag set, so `ApplyEncounterResult` was a no-op every time. Its + measured 0 words says nothing whatever about combat's RNG cost. **The largest unmeasured term in the turn + is untouched**, and the residual-0 result above holds only for turns with no battle. +* **Node-line expiry did not fire.** See §9 for the quantified distance rather than an absence. +* **A turn with a genuinely empty encounter vector was not observed** (§3). Every turn of `ref-turn2` in + contact produces exactly one sighting encounter. The tail-runs-every-turn claim is settled; the + no-encounters variant is still an inference, now a much narrower one. +* **`players` reads 8 on a 2-player save.** `(S+0x54)` enumerated as a `ServerPlayer*` vector gives 8 on + every record of a game the lobby shows as 2 players. Either the server always allocates a fixed slot + count, or this offset enumerates something else (spare capacity is the failure this campaign has already + paid for once, at `S+0x64`). **Nothing in the ledger depends on it** — it is decoration on the argument + record — but it should not be reused until someone resolves it. +* **Which of the twelve-to-fourteen `S+0x8` increments per turn come from where** (§5). +* The direct-call sweeps behind "node-line decay's downstream pair draws nothing" and + "`ProcessNodeSpaceTravel` draws nothing" do not model indirect calls. Both are now **also** behavioural + facts on this workload (0 words, eight observations of `ProcessNodeSpaceTravel` and four of node-line + decay), which is the stronger evidence of the two. +* **The ledger's block-chain machinery has never run live.** Every observation in both runs sat inside a + single MT block — `left` walked 432 → 413 → 395 → 375 → 357 on `ref-turn2` and 263 → 243 → 223 → 201 → 181 + on the Zuul save, never reaching 0. So every live word count reduces to `left_before − left_after`, and the + twist-and-index path that makes the instrument correct across block boundaries is exercised **only by the + host tests**. A turn that crosses a boundary (any turn spending more words than `left`) is the first real + test of it. This is the thinnest part of the instrument and the one to watch. +* Everything here is one game state per save, two saves, eight turns, with **158 words** of generator + movement in total (192 → 267 and 361 → 443). It is a thin workload measured precisely, not a broad one. + In particular the per-turn total moved only between 18 and 22 across eight turns: the *variation* is barely + sampled, and nothing here says what makes it 18 rather than 22. + +## 9. Node-line expiry — a distance, not an absence + +Lane O's `zuul-turn16-noderoute.sav` was pushed to the VM and played forward. Phase 11 draws one word per +**expired** node line; rather than report "we ran N turns and it never fired", the hook was extended to +classify the whole `NodePath` population at entry, using the same `RemainingLife` formula the original +tests. The classification is what makes the negative result usable: + +| turn | node paths | permanent (`npt == 0`) | immortal (`npdtn == INT_MAX`) | **mortal** | min remaining life | ≤ 5 | expired → words | +|---|---|---|---|---|---|---|---| +| 17 | 53 | 51 | 0 | **2** | 40 | 0 | 0 | +| 18 | 54 | 51 | 0 | **3** | 42 | 0 | 0 | +| 19 | 56 | 51 | 0 | **5** | 41 | 0 | 0 | +| 20 | 57 | 51 | 0 | **6** | 43 | 0 | 0 | + +Three things follow, none of which was knowable before: + +1. **51 of the 53 node lines on this map can never expire** — `npt == 0` takes `RemainingLife`'s first + early-out. The static map's node network is not a decay candidate at all. Only *dug* lines are, which is + why this is a Zuul save: the mortal count rises by roughly one per turn as the Zuul dig. +2. **Every mortal line is ~40 turns from expiry**, and the population's minimum stays in a 40–43 band while + new lines are added at full life. So the first phase-11 draw on this save is **tens of turns away**, not + one or two — and it is reachable, which "we saw nothing" would not have told anyone. +3. It explains why the campaign never noticed: no save in the corpus is within 40 turns of a decay event, + and the ones that could get there are the newest saves in it. + +**Phase 11's draw therefore remains a path no save exercises — a hypothesis, and labelled one.** What *is* +now instruction-verified is the predicate that decides it (§6.1), and what is measured is the distance. + +### 9.1 The model is stated but not yet tested + +`predict_words` is computed at hook entry, before the original runs, and recorded on every node-line-decay +record. It read **0** on every call and the measured delta was **0** on every call. That agreement is worth +exactly nothing as a test of the model — it is the "0 diverged while comparing nothing" shape this campaign +has already paid for — and it is reported that way rather than as a green tick. + +Compare mode was **not** run on this hook for the same reason: with a prediction of 0 and a measurement of 0, +`ours` would advance the scratch generator by nothing, diff clean, and prove only that the harness works. +The model becomes checkable the first time a line expires, and the descriptor is ready for that day. + diff --git a/ghidra/addresses.d/lane-z.json b/ghidra/addresses.d/lane-z.json new file mode 100644 index 0000000..4894b81 --- /dev/null +++ b/ghidra/addresses.d/lane-z.json @@ -0,0 +1,84 @@ +{ + "entries": [ + { + "name": "StrategyServer_NodeLineDecay", + "addr": "0x007ae010", + "convention": "thiscall", + "prototype": "void (StrategyServer* this /*base S*/) // phase 11 of OnAllCombatDone_Tail, called at 0x007d9714 as `mov ecx,esi; call`. 1117 B, three loops. LOOP 1 (0x007ae07a..0x007ae1e2) walks the 0x30-stride Game::NodePath records in the vector at (*(S+0x154))+0x8/+0xc, re-reading _Myfirst/_Mylast every iteration, and per record: (1) NodePath::RemainingLife(r, S->Frame) 0x006e2130, `test eax,eax; jg` -> not expired, NEXT RECORD, NO DRAW; (2) THE DRAW, `mov ecx,[esi+0x16c]; fld [0x009e2ea0] /*0.5f*/; call 0x008e6dd0` = Mars::RNG::Chance(0.5f), exactly one MT word; (3) `test al,al; je` -> roll failed, next record; (4) the 0x20000-fleet scan over S->Fleets (S+0x64/+0x68) calling 0x00703500(fleet,0x20000,0) then 0x0078c360(fleet,npid) and dropping the record when that returns 3; (5) push_back npid into a scratch vector. LOOP 2 collapses each collected line via 0x007a92e0(690 B) then 0x007a4700(2244 B); LOOP 3 posts the decay-stage events through NodePath::DecayStage 0x006e21b0. THE ONLY RNG SITE IN THE WHOLE 1117 BYTES: direct-call sweep to depth 5 over 140 functions from 0x007ae010 against {NextFloat 0x0047d830, NextInt 0x004271c0, Chance 0x008e6dd0, Twist 0x00426e00, Seed 0x0049fdf0} yields exactly one hit, 0x007ae010 -> 0x008e6dd0. Neither downstream function draws (138 and 49 functions reached, zero hits) -- caveat: direct calls only, their subtrees contain unresolved indirect sites", + "status": "verified", + "source": "findings/control-flow/tail-rng-ledger.md (lane Z 2026-09-08, loop 1 read from the instruction stream)" + }, + { + "name": "StrategyServer_NodeLineDecay_FleetSkipIsPostDraw", + "addr": "0x007ae0b2", + "convention": "site", + "prototype": "site, and a CORRECTION to findings/control-flow/combat-done-tail.md §3, which says \"the roll is skipped for a line if any fleet with flag 0x20000 is targeting it\". IT IS NOT: the fleet scan begins HERE, at 0x007ae0b2, which is 0x1d bytes AFTER the Chance(0.5f) call at 0x007ae0a5 and is reached only when the roll SUCCEEDED (`test al,al; je 0x007ae1e2` at 0x007ae0aa). The scan therefore cannot change the draw count -- it suppresses only the collapse (the 0x007a92e0 / 0x007a4700 pair), never the draw. The straight-line order in loop 1 is: expiry test -> DRAW -> roll gate -> fleet gate -> collect. Lane K's headline claim, one NextFloat per expired node line per turn, survives intact and is now pinned to a concrete expiry formula (NodePath_RemainingLife)", + "status": "verified", + "source": "findings/control-flow/tail-rng-ledger.md (lane Z 2026-09-08)" + }, + { + "name": "NodePath_RemainingLife", + "addr": "0x006e2130", + "convention": "thiscall", + "prototype": "int (NodePath* this, int turn) // RET 4, whole 122-byte body read as instructions. THE EXPIRY PREDICATE for node-line decay. `if (npt(+0x4) == 0) return INT_MAX; if (npdtn(+0x1c) == INT_MAX) return INT_MAX; aged = (npctm(+0x14) >= 0 && turn >= npctm) ? turn - npctm : 0; wear = (npdtf(+0x20) != INT_MAX && npdtf > 0) ? nptf(+0x24) / npdtf : 0; /* SIGNED idiv, nptf is never sign-checked */ rem = npdtn - wear - aged; return rem > 0 ? rem : 0;`. Writes nothing -- the lifetime is DERIVED from a creation stamp and a traffic accumulator, never ticked, so there is no decrement-ordering question. Two never-expire escape hatches (npt == 0, npdtn == INT_MAX). A line is expired exactly when this returns 0", + "status": "verified", + "source": "findings/control-flow/tail-rng-ledger.md (lane Z 2026-09-08, whole body from the instruction stream)" + }, + { + "name": "NodePath_DecayStage", + "addr": "0x006e21b0", + "convention": "thiscall", + "prototype": "int (NodePath* this, int turn) // 52 B. Wraps NodePath::RemainingLife and buckets it: <=2 -> 0, <=5 -> 1, <=10 -> 2, else 3. Called TWICE per record by loop 3 of node-line decay (once with turn-1, once with turn) to detect a stage transition and post the two decay-stage events. Draw-free -- but note that instrumenting node-line decay by counting RemainingLife CALLS rather than Chance calls over-counts badly because of this wrapper", + "status": "verified", + "source": "findings/control-flow/tail-rng-ledger.md (lane Z 2026-09-08)" + }, + { + "name": "StrategyServer_ProcessNodeSpaceTravel", + "addr": "0x007a0e20", + "convention": "thiscall", + "prototype": "void (StrategyServer* this /*base S*/) // 2945 B. Called TWICE per turn: StrategyServer::ProcessTurn phase 7 (0x007dc93a) and OnAllCombatDone_Tail phase 10 (0x007d970b). Body unread; hooked by lane Z only to measure whether it advances the strategic generator, because a draw inside it would be double-counted by anyone who modelled it as running once", + "status": "mapped", + "source": "findings/control-flow/turn-driver.md phase 7 + combat-done-tail.md phase 10; call shapes instruction-verified by those lanes" + }, + { + "name": "StrategyServer_off_NodeGraph", + "offset": "0x00000150", + "convention": "offset", + "prototype": "Game::ServerNodeGraph* -- the node-line graph. Stored frame (S+4), so it is S+0x154 in the frame OnAllCombatDone_Tail and ProcessTurn receive; node-line decay reads it as `mov eax,[esi+0x154]` at 0x007ae03a with esi = S. Dereferenced without a null check by the original", + "status": "verified", + "source": "findings/control-flow/tail-rng-ledger.md (lane Z 2026-09-08)" + }, + { + "name": "ServerNodeGraph_off_Paths", + "offset": "0x00000008", + "convention": "offset", + "prototype": "std::vector (_Myfirst @+0x8, _Mylast @+0xc). Element stride 0x30, confirmed four ways: the reciprocal 0x2aaaaaab / sar 3 at four sites in node-line decay, `add [ebp-0x14],0x30` in its loop 1, `add [ebp-0x18],0x30` in its loop 3, and `add eax,0x30` in ServerNodeGraph::FindPathById 0x006e23d0. Loop 1 of node-line decay re-reads both words every iteration but writes neither, so an entry snapshot is a valid prediction basis", + "status": "verified", + "source": "findings/control-flow/tail-rng-ledger.md (lane Z 2026-09-08)" + }, + { + "name": "ServerNodeGraph_FindPathById", + "addr": "0x006e23d0", + "convention": "thiscall", + "prototype": "NodePath* (ServerNodeGraph* this, int npid) // 59 B, linear scan of the 0x30-stride paths vector comparing npid(+0x8). Node-line decay collects npid HANDLES rather than record pointers in loop 1 and re-resolves them here in loop 2, which is how the original hedges against the collapse functions mutating the vector under it", + "status": "verified", + "source": "findings/control-flow/tail-rng-ledger.md (lane Z 2026-09-08)" + }, + { + "name": "StarFleet_HasFlagShips", + "addr": "0x00703500", + "convention": "thiscall", + "prototype": "bool (StarFleet* this, uint maskA, uint maskB) // RET 8, a 29-byte thunk onto 0x00702d70(this, maskA, maskB, out = 0). Returns count > 0 where count is the number of ships in the fleet's NShips vector (+0xa4/+0xa8) passing the two-mask ship-flag predicate 0x00814da0, itself gated on (fleet->+0xb8 & maskA) == maskA. Called from node-line decay's post-draw fleet scan with maskA = 0x20000", + "status": "verified", + "source": "findings/control-flow/tail-rng-ledger.md (lane Z 2026-09-08)" + }, + { + "name": "StarFleet_PathRelation", + "addr": "0x0078c360", + "convention": "cdecl", + "prototype": "int (StarFleet* fleet, int npid) // 234 B. Returns 3 exactly when the fleet's FRONT waypoint (the deque at fleet+0xc4, element +0x10) names this npid -- i.e. the fleet is currently riding this line; 0/1/2/4 otherwise. Node-line decay drops a rolled line when any 0x20000-flagged fleet returns 3 for it", + "status": "verified", + "source": "findings/control-flow/tail-rng-ledger.md (lane Z 2026-09-08)" + } + ] +} diff --git a/tools/rng_ledger_report.py b/tools/rng_ledger_report.py new file mode 100755 index 0000000..3c75b2f --- /dev/null +++ b/tools/rng_ledger_report.py @@ -0,0 +1,92 @@ +#!/usr/bin/env python3 +"""Per-turn RNG ledger from a lane-Z trace (docs: findings/control-flow/tail-rng-ledger.md). + +The lane-Z hooks declare the strategic Mars::RNG as a region whose `describe` reports an ABSOLUTE +WORD POSITION (see sots-engine/src/shim/hooks/rng_ledger.h). This tool turns those positions into +the three numbers a reimplementation needs: + + * how many 32-bit words the generator consumed inside each hooked call; + * the same, attributed by phase, using the fact that the hooks nest; + * the bracket total between the pre-turn autosave and the post-turn autosave -- the interval a + standalone has to reproduce -- and the RESIDUAL left over after the attributed subtotals. + +A residual > 0 is a draw site outside every hooked function. A `words: null` is a position the +ledger could not place and MUST NOT be read as zero. + + uv run python3 tools/rng_ledger_report.py +""" +import json, sys + +def val(tv): + if tv is None: return None + return tv.get('v') + +def sv(struct_tv, key): + if struct_tv is None or struct_tv.get('t') != 'struct': return None + return val(struct_tv['v'].get(key)) + +rows = [] +for i, line in enumerate(open(sys.argv[1])): + d = json.loads(line) + if 'meta' in d and 'hook' not in d: continue + if 'hook' not in d: continue + args = {a.get('n'): val(a) for a in d.get('args', []) if a.get('n')} + side = d.get('side', {}) + rng = side.get('rng', {}) + b, a2 = rng.get('before'), rng.get('after') + wb, wa = sv(b, 'words'), sv(a2, 'words') + rows.append(dict(cid=d['call_id'], depth=d.get('depth'), hook=d['hook'].split('::')[-1], + turn=args.get('turn'), pc=args.get('phase_counter'), + enc=args.get('encounters'), end_turn=args.get('end_turn'), + paths=args.get('node_paths'), + perm=args.get('np_permanent'), imm=args.get('np_immortal'), + mortal=args.get('np_mortal'), minlife=args.get('np_min_life'), + within5=args.get('np_within5'), res_nb=args.get('res_no_battle'), + predict=args.get('predict_words', args.get('predict_nodeline_words')), + wb=wb, wa=wa, lb=sv(b,'left'), la=sv(a2,'left'), + bb=sv(b,'block'), ba=sv(a2,'block'), + words=(wa - wb) if (wa is not None and wb is not None) else None, + err=d.get('err'), undecl=d.get('undeclared_total'))) + +hdr = f"{'cid':>4} {'d':>1} {'hook':<28} {'turn':>4} {'S+8':>4} {'enc':>4} {'paths':>5} {'pred':>4} {'w_in':>8} {'w_out':>8} {'WORDS':>6} {'left':>10}" +print(hdr); print('-'*len(hdr)) +for r in rows: + print(f"{r['cid']:>4} {r['depth']:>1} {r['hook']:<28} {str(r['turn']):>4} {str(r['pc']):>4} " + f"{str(r['enc']):>4} {str(r['paths']):>5} {str(r['predict']):>4} {str(r['wb']):>8} " + f"{str(r['wa']):>8} {str(r['words']):>6} {str(r['lb'])+'->'+str(r['la']):>10}" + + (f" ERR={r['err']}" if r['err'] else '')) + +# Brackets: Autosave(end_turn=1) .. Autosave(end_turn=0) +nl = [r for r in rows if r['hook'] == 'NodeLineDecay'] +if nl: + print() + print("node-line population (phase 11's draw is one word per EXPIRED line):") + print(f"{'turn':>5} {'paths':>6} {'permanent':>10} {'immortal':>9} {'mortal':>7} {'min_life':>9} {'<=5':>4} {'expired':>8} {'words':>6}") + for r in nl: + print(f"{str(r['turn']):>5} {str(r['paths']):>6} {str(r['perm']):>10} {str(r['imm']):>9} " + f"{str(r['mortal']):>7} {str(r['minlife']):>9} {str(r['within5']):>4} " + f"{str(r['predict']):>8} {str(r['words']):>6}") + +enc = [r for r in rows if r['hook'] == 'OnAllCombatDone_Tail'] +if enc: + print() + print("tail invocations (P1: does it run on every End Turn?):") + for r in enc: + print(f" turn {r['turn']}: encounters={r['enc']} words={r['words']}") + +print() +marks = [r for r in rows if r['hook'] == 'Autosave'] +for k in range(len(marks) - 1): + lo, hi = marks[k], marks[k+1] + if not (lo['end_turn'] is True and hi['end_turn'] is False): continue + if lo['wb'] is None or hi['wb'] is None: + print(f"bracket {k}: INCOMPLETE (pre-turn marker has no ledger position)"); continue + total = hi['wb'] - lo['wb'] + inner = [r for r in rows if lo['cid'] < r['cid'] < hi['cid'] and r['depth'] == 0 and r['words'] is not None] + acc = sum(r['words'] for r in inner) + # The autosave hook's `this` is a StrategyHost, so its record carries no turn; take the turn + # from the drivers the bracket encloses. + turn = next((r['turn'] for r in inner if r['turn'] is not None), None) + print(f"BRACKET turn {turn}: total={total} words attributed={acc} residual={total-acc}") + for r in inner: + print(f" {r['hook']:<30} {r['words']:>6}") diff --git a/verify/results/shim/tailrng/save_rng_delta.py b/verify/results/shim/tailrng/save_rng_delta.py new file mode 100644 index 0000000..2fd8abc --- /dev/null +++ b/verify/results/shim/tailrng/save_rng_delta.py @@ -0,0 +1,63 @@ +"""Independent check of the live ledger: pull the RNG blob out of two save files and +compute the word delta from the FILES, with no reference to the shim's numbers.""" +import sys, os +sys.path.insert(0, os.path.expanduser('~/sots-re/verify/save-reader')) +import save_reader as sr + +N, M = 624, 397 +def twist(mt): + mt = list(mt) + for kk in range(N - M): + y = (mt[kk] & 0x80000000) | (mt[kk+1] & 0x7fffffff) + mt[kk] = mt[kk+M] ^ (y >> 1) ^ (0x9908b0df if y & 1 else 0) + for kk in range(N - M, N - 1): + y = (mt[kk] & 0x80000000) | (mt[kk+1] & 0x7fffffff) + mt[kk] = mt[kk+(M-N)] ^ (y >> 1) ^ (0x9908b0df if y & 1 else 0) + y = (mt[N-1] & 0x80000000) | (mt[0] & 0x7fffffff) + mt[N-1] = mt[M-1] ^ (y >> 1) ^ (0x9908b0df if y & 1 else 0) + return mt + +def find_rng(node, out): + name = getattr(node, 'name', None) + if name == 'RNG': + out.append(node) + for c in getattr(node, 'children', []) or []: + find_rng(c, out) + +def rng_of(path): + res = sr.read_save(path) + hits = [] + find_rng(res.tree, hits) + if not hits: + raise SystemExit(f"no RNG frame in {path}") + n = hits[0] + raw = n.raw + if not raw and n.children: + raw = b"".join(c.raw for c in n.children) + if not raw: + raise SystemExit(f"RNG frame in {path} carries no bytes") + return raw + +def parse(raw): + # the blob is mt[624] then left, little-endian; tolerate a leading/trailing frame byte + for off in range(0, len(raw) - 2500 + 1): + if len(raw) - off < 2500: continue + mt = [int.from_bytes(raw[off+4*i:off+4*i+4], 'little') for i in range(N)] + left = int.from_bytes(raw[off+2496:off+2500], 'little', signed=True) + if 0 <= left <= N: + return mt, left, off, len(raw) + raise SystemExit(f"cannot parse RNG blob of {len(raw)} bytes") + +a, b = sys.argv[1], sys.argv[2] +ra, rb = rng_of(a), rng_of(b) +ma, la, oa, na = parse(ra) +mb, lb, ob, nb = parse(rb) +print(f"{os.path.basename(a)}: blob {na} B, offset {oa}, left={la}") +print(f"{os.path.basename(b)}: blob {nb} B, offset {ob}, left={lb}") +cur, tw = ma, 0 +while tw <= 64 and cur != mb: + cur = twist(cur); tw += 1 +if cur != mb: + raise SystemExit("second block is not on the first block's chain within 64 twists") +words = 624*tw + (la - lb) +print(f"twists={tw} words consumed between the two files = {words}") diff --git a/verify/results/shim/tailrng/z-t5-autosave.sav b/verify/results/shim/tailrng/z-t5-autosave.sav new file mode 100644 index 0000000..070b8e9 Binary files /dev/null and b/verify/results/shim/tailrng/z-t5-autosave.sav differ diff --git a/verify/results/shim/tailrng/z-t6-autosave.sav b/verify/results/shim/tailrng/z-t6-autosave.sav new file mode 100644 index 0000000..dc68aed Binary files /dev/null and b/verify/results/shim/tailrng/z-t6-autosave.sav differ diff --git a/verify/results/shim/tailrng/z-t6-endturn.sav b/verify/results/shim/tailrng/z-t6-endturn.sav new file mode 100644 index 0000000..8b51df9 Binary files /dev/null and b/verify/results/shim/tailrng/z-t6-endturn.sav differ diff --git a/verify/traces/tailrng-refturn2.jsonl.gz b/verify/traces/tailrng-refturn2.jsonl.gz new file mode 100644 index 0000000..3f826c4 Binary files /dev/null and b/verify/traces/tailrng-refturn2.jsonl.gz differ diff --git a/verify/traces/tailrng-zuul-noderoute.jsonl.gz b/verify/traces/tailrng-zuul-noderoute.jsonl.gz new file mode 100644 index 0000000..0558221 Binary files /dev/null and b/verify/traces/tailrng-zuul-noderoute.jsonl.gz differ