sots-re/findings/control-flow/alliance-mask-and-modcount.md
alex 728348b4a0 lane A2: the alliance mask read from the bytes, and every ModCount writer
Spine phase 4, byte for byte at 0x007dc871-0x007dc8c7. Three separate stores to
the same word -- clear, OR the self bit, then conditionally OR the alliance
mask -- which is what makes the alliance term an OR and is invisible in any
summary of the phase. The bit is the player's POSITION IN THE VECTOR.

Checked against the almem bytes the game archived: 72 of 80 player-records
agree, and the other 8 are predicted by the same model (FinalizeTurnRecords also
runs on load, and the load path does not run the spine, so every save's earliest
archived turn carries a zero mask -- true on all 11).

Reported as loudly: the corpus cannot separate `1 << vectorIndex` from
`1 << PlyrIdx` (0 of 80 records differ), cannot separate the OR from a plain
assignment (every observed alliance mask already contains its member's own bit),
and cannot separate the ALid guard from an AL != 0 guard. Those three are
instruction-stream readings only.

ModCount, named and enumerated. StrategyServer::Write tags both words itself, so
S+0x8 is the wire's ModCount and S+0xc is Frame -- addresses.json has the name
on the wrong word, and turn-driver.md's "they stay in lockstep" is corrected in
place. RTTI gives the reason for the two bases: Game::StrategySim is a base
sub-object at offset 4.

29 writer sites: 20 command handlers plus 6 inlined in the command-batch applier
(each an unconditional bump on ENTRY, before validation), the two turn drivers,
and the abandon/chaos check -- which is gated on Abdn, false on all 28 systems
of all 11 saves. So the per-turn delta is 2 + one per command applied out of
every player's TurnCommands block; the 0x1b4 container stride independently
confirms lane Q's block layout. It is not derivable from the pre-turn save.

The residual is a watchpoint, specified with its own written prediction (exactly
12 hits on turn1-state) and four falsifiers. Direct-call reachability is stated
as the lower bound it is, per lane V2's indirect-edge measurement.

Engine side: sots-engine wip/alliance 5e409cf.
2026-09-08 12:44:30 -04:00

354 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# The alliance mask (spine phase 4), and every writer of `ModCount`
Lane A2, 2026-09-08. Static + host, no VM held. Engine worktree `wip/alliance` off `main`
`b2bad30`; the prediction is `sots-engine/docs/A2-alliance-and-modcount.md`, committed
(`49ae628`) before any implementation. Program `sots` / "Sword of the Stars.exe",
ImageBase 0x00400000, all addresses VAs.
**Method.** Every control-flow and offset claim below was read from the instruction stream —
`objdump -b binary -m i386 -M intel` over the raw image, file offsets from the PE section table.
The decompiler was not used at all in this lane. Reachability came from a whole-image direct-call
edge sweep built with `tools/x86disp.py`'s decoder, sweeping each body **to the next function
start** (rule 17).
---
## 0. The two results, up front
1. **The alliance mask is closed.** `almem == (1 << vectorIndex) | (ALid != -1 ? AL : 0)`, and it
agrees with the bytes the game wrote on **72 of 80 player-records**; the other 8 are
*predicted* by the same model. `S04` is implemented and `T36`'s regression on the reference
pair drops from **17 to 9**.
2. **`ModCount` is a command counter, and it is not derivable from the pre-turn save.** It is
`StrategySim+0x4` (= `S+0x8`), bumped once on entry to each of **26 command-application
sites**, once by each turn driver, and once per abandoned-system check. The per-turn delta is
the number of commands applied, and the AI's commands are generated inside the turn. Static
reading has taken this as far as it goes; the residual is a **watchpoint**, specified in §3.
---
## 1. Spine phase 4 — the alliance / shared-vision mask
### 1.1 The bytes
`StrategyServer::ProcessTurn` 0x007dc871–0x007dc8c7, byte for byte:
```
007dc871 mov edx,[esi+0x58] ; Players._Mylast (esi = S)
007dc874 sub edx,[esi+0x54] ; - Players._Myfirst
007dc877 xor ecx,ecx ; i = 0
007dc879 test edx,0xfffffffc
007dc87f jle 0x7dc8c8 ; empty vector -> skip the whole phase
007dc881 mov eax,[esi+0x54]
007dc884 mov eax,[eax+ecx*4] ; p = Players[i]
007dc887 mov edx,[eax+0x3d8] ; rec = p->turnRecord
007dc88d mov edi,0x1
007dc892 shl edi,cl ; bit = 1 << i <-- the LOOP COUNTER
007dc894 mov [edx+0x8],ebx ; rec->almem = 0 (ebx == 0)
007dc897 mov edx,[eax+0x3d8] ; ... reloaded
007dc89d or [edx+0x8],edi ; rec->almem |= bit
007dc8a0 cmp DWORD PTR [eax+0x168],0xffffffff ; ALid == -1 ?
007dc8a7 je 0x7dc8b8
007dc8a9 mov edx,[eax+0x3d8] ; ... reloaded again
007dc8af mov eax,[eax+0x16c] ; AL
007dc8b5 or [edx+0x8],eax ; rec->almem |= AL
007dc8b8 mov edx,[esi+0x58]
007dc8bb sub edx,[esi+0x54]
007dc8be inc ecx
007dc8bf sar edx,0x2
007dc8c2 xor ebx,ebx
007dc8c4 cmp ecx,edx
007dc8c6 jl 0x7dc881
```
So:
```c
for (int i = 0; i < numPlayers; ++i) {
ServerPlayer* p = Players[i];
TurnRecord* rec = p->turnRecord; // +0x3d8
rec->almem = 0; // +0x8
rec->almem |= 1 << i;
if (p->ALid != -1) rec->almem |= p->AL; // +0x168, +0x16c
}
```
Three separate stores to the same word. That is what makes it an **OR** rather than an
assignment, and it is not visible in any summary of the phase. The record pointer is reloaded
from `p->+0x3d8` before each store.
**Lane T's reading of this phase (`turn-driver.md` §1 row 4) is correct in every particular**,
including the load-bearing one: the bit index is the player's **position in the server's player
vector**, not `PlyrIdx`. Lane T's `ServerPlayer_off_TurnRecord`, `_off_AllianceId` and
`_off_AllianceMask` already carry the rule; this lane's duplicate address entries were dropped
and the agreement recorded instead. The only thing added here is the byte-level range (lane T's
table gives it as `0x007dc8c8–0x007dc8c6`, which is a transcription slip — the phase runs
0x007dc871–0x007dc8c7 and 0x007dc8c8 is where phase 5 begins).
### 1.2 The output is on the wire, and it agrees
`p->+0x3d8` is the per-player turn record that tail phase 36 archives into
`/Sim/turnstats/nply[]/hist/stats[]`; `almem` is the archive element's second int. So the phase
is checkable against bytes the original wrote, with no game.
Over the eleven-save corpus, one element per player per save at the save's own frame:
| | records | result |
|---|---:|---|
| `almem == (1 << i) \| (ALid != -1 ? AL : 0)` | **72** | agree |
| `almem == 0` on `turn1-state.sav` | **8** | **also predicted** — see below |
| disagree | **0** | |
`sots-engine/tests/app/test_turn_record.cpp` now compares **7** fields, not 6:
**11 saves, 80 player-records, 560 fields, 0 mismatches.**
### 1.3 The eight zeros are the model, not an exception
`turn1-state.sav` is the game's first frame and every one of its eight `almem` values is 0 while
the rule says 1, 2, 12, 12, 16, 32, 64, 128. That is falsifier **F5** of the written prediction,
and it fires exactly as the prediction said it would:
`FinalizeTurnRecords` 0x0078a0e0 is called from **two** places — the tail (0x007d98ba) and
`LoadGame` (0x007ddc40) — and `almem` is written **only** by spine phase 4. A record archived by
the load / new-game path therefore carries the turn record's initial zero. The corpus confirms
this on every save independently: the archive is cumulative, and in **all eleven** saves the
`trn == 1` element is `almem == 0` for every player while every `trn >= 2` element matches the
rule. `zuul-turn23-fleet23.sav` carries 23 elements per player and shows the same shape.
The engine expresses this as a positive claim rather than a skipped field: `BuildTurnRecord`
takes a `spineRan` flag, the caller sets it from `frame > EarliestArchivedTurn(hist)`, and the
model **predicts zero** for the load-written element. All 80 records are compared.
### 1.4 What the corpus does NOT separate — reported as loudly as the agreement
Three parts of the rule are instruction-stream readings that **no comparison against these saves
can test**, and the run prints the fact every time:
| claim | why the corpus cannot separate it |
|---|---|
| the bit is `1 << vectorIndex`, **not** `1 << PlyrIdx` | every player in every save has `PlyrIdx == i`. **0 of 80 records separate them.** |
| the alliance term is an **OR** with the self bit | every observed alliance mask already contains its own member's bit (`AL == 12` on players 2 and 3, `AL == 4` on player 2), so `self \| AL == AL` there |
| the guard is `ALid != -1`, **not** `AL != 0` | in the corpus `AL == 0` exactly when `ALid == -1`, so the two guards never disagree |
14 of 80 records do carry a live alliance id, so the alliance term is **exercised** — falsifier F7
did not bite — but it is exercised only in the shape where the three readings above coincide.
`sots-engine/tests/app/test_alliance.cpp` pins all three as unit cases with the separating inputs
the corpus lacks, so a later save that does separate them has something to disagree with.
One more untested branch, reproduced rather than corrected: `shl edi,cl` masks the count to five
bits, so a 32nd player would set bit 0. No save has more than 8 players.
---
## 2. `ModCount` — every writer
### 2.1 It is `S+0x8`, and `addresses.json` has the name on the wrong word
`StrategyServer::Write` 0x0079fa70 tags its own members. With `edi = this`:
```
0079fb2f lea edx,[edi+0x08] ; push "ModCount" (0x00a23844)
0079fb40 lea eax,[edi+0x0c] ; push "Frame" (0x00a2383c)
0079fb4f mov eax,[edi+0x14] ; push "GameID" (0x00a238f0)
0079fb90 mov eax,[edi+0x16c] ; push "RNG" (0x00a23828)
```
The last line settles the frame: `StrategyServer_off_RNGPtr` is `0x168` in the **S+4** frame, so
`[edi+0x16c]` means `edi = S`. Therefore
* **`S+0x8` is the wire's `ModCount`** — the word `StrategyServer::ProcessTurn`'s first
instruction bumps;
* **`S+0xc` is the wire's `Frame`** — the word `BeginProcessTurn` bumps, that `TechTree::SetResearched`
stamps as `turnResearched`, and that `FinalizeTurnRecords` uses as the archive key.
**Corrections that follow, stated plainly (rule 11):**
1. `ghidra/addresses.json`'s `StrategyServer_off_ModCount` (offset `0x8`, S+4 frame, i.e. `S+0xc`)
carries the name of the *other* word. It is `Frame`. **Flagged for the integrator rather than
edited here**, because `addresses.json` is the shared file the per-lane fragments exist to keep
out of. `ghidra/addresses.d/lane-a2.json` adds `StrategySim_off_ModCount` (0x4) and
`StrategySim_off_Frame` (0x8) with the correction in their prototypes.
2. `turn-driver.md` §0.1's "the word at `S+0x8` has never been named" and "both advance once per
turn … so they stay in lockstep" are wrong on both counts. Lane K corrected the lockstep half
in `combat-done-tail.md` §7.1 and lane Z named the word there. This lane re-derives the name
independently from the serializer and confirms it. §0.1 is corrected in place.
3. RTTI gives the reason the two bases exist: `Game::StrategyServer` has bases
`Mars::IStreamable` at offset 0 and **`Game::StrategySim` at offset 4** (vftables 0x00a26084
and 0x00a26034). The "RAW base" every `StrategyServer_off_*` is expressed in is the
`StrategySim` sub-object. `ModCount` is `StrategySim+0x4`.
### 2.2 The writers: a command counter
A whole-image sweep for `inc dword [reg+4]` and `inc dword [reg+8]` at real instruction
boundaries, with the base proved to be a `StrategySim` by an independent fingerprint — the same
register feeding `lea ecx,[base+0x80]; call 0x008b9240`, the entity-hash lookup that
`addresses.json` already places at `StrategySim+0x80`.
**Twenty command handlers, each bumping once on entry:**
| VA | the command, from its own log string |
|---|---|
| 0x00821a80 | set research rate |
| 0x00821b40 | boost research |
| 0x00821b90 | surrender |
| 0x00821c30 | abandon system |
| 0x00821cf0 | set fleet auto-support |
| 0x00821d70 | set fleet to guard |
| 0x00821e20 | set fleet to raid |
| 0x0083d5d0 | remove design |
| 0x0083d6e0 | set research project |
| 0x00842f40 | cancel build order |
| 0x00849460 | (no log string; same shape) |
| 0x00849520 | set ship action |
| 0x00849570 | set fleet name |
| 0x0085b6d0 | ship requesting support |
| 0x00865780 | **move fleet** ("StrategySim: … cannot move fleet") |
| 0x00865910 | set fleet layout |
| 0x0086c3e0 | set system rates |
| 0x008784e0 | set player notes |
| 0x00882910 | create design |
| 0x0088bed0 | transfer ships |
Plus **six more inlined** inside `StrategySim::ApplyTurnCommandBatch` 0x0088f9b0 (0x0088fe0a,
0x008902fe, 0x008903b9, 0x0089046c, 0x008905c8, 0x008907bc) — cancel project, set civilian
ratios, set performance mods, set weapon groups, and two more its strings name.
**The bump is unconditional and precedes validation.** In every one of the twenty it is the
second or third instruction of the body, *before* the handle lookup that decides whether the
command is even applicable. A command naming a player that does not exist still advances
`ModCount`.
**The shape of a turn**, from `StrategyNetworkClient::OnMessage` 0x00784640 — the single direct
caller of all three:
```
0x00784904 StrategyServer::ApplyAllTurnCommands(S) ; -> ApplyTurnCommandBatch(S+4, blocks, nPlayers)
0x0078491c StrategyServer::ProcessTurn(S) ; +1
0x00784d07 StrategyServer::OnAllCombatDone_Tail(S) ; +1
```
`ApplyAllTurnCommands` 0x0078f6a0 computes `count = (S->+0x178 − S->+0x174) / 0x1b4` — signed
magic `0x964fda6d`, `sar 8`, the reciprocal of **436** — and hands the run to the batch applier.
**0x1b4 = sizeof(`Game::TurnCommands`)**, which independently confirms lane Q's field walk
(27 `std::list` members at 0x70…0x1a8, stride 0xc, so the class ends at 0x1b4). So `ModCount`'s
per-turn delta is:
> **2 (the two drivers) + one per command applied out of every player's `TurnCommands` block**
> (+ one per abandoned system, §2.3).
That is exactly what "12 to 44 times a turn" looks like, and it is why the delta is not a
function of the board: `human-turn2→turn3` is 28 and `zuul-turn16→turn17` is 16 on boards with
**identical** system and player counts (28 and 7).
### 2.3 The one writer inside the turn drivers, and it is gated shut
`FUN_007b9df0` — the abandon/chaos check that spine **phase 1** calls once per system — bumps
`ModCount` as its fourth instruction (`inc [esi+8]` @0x007b9e20, `esi = ecx = S`, entered from
0x007dc7fd `push edi; mov ecx,esi; call 0x7b9df0`).
The phase-1 loop gate is `cmp BYTE PTR [edi+0xc4],0; je <next>` and `ServerSystem+0xc4` is
**`Abdn`**. `Abdn` is **false on all 28 systems of all 11 saves**, so this writer contributes
**0** on every measured turn. Per rule 6 that makes its per-turn cost a hypothesis, not a
measurement — but it also means it is not the missing term.
A direct-call reachability sweep from each driver over the whole image:
| root | closure | `ModCount` writers reachable |
|---|---:|---|
| `StrategyServer::ProcessTurn` 0x007dc6c0 | 1382 | `FUN_007b9df0` only |
| `OnAllCombatDone_Tail` 0x007d92a0 | 1369 | `FUN_007b9df0`, `MoveFleetCommand` 0x00865780 |
| `ServerPlayer::ProcessTurn` 0x00891340 | 272 | `FUN_007b9df0` only |
**This is a lower bound and is labelled as one.** Lane V2 measured that 5,045 of 5,207
vtable-named functions have zero direct call sites, and that sweeping for jumps into another
function's start yields 14,958 further edges. A writer reached only through a vtable slot or a
tail-call thunk is invisible to the table above — which is precisely the failure mode rule 18 was
written about.
### 2.4 Three sites examined and ruled out, named rather than dropped
| site | why it is not `ModCount` |
|---|---|
| `StrategyNetworkClient::OnMessage` 0x007850d5 / 0x0078514b / 0x00785224 | its `this` is not a `StrategySim`: the class holds the server at `+0x54` (`mov ecx,[esi+0x54]` before both driver calls) and its own entity hash at `+0x84`, not `+0x80`. `[this+4]` is a client-side word. **Unresolved, not counted.** |
| `FUN_007b10c0` 0x007b1304 | `inc [eax+4]` with `eax` from a stack slot; the body's fingerprint is 0x74-stride team records and `[esi+0x250/0x254]`, not the `StrategySim` layout. **Unresolved.** |
| `FUN_00890d50` 0x00890f61 | `eax = this->+0x1e8 − 0x14; if (eax) inc [eax+8]`. The −0x14 adjust is not the +4 that reaches a `StrategySim`. **Unresolved.** |
### 2.5 The corpus numbers
| pair | `ModCount` | Δ | note |
|---|---|---:|---|
| `turn1-state` → `turn2-state` | 0 → 12 | **12** | no queued commands in either block |
| `turn2-state` → `turn3-state` | 12 → 24 | **12** | ditto |
| `human-turn2-orders` → `human-turn3-noderoute` | 25 → 53 | **28** | 20 owned systems |
| `zuul-turn15-orders` → `zuul-turn16` | 210 → 241 | **31** | |
| `zuul-turn16` → `zuul-turn17` | 241 → 257 | **16** | |
| `zuul-turn17` → `zuul-turn23` | 257 → 412 | 155 / 6 turns | |
The two 12s look like a state function and are the strongest evidence *against* the reading
above; they are also both turns on which the human issued nothing, so the 10 non-driver bumps are
the AI's commands on a two-colony board, and a stable board plausibly produces a stable command
count. The `TurnCommands_v5` block in each of these saves is the **empty** 35-item block
(lane Q §1c), so the commands that produced the 12 were issued *after* the autosave that wrote
the input file — which is exactly why the pre-turn save cannot predict them.
---
## 3. What needs the VM — the watchpoint, specified (rule 18)
`ModCount` is a single 4-byte counter with an unknown writer set. That is the textbook shape for
a hardware watchpoint, and it settles in one turn what static reading has spent this lane
bounding.
**Probe 1 — write watchpoint on `ModCount`.**
* **Address.** `ModCount = (char*)S + 8`, where `S` is `StrategyServer::ProcessTurn`'s `this`.
Equivalently `*(void**)(ServerPlayer + 0x8) + 4`, or `*(void**)(ServerSystem + 0x10) + 4`.
Read it once at a breakpoint on 0x007dc6c0 and set the watchpoint from there; do **not** try to
compute it from a global, and do **not** use `S+0xc` — that is `Frame`, and getting the two
the wrong way round is exactly the error this document corrects.
* **Type.** DR0 as a 4-byte data-write watchpoint. One debug register suffices.
* **Record per hit.** EIP, the two return addresses up the stack, the value written, and a
monotonic hit index.
* **Workload.** Load `turn1-state.sav`, press End Turn once, wait for the post-turn autosave.
* **Prediction, written before the run:** **exactly 12 hits.** Two of them at 0x007dc6f0 and
0x007d92ca. Zero of them at 0x007b9e20 (`Abdn` is false everywhere). The other **10** inside
the twenty handlers of §2.2 or the six inlined sites of `ApplyTurnCommandBatch`, all with a
return address in `StrategySim::ApplyTurnCommandBatch` 0x0088f9b0 or its callers, and all
**before** `ProcessTurn` is entered.
* **Falsifiers.** (a) more or fewer than 12 hits ⇒ something writes `ModCount` that is not an
increment (check `StrategyServer::Read` on load first); (b) a hit whose EIP is not in the 29
enumerated sites ⇒ this static sweep missed a writer, which is the result worth having, and
lane V2's indirect-edge map is where to look next; (c) hits *after* `ProcessTurn` is entered ⇒
commands are applied during the turn, not before it, and the "the queue is flushed first"
reading is wrong; (d) hits at 0x007850d5 / 0x0078514b / 0x00785224 ⇒ §2.4's exclusion of
`OnMessage` is wrong and `StrategyNetworkClient` shares the base.
**Probe 2 — cheaper, and it answers the modelling question rather than the enumeration one.**
Break on `StrategyServer::ApplyAllTurnCommands` 0x0078f6a0 entry and log, for each of the `count`
blocks at `S->+0x174`, the six prologue gate bools and the 27 list sizes. That is the exact term
`ModCount` counts, in the exact layout lane Q recovered, and it says directly whether the
standalone could ever produce the number. **If probe 2 shows the AI's blocks are non-empty at
this point, `ModCount` is downstream of the whole AI and is the wrong leaf to chase before it;
if they are empty, the bumps come from somewhere else entirely and probe 1's hit list names it.**
Both probes are minutes of work for whoever holds VM140 next. Neither is worth another lane of
reading.
---
## 4. What this lane did not do
* **Nothing was compared against the live game.** The alliance rule is checked against *bytes the
original wrote*, which is stronger than nothing and weaker than a live compare, and the phase
catalog's `verified` count stays 0.
* **The alliance rule's three unseparated readings** (§1.4) rest on the instruction stream alone.
A save with a player whose `PlyrIdx` differs from its vector position would settle the first
one; none exists, and one could be **manufactured** (rule 6) by editing `PlyrIdx` on a save,
which is a named wire field.
* **The `ModCount` writer set is a lower bound.** It is complete for *direct* `inc [this+4]` /
`inc [this+8]` forms with a proved `StrategySim` base. A writer that stores rather than
increments, or that reaches the counter through a pointer this sweep could not type, is not
excluded. §2.4 lists the three it could not type.
* **The abandon/chaos writer has never fired** on any save in the corpus. Its per-turn cost is a
hypothesis.
* No shim TU was touched, so no CT111 cross-build was required.