sots-engine/docs/B2.md

# B2 — `ServerPlayer::OnTechResearched` old-vs-new, with the player's fields as named regions

**Result (2026-09-08): verified on the live game.** Trace: 2 completions captured, `tracecmp.py`
exit 0, 0 invalid. Compare: **3 completions compared, 0 divergences, 0 errors.** Replace: one
End Turn from `ref-turn2.sav` reproduces the determinism oracle byte for byte
(`(Autosave).sav` = `978041acd168b56e…`, `(Autosave EndTurn).sav` = `bb4fd9ac89f41e3b…`) — a weak
check by construction, because no tech completes on that turn so the hook never fires; what it
proves is that the hook installs and perturbs nothing. Game left on VM140 at the main menu in
`hooks=trace`.

Build `b2-646e4e8-dirty-20260908T0434Z`, staged in `/srv/re-lab/shim/dist-b2`, deployed from
`C:\SOTS\shimdist-b2`. Evidence in `sots-re/verify/traces/b2-*` and
`verify/results/compare/b2-techfx-golden.json`.

All three runs are archived: `sots-re/verify/traces/b2-techfx-golden.jsonl`,
`b2-techfx-compare.jsonl`, `b2-shim.log`, `b2-final-menu.png`, with the `tracecmp.py` reports in
`verify/results/compare/b2-techfx-{golden,compare}.json`.

The point of the target: it is the single place where 36 hard-coded strategic effects, the
per-species xenotech flag words, the two design-option masks and the node-bore parameters
are all written. One call validates the whole `src/game/effects` layer at once.

## What was hooked

| hook name (record `hook`) | RVA | prototype |
|---|---|---|
| `Game::ServerPlayer::OnTechResearched` | 0x00491790 | `void (ServerPlayer*, TechDef* def, bool silent)` |

`__thiscall`, `[verified]`, vft slot 4, so it goes through `Hook<>` with
`CallConv::Thiscall`. Source: `src/shim/hooks/tech_effects.{h,cpp}` (the descriptor and the
delegation into the game's tech tree) over `src/shim/hooks/tech_effect_fields.{h,cpp}` (the
byte-level field adapter, host-tested — the split B1 made for `budget_inputs`). Installed
from `src/shim/main.cpp` after the B3 hook.

`TechTree::SetResearched` is the only caller. It fires **once per tech completion**, which
on a typical turn is zero times — see "Coverage" below, because that is the hard part of
this milestone.

### Region model

Fourteen regions, one per field group the callback writes, each with a struct describer, so
a divergence reads `side.modifiers.after.v.out_mod` rather than a byte offset in a blob:

| region | offset | what the describer names |
|---|---|---|
| `suit` | +0xb4, 8 B | `suit_tol`, `max_overharvest` |
| `res_mod` | +0xc0, 4 B | `res_mod` |
| `abilities` | +0xfc, 12 B | `reb_ai`, `ai_benefit`, `trade_allowed`, `commerce_raiding`, `view_intel`, `grav_synth`, `advanced_sensors`, `arcology`, + 4 unmodelled bytes |
| `modifiers` | +0x108, 0x54 B | `pddm`, `con_mod[3]`, `sav_mod[3]`, `out_mod`, `pop_mod`, `terra_mod`, `asteroid_mining`, `min_rate`, `per_gate_traffic`, `cast_range/efficiency/threshold`, + 5 unmodelled words |
| `design_masks` | +0x190, 8 B | `design_options_a`, `design_options_b` |
| `translation` | +0x1a4, 4 B | `translation_known` |
| `vaccines` | +0x288, 12 B | `has_vaccine`, `has_immunity`, `node_track_mask` |
| `research_target` | +0x294, 4 B | `research_target`, `research_target_set` |
| `node_bore_ptr` | +0x308, 4 B | the block pointer, as an opaque `ptr` |
| `inc_mod` | +0x30c, 4 B | `inc_mod` |
| `capture_designs` | +0x330, 4 B | `capture_designs` |
| `species_flags` | +0x348, 0x20 B | `species_flags[7]`, `count` |
| `roll` | +0x3b4, 8 B | `research_roll_pending`, `ai_rebellion` |
| `node_bore` | `*(+0x308)`, 12 B | `node_bore_params[3]` — declared **only when the block already exists** |

The unmodelled words are emitted as raw `u32` on purpose: if the original writes one of
them, that is a real finding and shows up as a divergence instead of going unseen. Pointer
words are emitted as `ptr` (ignored by the default policy) because they differ run to run.
`tests/shim_techfx` asserts that no two regions overlap and that each stays inside the
declared span.

### Comparison design

The regions do not cover the whole object, so there is no "scratch `ServerPlayer`" to hand
`ours`. Instead `rebind` passes `self` through unchanged and records the scratch buffer of
each region; `ours` then reads and writes **every field the callback can touch** through
those buffers, and reads only never-written fields (species, the tech-tree pointer) off the
live object, where "live" and "before" are the same value. `techfx::Views` is that
indirection, and it is what makes replace mode (no regions at all) the same code path.

Two things `ours` delegates to the game, in the same spirit as B3 delegating
`TechTree::Cost`:

* **the researched set** — 196 calls to `TechTree::HasResearched`, a verified, read-only
  three-compare function. In compare mode the tree is the live, post-original one, which
  is exactly the state the original's own tail saw (the completing node is already state 4).
* **the TechId of the definition** — 196 calls to `MasterTechTree::IsTech`, for the reason
  in "Findings" below.

## What `ours` deliberately does not do

Compare mode must not touch live game state or consume randomness, so `ours` reproduces the
player-field writes and nothing else. Not reproduced, and each reported in `shim.log`
instead (one `techfx: ours ...` line per completion):

* the three completion events (`EVENT_RESEARCH_COMPLETE` / `_UNDERBUDGET` / `_TEMPERANCE`);
* the pending plague-cure roll — it draws from the strategic generator, so `ours` clears
  the two words it guards and logs whether it would have fired;
* the writes to *other* objects: every owned system's AI flag (`CCC_AIVrus` / `CCC_AISlv`),
  the arcology civilian-cap re-evaluation, the addiction cure behind the temperance sweep,
  and the plague clear across systems and ships;
* `TechTree::SetResearched` for the Zuul boarding-pod grant (it would mutate the tree).

None of those touch a declared region, so **a compare stays clean through them** — but it
also does not check them. That is the honest boundary of this milestone: the compare proves
the player's fields, the log line proves what our layer decided about the rest.

One consequence worth stating: the Zuul grant makes the original call itself recursively.
The inner call rewrites the same tail fields with `IND_BrdPod` researched — and `IND_BrdPod`
is in neither mask table, has no chain branch and no xenotech bit, so the inner call's
writes equal the outer's. The compare is unaffected either way, since `ours` reads the
post-original tree.

## Findings — what the binary says that the notes did not

Read off the instruction stream first; the compare is confirmation, not discovery (B3's
lesson). Ten corrections, all of which changed code.

### 1. Every modifier is float32, and every constant is a widened float literal

The arithmetic is `fld dword [field]; fadd qword [k]; fstp dword [field]`: a float32 field,
combined with a *double* constant, stored back through float32. And every one of those
constants is a float32 value widened to double, not the decimal it looks like:

| written as | the double actually in the image | equals |
|---|---|---|
| 0.05 | 0x3FA99999A0000000 | `(double)0.05f` |
| 0.06 | 0x3FAEB851E0000000 | `(double)0.06f` |
| 0.10 | 0x3FB99999A0000000 | `(double)0.1f` |
| 0.15 | 0x3FC3333340000000 | `(double)0.15f` |
| 0.20 | 0x3FC99999A0000000 | `(double)0.2f` |
| 0.30 | 0x3FD3333340000000 | `(double)0.3f` |
| 0.45 | 0x3FDCCCCCC0000000 | `(double)0.45f` |
| 0.60 | 0x3FE3333340000000 | `(double)0.6f` |
| 0.90 | 0x3FECCCCCC0000000 | `(double)0.9f` |
| 0.35 | 0x3FD6666660000000 | `(double)0.35f` |
| 0.25, 0.5, 0.75, 1.0, 1.5 | exact | unaffected |

Ours held the state in `double` and used exact decimals. Rounding once at the end instead
of at every step drifts: `PlayerEconomyState` now holds `float`s, the apply layer rounds
through float32 at each step (`AddF` / `MulF`), and the table's literals carry an `f`
suffix so they widen to the same double bit pattern. `tests/game_effects` pins the six-tech
terraform chain against the stepwise result, and `tests/shim_techfx` pins the exact bit
patterns at the exact offsets.

Caveat, the same one B3 records: at 53-bit x87 precision control (the MSVC default) the
middle step rounds to double and the store rounds again, which is exactly what
`(float)((double)a + k)` gives on both the i386 target and the host. At 24-bit precision it
would not be. The hook records `fpu_cw` on every call, so the first trace settles it.

### 2. The AI-benefit bonus values are 0.5 — the table is dumped

`g_AITechValueTable` (6 × 12 bytes, `{int techId, float rebellionOdds, float bonus}`) was
"values not dumped" in the notes. It reads:

| tech | rebellion odds | bonus |
|---|---|---|
| CCC_AI, CCC_AIAdmin, CCC_AIFac | 0.1f | **0.5f** each |
| CCC_AIFRCON (10083) | 0.2f | 0 |
| CCC_AIVrus, CCC_AISlv | 0 | 0 |

So `AiBonusValues` now defaults to 0.5/0.5/0.5, and `AiRebellionOdds(id)` carries the odds
column. `ApplyAITechBonus` multiplies the value by `AIBn ? 1.0f : -1.0f` and narrows the
product to float32 before adding, which the apply layer now does too. `CCC_AIFRCON` is in
the table for its odds only — no branch reads its bonus.

### 3. `PrGtTrf` is an integer

The gate techs do `mov ecx,[cfg]; mov edx,[player+0x148]; cmp edx,[ecx]; jl ...` — a signed
**integer** max against an integer config word, not a float compare. Ours modelled the field
and both `PERGATETRAFFIC_*` tuning keys as `double`. Both are `int` now.

### 4. The node-bore parameters live behind a pointer, and the rule is "highest researched"

`ServerPlayer+0x308` is not three inline words: it is a pointer to a separately allocated
3-word block. The updater runs on **every** completion, allocates the block on first use and
`operator delete`s it when no bore drive is researched. The selector tests `DRV_RAD`
{95,60,5}, then `DRV_REND` {65,35,4}, then `DRV_RIP` {45,15,3}, first hit wins — which
confirms "highest wins" and raises it from medium to high confidence. There is **no species
gate**: the notes call these Zuul parameters, but nothing in the code checks the species (the
techs themselves are Zuul-only by data). `PlayerEconomyState` now carries
`hasNodeBoreParams` so "absent" is a state of its own, and the selection is re-derived in
the tail rather than applied by the completing tech.

### 5. A sticky "translation known" mask at +0x1a4 that was not in the notes

`RebuildSpeciesTechFlags` has a **second pass** the catalog does not mention: for each
species except the NPC race, if the species' flag word has bit 0 (level-1 translation), it
ORs that species' bit into `ServerPlayer+0x1a4`. Only ever ORed — never cleared, so it
survives the first pass clearing a flag. Modelled as `translationKnownMask`.

### 6. Three effects are tail checks, not effects of their tech

* **capture designs** — `if (!cdp) { if (HasResearched(SpyBm) && HasResearched(SlvgTech)) cdp = 1; }`
  runs on *every* completion. Ours only fired when the completing tech was one of the two,
  which is the same outcome in the normal path but wrong for a save where both are already
  researched and the flag is not yet set.
* **the design-option masks** — recomputed wholesale from the researched set every time.
* **the node-bore parameters** — see above.

### 7. The completion callback has no already-researched guard

`ApplyTechEffect` returned early when the tech was already researched. The callback cannot
do that: `SetResearched` marks the node state 4 *before* invoking it, so the guard would
make every real call a no-op. `ApplyTechCompletion` is the unguarded form and is what the
hook calls; `ApplyTechEffect` keeps the guard for callers driving the layer themselves.

### 8. `TechDef`'s first word is **not** the TechId

`MasterTechTree::IsTech(def, id)` maps `id - 10000` into the master `TechDef*[196]` table and
compares pointers. `TechTree::HasResearched(id)` does the same map to reach a `TechDef`, and
*then* uses that def's first word as an index into the tree's node vector. So the first word
is a node index in a larger key space, and the TechId can only be got from the identity test.
`ours` therefore scans `IsTech` over 10000..10195 (196 three-compare calls, once per
completion) rather than trusting the word — and the record carries both, so a trace shows
the two key spaces side by side. The valid range is confirmed as exactly 10000..10195, with
197 as the "none" sentinel special-cased before the subtraction.

### 9. The 196-name table is dumped, so the key space is no longer reconstructed

`g_TechIdNames` is 196 × `{const char* name, int}`. Reading it out gives every name in
position order, which turns `tech_id.h` from a partial reconstruction (89 confirmed names,
~55 inferred, ~52 `Unresolved_NNN`) into the table itself. Sanity checks all pass: index 0
is `CCC_AdvSens` (the id the chain's first `push 0x2710` tests), index 30 `IND_HrdStrct`
(0x272e), indices 38..43 the six vaccines (0x2736..0x273b), index 66 `DRV_RAD` (0x2752).
Every one of the seven prefixes previously inferred for indices 100..107 was right, and so
were all 31 names the design-option mask tables would have implied.

Two guesses were **wrong**, and one of them was a bug:

* **The proliferate block has five entries, not six.** It is 158..162 (Human, Hiver,
  Tarkas, Liir, Morrigi — no Zuul, like the other 5-entry families), and 163/164 are
  `CCC_NDTRKHUM` / `CCC_NDTRKZUL`. Our block ran 158..163, so
  `XenoTechId(Proliferate, Morrigi)` returned 10163 — the *Human node-track tech*. Fixed.
* Index 130, the Zuul slot of the level-3 translation block, is `XNC_DOMZUUL`, not a
  translation by name. The position is unchanged, so nothing keyed on it moves.

The species order inside every block is Human, Hiver, Tarkas, Liir, [Zuul,] Morrigi, and
the families that omit the Zuul are Incorporate, Addict, Temperance, Accommodate **and
Proliferate** — all of which were medium/low confidence and are now high. The node-track
techs' ids are known for the first time (10163 / 10164), which is what lets the hook set
`NPTrk`: our layer resolves them by name, and before this the name did not resolve.

The two design-option mask tables (`{int techId, uint bit}`) were dumped in the same pass
and every id in them agrees with the name table, so `kDesignOptionIdsA/B` are in the
effects module and `ComputeDesignOptionMasks` has a by-id overload — which is what the hook
needs, since the game keys on ids.

### 10. About half the techs have no TechId at all, and the tail still runs for them

`resolve_tech_id` returns "none" for any definition outside the 196-name table — which is
roughly half the shipped `.tech` files. The callback has no branch for those, but it runs
its whole tail for them all the same: the bore selection, the flag words, the sticky
translation mask, the capture-designs pair test and the temperance sweep. `EffectsOf` is
empty for them, so an early return would have been easy to write and wrong.
`RunCompletionTail` is that tail on its own, and it is what `ours` calls in that case.

### Things the notes had right

The 36-branch chain is an if/else-if over ids 10000..10031 in exactly that order with no
gaps (`push 0x2710` … `push 0x272f`, then the tail's 0x2730..0x2733); at most one branch
runs. `IND_Waldo`/`IND_ExpSys` share a block, `BIO_EnvTail` falls through into
`BIO_TerBac`'s terraform add, and `IND_OrbDry` really does leave `ConMod[0]` alone. Every
magnitude in the catalog is confirmed. The plague-cure masks are bits 0..4 for ids
10038..10042 and 0x0f for 10043 — with one subtlety: the test is "this def **or a
descendant of it**", which our exact-id `PlagueCureMask` cannot express without the tree.

## Host tests

`ctest` 28/28 on the build box.

* `game_effects` — 399 checks (was 285). Added: the float32 state and the widened-literal
  identities, the multiplicative tech on top of an additive one, the integer gate-traffic
  max, node-bore highest-wins plus re-derivation on an unrelated completion, the
  capture-designs tail check from a save where both are already researched, the sticky
  translation mask (including that a rebuild does not take a bit back), that
  `ApplyTechCompletion` applies twice while `ApplyTechEffect` does not, the recovered AI
  bonus and odds, and that the by-name and by-id design-option builders agree bit for bit.
* `shim_techfx` (new) — 1473 checks. A synthetic `ServerPlayer` buffer driven through the
  real adapter: the region table (no overlaps, all inside the span, every region named and
  described), a full read/write round trip, **all 44 catalogued effects** byte-checked at
  the offsets the original writes, hand-written float32 bit patterns for the additive,
  multiplicative and exactly-representable cases, the integer gate max, the absent
  node-bore block, the species flag words and their count, the research-target clear, and
  that the describers emit field names a diff can point at.

Cross-build: `b2-9cd997d-dirty-20260908T0359Z`, exports 66 names identical to
`binkw32.dll`, staged in `/srv/re-lab/shim/dist-b2`. `tools/clean_room_check.sh` OK.

## Coverage — say it up front

**Only a tech that actually completes fires this hook.** The reference save is turn 2 of a
28-star game with 0 starting techs, so a given End Turn may produce no records at all. The
plan, in order of what it proves:

1. **Host-side, already done.** Every catalogued effect is applied to a synthetic player and
   byte-checked (`shim_techfx`, 1473 checks). This is the exhaustive coverage; it does not
   need the VM and it does not depend on which techs the save happens to finish.
2. **`shim.cfg.b2scout`** (staged alongside the three) turns on B3's `ProcessResearch` hook
   in trace mode as well, so one End Turn shows both the completions that fired and every
   node's `progress` against its `cost`. That says how many turns are needed before a
   completion, instead of guessing.
3. **Trace, then compare, driven by whatever completes.** Press End Turn repeatedly; each
   completion is one record. A run that yields zero records is a coverage failure to report,
   not a pass — `tracecmp.py` exiting 0 on an empty trace proves nothing.
4. If the reference save cannot be driven to a completion in a reasonable number of turns,
   the fallback is a fresh game with research set high (the game-setup research slider) and
   a cheap first tech, saved as its own reference. That is a new save, so its determinism
   oracle would have to be recorded before it is useful for step 4 of the run list.

## Gotchas

1. **`describe_args` runs before the original**, and therefore before `ours`. Anything
   `ours` decides cannot ride on the record; it goes to `shim.log` instead.
2. **The hook can nest**, because the Zuul branch calls `SetResearched` → the callback. The
   nested call happens *inside the original*, not inside the hook, so the per-call statics
   are only ever written by the outer hook invocation. `ours` never recurses.
3. **The node-bore block cannot be created by `ours`.** In compare mode the region is
   declared only when the block already exists, and the block pointer is described as an
   opaque `ptr` so the original allocating it is not a divergence. In replace mode `ours`
   calls the game's own updater to keep the allocation consistent — which means replace
   mode does not exercise our node-bore selection; compare and the host tests do.
4. **Replace mode for this hook is deliberately incomplete.** It writes the player's fields
   but raises no events, flags no systems, cures nothing and grants no boarding pods. On a
   turn where nothing completes the hook never fires and the determinism oracle should still
   hold — that is what step 4 below actually tests. On a turn where something completes, a
   changed save hash is expected and is not a finding.
5. A compare record is small (14 regions, none over 0x54 bytes), so unlike B3 the configs
   could leave other hooks on. They do not: the b2 configs switch everything else off so a
   trace holds nothing but this callback.
6. `ServerPlayer+0x308` is a 32-bit pointer field. Host tests cannot write a real pointer
   into a synthetic buffer at that offset (it would spill over `IncMod` at +0x30c), so they
   set the view's block pointer directly.

## Runs

| file | mode | records | result |
|---|---|---|---|
| `b2-techfx-golden.jsonl` | trace | 2 OnTechResearched (+18 ProcessResearch from the scout config) | `tracecmp.py` exit 0, 0 invalid |
| `b2-techfx-compare.jsonl` | compare | 3 OnTechResearched | 3 compared, **0 diverged**, 0 errors, exit 0 |
| replace (no log) | replace | 0 (no completion that turn) | End Turn from `ref-turn2.sav`: `(Autosave).sav` `978041ac…` / `(Autosave EndTurn).sav` `bb4fd9ac…` = oracle |
| `b2-shim.log` | | | install banners and the three `techfx: ours mode=compare` lines |

The completions that fired, all on the AI player (index 1, species 2 = Tarkas), driven by six to
eight End Turns from `ref-turn2.sav`: **`IND_Waldo` (10001)**, **`BIO_GnMod` (10010)** and
**`BIO_SPNDANI` (10070)**. The third is the interesting one: it is not in the effects table at
all, so only the tail ran — which is exactly the case finding 10 predicted and the case an early
return would have broken.

### What the live trace confirms

* **The float32 model, exactly.** `IND_Waldo` moved `con_mod` `1.0 → 0.899999976` and `out_mod`
  `1.10000002 → 1.25`; `BIO_GnMod` moved `pop_mod` `1.10000002 → 1.20000005`. Those are bit for
  bit `(float)(x + (double)0.1f)` / `(double)0.15f`, not the exact decimals. Finding 1 is now
  confirmed against the game and not only against the image.
* **`TechDef`'s first word is not the TechId** (finding 8): the records carry both, and they are
  `144` vs `10001` and `9` vs `10010`.
* **`fpu_cw` = `0x127f`** — 53-bit precision control, matching B3, so the rounding shape in
  `AddF`/`MulF` is the right one.
* **The gate-traffic config words are 3 and 6 at call time.** The hook's init banner prints
  `gate=0/0` because the data files have not been read when the DLL loads; the values are
  dereferenced per call, and the args show the live ones.
* **The RNG region resolves and is consistent**: `left` 413 with `next_index` 211, and
  `624 − 413 = 211`, which validates both the two-hop pointer chain to the generator and the
  describer's index arithmetic.
* The two unmodelled words at `+0x128`/`+0x12c` hold `1.0f`, and `+0x13c` holds `0.2f`. None of
  the unmodelled words moved on any call.

### The RNG draw B3 handed over

B3 saw one completion where the original drew one word more than its implementation and
attributed it to this callback. That is right, and it is the roll at the very top:
`ServerPlayer::RollResearchEvent` (`+0x3b4` set and the completing def is the current research
target) makes **exactly one `NextFloat`** from the strategic generator, unconditionally, before
any comparison — then fires the AI-rebellion/plague path only if `odds > roll`. `ours` reproduces
that draw against the scratch copy of the generator (B3's design), so the post-state is
comparable, and the whole `Mars::RNG` object is now a declared region.

**On this workload the branch never fired**: `research_roll_pending` was `false` on all three
completions, so the callback drew nothing and the `rng` region is unchanged before and after.
That means the compare could not exercise the draw — it is modelled and host-reasoned, not
behaviourally proven. The region is declared so the next run that does set the byte will check it.

### Where the compare is blind

Per B3's harness lesson: `ours` posts no events, and the player's event list is **not** a declared
region (its offset was not located in this window). Both non-silent completions raise
`EVENT_RESEARCH_COMPLETE` or `_UNDERBUDGET`, and a temperance completion would raise
`EVENT_TEMPERANCE`. So a clean compare here bounds the player's economy fields and nothing else,
and **replace mode is not oracle-clean on a turn where a tech completes** — the same failure mode
B3 hit, known in advance rather than discovered by the oracle. The replace run above deliberately
used a turn with no completion.

## What remains

1. **Locate the player's event list and declare it as a region**, then either post the events
   through the game's own API in `ours` (as B3 concluded for its own event) or accept a named,
   expected divergence there. Until then the compare's scope is the economy fields.
2. **Exercise the roll.** Find a turn where `+0x3b4` is set at completion (the record's
   `research_roll_pending` before-value says when) and confirm the `rng` post-state matches.
3. The Zuul branches — `IND_CruisCon` granting boarding pods, and the temperance sweep — need a
   Zuul player; the reference save has none.