sots-engine/docs/B2.md

B2 — ServerPlayer::OnTechResearched old-vs-new, with the player's fields as named regions

Result (2026-09-08): verified on the live game. Trace: 2 completions captured, tracecmp.py exit 0, 0 invalid. Compare: 3 completions compared, 0 divergences, 0 errors. Replace: one End Turn from ref-turn2.sav reproduces the determinism oracle byte for byte ((Autosave).sav = 978041acd168b56e…, (Autosave EndTurn).sav = bb4fd9ac89f41e3b…) — a weak check by construction, because no tech completes on that turn so the hook never fires; what it proves is that the hook installs and perturbs nothing. Game left on VM140 at the main menu in hooks=trace.

Build b2-646e4e8-dirty-20260908T0434Z, staged in /srv/re-lab/shim/dist-b2, deployed from C:\SOTS\shimdist-b2. Evidence in sots-re/verify/traces/b2-* and verify/results/compare/b2-techfx-golden.json.

All three runs are archived: sots-re/verify/traces/b2-techfx-golden.jsonl, b2-techfx-compare.jsonl, b2-shim.log, b2-final-menu.png, with the tracecmp.py reports in verify/results/compare/b2-techfx-{golden,compare}.json.

The point of the target: it is the single place where 36 hard-coded strategic effects, the per-species xenotech flag words, the two design-option masks and the node-bore parameters are all written. One call validates the whole src/game/effects layer at once.

What was hooked

hook name (record hook)	RVA	prototype
Game::ServerPlayer::OnTechResearched	0x00491790	void (ServerPlayer*, TechDef* def, bool silent)

__thiscall, [verified], vft slot 4, so it goes through Hook<> with CallConv::Thiscall. Source: src/shim/hooks/tech_effects.{h,cpp} (the descriptor and the delegation into the game's tech tree) over src/shim/hooks/tech_effect_fields.{h,cpp} (the byte-level field adapter, host-tested — the split B1 made for budget_inputs). Installed from src/shim/main.cpp after the B3 hook.

TechTree::SetResearched is the only caller. It fires once per tech completion, which on a typical turn is zero times — see "Coverage" below, because that is the hard part of this milestone.

Region model

Fourteen regions, one per field group the callback writes, each with a struct describer, so a divergence reads side.modifiers.after.v.out_mod rather than a byte offset in a blob:

region	offset	what the describer names
suit	+0xb4, 8 B	suit_tol, max_overharvest
res_mod	+0xc0, 4 B	res_mod
abilities	+0xfc, 12 B	reb_ai, ai_benefit, trade_allowed, commerce_raiding, view_intel, grav_synth, advanced_sensors, arcology, + 4 unmodelled bytes
modifiers	+0x108, 0x54 B	pddm, con_mod[3], sav_mod[3], out_mod, pop_mod, terra_mod, asteroid_mining, min_rate, per_gate_traffic, cast_range/efficiency/threshold, + 5 unmodelled words
design_masks	+0x190, 8 B	design_options_a, design_options_b
translation	+0x1a4, 4 B	translation_known
vaccines	+0x288, 12 B	has_vaccine, has_immunity, node_track_mask
research_target	+0x294, 4 B	research_target, research_target_set
node_bore_ptr	+0x308, 4 B	the block pointer, as an opaque ptr
inc_mod	+0x30c, 4 B	inc_mod
capture_designs	+0x330, 4 B	capture_designs
species_flags	+0x348, 0x20 B	species_flags[7], count
roll	+0x3b4, 8 B	research_roll_pending, ai_rebellion
node_bore	*(+0x308), 12 B	node_bore_params[3] — declared only when the block already exists

The unmodelled words are emitted as raw u32 on purpose: if the original writes one of them, that is a real finding and shows up as a divergence instead of going unseen. Pointer words are emitted as ptr (ignored by the default policy) because they differ run to run. tests/shim_techfx asserts that no two regions overlap and that each stays inside the declared span.

Comparison design

The regions do not cover the whole object, so there is no "scratch ServerPlayer" to hand ours. Instead rebind passes self through unchanged and records the scratch buffer of each region; ours then reads and writes every field the callback can touch through those buffers, and reads only never-written fields (species, the tech-tree pointer) off the live object, where "live" and "before" are the same value. techfx::Views is that indirection, and it is what makes replace mode (no regions at all) the same code path.

Two things ours delegates to the game, in the same spirit as B3 delegating TechTree::Cost:

• the researched set — 196 calls to TechTree::HasResearched, a verified, read-only three-compare function. In compare mode the tree is the live, post-original one, which is exactly the state the original's own tail saw (the completing node is already state 4).
• the TechId of the definition — 196 calls to MasterTechTree::IsTech, for the reason in "Findings" below.

What ours deliberately does not do

Compare mode must not touch live game state or consume randomness, so ours reproduces the player-field writes and nothing else. Not reproduced, and each reported in shim.log instead (one techfx: ours ... line per completion):

• the three completion events (EVENT_RESEARCH_COMPLETE / _UNDERBUDGET / _TEMPERANCE);
• the pending plague-cure roll — it draws from the strategic generator, so ours clears the two words it guards and logs whether it would have fired;
• the writes to other objects: every owned system's AI flag (CCC_AIVrus / CCC_AISlv), the arcology civilian-cap re-evaluation, the addiction cure behind the temperance sweep, and the plague clear across systems and ships;
• TechTree::SetResearched for the Zuul boarding-pod grant (it would mutate the tree).

None of those touch a declared region, so a compare stays clean through them — but it also does not check them. That is the honest boundary of this milestone: the compare proves the player's fields, the log line proves what our layer decided about the rest.

One consequence worth stating: the Zuul grant makes the original call itself recursively. The inner call rewrites the same tail fields with IND_BrdPod researched — and IND_BrdPod is in neither mask table, has no chain branch and no xenotech bit, so the inner call's writes equal the outer's. The compare is unaffected either way, since ours reads the post-original tree.

Findings — what the binary says that the notes did not

Read off the instruction stream first; the compare is confirmation, not discovery (B3's lesson). Ten corrections, all of which changed code.

1. Every modifier is float32, and every constant is a widened float literal

The arithmetic is fld dword [field]; fadd qword [k]; fstp dword [field]: a float32 field, combined with a double constant, stored back through float32. And every one of those constants is a float32 value widened to double, not the decimal it looks like:

written as	the double actually in the image	equals
0.05	0x3FA99999A0000000	(double)0.05f
0.06	0x3FAEB851E0000000	(double)0.06f
0.10	0x3FB99999A0000000	(double)0.1f
0.15	0x3FC3333340000000	(double)0.15f
0.20	0x3FC99999A0000000	(double)0.2f
0.30	0x3FD3333340000000	(double)0.3f
0.45	0x3FDCCCCCC0000000	(double)0.45f
0.60	0x3FE3333340000000	(double)0.6f
0.90	0x3FECCCCCC0000000	(double)0.9f
0.35	0x3FD6666660000000	(double)0.35f
0.25, 0.5, 0.75, 1.0, 1.5	exact	unaffected

Ours held the state in double and used exact decimals. Rounding once at the end instead of at every step drifts: PlayerEconomyState now holds floats, the apply layer rounds through float32 at each step (AddF / MulF), and the table's literals carry an f suffix so they widen to the same double bit pattern. tests/game_effects pins the six-tech terraform chain against the stepwise result, and tests/shim_techfx pins the exact bit patterns at the exact offsets.

Caveat, the same one B3 records: at 53-bit x87 precision control (the MSVC default) the middle step rounds to double and the store rounds again, which is exactly what (float)((double)a + k) gives on both the i386 target and the host. At 24-bit precision it would not be. The hook records fpu_cw on every call, so the first trace settles it.

2. The AI-benefit bonus values are 0.5 — the table is dumped

g_AITechValueTable (6 × 12 bytes, {int techId, float rebellionOdds, float bonus}) was "values not dumped" in the notes. It reads:

tech	rebellion odds	bonus
CCC_AI, CCC_AIAdmin, CCC_AIFac	0.1f	0.5f each
CCC_AIFRCON (10083)	0.2f	0
CCC_AIVrus, CCC_AISlv	0	0

So AiBonusValues now defaults to 0.5/0.5/0.5, and AiRebellionOdds(id) carries the odds column. ApplyAITechBonus multiplies the value by AIBn ? 1.0f : -1.0f and narrows the product to float32 before adding, which the apply layer now does too. CCC_AIFRCON is in the table for its odds only — no branch reads its bonus.

3. PrGtTrf is an integer

The gate techs do mov ecx,[cfg]; mov edx,[player+0x148]; cmp edx,[ecx]; jl ... — a signed integer max against an integer config word, not a float compare. Ours modelled the field and both PERGATETRAFFIC_* tuning keys as double. Both are int now.

4. The node-bore parameters live behind a pointer, and the rule is "highest researched"

ServerPlayer+0x308 is not three inline words: it is a pointer to a separately allocated 3-word block. The updater runs on every completion, allocates the block on first use and operator deletes it when no bore drive is researched. The selector tests DRV_RAD {95,60,5}, then DRV_REND {65,35,4}, then DRV_RIP {45,15,3}, first hit wins — which confirms "highest wins" and raises it from medium to high confidence. There is no species gate: the notes call these Zuul parameters, but nothing in the code checks the species (the techs themselves are Zuul-only by data). PlayerEconomyState now carries hasNodeBoreParams so "absent" is a state of its own, and the selection is re-derived in the tail rather than applied by the completing tech.

5. A sticky "translation known" mask at +0x1a4 that was not in the notes

RebuildSpeciesTechFlags has a second pass the catalog does not mention: for each species except the NPC race, if the species' flag word has bit 0 (level-1 translation), it ORs that species' bit into ServerPlayer+0x1a4. Only ever ORed — never cleared, so it survives the first pass clearing a flag. Modelled as translationKnownMask.

6. Three effects are tail checks, not effects of their tech

• capture designs — if (!cdp) { if (HasResearched(SpyBm) && HasResearched(SlvgTech)) cdp = 1; } runs on every completion. Ours only fired when the completing tech was one of the two, which is the same outcome in the normal path but wrong for a save where both are already researched and the flag is not yet set.
• the design-option masks — recomputed wholesale from the researched set every time.
• the node-bore parameters — see above.

7. The completion callback has no already-researched guard

ApplyTechEffect returned early when the tech was already researched. The callback cannot do that: SetResearched marks the node state 4 before invoking it, so the guard would make every real call a no-op. ApplyTechCompletion is the unguarded form and is what the hook calls; ApplyTechEffect keeps the guard for callers driving the layer themselves.

8. TechDef's first word is not the TechId

MasterTechTree::IsTech(def, id) maps id - 10000 into the master TechDef*[196] table and compares pointers. TechTree::HasResearched(id) does the same map to reach a TechDef, and then uses that def's first word as an index into the tree's node vector. So the first word is a node index in a larger key space, and the TechId can only be got from the identity test. ours therefore scans IsTech over 10000..10195 (196 three-compare calls, once per completion) rather than trusting the word — and the record carries both, so a trace shows the two key spaces side by side. The valid range is confirmed as exactly 10000..10195, with 197 as the "none" sentinel special-cased before the subtraction.

9. The 196-name table is dumped, so the key space is no longer reconstructed

g_TechIdNames is 196 × {const char* name, int}. Reading it out gives every name in position order, which turns tech_id.h from a partial reconstruction (89 confirmed names, ~55 inferred, ~52 Unresolved_NNN) into the table itself. Sanity checks all pass: index 0 is CCC_AdvSens (the id the chain's first push 0x2710 tests), index 30 IND_HrdStrct (0x272e), indices 38..43 the six vaccines (0x2736..0x273b), index 66 DRV_RAD (0x2752). Every one of the seven prefixes previously inferred for indices 100..107 was right, and so were all 31 names the design-option mask tables would have implied.

Two guesses were wrong, and one of them was a bug:

• The proliferate block has five entries, not six. It is 158..162 (Human, Hiver, Tarkas, Liir, Morrigi — no Zuul, like the other 5-entry families), and 163/164 are CCC_NDTRKHUM / CCC_NDTRKZUL. Our block ran 158..163, so XenoTechId(Proliferate, Morrigi) returned 10163 — the Human node-track tech. Fixed.
• Index 130, the Zuul slot of the level-3 translation block, is XNC_DOMZUUL, not a translation by name. The position is unchanged, so nothing keyed on it moves.

The species order inside every block is Human, Hiver, Tarkas, Liir, [Zuul,] Morrigi, and the families that omit the Zuul are Incorporate, Addict, Temperance, Accommodate and Proliferate — all of which were medium/low confidence and are now high. The node-track techs' ids are known for the first time (10163 / 10164), which is what lets the hook set NPTrk: our layer resolves them by name, and before this the name did not resolve.

The two design-option mask tables ({int techId, uint bit}) were dumped in the same pass and every id in them agrees with the name table, so kDesignOptionIdsA/B are in the effects module and ComputeDesignOptionMasks has a by-id overload — which is what the hook needs, since the game keys on ids.

10. About half the techs have no TechId at all, and the tail still runs for them

resolve_tech_id returns "none" for any definition outside the 196-name table — which is roughly half the shipped .tech files. The callback has no branch for those, but it runs its whole tail for them all the same: the bore selection, the flag words, the sticky translation mask, the capture-designs pair test and the temperance sweep. EffectsOf is empty for them, so an early return would have been easy to write and wrong. RunCompletionTail is that tail on its own, and it is what ours calls in that case.

Things the notes had right

The 36-branch chain is an if/else-if over ids 10000..10031 in exactly that order with no gaps (push 0x2710 … push 0x272f, then the tail's 0x2730..0x2733); at most one branch runs. IND_Waldo/IND_ExpSys share a block, BIO_EnvTail falls through into BIO_TerBac's terraform add, and IND_OrbDry really does leave ConMod[0] alone. Every magnitude in the catalog is confirmed. The plague-cure masks are bits 0..4 for ids 10038..10042 and 0x0f for 10043 — with one subtlety: the test is "this def or a descendant of it", which our exact-id PlagueCureMask cannot express without the tree.

Host tests

ctest 28/28 on the build box.

• game_effects — 399 checks (was 285). Added: the float32 state and the widened-literal identities, the multiplicative tech on top of an additive one, the integer gate-traffic max, node-bore highest-wins plus re-derivation on an unrelated completion, the capture-designs tail check from a save where both are already researched, the sticky translation mask (including that a rebuild does not take a bit back), that ApplyTechCompletion applies twice while ApplyTechEffect does not, the recovered AI bonus and odds, and that the by-name and by-id design-option builders agree bit for bit.
• shim_techfx (new) — 1473 checks. A synthetic ServerPlayer buffer driven through the real adapter: the region table (no overlaps, all inside the span, every region named and described), a full read/write round trip, all 44 catalogued effects byte-checked at the offsets the original writes, hand-written float32 bit patterns for the additive, multiplicative and exactly-representable cases, the integer gate max, the absent node-bore block, the species flag words and their count, the research-target clear, and that the describers emit field names a diff can point at.

Cross-build: b2-9cd997d-dirty-20260908T0359Z, exports 66 names identical to binkw32.dll, staged in /srv/re-lab/shim/dist-b2. tools/clean_room_check.sh OK.

Coverage — say it up front

Only a tech that actually completes fires this hook. The reference save is turn 2 of a 28-star game with 0 starting techs, so a given End Turn may produce no records at all. The plan, in order of what it proves:

1. Host-side, already done. Every catalogued effect is applied to a synthetic player and byte-checked (shim_techfx, 1473 checks). This is the exhaustive coverage; it does not need the VM and it does not depend on which techs the save happens to finish.
2. shim.cfg.b2scout (staged alongside the three) turns on B3's ProcessResearch hook in trace mode as well, so one End Turn shows both the completions that fired and every node's progress against its cost. That says how many turns are needed before a completion, instead of guessing.
3. Trace, then compare, driven by whatever completes. Press End Turn repeatedly; each completion is one record. A run that yields zero records is a coverage failure to report, not a pass — tracecmp.py exiting 0 on an empty trace proves nothing.
4. If the reference save cannot be driven to a completion in a reasonable number of turns, the fallback is a fresh game with research set high (the game-setup research slider) and a cheap first tech, saved as its own reference. That is a new save, so its determinism oracle would have to be recorded before it is useful for step 4 of the run list.

Gotchas

1. describe_args runs before the original, and therefore before ours. Anything ours decides cannot ride on the record; it goes to shim.log instead.
2. The hook can nest, because the Zuul branch calls SetResearched → the callback. The nested call happens inside the original, not inside the hook, so the per-call statics are only ever written by the outer hook invocation. ours never recurses.
3. The node-bore block cannot be created by ours. In compare mode the region is declared only when the block already exists, and the block pointer is described as an opaque ptr so the original allocating it is not a divergence. In replace mode ours calls the game's own updater to keep the allocation consistent — which means replace mode does not exercise our node-bore selection; compare and the host tests do.
4. Replace mode for this hook is deliberately incomplete. It writes the player's fields but raises no events, flags no systems, cures nothing and grants no boarding pods. On a turn where nothing completes the hook never fires and the determinism oracle should still hold — that is what step 4 below actually tests. On a turn where something completes, a changed save hash is expected and is not a finding.
5. A compare record is small (14 regions, none over 0x54 bytes), so unlike B3 the configs could leave other hooks on. They do not: the b2 configs switch everything else off so a trace holds nothing but this callback.
6. ServerPlayer+0x308 is a 32-bit pointer field. Host tests cannot write a real pointer into a synthetic buffer at that offset (it would spill over IncMod at +0x30c), so they set the view's block pointer directly.

Runs

file	mode	records	result
b2-techfx-golden.jsonl	trace	2 OnTechResearched (+18 ProcessResearch from the scout config)	tracecmp.py exit 0, 0 invalid
b2-techfx-compare.jsonl	compare	3 OnTechResearched	3 compared, 0 diverged, 0 errors, exit 0
replace (no log)	replace	0 (no completion that turn)	End Turn from ref-turn2.sav: (Autosave).sav 978041ac… / (Autosave EndTurn).sav bb4fd9ac… = oracle
b2-shim.log			install banners and the three techfx: ours mode=compare lines

The completions that fired, all on the AI player (index 1, species 2 = Tarkas), driven by six to eight End Turns from ref-turn2.sav: IND_Waldo (10001), BIO_GnMod (10010) and BIO_SPNDANI (10070). The third is the interesting one: it is not in the effects table at all, so only the tail ran — which is exactly the case finding 10 predicted and the case an early return would have broken.

What the live trace confirms

• The float32 model, exactly. IND_Waldo moved con_mod 1.0 → 0.899999976 and out_mod 1.10000002 → 1.25; BIO_GnMod moved pop_mod 1.10000002 → 1.20000005. Those are bit for bit (float)(x + (double)0.1f) / (double)0.15f, not the exact decimals. Finding 1 is now confirmed against the game and not only against the image.
• TechDef's first word is not the TechId (finding 8): the records carry both, and they are 144 vs 10001 and 9 vs 10010.
• fpu_cw = 0x127f — 53-bit precision control, matching B3, so the rounding shape in AddF/MulF is the right one.
• The gate-traffic config words are 3 and 6 at call time. The hook's init banner prints gate=0/0 because the data files have not been read when the DLL loads; the values are dereferenced per call, and the args show the live ones.
• The RNG region resolves and is consistent: left 413 with next_index 211, and 624 − 413 = 211, which validates both the two-hop pointer chain to the generator and the describer's index arithmetic.
• The two unmodelled words at +0x128/+0x12c hold 1.0f, and +0x13c holds 0.2f. None of the unmodelled words moved on any call.

The RNG draw B3 handed over

B3 saw one completion where the original drew one word more than its implementation and attributed it to this callback. That is right, and it is the roll at the very top: ServerPlayer::RollResearchEvent (+0x3b4 set and the completing def is the current research target) makes exactly one NextFloat from the strategic generator, unconditionally, before any comparison — then fires the AI-rebellion/plague path only if odds > roll. ours reproduces that draw against the scratch copy of the generator (B3's design), so the post-state is comparable, and the whole Mars::RNG object is now a declared region.

On this workload the branch never fired: research_roll_pending was false on all three completions, so the callback drew nothing and the rng region is unchanged before and after. That means the compare could not exercise the draw — it is modelled and host-reasoned, not behaviourally proven. The region is declared so the next run that does set the byte will check it.

Where the compare is blind

Per B3's harness lesson: ours posts no events, and the player's event list is not a declared region (its offset was not located in this window). Both non-silent completions raise EVENT_RESEARCH_COMPLETE or _UNDERBUDGET, and a temperance completion would raise EVENT_TEMPERANCE. So a clean compare here bounds the player's economy fields and nothing else, and replace mode is not oracle-clean on a turn where a tech completes — the same failure mode B3 hit, known in advance rather than discovered by the oracle. The replace run above deliberately used a turn with no completion.

What remains

1. Locate the player's event list and declare it as a region, then either post the events through the game's own API in ours (as B3 concluded for its own event) or accept a named, expected divergence there. Until then the compare's scope is the economy fields.
2. Exercise the roll. Find a turn where +0x3b4 is set at completion (the record's research_roll_pending before-value says when) and confirm the rng post-state matches.
3. The Zuul branches — IND_CruisCon granting boarding pods, and the temperance sweep — need a Zuul player; the reference save has none.