sots-engine/docs/U-unlock.md

U — the unlock cascade, and the prediction for the run that checks it

Lane P deliberately did not post EVENT_TECHS_UNLOCKED. It could have — the trigger was pinned — but evaluating it needs the cascade ours does not run, so it handed PostResearchPassEvents a nullptr unlock list ("no list", distinct from an empty one) and predicted the consequence: next_id short by exactly 1 on every completion call. Lane V measured that live, twice, exactly.

This lane removes the residual by running the cascade, not by posting an event when something completes. The distinction matters for one reason and lane P named it: the shortcut would score perfectly on this save and be wrong the first time a completion unlocks nothing.

Everything in §4 was written before the build was staged on VM140. §5 is the outcome.

---

1. What the cascade actually is

Read out of the instruction stream by this lane: TechTree::SetResearched (0x00581e10, whole function), TechTree::PrereqsMet (0x0057d8e0, whole function), the tail collector inside TechTree::ProcessResearch (0x00587cc3), and the head of ServerPlayer::OnTechResearched (0x00891790). Details and the decompiled shape are in sots-re findings/subsystems/setresearched-cascade.md; the module is src/game/sim/techgraph.{h,cpp}.

SetResearched(tree, def, flags):
    n = tree->nodes[def->techId]
    if (!n)             -> nothing (the flags&8 re-lookup path is not on the research path)
    if (n->state == 4)  -> return true, writing NOTHING, not even the order stamp
    if (!(flags & 2) && !PrereqsMet(def + 0x88)) -> nothing

    n->state          = 4
    n->turnResearched = owner ? ModCount : 0
    n->order          = tree->orderCounter++
    owner->OnTechResearched(def, (flags >> 2) & 1)      # flags = 2 from ProcessResearch: NOT silent

    for each edge in n->children:                       # sweep 1
        c = tree->nodes[edge->childDef->techId]
        if (c->state == 0) c->state = 1
        c->costRP = min(c->costRP, edge->costRP)         # signed, against an INT_MAX sentinel

    for each node m in tree->nodes:                     # sweep 2
        if (!m || m->def->[0xb0]) continue
        p = tree->nodes[m->def->techId]
        if (p && p->state == 1 && PrereqsMet(p->def + 0x88)):
            m->state = 2
            if (m->turnAvailable == -1 && owner) m->turnAvailable = ModCount
        if (m->state == 2 && Cost(m) == 0) SetResearched(m->def, flags)     # recursion

Four properties are easy to get wrong and each is pinned by a host test:

• turnAvailable is sticky. It is written only when it still reads −1. A node that returns to state 2 in a later turn keeps its first availability turn — which is the whole reason EVENT_TECHS_UNLOCKED can be computed from node state at all, and the reason a tech is never announced twice.
• The cost minimum is signed, against INT_MAX. The first researched parent sets the child's cost; later parents can only lower it.
• PrereqsMet is an AND over groups, each group an OR over techs — and a group with zero entries makes the whole test fail, because the inner loop cannot break and the outer one then exits with that group uncounted. A "vacuously true" reading gets this backwards. Zero groups, by contrast, is satisfied.
• Both sweeps and the collector re-resolve tree->nodes[node->def->techId] and test that node's state, not the iterated node's. For a well-formed tree it comes back to the same node; it is reproduced because the code has it.

def+0xb0 is a byte that excludes a node from sweep 2 entirely. Its write site was not found (MasterTechTree::ParseTech shows no reference to 0xb0), so addresses.d/lane-u.json names it for what it does — TechDef_off_NoAutoAvailable — and records the unlock_explicitly tech-file keyword as a hypothesis, not a fact. Nothing in this lane depends on which it is: the byte is read from the live def either way.

The tail collector

if (tree->owner) {
    turn = ModCount
    for each node n: if (n && n->def && (p = tree->nodes[n->def->techId]) &&
                        p->state == 2 && n->turnAvailable == turn) collect(n)
    if (!collected.empty()) PostEvent(EVENT_TECHS_UNLOCKED, ...)
}

Note the asymmetry — the state test is on the self-resolved node, the turn test on the iterated one — and note that it runs once, after the whole per-node loop and after the decay sweep.

The two halves of OnTechResearched this hook can see

OnTechResearched(player, def, silent):
    RecordObservedTech(...)                                  # FIRST statement, unconditional
    if (player->ResT == def) {
        if (player->ResearchRollPending) RollResearchEvent(player)   # exactly one NextFloat
        player->ResearchRollPending = 0
        player->ResT = 0
    }
    if (!silent) { ... EVENT_RESEARCH_COMPLETE / _UNDERBUDGET ... }

RecordObservedTech de-duplicates by tech name, so "the vector did not grow" is a real outcome and not a failure. RollResearchEvent draws one NextFloat unconditionally and only then tests it against the odds; clearing ResT is what makes a second completion in the same pass draw nothing. This is lane V's call-9 deviation, and it is now inside the compare rather than declared out of scope.

---

2. What changed

file	change
src/game/sim/techgraph.{h,cpp} (new)	the pure cascade: TechPrereqsMet, SetResearched, CollectNewlyAvailable
src/game/sim/research.{h,cpp}	one optional parameter: a completion hook called inside the allocation loop, where the original calls SetResearched — before the next entry's roll and before the decay sweep
src/shim/hooks/research.{h,cpp}	the pre-call scan the cascade needs, the live→graph transcription, the cascade wiring, four more node write-backs, the unlock list, the observed-tech append, the RNG draw, the rewritten coverage
tests/game_sim/test_techgraph.cpp (new)	13 cases, 92 checks
ghidra/addresses.d/lane-u.json (sots-re)	12 new entries; header regenerated 615 → 627

tools/clean_room_check.sh — OK. Host ctest — 34/34 (was 33/33; game_sim_techgraph is new). Run as separate commands. The shim TU is syntax-checked on the host (-std=c++17 -Wall -Wextra -fsyntax-only, clean) and cross-built on CT111 before deploy.

The four pre-call reads, and why each one is a trap

ours runs after the original in compare mode. Four inputs the cascade needs are things the original changes during the call, and each of them reads back a plausible wrong answer:

input	read after the original would give	consequence
TechTree+0x20 order counter	the next order	every node.order off by one per completion
ServerPlayer+0x294 (ResT)	0 — cleared by the callback	the extra RNG draw never modelled
ServerPlayer+0x3b4 (pending roll)	0 — cleared in the same block	same
ServerPlayer+0x274 observed-tech names	the completing tech already present	the append modelled as a de-duplication, agreeing with a count it did not compute

All four are taken in describe_args, which hook.h calls immediately before the original, and all four are reported as arguments (order_counter_in, research_target, roll_pending_in, observed_techs_in) so a run can be audited without trusting ours.

What is modelled, and what is deliberately not

The cascade runs in compare mode only. In replace mode every pointer is live game memory, and applying half of OnTechResearched — the observed-tech append and the research-event roll, but not the ninety-odd tech-effect field writes — would leave the player in a state no code path produces. Not running it leaves a player missing a cascade, which is a smaller and already declared lie. The same reasoning lane P used for the event counts.

TechTree::Cost is called, not re-derived: it is the game's own read-only function, and the cascade needs it on the node as it stands after the child-cost sweep. Guessing the cost multiplier would put a second unknown inside the thing being measured.

The unlock list is still an input to PostResearchPassEvents, and nullptr still means "this caller could not compute it" — now reached when the graph fails to transcribe or the tail's tree->owner != 0 gate is closed, rather than always. An empty vector means "computed, and nothing became available", which posts nothing. Keeping those apart is what stops a failure to read the tree from scoring as a correct silence.

---

3. Why a clean result here would not be vacuous

A hook that silently compares nothing is the failure mode to fear when a clean run is expected, and this lane is exactly that situation. Three things make it hard here:

1. The expected values are non-trivial and known in advance. cost_rp must come back 10000, 16000 and 8000 on three specific nodes, turn_available 4, order 22 and 23. A model that computed nothing leaves them at INT_MAX, 0 and −1 — which is precisely what lane V's report shows today. There is no "do nothing" answer that passes.
2. The collector runs on all fifteen calls, not just the two completions. On the thirteen calls with no completion it must come back empty; if the transcription were wrong in a way that over-collected, next_id would be too high and those calls would newly diverge. The quiet calls are as much of a check as the loud ones.
3. The shim log prints the counters. One line per call: cascade ok=… completions=… unlocked=… otch_appends=… roll_draws=… failures=… depth=… name_unreadable=…. A clean compare with all of those at zero on a completion call would be a clean compare of nothing, and it would be visible.

---

4. The prediction

Written before the build was staged. Recipe unchanged: shim.cfg.recapb3, ref-turn2.sav → Launch → End Turn, then four more End Turns in the same session.

4.1 First End Turn — nothing should change

Turn 1 of ref-turn2 completes no tech (lane V: observed_techs.bytes unchanged on all three calls), so the cascade never fires. 3 calls, 3 compared, 0 divergent, tracecmp exit 0 — identical to lane V's result, and that is the point: the new code must be inert when nothing completes.

new argument	call 0	calls 1, 2
observed_techs_in	10 (lane V measured 440 bytes = 10 × 44)	20 (880 bytes)
research_target	non-null (the player has a funded target)	either
roll_pending_in	false	false
order_counter_in	≥ 0, and not −1	≥ 0
observed_scan_failed / observed_scan_truncated	absent	absent

Shim log, all three calls: cascade ok=1 completions=0 unlocked=0 otch_appends=0 roll_draws=0 failures=0 depth=0 name_unreadable=0.

If ok=0 on any call the transcription failed and every clean field below is meaningless. If unlocked is non-zero on a call with no completion, the collector is over-collecting and next_id will be too high — the opposite failure from lane P's, and worth more than a pass.

4.2 The five-turn continuation — the headline

Predicted: 15 calls, 15 compared, 0 divergent, tracecmp exit 0. Every one of the 22 divergent fields lane V recorded, gone. Field for field, against verify/results/compare/eventlive-b3-t1-5.json:

call 3 field	orig	ours before	ours now
events.next_id	7	6	7
node[132].cost_rp / state / turn_available	10000 / 2 / 4	INT_MAX / 0 / −1	10000 / 2 / 4
node[136].cost_rp / state / turn_available	16000 / 2 / 4	INT_MAX / 0 / −1	16000 / 2 / 4
node[142].cost_rp / state / turn_available	8000 / 2 / 4	INT_MAX / 0 / −1	8000 / 2 / 4
node[144].order / turn_researched	22 / 4	−1 / −1	22 / 4
observed_techs.bytes	484	440	484
rng	not divergent	—	still not divergent

call 9 field	orig	ours before	ours now
events.next_id	12	11	12
node[133].cost_rp / state / turn_available	8000 / 2 / 6	INT_MAX / 0 / −1	8000 / 2 / 6
node[142].order / turn_researched	23 / 6	−1 / −1	23 / 6
observed_techs.bytes	528	484	528
rng.left / next_index	374 / 250	375 / 249	374 / 250

And the arguments that say it is right for the right reason — these are the sharp ones, because they are consequences of the model rather than restatements of it:

• order_counter_in = 22 on call 3 and 23 on call 9. If it is 23 and 24, the counter was read after the original and every order will be off by one.
• roll_pending_in = false on call 3, true on call 9. This is forced by lane V's data: call 3 completes with no extra draw and call 9 with one, and the funded tech is the research target on both, so the pending byte is the only thing that can differ. If call 3 comes back true the model of the gate is wrong and call 3's rng will newly diverge.
• observed_techs_in = 10 on call 3, 11 on call 9.
• Shim log on call 3: completions=1 unlocked=3 otch_appends=1 roll_draws=0 failures=0. On call 9: completions=1 unlocked=1 otch_appends=1 roll_draws=1 failures=0. failures, depth and name_unreadable must be 0 everywhere.

4.3 Coverage and guards — what must NOT change

No Result region was added or removed, so the guards see exactly what they saw:

• undeclared writes: 10 in 2 calls, the same eight spans — player+0x10c:3, +0x110:3, +0x114:3, +0x124:3, +0x294:4, +0x196:1, +0x3b4:1, tree_header+0x20:1. +0x294 and +0x3b4 stay undeclared even though ours now reads them: reading an input is not modelling a write. tree_header+0x20 likewise — the counter is seeded and advanced in the model, but the live word is only ever the original's.
• The coverage verdict stays partial, with 8 unmodelled notes (was 6): the two event-text notes, replace mode, the tech effects, the RollResearchEvent branch behind the draw, the ObservedTech element's own fields, the order counter, and the log line.

4.4 The End-Turn oracle

(Autosave EndTurn).sav = bb4fd9ac89f41e3b… and (Autosave).sav = 978041acd168b56e…, unchanged. The cascade writes only scratch memory, so the running game must be byte-identical to lane R's and lane V's runs. If either hash moves, the cascade is writing live memory and the result must be thrown away regardless of how clean the compare is.

4.5 Zuul (zuul-turn5.sav, one more End Turn)

Less certain — this is new ground, and lane V's save has no completion yet. Predicted: the End Turn reaches a species-5 completion; that call shows species = 5, the generator advancing by two for the completion roll (the Zuul double roll) plus one more if roll_pending_in is true, and 0 divergences. The double roll and the completion path have never been exercised together.

4.6 What would falsify the model, and how it would show

if wrong	symptom
def+0xb0 is not the sweep-2 exclusion	too few nodes unlocked → state/turn_available diverge and next_id is short again
the prereq offsets (0x88 / +0x00 / +0x10, stride 8)	PrereqsMet false everywhere → children stop at state 1 → same symptom
TechDef_off_Name (0x40)	name_unreadable=1 in the log, no observed-tech write-back, observed_techs.bytes diverges as before
the ResT == def gate	roll_draws=0 on call 9 → rng diverges as before
the collector's predicate is too loose	next_id too high on a quiet call — a new divergence, not an old one

---

5. Outcome

Build unlock-405ba41-20260908T1026Z, cross-built on CT111 (/srv/re-lab/build/sots-engine-u, exports byte-identical to the real binkw32.dll), staged C:\SOTS\shimdist-u on VM140. Recipe shim.cfg.recapb3, unchanged.

The residual is gone. All three runs are 0 divergent, tracecmp exit 0.

run	calls	compared	diverged	exit
first End Turn (ref-turn2 → Launch → End Turn)	3	3	0	0
five-turn continuation (turn 2 → 7)	15	15	0	0
Zuul (zuul-turn5 → 10 End Turns, turn 5 → 15)	20	20	0	0

Reports verify/results/compare/unlock-b3-{t1,t1-5,zuul}.md in the RE repo; traces verify/traces/unlock-b3-*; shim log verify/results/shim/unlock-shim.log.

5.1 The oracle first

(Autosave EndTurn).sav = bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948, (Autosave).sav = 978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 — the same two hashes lane R and lane V recorded. The cascade does not perturb the running game. That check comes first because a clean compare from a build that moved the game would be worthless.

5.2 First End Turn — §4.1 held field for field

3 calls, 0 divergent, exit 0, and every predicted argument:

	predicted	observed
observed_techs_in	10 / 20 / 20	10 / 20 / 20 ✓
roll_pending_in (call 0)	false	false ✓
order_counter_in (call 0)	≥ 0, not −1	22 ✓
research_target	non-null	0x049812f8 ✓
observed_scan_failed / _truncated	absent	absent ✓
shim log, all 3 calls	ok=1 completions=0 unlocked=0 otch_appends=0 roll_draws=0 failures=0 depth=0 name_unreadable=0	exactly that ✓

The collector ran on all three and came back empty, which is the §3.2 check: it is only silent because nothing became available, not because it was not asked.

5.3 Five turns — §4.2 held on the deterministic half, and the rest was better than predicted

Call 3 is exactly the call lane P and lane V pinned, and every number matched:

call 3	predicted	observed
events.next_id	5 → 7	5 → 7 ✓
node[132] state / cost_rp / turn_available	0→2 / INT_MAX→10000 / −1→4	✓
node[136]	0→2 / INT_MAX→16000 / −1→4	✓
node[142]	0→2 / INT_MAX→8000 / −1→4	✓
node[144] state / turn_researched / order	3→4 / 4 / 22	✓
observed_techs.bytes	440 → 484	✓
order_counter_in	22	22 ✓
roll_pending_in	false	false ✓
observed_techs_in	10	10 ✓
shim log	completions=1 unlocked=3 otch_appends=1 roll_draws=0	exactly that ✓

Call 9 was not the call I predicted, and that is the more interesting result. From turn 5 the AI picked a different research target than in lane V's session — lane R's documented trap #2, and the reason the board calls the continuation a run and not the run. Lane V's call 9 completed tech 142 and unlocked one node; mine completed tech 9 and unlocked three:

node[9]  state 3->4  progress 3064->6000  turn_researched -1->6  order -1->23
node[3]  state 0->2  cost_rp INT_MAX->13000  turn_available -1->6
node[12] state 0->2  cost_rp INT_MAX->35000  turn_available -1->6
node[18] state 0->2  cost_rp INT_MAX->4000   turn_available -1->6

next_id 10 → 12 and observed_techs.bytes 484 → 528 as predicted, but the three costs — 13000, 35000, 4000 — appear in no earlier report and were not predicted by anyone. The model reproduced them with zero divergences on a case it had never seen. An unrehearsed instance is worth more than a rehearsed one, and it is the answer to "did the model just memorise lane V's numbers": it cannot have, because lane V's numbers are not what happened.

Call 12 then allocates to node 18 — the 4000-cost tech the cascade had just unlocked — which is the cascade's output feeding the next turn's budget.

5.4 One prediction missed: the guard count

Predicted 10 undeclared writes in 2 calls, observed 9 in 2 calls, and the span list is not the same one:

lane V:  player+0x10c:3 +0x110:3 +0x114:3 +0x124:3 +0x294:4 +0x196:1 +0x3b4:1  tree_header+0x20:1
lane U:  player+0x10c:3 +0x110:3 +0x114:3 +0x124:3 +0x294:4 +0x130:3           tree_header+0x20:1

This is the workload difference, not the code. player+0x3b4 is gone because the pending-roll byte was already 0 when the completing call ran, so the original never wrote it (see §5.5). +0x196 (design-option mask B) is replaced by +0x130 (PopMod) because a different tech completed and different tech effects fired. Both are OnTechResearched's writes and both are declared unmodelled. tree_header+0x20 is still there and still undeclared, exactly as §4.3 said it would be: ours seeds the counter and advances its own copy, it does not model the live word.

Coverage verdict partial with 8 unmodelled notes, as predicted.

5.5 The RollResearchEvent draw — modelled, inside the compare, but NOT exercised

This is the honest limit of the run, and it should not be read as more than it is.

roll_draws was 0 on all 35 compared calls. Not one completion in three runs had the pending byte still set. The reason is visible in the trace and it is a real finding: ResearchRollPending is normally consumed by ServerPlayer::ProcessTurn before ProcessResearch runs, because that call site fires when the progress ratio crosses a threshold — which is precisely the turns approaching completion. In the Zuul run roll_pending_in is true on the funded call for turns 8–12 and flips to false on turn 13, the turn before the tech completes on turn 14.

So lane V's call-9 draw was the rare case (a tech that jumped past the threshold to completion in one turn), not the normal one — which also explains lane R's "RNG matched 15 of 15". Three sessions, three different answers: 0 extra draws, 1, and 0.

What can be claimed: the draw is implemented at the right point in the stream (inside the owner callback, between this entry's roll and the next entry's), its two inputs are read pre-call and reported in every record (research_target, roll_pending_in), and region:rng compared clean on all 35 calls — so the model is not drawing a word the original does not. What cannot be claimed: that the branch has been seen to fire live. The boundary is inside the compare, but the compare has not yet had the chance to test it. It needs a workload where a tech goes from below the ProcessTurn threshold to complete in a single turn.

5.6 Zuul — the double roll and the completion path, together

20 calls, 0 divergent, exit 0, turn 5 → 15, two Zuul completions. Call 2 is the one lane V's save was one End Turn short of:

call 2  turn 7  species 5  alloc {144, 1376}   left 449 -> 447   next_index 175 -> 177
        node[144] state 3->4  turn_researched 7  order 21
        node[132] state 0->2  cost_rp INT_MAX->10000  turn_available 7
        node[136] state 0->2  cost_rp INT_MAX->16000  turn_available 7
        events.next_id 10 -> 12      observed_techs 396 -> 440

The generator advances by two (the species-5 double roll) and the completion cascade runs, in the same call, and ours reproduces the post-state bit for bit. Note the Zuul tree unlocks only 132 and 136 from tech 144 where the Human tree also unlocks 142 — a per-species tree, so this is an independent instance of the cascade and not a repeat of §5.3. A second completion follows on call 16 (turn 14, next_id 26 → 28, observed_techs 440 → 484).

5.7 What is now owed

• The RollResearchEvent draw needs a workload that fires it (§5.5).
• TechTree::GetProgressRatio (0x0057e950) — the COMPLETE/UNDERBUDGET split is still analogy. Count-neutral, so no run can see it.
• def+0xb0's write site. The byte is read and honoured; where it comes from is not read.
• The 0x00889d60 branch behind the research-event roll, the temperance sweep, PruneOldTurns, and replace mode on this hook — all still untouched.
• The ObservedTech element's own fields. ours decides the append; it does not build the element.