sots-engine/docs/U-unlock.md

# U — the unlock cascade, and the prediction for the run that checks it

Lane P deliberately did **not** post `EVENT_TECHS_UNLOCKED`. It could have — the trigger was
pinned — but evaluating it needs the cascade `ours` does not run, so it handed
`PostResearchPassEvents` a **`nullptr`** unlock list ("no list", distinct from an empty one) and
predicted the consequence: `next_id` short by exactly 1 on every completion call. Lane V measured
that live, twice, exactly.

This lane removes the residual by running the cascade, not by posting an event when something
completes. The distinction matters for one reason and lane P named it: the shortcut would score
perfectly on this save and be wrong the first time a completion unlocks nothing.

**Everything in §4 was written before the build was staged on VM140.** §5 is the outcome.

---

## 1. What the cascade actually is

Read out of the instruction stream by this lane: `TechTree::SetResearched` (0x00581e10, whole
function), `TechTree::PrereqsMet` (0x0057d8e0, whole function), the tail collector inside
`TechTree::ProcessResearch` (0x00587cc3), and the head of `ServerPlayer::OnTechResearched`
(0x00891790). Details and the decompiled shape are in sots-re
`findings/subsystems/setresearched-cascade.md`; the module is `src/game/sim/techgraph.{h,cpp}`.

```
SetResearched(tree, def, flags):
    n = tree->nodes[def->techId]
    if (!n)             -> nothing (the flags&8 re-lookup path is not on the research path)
    if (n->state == 4)  -> return true, writing NOTHING, not even the order stamp
    if (!(flags & 2) && !PrereqsMet(def + 0x88)) -> nothing

    n->state          = 4
    n->turnResearched = owner ? ModCount : 0
    n->order          = tree->orderCounter++
    owner->OnTechResearched(def, (flags >> 2) & 1)      # flags = 2 from ProcessResearch: NOT silent

    for each edge in n->children:                       # sweep 1
        c = tree->nodes[edge->childDef->techId]
        if (c->state == 0) c->state = 1
        c->costRP = min(c->costRP, edge->costRP)         # signed, against an INT_MAX sentinel

    for each node m in tree->nodes:                     # sweep 2
        if (!m || m->def->[0xb0]) continue
        p = tree->nodes[m->def->techId]
        if (p && p->state == 1 && PrereqsMet(p->def + 0x88)):
            m->state = 2
            if (m->turnAvailable == -1 && owner) m->turnAvailable = ModCount
        if (m->state == 2 && Cost(m) == 0) SetResearched(m->def, flags)     # recursion
```

Four properties are easy to get wrong and each is pinned by a host test:

* **`turnAvailable` is sticky.** It is written only when it still reads −1. A node that returns to
  state 2 in a later turn keeps its *first* availability turn — which is the whole reason
  `EVENT_TECHS_UNLOCKED` can be computed from node state at all, and the reason a tech is never
  announced twice.
* **The cost minimum is signed, against `INT_MAX`.** The first researched parent sets the child's
  cost; later parents can only lower it.
* **`PrereqsMet` is an AND over groups, each group an OR over techs** — and a group with *zero*
  entries makes the whole test **fail**, because the inner loop cannot break and the outer one
  then exits with that group uncounted. A "vacuously true" reading gets this backwards. Zero
  groups, by contrast, is satisfied.
* **Both sweeps and the collector re-resolve `tree->nodes[node->def->techId]`** and test *that*
  node's state, not the iterated node's. For a well-formed tree it comes back to the same node;
  it is reproduced because the code has it.

`def+0xb0` is a byte that excludes a node from sweep 2 entirely. Its write site was **not** found
(`MasterTechTree::ParseTech` shows no reference to 0xb0), so `addresses.d/lane-u.json` names it
for what it *does* — `TechDef_off_NoAutoAvailable` — and records the `unlock_explicitly` tech-file
keyword as a hypothesis, not a fact. Nothing in this lane depends on which it is: the byte is read
from the live def either way.

### The tail collector

```
if (tree->owner) {
    turn = ModCount
    for each node n: if (n && n->def && (p = tree->nodes[n->def->techId]) &&
                        p->state == 2 && n->turnAvailable == turn) collect(n)
    if (!collected.empty()) PostEvent(EVENT_TECHS_UNLOCKED, ...)
}
```

Note the asymmetry — the state test is on the self-resolved node, the turn test on the iterated
one — and note that it runs **once, after the whole per-node loop and after the decay sweep**.

### The two halves of `OnTechResearched` this hook can see

```
OnTechResearched(player, def, silent):
    RecordObservedTech(...)                                  # FIRST statement, unconditional
    if (player->ResT == def) {
        if (player->ResearchRollPending) RollResearchEvent(player)   # exactly one NextFloat
        player->ResearchRollPending = 0
        player->ResT = 0
    }
    if (!silent) { ... EVENT_RESEARCH_COMPLETE / _UNDERBUDGET ... }
```

`RecordObservedTech` **de-duplicates by tech name**, so "the vector did not grow" is a real
outcome and not a failure. `RollResearchEvent` draws one `NextFloat` unconditionally and only then
tests it against the odds; clearing `ResT` is what makes a *second* completion in the same pass
draw nothing. This is lane V's call-9 deviation, and it is now inside the compare rather than
declared out of scope.

---

## 2. What changed

| file | change |
|---|---|
| `src/game/sim/techgraph.{h,cpp}` (new) | the pure cascade: `TechPrereqsMet`, `SetResearched`, `CollectNewlyAvailable` |
| `src/game/sim/research.{h,cpp}` | one optional parameter: a completion hook called *inside* the allocation loop, where the original calls `SetResearched` — before the next entry's roll and before the decay sweep |
| `src/shim/hooks/research.{h,cpp}` | the pre-call scan the cascade needs, the live→graph transcription, the cascade wiring, four more node write-backs, the unlock list, the observed-tech append, the RNG draw, the rewritten coverage |
| `tests/game_sim/test_techgraph.cpp` (new) | 13 cases, 92 checks |
| `ghidra/addresses.d/lane-u.json` (sots-re) | 12 new entries; header regenerated 615 → 627 |

`tools/clean_room_check.sh` — **OK**. Host `ctest` — **34/34** (was 33/33; `game_sim_techgraph` is
new). Run as separate commands. The shim TU is syntax-checked on the host
(`-std=c++17 -Wall -Wextra -fsyntax-only`, clean) and cross-built on CT111 before deploy.

### The four pre-call reads, and why each one is a trap

`ours` runs **after** the original in compare mode. Four inputs the cascade needs are things the
original changes during the call, and each of them reads back a *plausible* wrong answer:

| input | read after the original would give | consequence |
|---|---|---|
| `TechTree+0x20` order counter | the *next* order | every `node.order` off by one per completion |
| `ServerPlayer+0x294` (`ResT`) | 0 — cleared by the callback | the extra RNG draw never modelled |
| `ServerPlayer+0x3b4` (pending roll) | 0 — cleared in the same block | same |
| `ServerPlayer+0x274` observed-tech names | the completing tech already present | the append modelled as a de-duplication, agreeing with a count it did not compute |

All four are taken in `describe_args`, which `hook.h` calls immediately before the original, and
all four are reported as arguments (`order_counter_in`, `research_target`, `roll_pending_in`,
`observed_techs_in`) so a run can be audited without trusting `ours`.

### What is modelled, and what is deliberately not

The cascade runs in **compare mode only**. In replace mode every pointer is live game memory, and
applying half of `OnTechResearched` — the observed-tech append and the research-event roll, but
not the ninety-odd tech-effect field writes — would leave the player in a state no code path
produces. Not running it leaves a player missing a cascade, which is a smaller and already
declared lie. The same reasoning lane P used for the event counts.

`TechTree::Cost` is called, not re-derived: it is the game's own read-only function, and the
cascade needs it on the node *as it stands after the child-cost sweep*. Guessing the cost
multiplier would put a second unknown inside the thing being measured.

The unlock list is still an **input** to `PostResearchPassEvents`, and `nullptr` still means "this
caller could not compute it" — now reached when the graph fails to transcribe or the tail's
`tree->owner != 0` gate is closed, rather than always. An empty vector means "computed, and
nothing became available", which posts nothing. Keeping those apart is what stops a failure to
read the tree from scoring as a correct silence.

---

## 3. Why a clean result here would not be vacuous

A hook that silently compares nothing is the failure mode to fear when a clean run is *expected*,
and this lane is exactly that situation. Three things make it hard here:

1. **The expected values are non-trivial and known in advance.** `cost_rp` must come back 10000,
   16000 and 8000 on three specific nodes, `turn_available` 4, `order` 22 and 23. A model that
   computed nothing leaves them at `INT_MAX`, 0 and −1 — which is precisely what lane V's report
   shows today. There is no "do nothing" answer that passes.
2. **The collector runs on all fifteen calls, not just the two completions.** On the thirteen
   calls with no completion it must come back *empty*; if the transcription were wrong in a way
   that over-collected, `next_id` would be **too high** and those calls would newly diverge. The
   quiet calls are as much of a check as the loud ones.
3. **The shim log prints the counters.** One line per call:
   `cascade ok=… completions=… unlocked=… otch_appends=… roll_draws=… failures=… depth=…
   name_unreadable=…`. A clean compare with all of those at zero on a completion call would be a
   clean compare of nothing, and it would be visible.

---

## 4. The prediction

Written before the build was staged. Recipe unchanged: `shim.cfg.recapb3`, `ref-turn2.sav` →
Launch → End Turn, then four more End Turns in the same session.

### 4.1 First End Turn — nothing should change

Turn 1 of `ref-turn2` completes no tech (lane V: `observed_techs.bytes` unchanged on all three
calls), so the cascade never fires. **3 calls, 3 compared, 0 divergent, `tracecmp` exit 0** —
identical to lane V's result, and that is the point: the new code must be inert when nothing
completes.

| new argument | call 0 | calls 1, 2 |
|---|---|---|
| `observed_techs_in` | **10** (lane V measured 440 bytes = 10 × 44) | **20** (880 bytes) |
| `research_target` | non-null (the player has a funded target) | either |
| `roll_pending_in` | **false** | false |
| `order_counter_in` | ≥ 0, and not −1 | ≥ 0 |
| `observed_scan_failed` / `observed_scan_truncated` | **absent** | absent |

Shim log, all three calls: `cascade ok=1 completions=0 unlocked=0 otch_appends=0 roll_draws=0
failures=0 depth=0 name_unreadable=0`.

If `ok=0` on any call the transcription failed and every clean field below is meaningless. If
`unlocked` is non-zero on a call with no completion, the collector is over-collecting and
`next_id` will be too high — the opposite failure from lane P's, and worth more than a pass.

### 4.2 The five-turn continuation — the headline

**Predicted: 15 calls, 15 compared, 0 divergent, `tracecmp` exit 0.** Every one of the 22
divergent fields lane V recorded, gone. Field for field, against
`verify/results/compare/eventlive-b3-t1-5.json`:

| call 3 field | orig | ours before | ours now |
|---|---|---|---|
| `events.next_id` | 7 | 6 | **7** |
| `node[132].cost_rp / state / turn_available` | 10000 / 2 / 4 | INT_MAX / 0 / −1 | **10000 / 2 / 4** |
| `node[136].cost_rp / state / turn_available` | 16000 / 2 / 4 | INT_MAX / 0 / −1 | **16000 / 2 / 4** |
| `node[142].cost_rp / state / turn_available` | 8000 / 2 / 4 | INT_MAX / 0 / −1 | **8000 / 2 / 4** |
| `node[144].order / turn_researched` | 22 / 4 | −1 / −1 | **22 / 4** |
| `observed_techs.bytes` | 484 | 440 | **484** |
| `rng` | not divergent | — | still not divergent |

| call 9 field | orig | ours before | ours now |
|---|---|---|---|
| `events.next_id` | 12 | 11 | **12** |
| `node[133].cost_rp / state / turn_available` | 8000 / 2 / 6 | INT_MAX / 0 / −1 | **8000 / 2 / 6** |
| `node[142].order / turn_researched` | 23 / 6 | −1 / −1 | **23 / 6** |
| `observed_techs.bytes` | 528 | 484 | **528** |
| `rng.left / next_index` | 374 / 250 | 375 / 249 | **374 / 250** |

And the arguments that say it is right for the right reason — these are the sharp ones, because
they are consequences of the model rather than restatements of it:

* `order_counter_in` = **22** on call 3 and **23** on call 9. If it is 23 and 24, the counter was
  read after the original and every `order` will be off by one.
* `roll_pending_in` = **false on call 3, true on call 9**. This is forced by lane V's data: call 3
  completes with no extra draw and call 9 with one, and the funded tech is the research target on
  both, so the pending byte is the only thing that can differ. If call 3 comes back `true` the
  model of the gate is wrong and call 3's `rng` will newly diverge.
* `observed_techs_in` = **10** on call 3, **11** on call 9.
* Shim log on call 3: `completions=1 unlocked=3 otch_appends=1 roll_draws=0 failures=0`.
  On call 9: `completions=1 unlocked=1 otch_appends=1 roll_draws=1 failures=0`.
  `failures`, `depth` and `name_unreadable` must be 0 everywhere.

### 4.3 Coverage and guards — what must NOT change

No Result region was added or removed, so the guards see exactly what they saw:

* **undeclared writes: 10 in 2 calls**, the same eight spans — `player+0x10c:3`, `+0x110:3`,
  `+0x114:3`, `+0x124:3`, `+0x294:4`, `+0x196:1`, `+0x3b4:1`, `tree_header+0x20:1`. `+0x294` and
  `+0x3b4` stay undeclared even though `ours` now *reads* them: reading an input is not modelling
  a write. `tree_header+0x20` likewise — the counter is seeded and advanced in the model, but the
  live word is only ever the original's.
* The coverage verdict stays **partial**, with **8** unmodelled notes (was 6): the two event-text
  notes, replace mode, the tech effects, the `RollResearchEvent` branch behind the draw, the
  ObservedTech element's own fields, the order counter, and the log line.

### 4.4 The End-Turn oracle

`(Autosave EndTurn).sav` = `bb4fd9ac89f41e3b…` and `(Autosave).sav` = `978041acd168b56e…`,
unchanged. The cascade writes only scratch memory, so the running game must be byte-identical to
lane R's and lane V's runs. **If either hash moves, the cascade is writing live memory and the
result must be thrown away regardless of how clean the compare is.**

### 4.5 Zuul (`zuul-turn5.sav`, one more End Turn)

Less certain — this is new ground, and lane V's save has no completion yet. Predicted: the
End Turn reaches a species-5 completion; that call shows `species = 5`, the generator advancing by
**two** for the completion roll (the Zuul double roll) plus **one more** if `roll_pending_in` is
true, and **0 divergences**. The double roll and the completion path have never been exercised
together.

### 4.6 What would falsify the model, and how it would show

| if wrong | symptom |
|---|---|
| `def+0xb0` is not the sweep-2 exclusion | too few nodes unlocked → `state`/`turn_available` diverge and `next_id` is short again |
| the prereq offsets (0x88 / +0x00 / +0x10, stride 8) | `PrereqsMet` false everywhere → children stop at state 1 → same symptom |
| `TechDef_off_Name` (0x40) | `name_unreadable=1` in the log, no observed-tech write-back, `observed_techs.bytes` diverges as before |
| the `ResT == def` gate | `roll_draws=0` on call 9 → `rng` diverges as before |
| the collector's predicate is too loose | `next_id` **too high** on a quiet call — a new divergence, not an old one |

---

## 5. Outcome

Build `unlock-405ba41-20260908T1026Z`, cross-built on CT111
(`/srv/re-lab/build/sots-engine-u`, exports byte-identical to the real `binkw32.dll`), staged
`C:\SOTS\shimdist-u` on VM140. Recipe `shim.cfg.recapb3`, unchanged.

**The residual is gone. All three runs are 0 divergent, `tracecmp` exit 0.**

| run | calls | compared | diverged | exit |
|---|---|---|---|---|
| first End Turn (`ref-turn2` → Launch → End Turn) | 3 | 3 | **0** | 0 |
| five-turn continuation (turn 2 → 7) | 15 | 15 | **0** | 0 |
| Zuul (`zuul-turn5` → 10 End Turns, turn 5 → 15) | 20 | 20 | **0** | 0 |

Reports `verify/results/compare/unlock-b3-{t1,t1-5,zuul}.md` in the RE repo; traces
`verify/traces/unlock-b3-*`; shim log `verify/results/shim/unlock-shim.log`.

### 5.1 The oracle first

`(Autosave EndTurn).sav` = `bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948`,
`(Autosave).sav` = `978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921` — **the same two
hashes lane R and lane V recorded**. The cascade does not perturb the running game. That check
comes first because a clean compare from a build that moved the game would be worthless.

### 5.2 First End Turn — §4.1 held field for field

3 calls, 0 divergent, exit 0, and every predicted argument:

| | predicted | observed |
|---|---|---|
| `observed_techs_in` | 10 / 20 / 20 | **10 / 20 / 20** ✓ |
| `roll_pending_in` (call 0) | false | **false** ✓ |
| `order_counter_in` (call 0) | ≥ 0, not −1 | **22** ✓ |
| `research_target` | non-null | `0x049812f8` ✓ |
| `observed_scan_failed` / `_truncated` | absent | absent ✓ |
| shim log, all 3 calls | `ok=1 completions=0 unlocked=0 otch_appends=0 roll_draws=0 failures=0 depth=0 name_unreadable=0` | exactly that ✓ |

The collector ran on all three and came back empty, which is the §3.2 check: it is only silent
because nothing became available, not because it was not asked.

### 5.3 Five turns — §4.2 held on the deterministic half, and the rest was better than predicted

Call 3 is exactly the call lane P and lane V pinned, and every number matched:

| call 3 | predicted | observed |
|---|---|---|
| `events.next_id` | 5 → **7** | 5 → **7** ✓ |
| `node[132]` state / cost_rp / turn_available | 0→2 / INT_MAX→10000 / −1→4 | ✓ |
| `node[136]` | 0→2 / INT_MAX→16000 / −1→4 | ✓ |
| `node[142]` | 0→2 / INT_MAX→8000 / −1→4 | ✓ |
| `node[144]` state / turn_researched / order | 3→4 / 4 / **22** | ✓ |
| `observed_techs.bytes` | 440 → **484** | ✓ |
| `order_counter_in` | **22** | **22** ✓ |
| `roll_pending_in` | **false** | **false** ✓ |
| `observed_techs_in` | **10** | **10** ✓ |
| shim log | `completions=1 unlocked=3 otch_appends=1 roll_draws=0` | exactly that ✓ |

**Call 9 was not the call I predicted, and that is the more interesting result.** From turn 5 the
AI picked a different research target than in lane V's session — lane R's documented trap #2, and
the reason the board calls the continuation *a* run and not *the* run. Lane V's call 9 completed
tech 142 and unlocked one node; mine completed **tech 9** and unlocked **three**:

```
node[9]  state 3->4  progress 3064->6000  turn_researched -1->6  order -1->23
node[3]  state 0->2  cost_rp INT_MAX->13000  turn_available -1->6
node[12] state 0->2  cost_rp INT_MAX->35000  turn_available -1->6
node[18] state 0->2  cost_rp INT_MAX->4000   turn_available -1->6
```

`next_id` 10 → **12** and `observed_techs.bytes` 484 → **528** as predicted, but the three costs —
13000, 35000, 4000 — appear in no earlier report and were not predicted by anyone. The model
reproduced them with zero divergences on a case it had never seen. An unrehearsed instance is
worth more than a rehearsed one, and it is the answer to "did the model just memorise lane V's
numbers": it cannot have, because lane V's numbers are not what happened.

Call 12 then allocates to node 18 — the 4000-cost tech the cascade had just unlocked — which is
the cascade's output feeding the next turn's budget.

### 5.4 One prediction missed: the guard count

Predicted **10 undeclared writes in 2 calls**, observed **9 in 2 calls**, and the span list is not
the same one:

```
lane V:  player+0x10c:3 +0x110:3 +0x114:3 +0x124:3 +0x294:4 +0x196:1 +0x3b4:1  tree_header+0x20:1
lane U:  player+0x10c:3 +0x110:3 +0x114:3 +0x124:3 +0x294:4 +0x130:3           tree_header+0x20:1
```

This is the workload difference, not the code. `player+0x3b4` is gone because the pending-roll
byte was **already 0** when the completing call ran, so the original never wrote it (see §5.5).
`+0x196` (design-option mask B) is replaced by `+0x130` (`PopMod`) because a different tech
completed and different tech effects fired. Both are `OnTechResearched`'s writes and both are
declared unmodelled. `tree_header+0x20` is still there and still undeclared, exactly as §4.3 said
it would be: `ours` seeds the counter and advances its own copy, it does not model the live word.

Coverage verdict `partial` with **8** unmodelled notes, as predicted.

### 5.5 The `RollResearchEvent` draw — modelled, inside the compare, but NOT exercised

This is the honest limit of the run, and it should not be read as more than it is.

`roll_draws` was **0 on all 35 compared calls**. Not one completion in three runs had the pending
byte still set. The reason is visible in the trace and it is a real finding: `ResearchRollPending`
is normally consumed by `ServerPlayer::ProcessTurn` *before* `ProcessResearch` runs, because that
call site fires when the progress ratio crosses a threshold — which is precisely the turns
approaching completion. In the Zuul run `roll_pending_in` is `true` on the funded call for turns
8–12 and flips to **false** on turn 13, the turn before the tech completes on turn 14.

So lane V's call-9 draw was the *rare* case (a tech that jumped past the threshold to completion
in one turn), not the normal one — which also explains lane R's "RNG matched 15 of 15". Three
sessions, three different answers: 0 extra draws, 1, and 0.

What can be claimed: the draw is implemented at the right point in the stream (inside the owner
callback, between this entry's roll and the next entry's), its two inputs are read pre-call and
**reported in every record** (`research_target`, `roll_pending_in`), and `region:rng` compared
clean on all 35 calls — so the model is not drawing a word the original does not. What cannot be
claimed: that the branch has been seen to fire live. **The boundary is inside the compare, but the
compare has not yet had the chance to test it.** It needs a workload where a tech goes from below
the ProcessTurn threshold to complete in a single turn.

### 5.6 Zuul — the double roll and the completion path, together

20 calls, 0 divergent, exit 0, turn 5 → 15, **two** Zuul completions. Call 2 is the one lane V's
save was one End Turn short of:

```
call 2  turn 7  species 5  alloc {144, 1376}   left 449 -> 447   next_index 175 -> 177
        node[144] state 3->4  turn_researched 7  order 21
        node[132] state 0->2  cost_rp INT_MAX->10000  turn_available 7
        node[136] state 0->2  cost_rp INT_MAX->16000  turn_available 7
        events.next_id 10 -> 12      observed_techs 396 -> 440
```

The generator advances by **two** (the species-5 double roll) *and* the completion cascade runs,
in the same call, and `ours` reproduces the post-state bit for bit. Note the Zuul tree unlocks
only 132 and 136 from tech 144 where the Human tree also unlocks 142 — a per-species tree, so this
is an independent instance of the cascade and not a repeat of §5.3. A second completion follows on
call 16 (turn 14, `next_id` 26 → 28, `observed_techs` 440 → 484).

### 5.7 What is now owed

* The `RollResearchEvent` draw needs a workload that fires it (§5.5).
* `TechTree::GetProgressRatio` (0x0057e950) — the COMPLETE/UNDERBUDGET split is still analogy.
  Count-neutral, so no run can see it.
* `def+0xb0`'s write site. The byte is read and honoured; where it comes from is not read.
* The `0x00889d60` branch behind the research-event roll, the temperance sweep, `PruneOldTurns`,
  and replace mode on this hook — all still untouched.
* The ObservedTech element's own fields. `ours` decides the append; it does not build the element.