sots-re/findings/subsystems/observedtech-append.md

# `vector<ObservedTech> otch` — pinned, and the displacement scanner that pinned it

Lane X. Static/host only; VM140 was held by lane F and the game was never run.

Two things here: a general-purpose tool (`tools/x86disp.py`) for a class of cross-reference
Ghidra cannot index, and the `ServerPlayer+0x274` answer it was built to get.

---

## 1. The gap the tool fills

Ghidra indexes **immediate** operands. It does not index **ModRM displacements**. On a
41,411-function MSVC C++ binary `lea reg,[reg+disp]` is *the* idiom for taking the address of
a member — it is how every `std::vector`, every `std::string`, every embedded sub-object gets
passed to a method or a constructor. So `find-constant-uses 0x274` returned 13 unrelated hits
and none of them was the vector, and lane P correctly concluded the search was a limitation
rather than evidence of absence.

Lane E only found `EventStorage` at `+0x29c` because a whole accessor function happened to
exist (`lea eax,[ecx+0x29c]; ret`). We were blind to this whole reference class all campaign,
with ~1,600 classes still to go.

## 2. The tool — `tools/x86disp.py`

A real x86-32 **length decoder** (legacy prefixes, 1-/2-/3-byte opcodes, ModRM, SIB, disp8
sign-extended, disp32, every immediate form), swept from Ghidra's function starts so every
decode begins on a genuine instruction boundary. It indexes every memory operand that carries
a displacement, then answers "what code touches offset N off some object?" with the containing
function, instruction address, base register and decoded instruction.

Correctness points that would otherwise produce confident garbage, all handled:

* `mod=0,rm=5` and `mod=0,rm=4,sib.base=5` are **disp32 with no base** — an absolute global
  address, not a member offset. Excluded from member queries. (Without this, every
  `mov eax,[0x00a2bd88]` in the binary shows up as a "displacement".)
* `mod=1` disp8 is **sign-extended**: `-0x08` must not be reported as `+0xf8`.
* `mod=3` is a register operand with no displacement at all.
* `0x67` address-size prefix means 16-bit ModRM, a different layout entirely — refused rather
  than mis-decoded.

Commands: `build`, `query <disp>`, `cohort`, `func <va>`, `dis <va>`, `stats`, `brute`.

**Decode quality:**

```
functions swept : 41089
instructions    : 2174504
disp sites      : 612166
desyncs         : 70 (0.17% of functions)
code coverage   : 6142049/6142358 (100.0%)
```

Zero unknown-opcode desyncs — the opcode tables cover everything this binary contains. All 70
desyncs are truncations at a section/function boundary, not lost sync inside real code.

> Gotcha worth carrying: the first build clipped each sweep at `fva + Ghidra's sizeInBytes`
> and lost 11% of functions to mid-instruction truncation — Ghidra's `sizeInBytes` understates
> real bodies often enough to matter (it was cutting valid `mov esp,ebp; pop ebp; ret`
> epilogues in half). Sweeping to the **next function start** instead took coverage 89% → 100%.

The tool works off a local cache (`dumps/`, gitignored: the exe, the function list, the index)
so it never hammers the shared Ghidra box. `tools/cache_functions.py` pulls the function list
once.

## 3. Validation — it rediscovers what we already knew

Run before any new claim was made.

**GT1 — `ServerPlayer::GetEventStorage`, the one accessor we already had:**

```
$ uv run python3 tools/x86disp.py func 0x0080db00
ServerPlayer_GetEventStorage @ 0x0080db00 size 7
  0x0080db00  lea eax,[ecx+0x29c]   ServerPlayer_GetEventStorage+0x0  (8d819c020000)
```

**GT2 — lane E's two `OnTechResearched` sites, plus one it did not have:**

```
$ uv run python3 tools/x86disp.py query 0x29c --lea
 ProcessTurn  @0x00891340
  0x00891713  lea ecx,[esi+0x29c]   ProcessTurn+0x3d3          <-- NEW
 OnTechResearched  @0x00891790
  0x008919ab  lea ecx,[esi+0x29c]   OnTechResearched+0x21b     <-- known
  0x0089241d  lea ecx,[esi+0x29c]   OnTechResearched+0xc8d     <-- known
```

**GT3 — `EvNxID` at `ServerPlayer+0x2b0`:** 90 sites found; the `ServerPlayer` ones are present.
This one is also the honest illustration of the precision problem — see §5.

**GT4 — positive control for the cohort ranker.** Given the 50 `ServerPlayer` offsets already
in `struct-recovery.md`, `FUN_0087fac0` scores **50 co-hits out of 50** — it is the
`ServerPlayer` serializer, and nothing else in the binary comes close. A scanner that could
not surface that function from a bare offset query would not be worth using.

## 4. The answer: `sizeof(ObservedTech)` and the append site

### `sizeof(Game::ObservedTech) = 0x2c` (44 bytes) — verified, three independent ways

**(a)** `vector<ObservedTech>::operator=` at `0x00872380` divides the vector's byte span by a
constant using MSVC's magic-number sequence:

```
0x00872398  mov ecx,[edi+0x4]      ; src._Mylast
0x0087239b  mov edi,[edi]          ; src._Myfirst
0x0087239d  sub ecx,edi            ; byte span
0x0087239f  mov eax,0x2e8ba2e9
0x008723a4  imul ecx
0x008723a6  sar edx,3
```

`ceil(2^35 / 44) == 0x2e8ba2e9` **exactly**, and emulating the full sequence reproduces
n = 0,1,2,3,10,71,1000 from spans of 0,44,88,132,440,3124,44000. Divisor = 44, unambiguously.

**(b)** The same function then multiplies back by the literal stride:

```
0x0087243a  imul ecx,ecx,0x2c
0x0087243d  add ecx,[esi]          ; this->_Myfirst + n*44
0x00872441  mov [esi+0x4],ecx      ; this->_Mylast = ...
```

Same literal at `0x00872468` and at `0x007b735b` inside `push_back`.

**(c)** The linear search in the append function advances its iterator by `add edi,0x2c`
(`0x007ba257`).

This matches the on-disk lower bound *exactly* — 4 × `int` + one `0x1c` `std::string` = 44 —
so there is no padding slack anywhere in the element.

### The append site

```
FUN_007ba1a0 = RecordObservedTech   (direct callee of ServerPlayer::OnTechResearched 0x00891790)

0x007ba221  mov edi,[esi+0x274]     ; it = observer->otch._Myfirst
0x007ba227  cmp edi,[esi+0x278]     ; ... != _Mylast ?
0x007ba22d  je  0x007ba274          ; empty -> append
      loop: compare each element's name string (this = elem+0x0c, length = [elem+0x1c])
0x007ba257  add edi,0x2c            ; ++it   *** stride 44 ***
0x007ba25a  cmp edi,[esi+0x278]
0x007ba260  jne loop
0x007ba274  lea ecx,[ebp-0x3c]
0x007ba277  call 0x008562a0         ; ObservedTech::ObservedTech()  on the stack
0x007ba27e  push eax
0x007ba27f  lea ecx,[esi+0x274]     ; this = &player->otch      <<< THE APPEND
0x007ba288  call 0x007b7320         ; vector<ObservedTech>::push_back
```

`push_back` (`0x007b7320`) grows via `0x007b5820` when `_Mylast == _Myend` — **that realloc is
exactly why lane R's guard saw all three of `player+0x274/0x278/0x27c` move on a completion**,
rather than only `_Mylast`.

`RecordObservedTech` is called from `OnTechResearched` (`0x00891790`) and from `0x007be228`,
`0x007be4e1`, `0x007be535`. It is a **de-duplicating** append: it appends only if no existing
element already carries that tech name — worth knowing for the reimplementation, since a naive
`push_back` would diverge on a re-observation.

Neither `push_back` nor `operator=` is COMDAT-ambiguous: `0x007b7320` has exactly one caller
(`RecordObservedTech`, `ecx = player+0x274`) and `0x00872380` has two, both passing
`ServerPlayer+0x274`. `0x007b5820` **is** shared with `FUN_0086dec0` and may be a folded body,
so it is labelled `vector_44B_grow`, not as ObservedTech-specific.

### Element layout — as far as the evidence actually goes

The default ctor at `0x008562a0` writes vtable `0x00a2439c` at `+0x00`. RTTI:
COL `0x00a81c78` → type descriptor `0x00aeede4` → **`.?AVObservedTech@Game@@`**. So
`ObservedTech` is polymorphic and its first word is a vptr, not a data field — which the
on-disk shape does not tell you.

> **Superseded 2026-09-08 by lane S — see §9.** The table as first written called the
> embedded string 0x18 bytes and left `+0x08`, `+0x24`, `+0x28` unaccounted. `+0x24` is not a
> field: it is the string's own trailing allocator word. The corrected map is below; the
> original reasoning is kept in §9 because the way it went wrong is the useful part.

| offset | size | member | evidence |
|---|---|---|---|
| `+0x00` | 4 | vptr `0x00a2439c` | ctor writes it, RTTI-confirmed |
| `+0x04` | 2 | `uint16 otnF` (turn first observed) | `mov word [eax-0x28],cx` at `0x007ba2b0` (`eax` = `_Mylast`, element = `_Mylast-0x2c`), source `[ebx+0xc]`; tag from `ObservedTech::Write` |
| `+0x06` | 2 | `uint16 otnL` (turn last observed) | `mov word [eax-0x26],dx` at `0x007ba2c6`, **same source word** `[ebx+0xc]` — first sighting sets first == last |
| `+0x08` | 1 | `bool odet` | ctor stores a **byte** (`mov [esi+0x8],bl`, `0x008562f4`); copy-ctor copies a byte; serialised with `WriteBool`/`ReadBool` |
| `+0x0c..+0x27` | 0x1c | `std::string otch` (tech name) | object base `+0x0c`, MSVC `{_Bx[16] @0, _Mysize @0x10, _Myres @0x14, _Alval @0x18}`. Ctor writes `[+0x0c]=0`, `[+0x1c]=0`, `[+0x20]=0xf`; the search loop reads `[+0x1c]` as the length and calls compare with `this = +0x0c`; the post-append assign uses `lea ecx,[_Mylast-0x20]` = `+0x0c`. `+0x24` is `_Alval` — never read, never written, by anything |
| `+0x28` | 4 | `int owith` | ctor zeroes it; `ObservedTech::Write` emits it last |

`4 + 2 + 2 + 1(+3 pad) + 0x1c + 4 = 0x2c` exactly — sizeof is fully accounted for, with no
padding slack and no unaccounted field.

### Where the mapping came from — the serializer

`Game::ObservedTech`'s vftable `0x00a2439c` is the usual 3 slots
`{ [0] 0x00793610 scalar deleting dtor, [1] 0x00817c40 Read, [2] 0x00817cf0 Write }`.
`Write` enumerates the entire object, in order, and touches nothing else:

```
0x00817cff  movzx ecx,word [edi+0x04]   push 0xa2b38c "otnF"   -> stream vft+0x24 (int)
0x00817d0f  movzx ecx,word [edi+0x06]   push 0xa2b384 "otnL"   -> stream vft+0x24 (int)
0x00817d26  lea   eax,[edi+0x08]        push 0xa2b36c "odet"   -> WriteBool  0x008b9c20
0x00817d37  lea   ecx,[edi+0x0c]        push 0xa2b3e8 "otch"   -> WriteString 0x008b9d70
0x00817d46  mov   eax,[edi+0x28]        push 0xa2b364 "owith"  -> stream vft+0x24 (int)
```

`Read` (`0x00817c40`) is the exact mirror: `otnF`/`otnL` read as ints and stored back with
16-bit `mov word [ebx],ax`, `odet` through `ReadBool`, `otch` through `ReadString`, `owith`
reached as `add edi,0x28`. That matches the on-disk order `save_reader.py` already had
(`Otch = otnF otnL odet otch(string) owith`), which is a nice independent agreement between
the disassembly and the save oracle.

`Game::ObservedWeapon` (`Write` `0x00817bc0`, `Read` `0x00817b10`) is the same element shape
with tag `owep` (`0x00a2b3f0`) in place of `otch` — a second instance of the identical 0x2c
layout.

Lane P's `observed_techs` region byte delta remains a valid live cross-check and should come
back as exactly 44 per completion.

## 5. False-positive rate, honestly

Two different error rates, and the interesting one is not the one you'd expect.

**Decode-level error of the naive method is low.** A raw byte scan for `8D` + ModRM + the
displacement — the thing you'd write without a decoder — is mostly *right*:

| query | naive candidates | not actually an instruction | real sites the naive scan **missed** |
|---|---|---|---|
| `lea`, disp32 `0x274` | 19 | 0 (0.0%) | **80 of 99** |
| 11 common opcodes, disp32 `0x274` | 89 | 1 (1.1%) | 11 of 99 |
| `lea`, disp8 `0x14` | 828 | 1 (0.1%) | **13,784 of 14,611** |
| 11 common opcodes, disp8 `0x14` | 10,326 | 59 (0.6%) | 4,344 of 14,611 |

So the decoder's win is **recall, not decode precision**. A hand-written opcode set misses
80–94% of the real accesses, because a member is read, written, compared, and float-loaded far
more often than its address is taken, and you cannot enumerate those opcodes by hand.

**Class-level precision is the real problem, and it is poor.** The scanner knows the
displacement; it cannot know what class the base register holds. `query 0x274` returns 99 sites
in 45 functions and only about 6 of those functions are actually touching `ServerPlayer::otch`
— roughly **13% precision by function**. The honest way to state the value is as search-space
reduction: 41,411 functions → 45, about **900×**, down to a list a human reads in two minutes.

**The cohort ranker helps, and has a trap.** Scoring functions by how many *already-known*
offsets of the same class they touch pulls real class methods to the top (the serializer hits
50/50). But it would have **thrown away the correct answer**: `RecordObservedTech` touches only
`0x274` and `0x278` and nothing else on `ServerPlayer`, so any `--min>=1` filter drops it. Use
cohort as a ranker, never as a filter. This is now written into the tool's own docstring.

What actually closed the case was the plain `query 0x274` list **intersected with one call-graph
lookup** (`OnTechResearched`'s direct callees). Displacement scan for recall, call graph for
disambiguation — neither alone was enough.

## 6. Previously anonymous offsets — attribution results

The payoff was smaller than hoped, because most of the audit's anonymous offsets had already
been named by other lanes since the audit was written.

| offset | status before | after |
|---|---|---|
| `player+0x274/0x278/0x27c` | "vector grew, append site unknown" | **`RecordObservedTech+0xdf` (0x007ba27f)**, `sizeof` = 0x2c |
| `player+0x2b0` | already named by lane E | unchanged (`EvNxID`) |
| `TechTree+0x20` | already `TechTree_off_OrderCounter` | unchanged |
| `ServerPlayer+0x308` | already `ServerPlayer_off_NodeBore` | unchanged |
| `Budget+0x64` | harness-audit row 11: "the original writes it" | **not supported.** See below |

**`Budget+0x64` (harness-audit row 11) — a correction.** `ServerPlayer::ComputeBudget`
(`0x00863030`) writes its `Budget*` out-param through `esi`, and every such store lands in
`+0x00..+0x54` (the 22-int block). The only two `+0x64` accesses in the whole function are
**loads off a different base register** (`mov ecx,[eax+0x64]` at `0x0086328c`,
`mov edx,[eax+0x64]` at `0x0086335d`). There is no store to `Budget+0x64` in `ComputeBudget`.

Separately, `TechTree::ProcessResearch`'s `int* overbudget` fourth argument is **not**
`Budget+0x64`: its only caller is `ProcessTurn` (`0x00891340`) and the call at `0x008914a5`
passes `lea edx,[ebp-0x14]` — a stack local, consumed immediately after the call
(`mov eax,[ebp-0x14]; cmp eax,0; jle`).

This agrees with lane R, whose `budget_object` guard saw `Budget+0x64` change in **0 of 4284
calls**. Row 11 should be reclassified from "the original writes it and B1 never checked" to
"nothing has been shown to write it"; the remaining way to settle it is a write watchpoint on
that word, not another static search.

## 7. Verdict on the technique

Worth keeping, with its limits stated. It answered a question that had been parked, and the
`0x29c` validation turned up a `ProcessTurn` `EventStorage` site nobody had. But it is a
**recall** tool that produces a 20–100 line candidate list per offset, not an oracle: every
result still needs a call-graph or decompiler check before it is a fact. For the ~1,600
remaining classes the realistic workflow is `query <offset>` → read the list → confirm with one
cross-reference call.

## 8. Ghidra writeback

Labels: `RecordObservedTech` `0x007ba1a0`, `vector_ObservedTech_push_back` `0x007b7320`,
`ObservedTech_ctor` `0x008562a0`, `vector_ObservedTech_assign` `0x00872380`,
`vector_44B_grow` `0x007b5820`, `vftable_ObservedTech` `0x00a2439c`. Plate comments carrying
the stride evidence on the first four.

`addresses.json`: `ObservedTech_sizeof`, `ObservedTech_vftable`, `ObservedTech_off_Name`,
`RecordObservedTech`, `vector_ObservedTech_push_back`, `vector_ObservedTech_assign`,
`ObservedTech_ctor`, `vector_44B_grow`; `ServerPlayer_off_ObservedTechs` updated from
"NOT PINNED" to `verified`.

**Lane S round (2026-09-08).** Labels: `ObservedTech_Write` `0x00817cf0`, `ObservedTech_Read`
`0x00817c40`, `ObservedTech_scalar_deleting_dtor` `0x00793610`, `ObservedWeapon_Write`
`0x00817bc0`, `ObservedWeapon_Read` `0x00817b10`, `vector_ObservedTech_uninit_copy` `0x0079a150`,
`vector_string_find_by_name` `0x00699bd0`. Prototypes on the five class methods. Plate comments
carrying the full member map on the two serializers, the ctor, the dtor and the copy helper, and
the `sizeof(std::string) = 0x1c` fact on `Stream::WriteString` `0x008b9d70` and on the
`vector<string>` stride site `0x00699bd0` — the two places a future lane is most likely to look.
`addresses.json` +10 entries (`std_string_sizeof`, `ObservedTech_Read/_Write/_dtor/_copy_ctor`,
`ObservedTech_off_TurnFirst/_TurnLast/_Detected/_With`, `ObservedWeapon_Write`), 4 corrected.

---

## 9. `std::string` is 0x1c, not 0x18 — the correction, and why it mattered (lane S, 2026-09-08)

§4 above originally reported the embedded string as **0x18 bytes**, against the campaign-wide
`_Bx@0, _Mysize@0x10, _Myres@0x14, _Alval@0x18`, sizeof `0x1c`. Lane X flagged it as "worth
re-checking" rather than asserting it, which was the right call: **`0x1c` is correct, and it is
correct everywhere in this binary.** `ObservedTech+0x24` is the string's own trailing allocator
word, not a data member.

### Why the 0x18 reading looked right

Everything lane X observed was accurate. The string's *live* fields really do stop at `+0x14`
(`_Bx@0`, `_Mysize@0x10`, `_Myres@0x14`), because `_Alval` is `std::allocator<char>` — an empty
class. It occupies a word of the object but is never loaded or stored, so it is invisible to
any evidence based on **what the code touches**. Sizing a type from its accessed fields
undercounts it by exactly the tail padding. The same trap is live for `std::vector` in this
build, which is `{_Myfirst, _Mylast, _Myend, _Alval}` = `0x10` while only three words are ever
read (`events.md` already records the `_Alval` word at `EventStorage+0x10` and `+0x14`).

### The three things that settle it inside `ObservedTech`

Each is a *complete enumeration* of the object, which is the right instrument here — an
enumeration can show a field's **absence**, a touch-scan cannot.

1. **`ObservedTech::Write` `0x00817cf0`** serialises `+0x04, +0x06, +0x08, +0x0c, +0x28`.
   No `+0x24`. (`Read` `0x00817c40` mirrors it.)
2. **`ObservedTech_ctor` `0x008562a0`** initialises `+0x00, +0x04, +0x08, +0x0c(string), +0x28`.
   No `+0x24` — while zeroing every other scalar in the object.
3. **The copy constructor**, inlined at `0x0079a184` inside the vector's uninitialised-copy
   helper `FUN_0079a150`, copies `+0x04, +0x06, +0x08`, the string at `+0x0c`, and `+0x28`.
   No `+0x24`.

A 4-byte data member that the constructor, the copy constructor and the serializer all ignore
is not a data member.

### Binary-wide check — `tools/strfootprint.py`

One class is an anecdote, so the same question was put to the whole binary. Every serializer
hands member pointers to the `Mars::Stream` primitive helpers with a fixed idiom
(`lea r,[base+disp]; push r; push tag; push stream; call helper`), so the scanner recovers
`(base, disp, tag)` for every string site and then asks: does the same function touch any other
offset inside `(N, N+0x1c)` off the same base register?

```
string helper call sites            : 241
  resolved to a class member offset : 65
  stack temporaries (ebp/esp base)  : 30
  offset not reached by a plain lea : 146

members with a sibling inside (N, N+0x1c) : 0

gap from a string member to the next member on the same base:
  +0x1c   : 51
  +0x20   : 1
```

**65 string members across every serializer in the exe, zero collisions, and 51 of the 52
measurable gaps are exactly `0x1c`.** The single `+0x20` is `StrategyServer::KeyPath` at
`+0x134`, whose *Read* side gives `+0x1c` to `+0x150` — the writer simply skips a member.
There is no `0x18` instantiation, no empty-base-optimised variant, and no game-local string
class. One layout, `0x1c`.

Two exclusions matter or the scan reports noise, and both are why a naive version of this
would have "confirmed" 0x18:

* **`ebp`/`esp` bases are stack temporaries, not members.** A local string at `[ebp-0x2c]`
  shows accesses at `-0x1c` and `-0x18` — which read exactly like two sibling fields inside
  the span. All 30 such sites are excluded.
* **`N+0x10` and `N+0x14` are the string's own `_Mysize`/`_Myres`**, touched inline whenever
  the compiler expands the `_Myres >= 16 ? _Ptr : _Buf` test at a call site.

### Independent corroboration outside the scan

* `FUN_00699bd0` walks a `vector<std::string>` doing the SSO test at `[esi+0x14]`/`[esi]` —
  element base *is* string base — and advances `add esi,0x1c` (`0x00699c29`). The stride of a
  vector of strings **is** `sizeof(std::string)`.
* `EventStorage::PostEvent` `0x008862b0` takes two by-value `std::string`s at `[ebp+0x08]` and
  `[ebp+0x24]` (spacing `0x1c`) and `RET 0x4c` = `2*0x1c + 5*4`.
* `MoraleEvent` is 0x50 bytes with its name string at `+0x34`: `0x34 + 0x1c = 0x50` exactly.

### Blast radius — what actually had to change

Nothing in any recovered struct table. Every string-bearing layout in `struct-recovery.md`,
`save-editor-structs.md`, `events.md` and `schema-gaps-resolved.md` already used `0x1c` spans
and `0x1c` inter-field gaps, and the binary agrees with all of them. `ServerPlayer::pswd` at
`0x2dc..0x2f7` — the row §4 asked to re-check — is **correct**: `Write` `0x008563e0` puts the
next member exactly `0x1c` above it. The corrections were confined to three prose statements
that had propagated the 0x18 number (`loader-prototypes.md` conventions line, the
`GlobalConst_ParseString` prototype in `addresses.json`, and §4 here) plus one transposition in
`struct-recovery.md` §0 that said `size@0x14, res@0x18` where the verified offsets are
`0x10`/`0x14`.

The lesson worth carrying, and the reason this was worth chasing rather than reconciling: **do
not size a struct member from the offsets the code touches.** Trailing empty-allocator words in
this build's STL are invisible to a touch-scan and cost exactly 4 bytes every time. Size types
from an enumeration — a serializer, a constructor, a copy constructor, or a container stride.