From 460cb7ca2b91f89ea69ca1e604c1dd95ebdf8e32 Mon Sep 17 00:00:00 2001 From: alex Date: Tue, 8 Sep 2026 04:44:18 -0400 Subject: [PATCH] lane X: x86 displacement xref scanner; pin sizeof(ObservedTech) and its append site Ghidra does not index ModRM displacements, so `lea reg,[reg+disp]` -- the MSVC idiom for taking a member's address -- is invisible to find-constant-uses. That blind spot parked ServerPlayer+0x274 and covers every non-trivial member of the ~1,600 classes still to map. tools/x86disp.py: full x86-32 length decoder (prefixes, 1/2/3-byte opcodes, ModRM, SIB, sign-extended disp8, disp32, every immediate form) swept from Ghidra's 41,089 function starts so decodes begin on real instruction boundaries. 2,174,504 instructions, 612,166 displacement sites, 100.0% code coverage, 70 desyncs (0.17%), zero unknown opcodes. Excludes no-base disp32 forms (mod=0/rm=5, sib.base=5) which are absolute globals, not member offsets. Commands: build/query/cohort/func/dis/stats/brute. Works off a gitignored local cache in dumps/ rather than hammering CT111. Validated before use: re-finds lea eax,[ecx+0x29c] in ServerPlayer::GetEventStorage (0x0080db00) and both known OnTechResearched +0x29c sites, plus a new one in ProcessTurn. Positive control: the ServerPlayer serializer scores 50/50 known offsets. sizeof(Game::ObservedTech) = 0x2c (44), proven three ways: the exact magic divide 0x2e8ba2e9 sar 3 at 0x0087239f, imul reg,reg,0x2c at 0x0087243a and 0x007b735b, and the search stride add edi,0x2c at 0x007ba257. Append site: RecordObservedTech+0xdf (0x007ba27f) -- lea ecx,[player+0x274]; call vector_ObservedTech_push_back 0x007b7320 RecordObservedTech (0x007ba1a0) is a direct callee of OnTechResearched and de-duplicates by tech name before appending. The realloc through 0x007b5820 is why lane R's guard saw all three vector words move. Element carries a vptr (RTTI .?AVObservedTech@Game@@) at +0 and a 0x18-byte std::string at +0x0c; the four on-disk ints map onto +0x04/+0x06/+0x08/+0x24/+0x28 in an order this read does NOT determine, and is not guessed. Also corrects harness-audit row 11: ComputeBudget has no store to Budget+0x64 (its only +0x64 accesses are loads off a different base), and ProcessResearch's int* overbudget arg is a ProcessTurn stack local, not Budget+0x64. Agrees with lane R's guard seeing 0 changes in 4284 calls. Honest limits are recorded in the note and the board: this is a recall tool, not an oracle. Class-level precision at 0x274 is ~13% by function, i.e. a ~900x search-space cut that still needs one call-graph check. Cohort ranking must not be used as a hard filter -- it would have discarded the correct answer here. Ghidra writeback: labels + plate comments on RecordObservedTech, vector_ObservedTech_push_back, ObservedTech_ctor, vector_ObservedTech_assign, vector_44B_grow, vftable_ObservedTech. --- campaign/board.md | 5 +- findings/subsystems/golden-trace-recapture.md | 5 + findings/subsystems/observedtech-append.md | 278 ++++++ findings/subsystems/setresearched-cascade.md | 10 +- ghidra/addresses.json | 70 +- ghidra/generated/sots_addresses.h | 20 +- tools/cache_functions.py | 57 ++ tools/x86disp.py | 798 ++++++++++++++++++ 8 files changed, 1235 insertions(+), 8 deletions(-) create mode 100644 findings/subsystems/observedtech-append.md create mode 100644 tools/cache_functions.py create mode 100644 tools/x86disp.py diff --git a/campaign/board.md b/campaign/board.md index dd5c476..253d8af 100644 --- a/campaign/board.md +++ b/campaign/board.md @@ -68,7 +68,7 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`). | RollResearchEvent draw (behavioural) | verify | verified | high | 100% | 2026-09-08 | **CLOSED by lane R.** On turn 6 (`IND_TRKSTL`, tech 10094) `research_roll_pending` was true going in; the original drew exactly one word (rng `left` 375->374, `next_index` 249->250) and cleared the flag, and `ours` reproduced both bit-for-bit on the scratch generator. 0 divergences on that call. Needs 5 End Turns from ref-turn2 to reach - the reference turn has no completion at all | | golden-trace recapture (post-guards) | verify | verified | high | 100% | 2026-09-08 | DONE on the live game, build `recap-7584bad-20260908T0615Z` (NO source change needed - the audit's machinery did all of it). **B3 ProcessResearch: the defect is VISIBLE** - `side.events.after.v.next_id orig=4 ours=3`, one divergent call of 3 on the reference turn and its ONLY divergent field; bit-for-bit the `EvNxID 4->3` that previously needed a 609 KB save diff. Over 5 turns 15 calls / 3 diverged, **RNG matched 15/15**, and the two completion calls miss TWO event ids each. **B1 ComputeBudget: verdict held** - 4284 compared, 0 diverged, exit 0, `budget_object` guard caught 0 undeclared writes (Budget+0x64 never changed value). **MoveFleet: 8 of 45 diverge by 1 ULP of position** (new; see its own row). First guarded captures for OnTechResearched (2 calls), ServerSystem::ProcessTurn (140 calls) and MoveFleet (45). Guards mapped SetResearched live (ConMod[0..2], OutMod, PopMod, ResTNm, TechTree+0x20 order counter) and found an UNDECLARED `vector otch` append at player+0x274. Oracle held on every run's first End Turn. Report `findings/subsystems/golden-trace-recapture.md`; engine `docs/R-recapture.md`; traces `verify/traces/recap-*`, reports `verify/results/compare/recap-*`. sots-engine branch `wip/recapture` e50d5e5 (merged with main 82ef52f; ctest 32/32, clean-room OK) | | MoveFleet position rounding (1 ULP) | verify | verified | high | 100% | 2026-09-08 | **CLOSED by lane M.** Mechanism read off the instruction stream, not fitted: the engine's `Mars_Vec3_Normalize` (0x00422520, 123 callers) narrows to float32 FOUR times - `sumsq = f32(x*x+y*y+z*z)` (products/adds stay in 53-bit regs, only the SUM is stored), `len = f32(sqrt(sumsq))`, `inv = f32(1.0/len)` a RECIPROCAL that is MULTIPLIED through rather than three divides, and `dir.c = f32(delta.c*inv)`; and MoveFleet stores each `dest.c - pos.c` BACK TO A FLOAT32 SLOT before calling it, and takes the leg distance from that same call's return value. `ours` did all of it in double. The position tail (`f32(pos + f32(dir*move))`) was already right, which is exactly why the error was a constant ABSOLUTE ~1.2e-7. Confirmed OFFLINE first (an arrival copies the destination verbatim, so calls 115/155 hand you fleet 34's and fleet 50's exact float32 destinations = 8 fully determined legs; the 5-narrowing model reproduces the ORIGINAL bit-for-bit on all 8, the old double model reproduces `ours` on the 3 divergent ones), then LIVE: control run 8/45 diverged exit 1, fixed run **0/45 diverged exit 0**, with identical args, identical pos.before and identical ORIGINAL pos.after on all 45 calls. Report `findings/subsystems/movefleet-position-rounding.md`, engine `docs/M-movefleet.md`, branch `wip/movefleet` 2aa8cba | -| undeclared ObservedTech append | verify | backlog | — | 0% | 2026-09-08 | NEW (lane R). A tech completion grows `vector otch` at ServerPlayer+0x274 (all three vector words move = a realloc). Seen as an undeclared write by BOTH the ProcessResearch `player` guard and the OnTechResearched `player` guard. It is serialized ServerPlayer state and it is in NO coverage note anywhere - a third list append in the same neighbourhood as the event list. B3's replace oracle never saw it because turn 1 of ref-turn2 has no completion | +| undeclared ObservedTech append | verify | backlog | — | 0% | 2026-09-08 | NEW (lane R). A tech completion grows `vector otch` at ServerPlayer+0x274 (all three vector words move = a realloc). Seen as an undeclared write by BOTH the ProcessResearch `player` guard and the OnTechResearched `player` guard. It is serialized ServerPlayer state and it is in NO coverage note anywhere - a third list append in the same neighbourhood as the event list. B3's replace oracle never saw it because turn 1 of ref-turn2 has no completion | **APPEND SITE NOW NAMED (lane X): `RecordObservedTech` 0x007ba1a0, called from OnTechResearched; stride 0x2c.** Still needs a declared region + a model in ours. | unnamed offsets from guard hits | verify | backlog | — | 0% | 2026-09-08 | NEW (lane R). Three spans the guards report every run and no addresses.json entry names: **ServerSystem+0xd8 (1 B)** and **ServerSystem+0x238 (4 B)** - written by the AI home system on every colony turn, alongside the fleet-vector growth; **StarFleet+0xdc (1 B)** - written on every moving MoveFleet call, just past Speed (FPsp2 @0xd8). Cheap wins for the contract | | waypoint types 2-5 have no coverage | verify | backlog | — | 0% | 2026-09-08 | NEW (lane M, promoted from a coverage line to its own row because it is now the biggest gap in `MoveFleet`). Types 2 (node line), 3 (node route), 4 (gate teleport) and 5 (probabilistic jump) have NEVER fired in any capture, and the node-line step is WRONG BY CONSTRUCTION - `sim::NodeLineStep` and `sim::BuildStutterSegments` are written and host-tested but are NOT wired into the hook, which steps every waypoint type as `speed x dt`. **`ref-turn2` structurally cannot exercise them**: lane M held the VM and tried. The only mover in that save is the AI, which travels straight runs; the player that would travel a node line has `DE 00 CR 00 DN 00` at its home system (screenshot `verify/results/shim/mf-human-home-no-ships.png`), so Move/Manage Fleets are greyed out on every turn. Needs a ship built over several turns, or - much cheaper - a PURPOSE-BUILT SAVE with a fleet already in orbit next to a node line. Same save would unblock the gate-traffic and probabilistic-jump rows. Also owed on that path: `sim::Distance` is still plain double, and `Mars_Vec3_Length` (0x004224b0) says every vector length in the engine is float32-narrowed twice, so the stutter geometry is probably 1 ULP out the same way the position update was - deliberately left alone by lane M because there is no behavioural evidence to correct it against | | ref-turn2 has no tech completion | meta | verified | high | 100% | 2026-09-08 | TRAP for anyone writing a workload (lane R). The documented one-End-Turn recipe produces **zero** `OnTechResearched` calls - an empty log that still passes. It takes 5 End Turns (to turn 7) to reach a completion. Also: only the FIRST End Turn is reproducible - it hashed to the oracle on all four runs and its research calls reproduce docs/B3.md exactly, but from turn 4 the AI picks a different research target than B3 recorded while the point totals stay nearly identical. Treat anything past turn 1 as *a* run, not *the* run | @@ -85,4 +85,5 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`). | MoveFleet waypoint types 2-5 | verify | backlog | — | 0% | 2026-09-08 | Still ZERO behavioural coverage after lane M. Not for lack of trying: the only mover in ref-turn2 is the AI (straight runs only), and the player that would travel a node line has DE/CR/DN all 00 at its home system, so Move/Manage Fleets are greyed out every turn - there is nothing to send along the node lines the map draws. Needs a ship built over several turns or a purpose-built save. The type-2 node-line step is still WRONG BY CONSTRUCTION (B4). Also: sim::Distance deliberately left in double (only stutter geometry uses it); Mars_Vec3_Length says it is probably 1 ULP out the same way, but there is zero behavioural evidence to correct it against - do not "fix" it blind | | P2-P event posting in ours | phase2 | mapped | high | 80% | 2026-09-08 | HOST-VERIFIED, VM RUN QUEUED (lane F holds VM140). next_id reaches 4 in a host reproduction of recap-b3 call 0, fixture rebuilt from raw bytes at the real 0x1c/0x18/0x74 strides and cross-checked against turn3-state.sav with lane C's state_checksum --tree. Count-only (lane E option a): ours never calls the game's PostEvent and REPLACE MODE WRITES NOTHING - a bumped EvNxID with no record behind it would corrupt the very save the oracle hashes. Three design points: the event scan is taken in describe_args BEFORE the original (taken after, ours would dedup against the original's own posts and agree for the wrong reason); dedup risk is MEASURED and reported as events_dedup_risk, not assumed; KeylessEventText resolves keys to "%s" so the shim carries no prose. VERIFIED from the instruction stream: SetResearched 0x00581e10 calls owner vft+0x10 with (flags>>2)&1 and ProcessResearch passes flags=2, so silent=false and the completion event IS posted - previously only inferable from "EvNxID moved by two". ctest 33/33, shim cross-builds on CT111 (lane P could only syntax-check) | | EVENT_TECHS_UNLOCKED not posted (predicted residual) | verify | backlog | — | 0% | 2026-09-08 | Lane P FLAGGED RATHER THAN GUESSED. Trigger IS pinned (SetResearched's sweep sets state=2 + stamps turnAvailable sticky at -1; tail loop collects state==2 && turnAvailable==currentTurn) but evaluating it needs the unlock cascade ours deliberately does not run. The driver takes the unlock list as an INPUT and is handed nullptr ("no list") - deliberately distinct from an empty list ("computed, empty"). PREDICTED RESIDUAL: next_id short by exactly 1 on every completion call. Posting it "whenever something completed" would score on this save and be WRONG the first time a completion unlocks nothing - the exact false-pass shape this project keeps catching | -| sizeof(ObservedTech) unpinned | objects | backlog | — | 0% | 2026-09-08 | Lane P declared the region at ServerPlayer+0x274 but did NOT model the bytes. The append site is in neither OnTechResearched's nor SetResearched's decompilation, and `find-constant-uses` for 0x274 returns only unrelated objects because GHIDRA DOES NOT INDEX `lea` DISPLACEMENTS - a systemic blind spot, not a one-off. The declared region's byte delta on a completion call will MEASURE the struct size. Worth a raw-byte scanner for lea-displacement xrefs as a reusable tool | +| sizeof(ObservedTech) unpinned | objects | verified | high | 100% | 2026-09-08 | **PINNED (lane X).** `sizeof(Game::ObservedTech) = 0x2c (44)` -- three independent proofs: the magic divide `0x2e8ba2e9 sar 3` (= /44, exact) at 0x0087239f, `imul reg,reg,0x2c` at 0x0087243a / 0x007b735b, and the search stride `add edi,0x2c` at 0x007ba257. **Append site = `RecordObservedTech+0xdf` (0x007ba27f): `lea ecx,[player+0x274]; call vector_ObservedTech_push_back 0x007b7320`** -- a de-duplicating append, direct callee of OnTechResearched 0x00891790; the realloc through 0x007b5820 is why all three vector words move. Element: vptr 0x00a2439c at +0 (RTTI `.?AVObservedTech@Game@@`), std::string at +0x0c (0x18 B); +0x04/+0x06 (16-bit) and +0x08/+0x24/+0x28 hold the four on-disk ints in an order NOT yet determined. Built the general tool the row asked for: `tools/x86disp.py`, an x86 displacement xref scanner (100% code coverage, 0.17% desync). `findings/subsystems/observedtech-append.md` | +| lea-displacement xref scanner | meta | verified | high | 100% | 2026-09-08 | `tools/x86disp.py` -- fixes the systemic blind spot that Ghidra does not index ModRM displacements. Full x86-32 length decoder swept from Ghidra's 41,089 function starts: 2,174,504 instructions, 612,166 displacement sites, **100.0% code coverage, 70 desyncs (0.17%), zero unknown opcodes**. Validated against ground truth before use (re-finds `lea eax,[ecx+0x29c]` in GetEventStorage, both OnTechResearched +0x29c sites, and one NEW ProcessTurn site). HONEST LIMITS: it is a **recall** tool, not an oracle -- class-level precision at 0x274 is ~13% by function (99 sites / 45 functions, ~6 real), i.e. a 900x search-space cut that still needs one call-graph check. The naive byte scan it replaces is not wrong so much as **blind**: it misses 80/99 real sites at 0x274 and 13,784/14,611 at disp8 0x14. `cohort` ranking must never be used as a hard filter -- it would have discarded the correct ObservedTech answer. Works off a gitignored local cache in `dumps/`, so it does not hammer CT111 | diff --git a/findings/subsystems/golden-trace-recapture.md b/findings/subsystems/golden-trace-recapture.md index 39a4057..2553196 100644 --- a/findings/subsystems/golden-trace-recapture.md +++ b/findings/subsystems/golden-trace-recapture.md @@ -284,6 +284,11 @@ said "an arriving call was simply clean"; that is still true and still means not pinned against the instruction stream. * `ServerSystem+0xd8` and `ServerSystem+0x238` are unnamed and are written every turn by the AI home system. `StarFleet+0xdc` likewise, on every move. +> **Follow-up 2026-09-08 (lane X):** the append is now named — `RecordObservedTech` `0x007ba1a0`, +> called directly from `OnTechResearched`; `sizeof(ObservedTech) = 0x2c`; the realloc that moves all +> three words is `vector_44B_grow` `0x007b5820`. See `findings/subsystems/observedtech-append.md`. +> That note also corrects harness-audit row 11: nothing in `ComputeBudget` stores to `Budget+0x64`. + * `vector otch` (`ServerPlayer+0x274`) is undeclared everywhere and is save state. * Nothing here exercises: replace mode (any hook), a Zuul double roll, a rebellion, an diff --git a/findings/subsystems/observedtech-append.md b/findings/subsystems/observedtech-append.md new file mode 100644 index 0000000..0b2a722 --- /dev/null +++ b/findings/subsystems/observedtech-append.md @@ -0,0 +1,278 @@ +# `vector otch` — pinned, and the displacement scanner that pinned it + +Lane X. Static/host only; VM140 was held by lane F and the game was never run. + +Two things here: a general-purpose tool (`tools/x86disp.py`) for a class of cross-reference +Ghidra cannot index, and the `ServerPlayer+0x274` answer it was built to get. + +--- + +## 1. The gap the tool fills + +Ghidra indexes **immediate** operands. It does not index **ModRM displacements**. On a +41,411-function MSVC C++ binary `lea reg,[reg+disp]` is *the* idiom for taking the address of +a member — it is how every `std::vector`, every `std::string`, every embedded sub-object gets +passed to a method or a constructor. So `find-constant-uses 0x274` returned 13 unrelated hits +and none of them was the vector, and lane P correctly concluded the search was a limitation +rather than evidence of absence. + +Lane E only found `EventStorage` at `+0x29c` because a whole accessor function happened to +exist (`lea eax,[ecx+0x29c]; ret`). We were blind to this whole reference class all campaign, +with ~1,600 classes still to go. + +## 2. The tool — `tools/x86disp.py` + +A real x86-32 **length decoder** (legacy prefixes, 1-/2-/3-byte opcodes, ModRM, SIB, disp8 +sign-extended, disp32, every immediate form), swept from Ghidra's function starts so every +decode begins on a genuine instruction boundary. It indexes every memory operand that carries +a displacement, then answers "what code touches offset N off some object?" with the containing +function, instruction address, base register and decoded instruction. + +Correctness points that would otherwise produce confident garbage, all handled: + +* `mod=0,rm=5` and `mod=0,rm=4,sib.base=5` are **disp32 with no base** — an absolute global + address, not a member offset. Excluded from member queries. (Without this, every + `mov eax,[0x00a2bd88]` in the binary shows up as a "displacement".) +* `mod=1` disp8 is **sign-extended**: `-0x08` must not be reported as `+0xf8`. +* `mod=3` is a register operand with no displacement at all. +* `0x67` address-size prefix means 16-bit ModRM, a different layout entirely — refused rather + than mis-decoded. + +Commands: `build`, `query `, `cohort`, `func `, `dis `, `stats`, `brute`. + +**Decode quality:** + +``` +functions swept : 41089 +instructions : 2174504 +disp sites : 612166 +desyncs : 70 (0.17% of functions) +code coverage : 6142049/6142358 (100.0%) +``` + +Zero unknown-opcode desyncs — the opcode tables cover everything this binary contains. All 70 +desyncs are truncations at a section/function boundary, not lost sync inside real code. + +> Gotcha worth carrying: the first build clipped each sweep at `fva + Ghidra's sizeInBytes` +> and lost 11% of functions to mid-instruction truncation — Ghidra's `sizeInBytes` understates +> real bodies often enough to matter (it was cutting valid `mov esp,ebp; pop ebp; ret` +> epilogues in half). Sweeping to the **next function start** instead took coverage 89% → 100%. + +The tool works off a local cache (`dumps/`, gitignored: the exe, the function list, the index) +so it never hammers the shared Ghidra box. `tools/cache_functions.py` pulls the function list +once. + +## 3. Validation — it rediscovers what we already knew + +Run before any new claim was made. + +**GT1 — `ServerPlayer::GetEventStorage`, the one accessor we already had:** + +``` +$ uv run python3 tools/x86disp.py func 0x0080db00 +ServerPlayer_GetEventStorage @ 0x0080db00 size 7 + 0x0080db00 lea eax,[ecx+0x29c] ServerPlayer_GetEventStorage+0x0 (8d819c020000) +``` + +**GT2 — lane E's two `OnTechResearched` sites, plus one it did not have:** + +``` +$ uv run python3 tools/x86disp.py query 0x29c --lea + ProcessTurn @0x00891340 + 0x00891713 lea ecx,[esi+0x29c] ProcessTurn+0x3d3 <-- NEW + OnTechResearched @0x00891790 + 0x008919ab lea ecx,[esi+0x29c] OnTechResearched+0x21b <-- known + 0x0089241d lea ecx,[esi+0x29c] OnTechResearched+0xc8d <-- known +``` + +**GT3 — `EvNxID` at `ServerPlayer+0x2b0`:** 90 sites found; the `ServerPlayer` ones are present. +This one is also the honest illustration of the precision problem — see §5. + +**GT4 — positive control for the cohort ranker.** Given the 50 `ServerPlayer` offsets already +in `struct-recovery.md`, `FUN_0087fac0` scores **50 co-hits out of 50** — it is the +`ServerPlayer` serializer, and nothing else in the binary comes close. A scanner that could +not surface that function from a bare offset query would not be worth using. + +## 4. The answer: `sizeof(ObservedTech)` and the append site + +### `sizeof(Game::ObservedTech) = 0x2c` (44 bytes) — verified, three independent ways + +**(a)** `vector::operator=` at `0x00872380` divides the vector's byte span by a +constant using MSVC's magic-number sequence: + +``` +0x00872398 mov ecx,[edi+0x4] ; src._Mylast +0x0087239b mov edi,[edi] ; src._Myfirst +0x0087239d sub ecx,edi ; byte span +0x0087239f mov eax,0x2e8ba2e9 +0x008723a4 imul ecx +0x008723a6 sar edx,3 +``` + +`ceil(2^35 / 44) == 0x2e8ba2e9` **exactly**, and emulating the full sequence reproduces +n = 0,1,2,3,10,71,1000 from spans of 0,44,88,132,440,3124,44000. Divisor = 44, unambiguously. + +**(b)** The same function then multiplies back by the literal stride: + +``` +0x0087243a imul ecx,ecx,0x2c +0x0087243d add ecx,[esi] ; this->_Myfirst + n*44 +0x00872441 mov [esi+0x4],ecx ; this->_Mylast = ... +``` + +Same literal at `0x00872468` and at `0x007b735b` inside `push_back`. + +**(c)** The linear search in the append function advances its iterator by `add edi,0x2c` +(`0x007ba257`). + +This matches the on-disk lower bound *exactly* — 4 × `int` + one `0x1c` `std::string` = 44 — +so there is no padding slack anywhere in the element. + +### The append site + +``` +FUN_007ba1a0 = RecordObservedTech (direct callee of ServerPlayer::OnTechResearched 0x00891790) + +0x007ba221 mov edi,[esi+0x274] ; it = observer->otch._Myfirst +0x007ba227 cmp edi,[esi+0x278] ; ... != _Mylast ? +0x007ba22d je 0x007ba274 ; empty -> append + loop: compare each element's name string (this = elem+0x0c, length = [elem+0x1c]) +0x007ba257 add edi,0x2c ; ++it *** stride 44 *** +0x007ba25a cmp edi,[esi+0x278] +0x007ba260 jne loop +0x007ba274 lea ecx,[ebp-0x3c] +0x007ba277 call 0x008562a0 ; ObservedTech::ObservedTech() on the stack +0x007ba27e push eax +0x007ba27f lea ecx,[esi+0x274] ; this = &player->otch <<< THE APPEND +0x007ba288 call 0x007b7320 ; vector::push_back +``` + +`push_back` (`0x007b7320`) grows via `0x007b5820` when `_Mylast == _Myend` — **that realloc is +exactly why lane R's guard saw all three of `player+0x274/0x278/0x27c` move on a completion**, +rather than only `_Mylast`. + +`RecordObservedTech` is called from `OnTechResearched` (`0x00891790`) and from `0x007be228`, +`0x007be4e1`, `0x007be535`. It is a **de-duplicating** append: it appends only if no existing +element already carries that tech name — worth knowing for the reimplementation, since a naive +`push_back` would diverge on a re-observation. + +Neither `push_back` nor `operator=` is COMDAT-ambiguous: `0x007b7320` has exactly one caller +(`RecordObservedTech`, `ecx = player+0x274`) and `0x00872380` has two, both passing +`ServerPlayer+0x274`. `0x007b5820` **is** shared with `FUN_0086dec0` and may be a folded body, +so it is labelled `vector_44B_grow`, not as ObservedTech-specific. + +### Element layout — as far as the evidence actually goes + +The default ctor at `0x008562a0` writes vtable `0x00a2439c` at `+0x00`. RTTI: +COL `0x00a81c78` → type descriptor `0x00aeede4` → **`.?AVObservedTech@Game@@`**. So +`ObservedTech` is polymorphic and its first word is a vptr, not a data field — which the +on-disk shape does not tell you. + +| offset | size | evidence | +|---|---|---| +| `+0x00` | 4 | vptr `0x00a2439c`; ctor writes it, RTTI-confirmed | +| `+0x04` | 2 | `mov word [eax-0x28],cx` at `0x007ba2b0` (`eax` = `_Mylast`, element = `_Mylast-0x2c`), source `[ebx+0xc]` | +| `+0x06` | 2 | `mov word [eax-0x26],dx` at `0x007ba2c6`, **same source word** `[ebx+0xc]` | +| `+0x08` | 4 | **unaccounted** | +| `+0x0c..+0x23` | 0x18 | `std::string` (the `otch` tech name). Object base is `+0x0c`, MSVC layout `{_Bx[16] @+0x00, _Mysize @+0x10, _Myres @+0x14}`. Ctor writes `[+0x0c]=0`, `[+0x1c]=0`, `[+0x20]=0xf`; the search loop reads `[+0x1c]` as the length and calls the compare with `this = +0x0c`; the post-append assign uses `lea ecx,[_Mylast-0x20]` = `+0x0c` | +| `+0x24` | 4 | **unaccounted** | +| `+0x28` | 4 | **unaccounted** | + +Note the string here is **0x18 bytes**, not the 0x1c the `ServerPlayer::pswd` row in +`struct-recovery.md` implies — three separate reads inside this element agree on +`{buf16, _Mysize@+0x10, _Myres@+0x14}`. Worth re-checking `pswd` against that. + +The four on-disk ints (`otnF`, `otnL`, `odet`, `owith`) have to map onto `+0x04`/`+0x06` (two +16-bit fields) and the three 4-byte slots `+0x08`, `+0x24`, `+0x28`. **That mapping is not +determined here and is deliberately not guessed.** The one suggestive observation, flagged as a +*hypothesis only*: `+0x04` and `+0x06` are two adjacent 16-bit fields written from the same +source word on first observation, which is the shape you would expect of a first-seen / +last-seen turn pair — but nothing here proves it. + +What would settle the mapping: read `ObservedTech`'s `Read`/`Write` serializer, where the +member order is explicit. Lane P's `observed_techs` region byte delta remains a valid live +cross-check and should now come back as exactly 44 per completion. + +## 5. False-positive rate, honestly + +Two different error rates, and the interesting one is not the one you'd expect. + +**Decode-level error of the naive method is low.** A raw byte scan for `8D` + ModRM + the +displacement — the thing you'd write without a decoder — is mostly *right*: + +| query | naive candidates | not actually an instruction | real sites the naive scan **missed** | +|---|---|---|---| +| `lea`, disp32 `0x274` | 19 | 0 (0.0%) | **80 of 99** | +| 11 common opcodes, disp32 `0x274` | 89 | 1 (1.1%) | 11 of 99 | +| `lea`, disp8 `0x14` | 828 | 1 (0.1%) | **13,784 of 14,611** | +| 11 common opcodes, disp8 `0x14` | 10,326 | 59 (0.6%) | 4,344 of 14,611 | + +So the decoder's win is **recall, not decode precision**. A hand-written opcode set misses +80–94% of the real accesses, because a member is read, written, compared, and float-loaded far +more often than its address is taken, and you cannot enumerate those opcodes by hand. + +**Class-level precision is the real problem, and it is poor.** The scanner knows the +displacement; it cannot know what class the base register holds. `query 0x274` returns 99 sites +in 45 functions and only about 6 of those functions are actually touching `ServerPlayer::otch` +— roughly **13% precision by function**. The honest way to state the value is as search-space +reduction: 41,411 functions → 45, about **900×**, down to a list a human reads in two minutes. + +**The cohort ranker helps, and has a trap.** Scoring functions by how many *already-known* +offsets of the same class they touch pulls real class methods to the top (the serializer hits +50/50). But it would have **thrown away the correct answer**: `RecordObservedTech` touches only +`0x274` and `0x278` and nothing else on `ServerPlayer`, so any `--min>=1` filter drops it. Use +cohort as a ranker, never as a filter. This is now written into the tool's own docstring. + +What actually closed the case was the plain `query 0x274` list **intersected with one call-graph +lookup** (`OnTechResearched`'s direct callees). Displacement scan for recall, call graph for +disambiguation — neither alone was enough. + +## 6. Previously anonymous offsets — attribution results + +The payoff was smaller than hoped, because most of the audit's anonymous offsets had already +been named by other lanes since the audit was written. + +| offset | status before | after | +|---|---|---| +| `player+0x274/0x278/0x27c` | "vector grew, append site unknown" | **`RecordObservedTech+0xdf` (0x007ba27f)**, `sizeof` = 0x2c | +| `player+0x2b0` | already named by lane E | unchanged (`EvNxID`) | +| `TechTree+0x20` | already `TechTree_off_OrderCounter` | unchanged | +| `ServerPlayer+0x308` | already `ServerPlayer_off_NodeBore` | unchanged | +| `Budget+0x64` | harness-audit row 11: "the original writes it" | **not supported.** See below | + +**`Budget+0x64` (harness-audit row 11) — a correction.** `ServerPlayer::ComputeBudget` +(`0x00863030`) writes its `Budget*` out-param through `esi`, and every such store lands in +`+0x00..+0x54` (the 22-int block). The only two `+0x64` accesses in the whole function are +**loads off a different base register** (`mov ecx,[eax+0x64]` at `0x0086328c`, +`mov edx,[eax+0x64]` at `0x0086335d`). There is no store to `Budget+0x64` in `ComputeBudget`. + +Separately, `TechTree::ProcessResearch`'s `int* overbudget` fourth argument is **not** +`Budget+0x64`: its only caller is `ProcessTurn` (`0x00891340`) and the call at `0x008914a5` +passes `lea edx,[ebp-0x14]` — a stack local, consumed immediately after the call +(`mov eax,[ebp-0x14]; cmp eax,0; jle`). + +This agrees with lane R, whose `budget_object` guard saw `Budget+0x64` change in **0 of 4284 +calls**. Row 11 should be reclassified from "the original writes it and B1 never checked" to +"nothing has been shown to write it"; the remaining way to settle it is a write watchpoint on +that word, not another static search. + +## 7. Verdict on the technique + +Worth keeping, with its limits stated. It answered a question that had been parked, and the +`0x29c` validation turned up a `ProcessTurn` `EventStorage` site nobody had. But it is a +**recall** tool that produces a 20–100 line candidate list per offset, not an oracle: every +result still needs a call-graph or decompiler check before it is a fact. For the ~1,600 +remaining classes the realistic workflow is `query ` → read the list → confirm with one +cross-reference call. + +## 8. Ghidra writeback + +Labels: `RecordObservedTech` `0x007ba1a0`, `vector_ObservedTech_push_back` `0x007b7320`, +`ObservedTech_ctor` `0x008562a0`, `vector_ObservedTech_assign` `0x00872380`, +`vector_44B_grow` `0x007b5820`, `vftable_ObservedTech` `0x00a2439c`. Plate comments carrying +the stride evidence on the first four. + +`addresses.json`: `ObservedTech_sizeof`, `ObservedTech_vftable`, `ObservedTech_off_Name`, +`RecordObservedTech`, `vector_ObservedTech_push_back`, `vector_ObservedTech_assign`, +`ObservedTech_ctor`, `vector_44B_grow`; `ServerPlayer_off_ObservedTechs` updated from +"NOT PINNED" to `verified`. diff --git a/findings/subsystems/setresearched-cascade.md b/findings/subsystems/setresearched-cascade.md index ca80269..1f8b73c 100644 --- a/findings/subsystems/setresearched-cascade.md +++ b/findings/subsystems/setresearched-cascade.md @@ -109,7 +109,15 @@ New `addresses.json` entries from this read: `TechNode_off_TurnAvailable` 0x20, `TechNode_off_TurnResearched` 0x24, `TechNode_off_Order` 0x28, `TechNode_off_Children` 0x04, `TechEdge_off_CostRP` 0x1c, `TechEdge_off_ChildDef` 0x40, `TechTree_off_OrderCounter` 0x20. -## 5. `vector otch` at `ServerPlayer+0x274` — still not pinned +## 5. `vector otch` at `ServerPlayer+0x274` — ~~still not pinned~~ **PINNED** + +> **RESOLVED 2026-09-08 by lane X** — see `findings/subsystems/observedtech-append.md`. +> `sizeof(Game::ObservedTech) = 0x2c (44)`. The append is `RecordObservedTech+0xdf` (`0x007ba27f`): +> `lea ecx,[player+0x274]; call vector_ObservedTech_push_back 0x007b7320`. `RecordObservedTech` +> (`0x007ba1a0`) is a **direct callee of `OnTechResearched`** and de-duplicates by tech name before +> appending. The suspicion recorded below was right: `find-constant-uses` missed it because the +> encoding is a `lea` displacement. `tools/x86disp.py` now indexes those. +> The rest of this section is left as written, as the record of what was known at the time. Lane R's guards caught all three vector words moving on every tech completion, in both the `ProcessResearch` and the `OnTechResearched` player guards, and it is serialized `ServerPlayer` diff --git a/ghidra/addresses.json b/ghidra/addresses.json index 69238f2..885af86 100644 --- a/ghidra/addresses.json +++ b/ghidra/addresses.json @@ -1454,9 +1454,9 @@ "name": "ServerPlayer_off_ObservedTechs", "offset": "0x274", "convention": "offset", - "prototype": "std::vector otch (3 words {first,last,end}); save tag otch, element {int otnF, otnL, odet; string otch; int owith}. All three words move on every tech completion (a realloc) -- observed live by both the ProcessResearch and the OnTechResearched player guards. NOT PINNED: the append call site and sizeof(ObservedTech); the byte span the observed_techs region reports is what will measure the stride", - "status": "verified-by-save", - "source": "findings/objects/struct-recovery.md#2 (Read/Write serializers) + findings/subsystems/golden-trace-recapture.md (player guard, both completion calls)" + "prototype": "std::vector otch -- 3 words {_Myfirst@0x274,_Mylast@0x278,_Myend@0x27c}; save tag otch. sizeof(ObservedTech) = 0x2c (44), PINNED three ways: magic divide 0x2e8ba2e9 sar 3 (=/44) in vector_ObservedTech_assign 0x0087239f, imul reg,reg,0x2c at 0x0087243a / 0x007b735b, and the linear search stride `add edi,0x2c` at 0x007ba257. Append site = RecordObservedTech+0xdf (0x007ba27f): lea ecx,[player+0x274]; call vector_ObservedTech_push_back 0x007b7320. All three words move because push_back reallocs through vector_44B_grow 0x007b5820.", + "status": "verified", + "source": "findings/subsystems/observedtech-append.md (lane X, tools/x86disp.py displacement scan)" }, { "name": "ServerPlayer_off_IncMod", @@ -3169,6 +3169,70 @@ "prototype": "site inside StrategyServer::MoveFleet /* the move != distance branch: exactly two roundings per component, tmp.c = float32(dir.c * move) then pos.c = float32(pos.c + tmp.c). `move` is reloaded from a float32 slot. The sibling branch (move == distance, an EXACT float compare) copies the destination's three words verbatim with mov, so an arrival never steps onto its destination. */", "status": "verified", "source": "lane M own disassembly pass 2026-09-08" + }, + { + "name": "ObservedTech_sizeof", + "offset": "0x2c", + "convention": "constant", + "prototype": "sizeof(Game::ObservedTech) = 0x2c (44 bytes). Confirmed independently by (a) the compiler's magic division by 44 (mov eax,0x2e8ba2e9; imul; sar edx,3) at 0x0087239f and 0x007b7339, (b) imul reg,reg,0x2c at 0x0087243a, 0x00872468, 0x007b735b, and (c) the iterator advance `add edi,0x2c` in RecordObservedTech's linear search at 0x007ba257. Matches the on-disk lower bound exactly (4 int + one 0x1c std::string = 44), so there is no padding slack.", + "status": "verified", + "source": "findings/subsystems/observedtech-append.md (lane X, tools/x86disp.py displacement scan)" + }, + { + "name": "ObservedTech_vftable", + "addr": "0x00a2439c", + "convention": "data", + "prototype": "Game::ObservedTech vftable. RTTI COL 0x00a81c78 -> type descriptor 0x00aeede4 = '.?AVObservedTech@Game@@'. Written to element+0x00 by ObservedTech_ctor 0x008562a0 -- so ObservedTech is polymorphic and its first word is the vptr, not a data field.", + "status": "verified", + "source": "findings/subsystems/observedtech-append.md (lane X, tools/x86disp.py displacement scan)" + }, + { + "name": "ObservedTech_off_Name", + "offset": "0x0c", + "convention": "offset", + "prototype": "std::string (save tag `otch`, the tech name) at ObservedTech+0x0c, 0x18 bytes, spanning +0x0c..+0x23. MSVC layout relative to the string object: _Bx[16] @+0x00, _Mysize @+0x10, _Myres @+0x14 -- i.e. ObservedTech+0x1c is the length and +0x20 the capacity. Evidence: ObservedTech_ctor 0x008562a0 writes [elem+0x0c]=0, [elem+0x1c]=0, [elem+0x20]=0xf; RecordObservedTech's search reads [elem+0x1c] as the length and calls compare with this=elem+0x0c; the post-append assign uses lea ecx,[_Mylast-0x20]. NOTE this string is 0x18 bytes, not the 0x1c implied by the ServerPlayer pswd row in struct-recovery.md. UNACCOUNTED in the element: +0x08, +0x24, +0x28 (4 bytes each) plus two 16-bit fields at +0x04/+0x06 written from one source word at 0x007ba2b0 / 0x007ba2c6 -- the mapping of the four on-disk ints (otnF, otnL, odet, owith) onto those slots is NOT determined.", + "status": "verified", + "source": "findings/subsystems/observedtech-append.md (lane X, tools/x86disp.py displacement scan)" + }, + { + "name": "RecordObservedTech", + "addr": "0x007ba1a0", + "convention": "thiscall", + "prototype": "void (this, ServerPlayer* observer, ?, std::string* techName) -- appends to observer->otch. Linear-searches observer->otch (ServerPlayer+0x274) with stride 0x2c comparing each element's name string; if not found, default-constructs an ObservedTech on the stack (0x008562a0) and push_backs it (0x007b7320 @0x007ba288), then writes two 16-bit fields into the new element at +0x04 and +0x06 from param_1+0xc. DIRECT CALLEE of ServerPlayer::OnTechResearched 0x00891790; also called from 0x007be228, 0x007be4e1, 0x007be535. This is the append lane P could not find.", + "status": "verified", + "source": "findings/subsystems/observedtech-append.md (lane X, tools/x86disp.py displacement scan)" + }, + { + "name": "vector_ObservedTech_push_back", + "addr": "0x007b7320", + "convention": "thiscall", + "prototype": "void (std::vector* this, ObservedTech* value) RET 4. Stride 0x2c. Calls vector_44B_grow 0x007b5820 when _Mylast==_Myend -- the realloc that moves all three vector words. Exactly one caller (RecordObservedTech), so this instantiation is not COMDAT-ambiguous.", + "status": "verified", + "source": "findings/subsystems/observedtech-append.md (lane X, tools/x86disp.py displacement scan)" + }, + { + "name": "vector_ObservedTech_assign", + "addr": "0x00872380", + "convention": "thiscall", + "prototype": "std::vector& operator=(const std::vector&) RET 4. Both callers pass ServerPlayer+0x274 (0x00878091 in the settings copy-out, 0x00892507 in the copy-in).", + "status": "verified", + "source": "findings/subsystems/observedtech-append.md (lane X, tools/x86disp.py displacement scan)" + }, + { + "name": "ObservedTech_ctor", + "addr": "0x008562a0", + "convention": "thiscall", + "prototype": "Game::ObservedTech* (ObservedTech* this) -- default ctor; sets vptr 0x00a2439c and empties the name string.", + "status": "verified", + "source": "findings/subsystems/observedtech-append.md (lane X, tools/x86disp.py displacement scan)" + }, + { + "name": "vector_44B_grow", + "addr": "0x007b5820", + "convention": "thiscall", + "prototype": "vector::_Reserve/grow for a 44-byte element type. Shared with FUN_0086dec0, so it may be a COMDAT-folded body -- do NOT assume it is ObservedTech-specific.", + "status": "mapped", + "source": "findings/subsystems/observedtech-append.md (lane X, tools/x86disp.py displacement scan)" } ] } diff --git a/ghidra/generated/sots_addresses.h b/ghidra/generated/sots_addresses.h index c528e84..2adca21 100644 --- a/ghidra/generated/sots_addresses.h +++ b/ghidra/generated/sots_addresses.h @@ -1,5 +1,5 @@ // GENERATED — do not edit. Facts about Sword of the Stars.exe (GOG 1.8.1). -// Source: sots-re ghidra/addresses.json @ b0ef139, generated 2026-09-08 by tools/gen_addresses.py +// Source: sots-re ghidra/addresses.json @ 1a58bcd, generated 2026-09-08 by tools/gen_addresses.py // Runtime address = (uintptr_t)GetModuleHandle(NULL) + RVA (the exe is ASLR-relocated). #pragma once #include @@ -369,7 +369,7 @@ constexpr uint32_t ServerPlayer_off_SetupResearchMult = 0x0000022c; constexpr uint32_t ServerPlayer_off_Sav = 0x00000284; // offset Tech* current research target (ResT); NULL = none [verified-by-save] constexpr uint32_t ServerPlayer_off_ResearchTarget = 0x00000294; -// offset std::vector otch (3 words {first,last,end}); save tag otch, element {int otnF, otnL, odet; string otch; int owith}. All three words move on every tech completion (a realloc) -- observed live by both the ProcessResearch and the OnTechResearched player guards. NOT PINNED: the append call site and sizeof(ObservedTech); the byte span the observed_techs region reports is what will measure the stride [verified-by-save] +// offset std::vector otch -- 3 words {_Myfirst@0x274,_Mylast@0x278,_Myend@0x27c}; save tag otch. sizeof(ObservedTech) = 0x2c (44), PINNED three ways: magic divide 0x2e8ba2e9 sar 3 (=/44) in vector_ObservedTech_assign 0x0087239f, imul reg,reg,0x2c at 0x0087243a / 0x007b735b, and the linear search stride `add edi,0x2c` at 0x007ba257. Append site = RecordObservedTech+0xdf (0x007ba27f): lea ecx,[player+0x274]; call vector_ObservedTech_push_back 0x007b7320. All three words move because push_back reallocs through vector_44B_grow 0x007b5820. [verified] constexpr uint32_t ServerPlayer_off_ObservedTechs = 0x00000274; // offset float IncMod [verified-by-save] constexpr uint32_t ServerPlayer_off_IncMod = 0x0000030c; @@ -799,5 +799,21 @@ constexpr uint32_t Mars_Vec3_NormaliseEpsilon = 0x005e1ef8; constexpr uint32_t StrategyServer_MoveFleet_straight_leg = 0x003da0f2; // site site inside StrategyServer::MoveFleet /* the move != distance branch: exactly two roundings per component, tmp.c = float32(dir.c * move) then pos.c = float32(pos.c + tmp.c). `move` is reloaded from a float32 slot. The sibling branch (move == distance, an EXACT float compare) copies the destination's three words verbatim with mov, so an arrival never steps onto its destination. */ [verified] constexpr uint32_t StrategyServer_MoveFleet_position_update = 0x003da2ac; +// constant sizeof(Game::ObservedTech) = 0x2c (44 bytes). Confirmed independently by (a) the compiler's magic division by 44 (mov eax,0x2e8ba2e9; imul; sar edx,3) at 0x0087239f and 0x007b7339, (b) imul reg,reg,0x2c at 0x0087243a, 0x00872468, 0x007b735b, and (c) the iterator advance `add edi,0x2c` in RecordObservedTech's linear search at 0x007ba257. Matches the on-disk lower bound exactly (4 int + one 0x1c std::string = 44), so there is no padding slack. [verified] +constexpr uint32_t ObservedTech_sizeof = 0x0000002c; +// data Game::ObservedTech vftable. RTTI COL 0x00a81c78 -> type descriptor 0x00aeede4 = '.?AVObservedTech@Game@@'. Written to element+0x00 by ObservedTech_ctor 0x008562a0 -- so ObservedTech is polymorphic and its first word is the vptr, not a data field. [verified] +constexpr uint32_t ObservedTech_vftable = 0x0062439c; +// offset std::string (save tag `otch`, the tech name) at ObservedTech+0x0c, 0x18 bytes, spanning +0x0c..+0x23. MSVC layout relative to the string object: _Bx[16] @+0x00, _Mysize @+0x10, _Myres @+0x14 -- i.e. ObservedTech+0x1c is the length and +0x20 the capacity. Evidence: ObservedTech_ctor 0x008562a0 writes [elem+0x0c]=0, [elem+0x1c]=0, [elem+0x20]=0xf; RecordObservedTech's search reads [elem+0x1c] as the length and calls compare with this=elem+0x0c; the post-append assign uses lea ecx,[_Mylast-0x20]. NOTE this string is 0x18 bytes, not the 0x1c implied by the ServerPlayer pswd row in struct-recovery.md. UNACCOUNTED in the element: +0x08, +0x24, +0x28 (4 bytes each) plus two 16-bit fields at +0x04/+0x06 written from one source word at 0x007ba2b0 / 0x007ba2c6 -- the mapping of the four on-disk ints (otnF, otnL, odet, owith) onto those slots is NOT determined. [verified] +constexpr uint32_t ObservedTech_off_Name = 0x0000000c; +// thiscall void (this, ServerPlayer* observer, ?, std::string* techName) -- appends to observer->otch. Linear-searches observer->otch (ServerPlayer+0x274) with stride 0x2c comparing each element's name string; if not found, default-constructs an ObservedTech on the stack (0x008562a0) and push_backs it (0x007b7320 @0x007ba288), then writes two 16-bit fields into the new element at +0x04 and +0x06 from param_1+0xc. DIRECT CALLEE of ServerPlayer::OnTechResearched 0x00891790; also called from 0x007be228, 0x007be4e1, 0x007be535. This is the append lane P could not find. [verified] +constexpr uint32_t RecordObservedTech = 0x003ba1a0; +// thiscall void (std::vector* this, ObservedTech* value) RET 4. Stride 0x2c. Calls vector_44B_grow 0x007b5820 when _Mylast==_Myend -- the realloc that moves all three vector words. Exactly one caller (RecordObservedTech), so this instantiation is not COMDAT-ambiguous. [verified] +constexpr uint32_t vector_ObservedTech_push_back = 0x003b7320; +// thiscall std::vector& operator=(const std::vector&) RET 4. Both callers pass ServerPlayer+0x274 (0x00878091 in the settings copy-out, 0x00892507 in the copy-in). [verified] +constexpr uint32_t vector_ObservedTech_assign = 0x00472380; +// thiscall Game::ObservedTech* (ObservedTech* this) -- default ctor; sets vptr 0x00a2439c and empties the name string. [verified] +constexpr uint32_t ObservedTech_ctor = 0x004562a0; +// thiscall vector::_Reserve/grow for a 44-byte element type. Shared with FUN_0086dec0, so it may be a COMDAT-folded body -- do NOT assume it is ObservedTech-specific. [mapped] +constexpr uint32_t vector_44B_grow = 0x003b5820; } // namespace sots::addr diff --git a/tools/cache_functions.py b/tools/cache_functions.py new file mode 100644 index 0000000..1b6f77a --- /dev/null +++ b/tools/cache_functions.py @@ -0,0 +1,57 @@ +#!/usr/bin/env python3 +"""Cache the full Ghidra function list (address, size, name) to dumps/functions.json. + +Ghidra/ReVa is a shared resource -- pull the list once, then work offline. +Usage: uv run python3 tools/cache_functions.py [--refresh] +""" +import json +import os +import subprocess +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +REPO = os.path.dirname(HERE) +OUT = os.path.join(REPO, "dumps", "functions.json") +PROG = "/Sword of the Stars.exe" + + +def call(tool, args): + r = subprocess.run( + ["uv", "run", "python3", os.path.join(HERE, "reva_call.py"), tool, json.dumps(args)], + capture_output=True, text=True, cwd=REPO, timeout=300) + if r.returncode != 0: + sys.exit(f"reva_call {tool} failed: {r.stderr[:400]}") + return r.stdout + + +def main(): + if os.path.exists(OUT) and "--refresh" not in sys.argv: + print(f"{OUT} exists; use --refresh to re-pull") + return + funcs = {} + start = 0 + while True: + raw = call("get-functions", { + "programPath": PROG, "startIndex": start, "filterDefaultNames": False}) + lines = [ln for ln in raw.splitlines() if ln.strip()] + header = json.loads(lines[0]) + for ln in lines[1:]: + try: + f = json.loads(ln) + except json.JSONDecodeError: + continue + funcs[f["address"]] = [f["name"], f.get("sizeInBytes", 0)] + nxt = header.get("nextStartIndex") + total = header.get("totalCount") + print(f" {len(funcs)}/{total}", file=sys.stderr) + if nxt is None or nxt <= start or len(funcs) >= total: + break + start = nxt + os.makedirs(os.path.dirname(OUT), exist_ok=True) + with open(OUT, "w") as fh: + json.dump(funcs, fh) + print(f"wrote {len(funcs)} functions to {OUT}") + + +if __name__ == "__main__": + main() diff --git a/tools/x86disp.py b/tools/x86disp.py new file mode 100644 index 0000000..441be30 --- /dev/null +++ b/tools/x86disp.py @@ -0,0 +1,798 @@ +#!/usr/bin/env python3 +"""x86-32 displacement cross-referencer for the SOTS1 exe. + +Why this exists +--------------- +Ghidra indexes immediate operands but NOT ModRM displacements. On an MSVC C++ +binary `lea reg,[reg+disp]` is *the* idiom for taking the address of a member -- +it is how every std::vector / std::string / embedded sub-object is passed to a +method or ctor. `find-constant-uses 0x274` therefore cannot see the append site +for a vector member at +0x274. This tool answers the question Ghidra can't: + + "what code takes the address of, or accesses, offset N off some object?" + +How it works +------------ +x86 is variable-length and not self-synchronising, so a naive byte scan for +"8D 8E " produces confident garbage. Instead we run a full instruction +*length* decoder (prefixes / 1-,2-,3-byte opcodes / ModRM / SIB / disp / imm) +seeded from Ghidra's 41k function starts, so every decode begins on a real +instruction boundary. `--brute` runs the naive scan too, purely so the +false-positive rate of the naive method can be *measured* rather than guessed. + +Correctness notes that matter (each of these is a silent-garbage source): + * mod=0,rm=5 -> disp32 with NO base: an absolute global address, not a + member offset. Excluded from member queries by default. + * mod=0,rm=4, + sib.base=5 -> disp32, no base, index only. Same: excluded. + * mod=1 -> disp8 SIGN-EXTENDED. -0x08 must not be reported as 0xf8. + * mod=3 -> register operand, no memory, no displacement at all. + * 0x67 addr-size -> 16-bit ModRM, entirely different layout. Skipped, flagged. + +Usage +----- + uv run python3 tools/x86disp.py build # decode + cache index + uv run python3 tools/x86disp.py query 0x274 # who touches +0x274 + uv run python3 tools/x86disp.py query 0x29c --lea # lea only + uv run python3 tools/x86disp.py func 0x0080db00 # dump one function + uv run python3 tools/x86disp.py stats +""" +import bisect +import json +import os +import struct +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +REPO = os.path.dirname(HERE) +EXE = os.path.join(REPO, "dumps", "sots.exe") +FUNCS = os.path.join(REPO, "dumps", "functions.json") +INDEX = os.path.join(REPO, "dumps", "dispindex.json") + +R32 = ["eax", "ecx", "edx", "ebx", "esp", "ebp", "esi", "edi"] + +# ---------------------------------------------------------------- opcode maps +# value: (has_modrm, imm_kind) +# imm kinds: 0 none | 'b' 1 | 'w' 2 | 'd' 4 | 'z' 2-if-66-else-4 +# 'p' far ptr 6 (4 with 66) | 'a' moffs (addr-size) | 'e' enter 3 +# 'g6' F6 group (b if reg<2) | 'g7' F7 group (z if reg<2) +_ONE = {} + + +def _fill(rng, modrm, imm): + for o in rng: + _ONE[o] = (modrm, imm) + + +# 00..3F: the eight ALU ops, each /r /r /r /r AL,ib eAX,iz + 2 seg ops +for _base in (0x00, 0x08, 0x10, 0x18, 0x20, 0x28, 0x30, 0x38): + _fill(range(_base, _base + 4), True, 0) + _ONE[_base + 4] = (False, "b") + _ONE[_base + 5] = (False, "z") + _ONE[_base + 6] = (False, 0) # PUSH seg / prefix-or-ascii-adjust + _ONE[_base + 7] = (False, 0) +_fill(range(0x40, 0x60), False, 0) # INC/DEC/PUSH/POP r32 +_fill([0x60, 0x61], False, 0) # PUSHA/POPA +_ONE[0x62] = (True, 0) # BOUND +_ONE[0x63] = (True, 0) # ARPL +_ONE[0x68] = (False, "z") # PUSH iz +_ONE[0x69] = (True, "z") # IMUL r,Ev,iz +_ONE[0x6A] = (False, "b") # PUSH ib +_ONE[0x6B] = (True, "b") # IMUL r,Ev,ib +_fill(range(0x6C, 0x70), False, 0) # INS/OUTS +_fill(range(0x70, 0x80), False, "b") # Jcc rel8 +_ONE[0x80] = (True, "b") +_ONE[0x81] = (True, "z") +_ONE[0x82] = (True, "b") +_ONE[0x83] = (True, "b") +_fill(range(0x84, 0x90), True, 0) # TEST/XCHG/MOV/MOV-seg/LEA/POP Ev +_fill(range(0x90, 0x9A), False, 0) # NOP/XCHG/CWDE/CDQ +_ONE[0x9A] = (False, "p") # CALLF +_fill(range(0x9B, 0xA0), False, 0) +_fill(range(0xA0, 0xA4), False, "a") # MOV moffs +_fill(range(0xA4, 0xA8), False, 0) # MOVS/CMPS +_ONE[0xA8] = (False, "b") +_ONE[0xA9] = (False, "z") +_fill(range(0xAA, 0xB0), False, 0) # STOS/LODS/SCAS +_fill(range(0xB0, 0xB8), False, "b") # MOV r8,ib +_fill(range(0xB8, 0xC0), False, "z") # MOV r32,iz +_ONE[0xC0] = (True, "b") +_ONE[0xC1] = (True, "b") +_ONE[0xC2] = (False, "w") # RET imm16 +_ONE[0xC3] = (False, 0) +_ONE[0xC4] = (True, 0) # LES +_ONE[0xC5] = (True, 0) # LDS +_ONE[0xC6] = (True, "b") # MOV Eb,Ib +_ONE[0xC7] = (True, "z") # MOV Ev,Iz +_ONE[0xC8] = (False, "e") # ENTER iw,ib +_ONE[0xC9] = (False, 0) +_ONE[0xCA] = (False, "w") # RETF imm16 +_fill([0xCB, 0xCC], False, 0) +_ONE[0xCD] = (False, "b") # INT ib +_fill([0xCE, 0xCF], False, 0) +_fill(range(0xD0, 0xD4), True, 0) # shift group by 1 / by CL +_ONE[0xD4] = (False, "b") # AAM +_ONE[0xD5] = (False, "b") # AAD +_fill([0xD6, 0xD7], False, 0) +_fill(range(0xD8, 0xE0), True, 0) # x87 -- always ModRM +_fill(range(0xE0, 0xE4), False, "b") # LOOP*/JECXZ +_fill(range(0xE4, 0xE8), False, "b") # IN/OUT ib +_ONE[0xE8] = (False, "z") # CALL rel32 +_ONE[0xE9] = (False, "z") # JMP rel32 +_ONE[0xEA] = (False, "p") # JMPF +_ONE[0xEB] = (False, "b") # JMP rel8 +_fill(range(0xEC, 0xF0), False, 0) # IN/OUT DX +_fill(range(0xF0, 0xF6), False, 0) # LOCK/INT1/REP*/HLT/CMC +_ONE[0xF6] = (True, "g6") +_ONE[0xF7] = (True, "g7") +_fill(range(0xF8, 0xFE), False, 0) +_ONE[0xFE] = (True, 0) +_ONE[0xFF] = (True, 0) + +_TWO = {} + + +def _fill2(rng, modrm, imm): + for o in rng: + _TWO[o] = (modrm, imm) + + +_fill2(range(0x00, 0x05), True, 0) +_fill2(range(0x05, 0x0D), False, 0) +_TWO[0x0D] = (True, 0) +_TWO[0x0E] = (False, 0) +_TWO[0x0F] = (True, "b") # 3DNow! +_fill2(range(0x10, 0x18), True, 0) +_fill2(range(0x18, 0x20), True, 0) # hint-NOP / prefetch +_fill2(range(0x20, 0x25), True, 0) +_fill2(range(0x28, 0x30), True, 0) +_fill2(range(0x30, 0x38), False, 0) +_fill2(range(0x40, 0x50), True, 0) # CMOVcc +_fill2(range(0x50, 0x70), True, 0) # SSE/MMX +_TWO[0x70] = (True, "b") +_fill2(range(0x71, 0x74), True, "b") +_fill2(range(0x74, 0x77), True, 0) +_TWO[0x77] = (False, 0) # EMMS +_fill2(range(0x78, 0x80), True, 0) +_fill2(range(0x80, 0x90), False, "z") # Jcc rel32 +_fill2(range(0x90, 0xA0), True, 0) # SETcc +_fill2([0xA0, 0xA1, 0xA2], False, 0) +_TWO[0xA3] = (True, 0) # BT +_TWO[0xA4] = (True, "b") # SHLD ib +_TWO[0xA5] = (True, 0) # SHLD CL +_fill2([0xA8, 0xA9, 0xAA], False, 0) +_TWO[0xAB] = (True, 0) # BTS +_TWO[0xAC] = (True, "b") # SHRD ib +_fill2([0xAD, 0xAE, 0xAF], True, 0) +_fill2(range(0xB0, 0xBA), True, 0) +_TWO[0xBA] = (True, "b") # group8 BT/BTS/BTR/BTC ib +_fill2(range(0xBB, 0xC0), True, 0) +_fill2([0xC0, 0xC1], True, 0) # XADD +_TWO[0xC2] = (True, "b") # CMPPS +_TWO[0xC3] = (True, 0) # MOVNTI +_fill2([0xC4, 0xC5, 0xC6], True, "b") +_TWO[0xC7] = (True, 0) # group9 CMPXCHG8B +_fill2(range(0xC8, 0xD0), False, 0) # BSWAP +_fill2(range(0xD0, 0x100), True, 0) # MMX/SSE bulk + +# minimal mnemonics -- enough to read a report, not a full disassembler +_ALU = ["add", "or", "adc", "sbb", "and", "sub", "xor", "cmp"] +_G1 = _ALU +_G5 = ["inc", "dec", "call", "callf", "jmp", "jmpf", "push", "?"] +_G3 = ["test", "test", "not", "neg", "mul", "imul", "div", "idiv"] +_SHIFT = ["rol", "ror", "rcl", "rcr", "shl", "shr", "shl", "sar"] + + +def mnemonic(op2, op, reg): + """Best-effort mnemonic. op2 True => the opcode was 0F-escaped.""" + if op2: + if 0x10 <= op <= 0x17 or 0x28 <= op <= 0x2F or 0x51 <= op <= 0x5F: + return "sse" + if 0x40 <= op <= 0x4F: + return "cmov" + if 0x90 <= op <= 0x9F: + return "setcc" + if op in (0xB6, 0xB7): + return "movzx" + if op in (0xBE, 0xBF): + return "movsx" + if op == 0xAF: + return "imul" + if op in (0x6E, 0x6F, 0x7E, 0x7F, 0xD6): + return "movq/movd" + return f"0f{op:02x}" + if op < 0x40 and (op & 7) < 6: + return _ALU[op >> 3] + if op in (0x88, 0x89, 0x8A, 0x8B, 0xC6, 0xC7): + return "mov" + if op == 0x8D: + return "lea" + if op in (0x84, 0x85): + return "test" + if op in (0x86, 0x87): + return "xchg" + if op in (0x80, 0x81, 0x82, 0x83): + return _G1[reg] + if op in (0xC0, 0xC1, 0xD0, 0xD1, 0xD2, 0xD3): + return _SHIFT[reg] + if op in (0xF6, 0xF7): + return _G3[reg] + if op == 0xFF: + return _G5[reg] + if op == 0xFE: + return ["inc", "dec"][reg] if reg < 2 else "?" + if op == 0x8F: + return "pop" + if op in (0x69, 0x6B): + return "imul" + if 0xD8 <= op <= 0xDF: + return "x87" + if op == 0x62: + return "bound" + return f"{op:02x}" + + +class Desync(Exception): + pass + + +def decode(buf, i, end): + """Decode one instruction at buf[i]. Returns (length, info|None). + + info = dict(mnem, base, index, scale, disp, dispsize, modrm_reg, is_lea) + for instructions with a memory operand carrying a displacement; None + otherwise. Raises Desync on an unknown/invalid opcode. + """ + start = i + opsize66 = False + addr67 = False + while i < end: + b = buf[i] + if b == 0x66: + opsize66 = True + i += 1 + elif b == 0x67: + addr67 = True + i += 1 + elif b in (0xF0, 0xF2, 0xF3, 0x2E, 0x36, 0x3E, 0x26, 0x64, 0x65): + i += 1 + else: + break + else: + raise Desync("prefix run to end") + if i - start > 14: + raise Desync("prefix flood") + + op = buf[i] + i += 1 + op2 = op3 = False + if op == 0x0F: + if i >= end: + raise Desync("truncated 0f") + op = buf[i] + i += 1 + op2 = True + if op in (0x38, 0x3A): + three_imm = "b" if op == 0x3A else 0 + if i >= end: + raise Desync("truncated 0f3x") + op = buf[i] + i += 1 + op3 = True + has_modrm, imm = True, three_imm + else: + ent = _TWO.get(op) + if ent is None: + raise Desync(f"unknown 0f{op:02x}") + has_modrm, imm = ent + else: + ent = _ONE.get(op) + if ent is None: + raise Desync(f"unknown {op:02x}") + has_modrm, imm = ent + + info = None + if has_modrm: + if i >= end: + raise Desync("truncated modrm") + modrm = buf[i] + i += 1 + mod = modrm >> 6 + reg = (modrm >> 3) & 7 + rm = modrm & 7 + if imm == "g6": + imm = "b" if reg < 2 else 0 + elif imm == "g7": + imm = "z" if reg < 2 else 0 + + if mod != 3: + if addr67: + # 16-bit ModRM: different table entirely. Rare in MSVC code and + # never the member-address idiom -- skip rather than mis-decode. + raise Desync("16-bit addressing (0x67)") + base = index = None + scale = 1 + if rm == 4: + if i >= end: + raise Desync("truncated sib") + sib = buf[i] + i += 1 + scale = 1 << (sib >> 6) + idx = (sib >> 3) & 7 + bse = sib & 7 + index = None if idx == 4 else R32[idx] + if bse == 5 and mod == 0: + base = None # disp32 absolute + index + else: + base = R32[bse] + elif rm == 5 and mod == 0: + base = None # disp32 absolute + else: + base = R32[rm] + + disp = 0 + dispsize = 0 + if mod == 1: + if i >= end: + raise Desync("truncated disp8") + disp = struct.unpack_from(" end: + raise Desync("truncated disp32") + disp = struct.unpack_from(" end: + raise Desync("truncated imm") + if i == start: + raise Desync("zero length") + return i - start, info + + +# ------------------------------------------------------------------- PE / IO +def load_pe(path): + """Return (image_base, [(va_start, va_end, bytes, name)]) for exec sections.""" + data = open(path, "rb").read() + pe = struct.unpack_from("= 0 else '-'}0x{abs(disp):x}]" + txt = f"{mnem} {dst},{ea}" if lea else f"{mnem} {ea}" + return (f" 0x{va:08x} {txt:<32} " + f"{fname}+0x{va - fva:x}{'' if inbody else ' [past-body]'} ({hx})") + + +def query(argv): + want = int(argv[0], 0) + only_lea = "--lea" in argv + only_this = "--this" in argv + show_all = "--all" in argv + d = load_index() + funcs = d["funcs"] + keys = d["byDisp"].get(str(want), []) + hits = [d["sites"][k] for k in keys] + if only_lea: + hits = [h for h in hits if h[6]] + if only_this: + hits = [h for h in hits if h[2] in THIS_REGS] + hits.sort(key=lambda h: h[0]) + + ranked, other = [], [] + for h in hits: + (ranked if h[2] in THIS_REGS else other).append(h) + print(f"displacement 0x{want:x} ({want}): {len(hits)} site(s)") + print(f"\n== base is a likely `this` ({'/'.join(THIS_REGS)}): {len(ranked)} ==") + byfn = {} + for h in ranked: + byfn.setdefault(h[7], []).append(h) + for fi in sorted(byfn, key=lambda f: funcs[f][0]): + print(f" {funcs[fi][1]} @0x{funcs[fi][0]:08x}") + for h in byfn[fi]: + print(fmt(h, funcs)) + if other: + print(f"\n== other base regs (esp/ebp = locals, eax/edx = temps): {len(other)} ==") + if show_all: + for h in other: + print(fmt(h, funcs)) + else: + cnt = {} + for h in other: + cnt[h[2]] = cnt.get(h[2], 0) + 1 + print(" " + ", ".join(f"{k}:{v}" for k, v in sorted(cnt.items())) + + " (--all to list)") + + +def func_dump(argv): + fva = int(argv[0], 0) + d = load_index() + funcs = d["funcs"] + fi = next((i for i, f in enumerate(funcs) if f[0] == fva), None) + if fi is None: + sys.exit(f"no function at 0x{fva:08x}") + print(f"{funcs[fi][1]} @ 0x{fva:08x} size {funcs[fi][2]}") + for s in d["sites"]: + if s[7] == fi: + print(fmt(s, funcs)) + + +def stats(): + d = load_index() + st = d["stats"] + for k, v in st.items(): + print(f"{k:16}: {v}") + hist = {} + for s in d["sites"]: + if s[2] in THIS_REGS: + hist[s[1]] = hist.get(s[1], 0) + 1 + print(f"\ndistinct displacements off ecx/esi/edi/ebx: {len(hist)}") + top = sorted(hist.items(), key=lambda kv: -kv[1])[:15] + print("most common: " + ", ".join(f"0x{k:x}({v})" for k, v in top)) + + +_STORE = {0x89, 0x01, 0x29, 0x39, 0x09, 0x21, 0x31, 0x19, 0x11, + 0x88, 0x00, 0x08, 0x10, 0x18, 0x20, 0x28, 0x30, 0x38} + + +def _reg_form(raw): + """Render the mod=3 (register-to-register) forms, which are exactly the + ones that carry the arithmetic you care about when recovering a struct + stride: `imul ecx,ecx,0x2c`, `sar edx,3`, `sub ecx,edi`.""" + p = 0 + while p < len(raw) and raw[p] in (0x66, 0x67, 0xF0, 0xF2, 0xF3, + 0x2E, 0x36, 0x3E, 0x26, 0x64, 0x65): + p += 1 + op = raw[p] + if op == 0x0F or p + 1 >= len(raw): + return mnemonic(op == 0x0F, raw[p + 1] if op == 0x0F else op, 0) + m = raw[p + 1] + mod, reg, rm = m >> 6, (m >> 3) & 7, m & 7 + if mod != 3: + # memory operand: render it and show which register is the other half, + # so a load reads `mov ecx,[edi+4]` not a bare `mov`. + q = p + 2 + ea = "[" + if rm == 4: + sib = raw[q]; q += 1 + bse, ix = sib & 7, (sib >> 3) & 7 + ea += ("" if (bse == 5 and mod == 0) else R32[bse]) + if ix != 4: + ea += f"+{R32[ix]}*{1 << (sib >> 6)}" + elif not (rm == 5 and mod == 0): + ea += R32[rm] + if mod == 1 and q < len(raw): + dv = struct.unpack_from("= 0 else '-'}0x{abs(dv):x}" + elif mod == 2 and q + 4 <= len(raw): + dv = struct.unpack_from("= 0 else '-'}0x{abs(dv):x}" + ea += "]" + nm = mnemonic(False, op, reg) + if op in _STORE: + return f"{nm} {ea},{R32[reg]}" + if op in (0x80, 0x81, 0x83, 0xC6, 0xC7, 0xF6, 0xF7, 0xFF, 0xFE, + 0xC0, 0xC1, 0xD0, 0xD1, 0xD2, 0xD3) or op >= 0xD8: + return f"{nm} {ea}" + return f"{nm} {R32[reg]},{ea}" + r, x = R32[reg], R32[rm] + if op in (0x89, 0x01, 0x29, 0x39, 0x09, 0x21, 0x31, 0x19, 0x11, 0x85, 0x87): + a, b = x, r # Ev,Gv -> dst is rm + else: + a, b = r, x # Gv,Ev -> dst is reg + nm = mnemonic(False, op, reg) + if op in (0xC0, 0xC1): # shift group, imm8 + return f"{_SHIFT[reg]} {x},{raw[p + 2]}" + if op in (0xD1, 0xD3): + return f"{_SHIFT[reg]} {x},{'cl' if op == 0xD3 else '1'}" + if op == 0x6B: # imul r,rm,imm8 <- the stride! + return f"imul {r},{x},0x{struct.unpack_from(' [count]`. Length-accurate; the + mnemonics are minimal but call/jmp targets are resolved to function names, + which is what you actually need to follow a `lea this,[obj+N]; call` pair.""" + va = int(argv[0], 0) + n = int(argv[1]) if len(argv) > 1 else 24 + _, secs = load_pe(EXE) + d = load_index() + fmap = {f[0]: f[1] for f in d["funcs"]} + sec = next((s for s in secs if s[0] <= va < s[1]), None) + if sec is None: + sys.exit("address not in an executable section") + sva, eva, buf, _ = sec + i = va - sva + for _ in range(n): + try: + ln, info = decode(buf, i, len(buf)) + except Desync as e: + print(f"0x{sva + i:08x} ") + return + raw = buf[i:i + ln] + cur = sva + i + txt = "" + # resolve the two rel32 forms and the rel8 jumps by hand + if raw[0] in (0xE8, 0xE9) and ln == 5: + tgt = cur + 5 + struct.unpack_from("=1 setting drops it. + Read the full `query` output before trusting a cohort shortlist. + """ + want = int(argv[0], 0) + known = [] + minhits = 2 + for a in argv[1:]: + if a.startswith("--known="): + known = [int(x, 0) for x in a[8:].split(",")] + elif a.startswith("--min="): + minhits = int(a[6:]) + if not known: + sys.exit("need --known=") + d = load_index() + funcs = d["funcs"] + kset = set(known) + touched = {} # funcidx -> set(offset) + target = {} # funcidx -> [sites at `want`] + for s in d["sites"]: + if s[2] not in THIS_REGS: + continue + if s[1] in kset: + touched.setdefault(s[7], set()).add(s[1]) + if s[1] == want: + target.setdefault(s[7], []).append(s) + rows = [] + for fi, sites in target.items(): + hits = touched.get(fi, set()) - {want} + if len(hits) >= minhits: + rows.append((len(hits), fi, sites, hits)) + rows.sort(key=lambda r: -r[0]) + print(f"functions touching 0x{want:x} AND >={minhits} other known offsets " + f"of this class: {len(rows)} of {len(target)} candidates " + f"({100.0 * len(rows) / max(1, len(target)):.0f}% kept)") + for n, fi, sites, hits in rows: + fva, fname, _ = funcs[fi] + hx = ",".join(f"0x{h:x}" for h in sorted(hits)) + print(f"\n {fname} @0x{fva:08x} [{n} co-hits: {hx}]") + for s in sites: + print(fmt(s, funcs)) + + +# ------------------------------------------------- naive scan, for FP measure +def brute(argv): + """Naive 'search the raw bytes' scan -- the thing you'd write without a + decoder. Only exists so we can put a NUMBER on how wrong it is. + + `--ops 8d` (default) = lea only. `--ops 8d,89,8b,01,03,39,3b` = the wider + scan you would actually need, since a member is read/written far more often + than its address is taken. The wider the opcode set and the smaller the + displacement, the worse the naive method gets -- that is the point. + """ + want = int(argv[0], 0) + ops = {0x8D} + for a in argv[1:]: + if a.startswith("--ops="): + ops = {int(x, 16) for x in a[6:].split(",")} + _, secs = load_pe(EXE) + d = load_index() + real = {s[0] for s in d["sites"] if s[1] == want} + found = [] + d8 = want if -128 <= want <= 127 else None + pat32 = struct.pack("> 6, m & 7 + if mod == 3 or mod == 0: + continue + k = i + 2 + (1 if rm == 4 else 0) + if mod == 1 and d8 is not None: + if struct.unpack_from("