sots-re/tools
alex d7ea0a048c lane D: automated struct recovery from the IStreamable serializers
Every serializable class carries an enumeration of its own fields -- its
Write(Stream&), walking the members in order with a 4-char tag. This decodes
that idiom mechanically for the whole binary in 0.35 s.

Validation first (tools/serializers.py validate), against answers the campaign
already had before the tool existed:
  A  305/307 field offsets+kinds exact across 17 classes, 0 WRONG, vs
     struct-recovery.md 1-4 and observedtech-append.md
  B  sizeof from the container-stride divides: ObservedTech 0x2c, MoraleEvent
     0x50, PlayerReport 0x30, DiplomacyStats 0x24 -- all matching
  C  22 of save_reader.py's shapes, tag order identical (Sys 78 tags,
     Player 104, CreateParams 25, Ship 22): 22 agree, 0 disagree
  D  Read/Write cross-check on every class: 437/437 field offsets agree

At scale: 386 classes with a Write, 1,682 member fields.
  verified 87 (542 fields) | clean 77 (328) | unnamed 176 (471)
  partial 31 (341) | empty 15
  58 classes with a sizeof corroborated by a second line of evidence
  (45 container stride, 13 enumeration meeting the embedding bound); the rest
  report a lower bound and say so.

Four things each worth 10-170 classes: the RTTI class hierarchy descriptor as
the only honest "is this an IStreamable" test (a 3-slot vftable also matches
TacAISquadRule_* and the row parsers); mod=0 memory operands, which x86disp.py
cannot index and which hide every field at offset 0; the member->id pointer
idiom behind every handle field; and sub-writers, both base-class and private
(StrategyServer's six id lists live in FUN_00794cd0).

Failure classes are enumerated in the finding -- 176 anonymous-tag classes are
a hard limit on names but not on layout, and the other 64 are bounded
mechanical fixes. Two fields lost to a value assembled across a branch were
left unrecovered rather than patched with an unverifiable heuristic.

Write-back: 288 structures + 328 labels into Ghidra (0 failures), +201
addresses.json entries, header regenerated with tools/gen_addresses.py.

Note: ghidra/addresses.json also carries lane V's already-written live
confirmation text on ObservedTech_sizeof and ServerPlayer_off_ObservedTechs --
their edit, swept in only because we share the file.
2026-09-08 05:51:37 -04:00
..
cache_functions.py lane X: x86 displacement xref scanner; pin sizeof(ObservedTech) and its append site 2026-09-08 04:44:18 -04:00
dashboard.py dashboard: coverage generator + first render 2026-09-07 17:36:27 -04:00
DASHBOARD_README.md dashboard: coverage generator + first render 2026-09-07 17:36:27 -04:00
gen_addresses.py b3 static findings: rng draw mapping, NextInt inclusive, float32 odds; address contract +17 2026-09-07 23:14:51 -04:00
reva_call.py reva_call.py: HTTP fallback for Ghidra when the MCP link drops; claim VM140 for recapture lane 2026-09-08 02:12:16 -04:00
rtti_map.py lane D: automated struct recovery from the IStreamable serializers 2026-09-08 05:51:37 -04:00
serializers.py lane D: automated struct recovery from the IStreamable serializers 2026-09-08 05:51:37 -04:00
serializers_ghidra.py lane D: automated struct recovery from the IStreamable serializers 2026-09-08 05:51:37 -04:00
serializers_golden.py lane D: automated struct recovery from the IStreamable serializers 2026-09-08 05:51:37 -04:00
strfootprint.py lane S: std::string is 0x1c binary-wide; ObservedTech element fully mapped 2026-09-08 05:09:33 -04:00
x86disp.py lane X: x86 displacement xref scanner; pin sizeof(ObservedTech) and its append site 2026-09-08 04:44:18 -04:00