d7ea0a048c
Every serializable class carries an enumeration of its own fields -- its
Write(Stream&), walking the members in order with a 4-char tag. This decodes
that idiom mechanically for the whole binary in 0.35 s.
Validation first (tools/serializers.py validate), against answers the campaign
already had before the tool existed:
A 305/307 field offsets+kinds exact across 17 classes, 0 WRONG, vs
struct-recovery.md 1-4 and observedtech-append.md
B sizeof from the container-stride divides: ObservedTech 0x2c, MoraleEvent
0x50, PlayerReport 0x30, DiplomacyStats 0x24 -- all matching
C 22 of save_reader.py's shapes, tag order identical (Sys 78 tags,
Player 104, CreateParams 25, Ship 22): 22 agree, 0 disagree
D Read/Write cross-check on every class: 437/437 field offsets agree
At scale: 386 classes with a Write, 1,682 member fields.
verified 87 (542 fields) | clean 77 (328) | unnamed 176 (471)
partial 31 (341) | empty 15
58 classes with a sizeof corroborated by a second line of evidence
(45 container stride, 13 enumeration meeting the embedding bound); the rest
report a lower bound and say so.
Four things each worth 10-170 classes: the RTTI class hierarchy descriptor as
the only honest "is this an IStreamable" test (a 3-slot vftable also matches
TacAISquadRule_* and the row parsers); mod=0 memory operands, which x86disp.py
cannot index and which hide every field at offset 0; the member->id pointer
idiom behind every handle field; and sub-writers, both base-class and private
(StrategyServer's six id lists live in FUN_00794cd0).
Failure classes are enumerated in the finding -- 176 anonymous-tag classes are
a hard limit on names but not on layout, and the other 64 are bounded
mechanical fixes. Two fields lost to a value assembled across a branch were
left unrecovered rather than patched with an unverifiable heuristic.
Write-back: 288 structures + 328 labels into Ghidra (0 failures), +201
addresses.json entries, header regenerated with tools/gen_addresses.py.
Note: ghidra/addresses.json also carries lane V's already-written live
confirmation text on ObservedTech_sizeof and ServerPlayer_off_ObservedTechs --
their edit, swept in only because we share the file.
|
||
|---|---|---|
| campaign | ||
| findings | ||
| ghidra | ||
| guides | ||
| notes | ||
| objects | ||
| scripts | ||
| tools | ||
| verify | ||
| .gitignore | ||
| README.md | ||
sots-re
Reverse-engineering worklog for Sword of the Stars (2006, SOTS1) — the 32-bit DX9 original + expansions. The nitty-gritty: static/dynamic analysis notes, Ghidra & ReVa scripts, function/struct maps, D3D9 call traces, decomp progress, findings.
Where this runs
Analysis lab on spicy (PVE, 192.168.3.201):
- CT111
sots-re— Linux workspace: Ghidra + headless ReVa server, radare2/rizin/cutter, binwalk. Hosts the Samba share and this repo's working tree at/srv/re-lab/notes. - VM140
sots-re-win10— Win10 runtime + dynamic analysis (x64dbg, Cheat Engine, RenderDoc/apitrace, DXVK→CPU-Vulkan for GPU-less rendering).
Infra (guests, storage, network, share, ReVa endpoint) is documented from the
system-maintainer POV in trikilli → services/re-lab.md. This repo is everything else.
Layout
findings/— the running findings log (append-only), one file per subsystem.ghidra/— exported scripts, data-type archives, struct definitions.traces/— D3D9 / Win32 API call captures + analysis.scripts/— helper tooling (loaders, extractors, parsers).notes/— session notes, scratch, hypotheses.
Ownership / legality
Game binaries come from the owner's own GOG/Steam copy. RE is for personal
interoperability, bug-fixing, and preservation. Binaries themselves are not committed
here (see .gitignore) — they live on the lab's Samba share /srv/re-lab/samples.
Campaign (how this repo is run)
A 4-agent crew (defined in ~/.claude/agents/re-*.md) runs the exploration:
re-quartermaster (backlog + board) → re-analyst (maps via ReVa) →
re-verifier (proves vs real data; old-vs-new differential once reimpl starts) →
re-scribe (files the note, links it, commits).
campaign/board.md— live status board (start here).- Live tracking = Forgejo issues on
alex/sots-re(labelsstatus/*are the kanban columns;type/*,conf/*).campaign/board.mdis the editable mirror — publish withscripts/forgejo_campaign.py bootstrap(needsFORGEJO_TOKEN). campaign/backlog.md— prioritized target queue.campaign/open-questions.md— unresolved threads.findings/_template.md— the record format every finding follows.findings/{objects,control-flow,subsystems}/— the growing engine map.verify/{parsers,traces,harness,results}/— validation: struct parsers now, golden-trace replay + shim compare-mode for reimplementation.ghidra/— exported scripts + datatype archives.
Approach & north star: findings/00-strategy.md. Binary facts: findings/01-fingerprint.md.
Sibling repo
alex/sots-engine — the from-scratch engine source (clean-room, public-capable). This repo keeps the
evidence + planning for both; binary facts cross over only via ghidra/addresses.json → tools/gen_addresses.py.
guides/re-windows-2000s-howto.md— annotated bibliography + how-to for RE of mid-2000s MSVC/DX9 Windows games, with our-experience call-outs.