From 3d05f954f8c69766200b5f1b1f4556003b6d384e Mon Sep 17 00:00:00 2001 From: lab Date: Mon, 7 Sep 2026 17:12:02 +0000 Subject: [PATCH] add RE strategy: annotated-RE + patching core; matching-decomp rejected --- findings/00-strategy.md | 63 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 findings/00-strategy.md diff --git a/findings/00-strategy.md b/findings/00-strategy.md new file mode 100644 index 0000000..b257043 --- /dev/null +++ b/findings/00-strategy.md @@ -0,0 +1,63 @@ +# SOTS1 — RE strategy + +Founding strategy for reverse-engineering **Sword of the Stars (2006)** — original + expansions, +GOG Complete Collection v1.8.1, a 32-bit DirectX 9 MSVC C++ Windows game. Retail binary only; +no source was released. Personal, owned copy. + +Lab: Ghidra 12 + ReVa (AI-assisted, MCP) in CT111; Win10 VM140 with x64dbg / Cheat Engine / +RenderDoc / apitrace / DXVK; Samba share `/srv/re-lab`. See trikilli `services/re-lab.md` for infra. + +## Goals (priority order) +1. Understand the engine and game logic. +2. Enable modding and bug-fixing — the slow tactical-battle load on many-core CPUs; the 2 GB/4 GB memory ceiling. +3. Preservation — keep it runnable on modern Windows. +4. Interop tools — save/mod editors, file-format parsers. + +## Approach decision +Five styles evaluated: + +| Style | Verdict | Why | +|---|---|---| +| Matching (byte-identical) decomp | **Reject** | Optimized MSVC C++ (inlining, STL, RTTI); no PC-C++ prior art; no payoff until ~complete | +| Functional reimplementation | **Defer** | Right end-state, wrong first move; if ever, OpenRCT2 incremental shim-DLL method, ship engine only never assets | +| Ghidra/ReVa annotated RE | **Core** | Highest for goal 1; our lab is built for it; substrate for everything else | +| Binary patching / modding | **Do first** | Highest for goals 2 & 3; both flagship bugs already have community mitigations | +| File-format / interop RE | **Opportunistic** | Safest legally; partly already done by the community | + +## Known ground (community — don't re-derive) +- **Archives:** `sots.gob` and expansion `.gob`s are **renamed, uncompressed ZIPs** — 7-Zip opens them. No archive RE needed. +- **Modding is file-override:** drop a file at the same relative path under `./Mods/`; `sots.ini` load order prefers it over the `.gob`. Textures = DDS; models = custom format with community Xporter v1.19. +- **Save format partly RE'd:** BardezAnAvatar/Sots.Sots1.SavedGameEditor (C#, v1.8+, player/tech-tree/system structs), ghbplayer/SOTSedit, a SourceForge editor. Use as a Rosetta Stone for the in-memory object model, not just on-disk saves. +- **Bug mitigations to validate on our GOG copy (behavioral baseline):** + - LAA/4GB: flip the Large-Address-Aware PE flag (safe on GOG; the Steam build breaks with checksum "error 51"). + - Battle load: `[CPU] ForceSingleCore=1` in `sots.ini` — the engine is single-threaded. Goal is to find *why* many cores hurt, not just set the flag. +- **Compiler:** confirm from the binary, don't assume — read the PE **Rich header** + CRT imports (`msvcr71/80/90`). Candidates VS2003/2005/2008. MSVC C++ ⇒ **RTTI present** (Ghidra's Windows x86 PE RTTI Analyzer recovers vftables + class hierarchy — big head start); MSVC name mangling (Ghidra demangles); expect heavily inlined STL. + +## Phased plan +- **Phase 0 — baseline & fingerprint.** Unpack `sots.gob`; inventory data + `sots.ini` keys. Ghidra auto-analysis + **RTTI analyzer** + demangle; read Rich header / CRT imports to pin the compiler. Validate LAA + `ForceSingleCore` to get a known-good behavioral baseline. Commit Ghidra project + notes. +- **Phase 1 — annotated RE of the three hot paths** (core program): + 1. **Battle-load path** (top priority). Timestamp the tactical-load transition under apitrace; break in x64dbg, walk up into Ghidra. Diff `ForceSingleCore=0` vs `=1` to localize core-count-sensitive code (`GetSystemInfo`/`dwNumberOfProcessors`, `CreateThread` loops, `QueryPerformanceCounter` spin-waits, per-core allocation). + 2. **Memory-allocation path** (LAA). Find the global allocator/arena; instrument peak use in the large-battle crash; decide whether LAA alone suffices or a deeper allocator shim is warranted. + 3. **D3D9 device init.** RenderDoc/apitrace `Direct3DCreate9`→`CreateDevice`; xref `d3d9.dll` imports to the renderer bootstrap + present loop. Anchor for future shims and compat fixes. + Cross-reference recovered structs against the existing save-editor field defs to name core objects fast. +- **Phase 2 — ship fixes as patches/shims.** Package LAA + `sots.ini` tweaks as a *patcher*, not a patched binary. Prototype the battle-load fix as an asm/hex patch or DLL wrapper (ASI loader / DXVK-style shim) — reversible and legally clean. +- **Phase 3 — interop tools.** Extend the save-format work into a maintained editor + a published format spec (format facts are safe to publish). Add `.gob`/DDS/model helpers only where community tooling is thin. +- **Deferred:** matching decomp (skip); full reimplementation (someday, engine-only, users bring assets). + +## First concrete targets +Battle-load path · memory-allocation path · D3D9 device init. + +## Key sources +- SOTS1 modding & file structure — https://swordofthestars.fandom.com/wiki/Introduction_to_modding , https://swordofthestars.fandom.com/wiki/Community_Mods +- Fixes/compat — https://www.pcgamingwiki.com/wiki/Sword_of_the_Stars +- Battle-load / threading — https://steamcommunity.com/app/42890/discussions/0/615086038683223062 +- Save-format head start — https://github.com/BardezAnAvatar/Sots.Sots1.SavedGameEditor , https://github.com/ghbplayer/SOTSedit +- MSVC C++ RE technique — https://dennisbabkin.com/blog/?t=reverse-engineer-virtual-functions-vs-cpp-compiler-vtable-purecall-cfg , https://www.christophbrill.de/en/posts/ghidra_msvc_win32/ +- Incremental reimplementation model — https://desosa.nl/projects/openrct2/2020/03/02/openrct2,-porting-rollercoaster-tycoon-into-2020.html + +## Open uncertainties +- Exact compiler/CRT of the v1.8.1 binary unconfirmed — resolve from Rich header + CRT imports first. +- Many-core battle-load root cause is inferred (single-threaded engine + core-scaling logic) — confirm by the Phase-1 diff. +- Community save editors cover what their authors needed (tech tree, players, systems); fleet/combat-state coverage may be partial. + +_Research 2026-09-07._