add RE strategy: annotated-RE + patching core; matching-decomp rejected

findings/00-strategy.md

@@ -0,0 +1,63 @@
+# SOTS1 — RE strategy
+
+Founding strategy for reverse-engineering **Sword of the Stars (2006)** — original + expansions,
+GOG Complete Collection v1.8.1, a 32-bit DirectX 9 MSVC C++ Windows game. Retail binary only;
+no source was released. Personal, owned copy.
+
+Lab: Ghidra 12 + ReVa (AI-assisted, MCP) in CT111; Win10 VM140 with x64dbg / Cheat Engine /
+RenderDoc / apitrace / DXVK; Samba share `/srv/re-lab`. See trikilli `services/re-lab.md` for infra.
+
+## Goals (priority order)
+1. Understand the engine and game logic.
+2. Enable modding and bug-fixing — the slow tactical-battle load on many-core CPUs; the 2 GB/4 GB memory ceiling.
+3. Preservation — keep it runnable on modern Windows.
+4. Interop tools — save/mod editors, file-format parsers.
+
+## Approach decision
+Five styles evaluated:
+
+| Style | Verdict | Why |
+|---|---|---|
+| Matching (byte-identical) decomp | **Reject** | Optimized MSVC C++ (inlining, STL, RTTI); no PC-C++ prior art; no payoff until ~complete |
+| Functional reimplementation | **Defer** | Right end-state, wrong first move; if ever, OpenRCT2 incremental shim-DLL method, ship engine only never assets |
+| Ghidra/ReVa annotated RE | **Core** | Highest for goal 1; our lab is built for it; substrate for everything else |
+| Binary patching / modding | **Do first** | Highest for goals 2 & 3; both flagship bugs already have community mitigations |
+| File-format / interop RE | **Opportunistic** | Safest legally; partly already done by the community |
+
+## Known ground (community — don't re-derive)
+- **Archives:** `sots.gob` and expansion `.gob`s are **renamed, uncompressed ZIPs** — 7-Zip opens them. No archive RE needed.
+- **Modding is file-override:** drop a file at the same relative path under `./Mods/`; `sots.ini` load order prefers it over the `.gob`. Textures = DDS; models = custom format with community Xporter v1.19.
+- **Save format partly RE'd:** BardezAnAvatar/Sots.Sots1.SavedGameEditor (C#, v1.8+, player/tech-tree/system structs), ghbplayer/SOTSedit, a SourceForge editor. Use as a Rosetta Stone for the in-memory object model, not just on-disk saves.
+- **Bug mitigations to validate on our GOG copy (behavioral baseline):**
+  - LAA/4GB: flip the Large-Address-Aware PE flag (safe on GOG; the Steam build breaks with checksum "error 51").
+  - Battle load: `[CPU] ForceSingleCore=1` in `sots.ini` — the engine is single-threaded. Goal is to find *why* many cores hurt, not just set the flag.
+- **Compiler:** confirm from the binary, don't assume — read the PE **Rich header** + CRT imports (`msvcr71/80/90`). Candidates VS2003/2005/2008. MSVC C++ ⇒ **RTTI present** (Ghidra's Windows x86 PE RTTI Analyzer recovers vftables + class hierarchy — big head start); MSVC name mangling (Ghidra demangles); expect heavily inlined STL.
+
+## Phased plan
+- **Phase 0 — baseline & fingerprint.** Unpack `sots.gob`; inventory data + `sots.ini` keys. Ghidra auto-analysis + **RTTI analyzer** + demangle; read Rich header / CRT imports to pin the compiler. Validate LAA + `ForceSingleCore` to get a known-good behavioral baseline. Commit Ghidra project + notes.
+- **Phase 1 — annotated RE of the three hot paths** (core program):
+  1. **Battle-load path** (top priority). Timestamp the tactical-load transition under apitrace; break in x64dbg, walk up into Ghidra. Diff `ForceSingleCore=0` vs `=1` to localize core-count-sensitive code (`GetSystemInfo`/`dwNumberOfProcessors`, `CreateThread` loops, `QueryPerformanceCounter` spin-waits, per-core allocation).
+  2. **Memory-allocation path** (LAA). Find the global allocator/arena; instrument peak use in the large-battle crash; decide whether LAA alone suffices or a deeper allocator shim is warranted.
+  3. **D3D9 device init.** RenderDoc/apitrace `Direct3DCreate9`→`CreateDevice`; xref `d3d9.dll` imports to the renderer bootstrap + present loop. Anchor for future shims and compat fixes.
+  Cross-reference recovered structs against the existing save-editor field defs to name core objects fast.
+- **Phase 2 — ship fixes as patches/shims.** Package LAA + `sots.ini` tweaks as a *patcher*, not a patched binary. Prototype the battle-load fix as an asm/hex patch or DLL wrapper (ASI loader / DXVK-style shim) — reversible and legally clean.
+- **Phase 3 — interop tools.** Extend the save-format work into a maintained editor + a published format spec (format facts are safe to publish). Add `.gob`/DDS/model helpers only where community tooling is thin.
+- **Deferred:** matching decomp (skip); full reimplementation (someday, engine-only, users bring assets).
+
+## First concrete targets
+Battle-load path · memory-allocation path · D3D9 device init.
+
+## Key sources
+- SOTS1 modding & file structure — https://swordofthestars.fandom.com/wiki/Introduction_to_modding , https://swordofthestars.fandom.com/wiki/Community_Mods
+- Fixes/compat — https://www.pcgamingwiki.com/wiki/Sword_of_the_Stars
+- Battle-load / threading — https://steamcommunity.com/app/42890/discussions/0/615086038683223062
+- Save-format head start — https://github.com/BardezAnAvatar/Sots.Sots1.SavedGameEditor , https://github.com/ghbplayer/SOTSedit
+- MSVC C++ RE technique — https://dennisbabkin.com/blog/?t=reverse-engineer-virtual-functions-vs-cpp-compiler-vtable-purecall-cfg , https://www.christophbrill.de/en/posts/ghidra_msvc_win32/
+- Incremental reimplementation model — https://desosa.nl/projects/openrct2/2020/03/02/openrct2,-porting-rollercoaster-tycoon-into-2020.html
+
+## Open uncertainties
+- Exact compiler/CRT of the v1.8.1 binary unconfirmed — resolve from Rich header + CRT imports first.
+- Many-core battle-load root cause is inferred (single-threaded engine + core-scaling logic) — confirm by the Phase-1 diff.
+- Community save editors cover what their authors needed (tech tree, players, systems); fleet/combat-state coverage may be partial.
+
+_Research 2026-09-07._