# SOTS1 — RE strategy

Founding strategy for reverse-engineering **Sword of the Stars (2006)** — original + expansions,
GOG Complete Collection v1.8.1, a 32-bit DirectX 9 MSVC C++ Windows game. Retail binary only;
no source was released. Personal, owned copy.

Lab: Ghidra 12 + ReVa (AI-assisted, MCP) in CT111; Win10 VM140 with x64dbg / Cheat Engine /
RenderDoc / apitrace / DXVK; Samba share `/srv/re-lab`. Infra: trikilli `services/re-lab.md`.

## North star (decided 2026-09-07)
**A functional reimplementation of the engine — behavior-equivalent, NOT byte-for-byte** — reached
by broad engine understanding first. Think OpenRCT2 / OpenMW / DevilutionX: a re-written engine that
reads the *original* assets and reproduces behavior, not a byte-identical source rebuild. Byte-matching
decompilation is explicitly **out of scope**. Bug-fixes and mods are welcome byproducts and useful
correctness checks along the way, not the primary goal.

## Goals (priority order)
1. **Broad engine understanding** — map the whole architecture, not just hot spots: main loop & tick,
   subsystem boundaries, the core object model (empires, systems, fleets, tech), turn/AI logic,
   the renderer, and save/load + asset I/O.
2. Functional reimplementation, built incrementally on that understanding (see method below).
3. Bug-fixing / modding as validation and early wins (battle-load slowness; 2 GB/4 GB ceiling).
4. Preservation and interop tools (save/mod editors, format parsers).

## Approach decision
Five styles evaluated:

| Style | Verdict | Why |
|---|---|---|
| Matching (byte-identical) decomp | **Reject** | Optimized MSVC C++ (inlining, STL, RTTI); no PC-C++ prior art; and not our goal |
| **Functional reimplementation** | **North star** | Behavior-equivalent engine reading original assets; incremental, community-scalable |
| Ghidra/ReVa annotated RE | **Core near-term** | Builds the broad understanding every later step needs; our lab is built for it |
| Binary patching / modding | **Byproduct** | Fast wins + behavioral baselines; not the driver |
| File-format / interop RE | **Opportunistic** | Safe, partly already done by the community |

## Reimplementation method (the bridge from RE → new engine)
Follow the **OpenRCT2 incremental model**: as understanding of a subsystem solidifies, reimplement it
in modern C++ in a **shim DLL that the original `sots.exe` calls into**, verifying *behavior* (not bytes)
against the original at each step; grow the reimplemented surface until the original code is fully
displaced. This keeps a runnable, testable artifact at every stage instead of a big-bang rewrite.
Golden rule for any future distribution: **ship engine code only, never original assets — users bring
their owned copy.** Clean-room boundaries where it matters.

## Known ground (community — don't re-derive)
- **Archives:** `sots.gob` and expansion `.gob`s are **renamed, uncompressed ZIPs** — 7-Zip opens them.
- **Modding is file-override:** files under `./Mods/` at matching relative paths win per `sots.ini` load order. Textures = DDS; models = custom format with community Xporter v1.19.
- **Save format partly RE'd:** BardezAnAvatar/Sots.Sots1.SavedGameEditor (C#, v1.8+, player/tech/system structs), ghbplayer/SOTSedit. A Rosetta Stone for the in-memory object model — accelerates goal 1.
- **Compiler:** confirm from the binary — PE **Rich header** + CRT imports (`msvcr71/80/90`). MSVC C++ ⇒ **RTTI present** (Ghidra Windows x86 PE RTTI Analyzer recovers vftables + class hierarchy — a major head start toward the object model); MSVC name mangling; heavily inlined STL.
- **Bug baselines to validate:** LAA/4GB PE flag (safe on GOG; Steam build breaks, "error 51"); `[CPU] ForceSingleCore=1` (engine is single-threaded).

## Phased plan
- **Phase 0 — baseline & fingerprint.** Unpack `sots.gob`; inventory data + `sots.ini`. Ghidra auto-analysis
  + **RTTI analyzer** + demangle; pin the compiler (Rich header / CRT). Import the save-editor struct
  definitions as a naming seed. Validate LAA + `ForceSingleCore` for behavioral baselines. Commit project + notes.
- **Phase 1 — broad architecture map** (core near-term work). Recover and document, breadth-first:
  the entry/init path and **main loop**; the **object model** (empires, star systems, fleets, tech tree —
  cross-referenced to the save structs); the **turn/AI** update; the **renderer** (D3D9 init → present loop);
  **save/load** and **asset/gob I/O**. Produce a living architecture doc + a class/subsystem map in the repo.
  Good early anchors (they cut across the above): the battle-load path, the allocator/arena, D3D9 device init.
- **Phase 2 — reimplement incrementally.** Start with the most self-contained, well-understood subsystem
  (likely a file-format parser or a leaf subsystem) as a shim-DLL reimplementation; verify behavior; expand.
- **Phase 3 — fixes, tools, preservation** (ongoing byproducts). Patcher for LAA + battle-load; save editor;
  published format specs.
- **Deferred / out of scope:** byte-matching decompilation.

## First concrete targets
Compiler fingerprint · object-model structs (seeded from save editor) · main loop · D3D9 init ·
battle-load path · allocator. Breadth first — these anchor the wider map.

## Key sources
- Modding & file structure — https://swordofthestars.fandom.com/wiki/Introduction_to_modding , https://swordofthestars.fandom.com/wiki/Community_Mods
- Fixes/compat — https://www.pcgamingwiki.com/wiki/Sword_of_the_Stars
- Save-format head start — https://github.com/BardezAnAvatar/Sots.Sots1.SavedGameEditor , https://github.com/ghbplayer/SOTSedit
- MSVC C++ RE — https://dennisbabkin.com/blog/?t=reverse-engineer-virtual-functions-vs-cpp-compiler-vtable-purecall-cfg , https://www.christophbrill.de/en/posts/ghidra_msvc_win32/
- Incremental reimplementation model — https://desosa.nl/projects/openrct2/2020/03/02/openrct2,-porting-rollercoaster-tycoon-into-2020.html

## Open uncertainties
- Exact compiler/CRT unconfirmed — resolve from Rich header + CRT imports first.
- Many-core battle-load root cause inferred (single-threaded engine + core-scaling) — confirm empirically.
- Save-editor struct coverage may be partial (strong hint set, not a full spec).
- Reimplementation is a long arc; the value gate is the *understanding* (goal 1), which stands on its own.

_Strategy set 2026-09-07. Direction: broad understanding → functional reimplementation (not byte-matching)._

## Phase 2 — decided 2026-09-07
- **Two repos.** This repo = evidence + planning (private, binary-derived). **`alex/sots-engine`** =
  the from-scratch engine source, **public-capable from day one**: clean-room (own code only; no game
  data, decompiles, saves), MIT, C++17/CMake, assets external via `$SOTS_DATA_DIR`. Tracking for both
  stays here. Binary facts cross over only via `ghidra/addresses.json` → `tools/gen_addresses.py` →
  `sots-engine/include/generated/sots_addresses.h`.
- **Straight to the C++ shim** (no Python prototype): a proxy `binkw32.dll` (MinGW i686 on CT111,
  MinHook) with per-hook `off|trace|compare|replace` modes; compare runs original + ours on a
  snapshot and diffs; deterministic saves are the strongest equivalence oracle. The shim is the
  engine's first frontend (`src/shim/`); a standalone `src/app/` comes when enough engine exists.
- **Slow accrual:** scaffold only what each milestone needs. Order: M0 bootstrap → M1 flat-KV config
  loader → M2 id manifests → M3 Mars brace-block parser → M4 gobio VFS → serializers/RNG → turn pipeline.
- **Battle-load bug is parked** (not on the critical path); resume recipe in the backlog.