From-scratch reimplementation of the Sword of the Stars (2006) engine. Clean-room; engine code only — bring your own game copy. Planning/RE evidence live in sots-re.
c662f73b69
The last opaque block of the save format. Reconciles lane W's provable negative
against lane O's issued-order saves: the two disagreed because three different
things were being compared as one.
Read out of Game::TurnCommands::Write (0x00842540, 764 bytes, no loops of its
own). Two halves:
* a PROLOGUE of six flag-gated groups. Each is a WriteBool on a member,
followed only when set by that command's payload. Write order is NOT offset
order -- each gate sits after its payload in the struct -- so the class is
one of the 89 whose offset-sorted layout view cannot be aligned to the wire.
That, not a branch, is why the sorted view showed an i32 where the save has
a bool at item 4.
* TWENTY-SEVEN std::list<T> members (0x70..0x1a8, stride 0xc, allocator-last),
each written by its own helper as WriteInt(size) then size element records.
All 27 are always written, so an empty list still costs one zero int.
So the recovered 44 items are 17 member writes plus ONE ITEM PER CONTAINER CALL
SITE: the linear recovery keeps the call site, guesses its kind from an element
field it could resolve, and drops the count word. 44 - 17 = 27 = the number of
lists. The "27 trailing ints with only 22 i32 slots" objection is that same 27
seen from both sides -- a kind mismatch in the table, not a structural
impossibility. And 8 prologue items + 27 zero counts = the 35-item block every
no-orders save carries bit-identically.
Item arithmetic closes to the unit on all five distinct workloads (35/38/41/61/
123). One correction to the provisional layout: the fleet-move element ends in a
COUNTED route vector {fleetId, nHops, nHops x systemId}, not a fixed quadruple --
with a four-item element the writer would need 26 lists on one save and 28 on
another, and it has 27.
Conformance: the generic check cannot be used here. Every tag is "." so the LCS
degenerates to a strict positional compare in which any primitive disagreement is
fatal, and the table's tail describes elements where the wire has counts. A
dedicated check states what is checkable instead: the 17-item prologue item for
item (17/17, and SchemaProbe takes every branch, so this is real evidence that
the conditional structure read from the instruction stream is the one the
recovery flattened) and the tail count (27 lists vs 27 table items). The tail is
reported as wire-only, never claimed as matched.
86 shapes / 838 items -> 87 / 856, still 0 MISMATCH.
Twenty-two of the 27 lists are HYPOTHESES: the scalar sequence comes straight off
the helper, but no save exercises them. Nested element bodies not otherwise
modelled here (ShipDesignDef, FleetLayout, WeaponGroups, DefenceLayout,
RaidTargets) are carried as opaque Nodes rather than guessed at, so a wrong body
cannot desynchronise a reader. select() matches the exact ".TurnCommands_v5"
suffix, so a future _v6 falls back to the carried Node.
SECOND DEFECT, present at main and unrelated to this block: zuul-turn23-fleet23
is the first save with a non-empty NVs list, and its element's leading id was
typed positionally as "." where the real tag is PID. The typed round trip on that
save differed at 0x89c14. Rule 6 exactly -- a path no save exercised was a
hypothesis flying as a fact. Fixed, and independently corroborated: the recovered
table for Game::ServerSystem names that item PID, and the Sys row moves from
102 matched / 1 wire-only / 3 shape-only to 103 / 0 / 2.
Coverage (CoverageArchive typed-vs-carried, not round-trip success): all eleven
saves 99.7-99.9% -> 100.0%; opaque items 37/43/63/132 -> 2 everywhere, and those
two are the deliberately-carried MT19937 block. Nothing else in any save we hold
is untyped. Ratchet 99.8 -> 99.99. Round trip byte-identical on all eleven
(newly so on zuul-turn23-fleet23).
test_save now asserts, on every real save, that each TurnCommands_v5 block is
consumed by the prologue plus the 27 lists with nothing left over -- the
item-granular statement a wrong list count or element width breaks first.
clean-room OK; host ctest 36/36; test_save 11 saves 0 failures. src/shim/ not
touched, so no cross-build was needed.
|
||
|---|---|---|
| cmake | ||
| docs | ||
| include/generated | ||
| src | ||
| tests | ||
| third_party | ||
| tools | ||
| .gitignore | ||
| CMakeLists.txt | ||
| CMakePresets.json | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| README.md | ||
sots-engine
A from-scratch, functional reimplementation of the engine behind Sword of the Stars (2006). Not a byte-for-byte decompilation: behavior-equivalent code, built up one verified piece at a time (OpenRCT2-style) until the tree can build the whole application on its own.
Bring your own game. This repository contains engine code only. Game data, assets, saves, and
the original binaries are never included; tests and tools read an owner-supplied copy via
$SOTS_DATA_DIR. See CONTRIBUTING.md for the clean-room rules.
Status
Phase 2. M0 done — the shim frontend (src/shim/, a proxy binkw32.dll the original game
loads) builds, deploys, hooks, and logs from a real game launch. Engine code accrues under
src/mars/ and src/game/; each module is oracle-verified against the owner's game data before merge:
mars/parse— brace-block +.effectreaders (1,531/1,531 files agree with the reference)mars/text— flat key/value tables, id manifests, CSV (64/64 files agree)game/sim— strategic formulas (economy, research, colonies, movement) as pure functions; 356 hand-computed checksmars/vfs—.gob(ZIP) archive reader + loose-file override; entry counts and bytes verified againstunzipmars/stream— the game's self-describing save/serialization format (reader, writer, typed shapes) + gzip; three real saves round-trip byte-identicalmars/rng— MT19937 with save-state load/store; layout confirmed against real saves, draw mappings read off the binary (docs/mars-rng.md)game/data— typed catalogs (weapons, ship sections, turrets, id registries, tech tree, strings) with cross-reference checks; 229k values agree with the referencegame/design— ship-design assembly/fit/tech-gating rules and derived stats; validates all 127 stock designs from real saves
Build: cmake --preset host && cmake --build --preset host && ctest --preset host (Linux);
tools/sync-build.sh cross-builds the shim on the lab box and stages it for deployment.
Layout (grows with the work)
src/shim/— binkw32 proxy + hooks + old-vs-new compare harness (frontend #1)src/mars/,src/game/— the engine and game reimplementation (accruing)include/generated/sots_addresses.h— binary facts (RVAs/prototypes), generated from the RE repotests/— host tests; real-data tests skip unless$SOTS_DATA_DIRis settools/— build (MinGW i686 cross) and deploy scripts
Planning, findings, and verification evidence are tracked in the private RE repo (sots-re).