12 KiB
Roadmap
Groomed 2026-09-08. Replaces the phase-0/phase-2 backlog, all of which shipped. The board
(board.md) is the ledger of findings; this is the plan. The product is sots-engine — our
code. RE findings are inputs to it, and the measure of progress is how much of the game runs in
our engine, verified against the original. Every number here is measured; the source is named.
1. The engine today (sots-engine main, ~33k LOC, ctest 46/46, 1,031 addresses, clean-room, MIT)
| module | LOC | what it is | verification |
|---|---|---|---|
mars/parse, mars/text |
1,774 | brace-block + flat-KV + CSV readers (the Mars::Script pull tokenizer, first-wins keys) |
oracle 1,531/1,531 + 64/64 |
mars/vfs |
788 | .gob ZIP + native-override VFS (the ./Mods mechanism) |
10,268 files CRC-clean |
mars/stream |
5,407 | save reader/writer, 386-class wire schema, SchemaProbe, CoverageArchive |
100% named coverage, byte-identical round-trip, 11 saves |
mars/rng |
244 | MT19937 + all seven entry points (NextFloat/NextInt/Chance/NextUInt/FloatRange/IntRangeBell/GaussianRange) |
live: every draw of a turn attributed |
game/data, game/design, game/config |
3,835 | weapons/sections/tech/strings catalogs, design rules, hull class, tuning | 229,042 values 0 diffs; 127/127 designs |
game/effects, game/events |
1,531 | tech effects, EventStorage (turn-bucketed, dedup) |
live: research slice 35 calls / 0 div |
game/sim |
3,382 | economy (ComputeBudget, output term), research + unlock cascade, colony, movement |
live: budget 4,437 / output 24,357 / research 35 / movement 45 calls — all 0 div on declared regions |
game/nav, game/combat |
1,163 | route classifier + failure bits; retreat planner | offline vs 58 waypoints / 46 plans, 0 failures; retreat never instrumented |
app (sots_turn) |
1,754 | the standalone: load save → walk all three drivers' phase order → write save; divergence + completion metric | 11/11 saves run; ref pair 209 → 158 leaves, closed 51 / regressed 0 |
shim |
769 + hooks | proxy binkw32.dll + MinHook trace/compare/replace harness — scaffolding, to be displaced |
it is the instrument, not the product |
What the engine can do end to end today: read every shipped asset and every save byte-for-byte, run the economy/research/movement/output pieces to the original's numbers, and step a turn with 16 of 44 driver phases and 5 of 37 tail phases modelled (10 committed). It cannot yet reproduce a turn.
2. The milestone, restated (again — 2026-09-08 evening)
sots_turnloads a save, runs one strategic turn, and writes an autosave that byte-matches the original's.
Lane L5 showed the original cannot always do this itself. Three runs of turn1-state → turn2
gave three different autosaves, differing in exactly four leaves — one AI empire's research pick and
the derived checksum — with the instrument exonerated (two runs had identical hooks and still
differed; hooks=off was a third value). The oracle stands on ref-turn2 → turn3, where every AI
already holds a target; it fails on the neighbour, where three AI players must choose one.
This is not a contradiction of the lockstep multiplayer design — it is the reason for it. The AI is a
StrategyClient, not part of the sim: it decides once, on one machine, and its decisions travel
as TurnCommands through the same host relay as a human's. Peers reproduce the sim's response to a
command, never the decision. The sim is deterministic (lane Z's zero-residual ledger is the strategic
generator, which the AI never touches); the decision layer is not, and does not need to be.
So the rungs are:
- Rung A — byte-match modulo the command stream. Every leaf except
ModCount,Summary.Checksum, and what the AI's own orders touch. No AI needed. Canonical reference pair is nowturn2-state → turn3-state— deterministic, oracle-hashed.turn1-state → turn2stays as a secondary pair with its four non-deterministic leaves masked. - Rung B — byte-match given a recorded command stream.
sots_turn --turn-commands <blocks>replays the AI's (and human's) orders captured from a real turn, and the output must match that turn's autosave byte-for-byte. This is what "the engine is the game" means when the game's own decision layer is not reproducible from a save. Lane AI1's fallback becomes the primary path. - Rung C — a deterministic
game/aiwhose choice is provably inside the original's tie set. Our engine need not inherit the original's tiebreak. If the variation is a tie broken per-process (the leading hypothesis), the original's possible outcomes are a small enumerable set; we pick canonically and verify that our choice is one of exactly k outcomes the original can produce, we can name all k, and every observed original run lands inside that set. That is stronger than behavioural equivalence and keeps the byte-match live on every run where the tiebreaks agree. (User's framing, 2026-09-08: make the AI deterministic and the old approach still pays.)
Mechanism, stated as inference: only one of three AI players varies, which looks like a tie broken by something per-process — most likely iteration order over a pointer-keyed container under ASLR — rather than a time seed (which would move all three). One probe separates them: read the AI client's generator state after construction in two processes. Routed to lane L1.
3. Engine work to Rung A (ranked by leaves closed per lane)
| # | engine deliverable | module | RE input (done unless noted) | closes |
|---|---|---|---|---|
| 1 | ComputeOutput turn pathS11 civilian growth. One input — imperial carrying capacity — gates the human's Sav, PvSav, BnkEl and BnkPr on both pairs. |
game/sim |
lanes E1/C3 | the budget leaves + unblocks T36 |
| 1b | S11 civilian growth — the new item 1 |
game/sim |
needs imperial carrying capacity | the human budget cluster |
| 1c | Ship::RepairCost 0x00815180 — the ninth output input, unread; taken as 0, so a colony with a damaged fleet reads high |
game/sim |
unread | output accuracy |
| 2 | ShipCensus + alliance mask into BuildTurnRecord |
app |
lanes D2/A2 | +24 turnstats leaves when item 1 lands |
| 3 | encounter-detection draws in the generator model | game/sim + app |
lane I bound; lane H measuring now | last 2 RNG words |
| 4 | nve visibility recordAFlags, not VFlags/EFlags, and all three agree on nearly every corpus system so the wrong one looks right. Remainder: TShn (10/pair, gate proven NOT AFlags, watchpoint specified), rcex (6/pair, unexplained). |
game/sim |
lane E3 | 51 leaves closed |
| 5 | post events into the save's turn bucket (P11 + tail event phases) | app + game/events |
model exists; wiring only | events subtree |
| 6 | Player.Status restore+0x164 — ProcessTurn's encounter loop writes 1, ResumePlaying writes 0 on load, and MarkPlayerTurnEnded writes 4 from the End-Turn submission paths, before the turn runs. It needs one predicate, not a watchpoint — and only two corpus saves carry a non-zero Status, so an entry probe should name the set (rule 20). |
app |
lane T2 | S31 regression |
| 7 | tail phases — bankruptcy decision half, turn results outbox swap, per-player sync | game/sim + app |
lane K map; 34/37 stubs | tail subtree |
| 8 | Rung B: --turn-commands replay from a captured block (then Rung C: game/ai behavioural equivalence) |
app, then game/ai |
lanes AI1–AI4, L4 | ModCount, Checksum given the stream |
Rules that hold for every item: formula-held-inputs-missing is evaluated and reported, not
written; closed and regressed reported separately, never netted; verified in the phase
table means compared against the live game and the build fails if it drifts silently.
4. Future engine modules (breadth, ranked by what it unblocks)
game/ai—AIPlayer, fuzzyIAIRule,StrategyAIAgent. Essentially unread; proven to gate Rung B. Known:AISystem::Writeemits nothing; the 3AIAgentsave blocks are cache state (typed, 10 element layouts unexercised).game/combatbeyond the planner — resolver (7,641 B, orchestration only) and retreat are mapped; the six pipeline callees and the tanker fold are unread; nothing here has run under an instrument. Combat simulation (GameCombatSim) stays parked.- trade + spy managers — all 11 virtual targets named; four can draw RNG and none has fired (no trade routes / spy program in the corpus).
net/— the lockstep protocol. Direct IP works in the original without GameSpy; our engine inherits determinism to the word, so a lockstep peer is mostly serialization we already own.- UI and renderer — not on any path; DXVK carries rendering for the original.
5. Workloads to manufacture (an unexercised path is a hypothesis)
- orders save → End Turn → autosave — the Rung-A reference (human commands in-file).
- trade routes + spy program — the four tail draw sites;
spies2/SysMem/mtselements. - a cruiser (2 census leaves), a DN platform (the third).
- a Liir fleet — waypoint type 2 is the Liir drive, not a node line; a Hiver game — types 4/5.
- a battle with
res->+4clear and a partial retreat — resolver + retreat instrumented; tests theNMnxprediction and the predicted zero-ship-fleet bug in the original. ResErrRolltrue enteringProcessResearch— save exists; needs one turn funding > ½ a tech.- expense sliders / debtor / aid — likely dead in 1.8; debtor ≈ 35 turns. Low priority.
6. Multiplayer revival (lane G2)
- Tier 0 (hours):
/concurrent /join 127.0.0.1:3369— two clients on one VM, no server. Predictions written; queued behind lane H for VM140. - Tier 1: availability responder, byte-specified; probably a no-op (fails open on DNS failure).
- Tier 3 (config, not code): self-hosted OpenSpy/UniSpy with the
swordots/Z5gR9Z/1381 row; fix queryport 3369 (not 6500) and add the 16 custom QR2 keys (ids 50–65). - No CD-key code exists; 333networks ruled out (v0 only).
7. Method and infrastructure
guides/method-rules.md(18 rules) is the contract for every lane brief. Rule 18 — measure first, the lab exists — is why the RNG ledger closed in one VM session after three static lanes.- Reachability claims older than lane V2 are lower bounds; re-check "no caller" findings with
tools/vtable_map.py. Never trust Ghidra's function end (five lanes bitten). - Regenerate the determinism-oracle autosaves on VM140 (lane N overwrote them; lane H doing it).
- Board rows superseded by later rows are marked
SUPERSEDEDat integration — keep doing it. - Displace the shim: each hook that reaches
replacewith 0 divergences is code the original no longer runs. Track that count; it is the honest "how much of the game is ours" number.
Parked
- Battle-load bug (thread contention) — resume only with a reason.
- Renderer — after the sim byte-matches.