From 1eb967756b7266ce504df60bff69e916b798290a Mon Sep 17 00:00:00 2001 From: alex Date: Mon, 7 Sep 2026 17:46:20 -0400 Subject: [PATCH] verified: End-Turn oracle byte-identical cross-process; stream tag corrections --- campaign/board.md | 4 +- campaign/open-questions.md | 2 + findings/subsystems/determinism-oracle.md | 135 ++++++++++++++++++++ verify/results/saves/determinism-hashes.txt | 27 ++++ 4 files changed, 166 insertions(+), 2 deletions(-) create mode 100644 findings/subsystems/determinism-oracle.md create mode 100644 verify/results/saves/determinism-hashes.txt diff --git a/campaign/board.md b/campaign/board.md index 1a93e16..4cae427 100644 --- a/campaign/board.md +++ b/campaign/board.md @@ -43,8 +43,8 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`). | engine: mars/parse | engine | verified | high | 100% | 2026-09-07 | sots-engine: brace-block + .effect readers; oracle 1531/1531; cross-compiles i686; wired into host+shim builds | | engine: mars/text | engine | verified | high | 100% | 2026-09-07 | flat-kv, id-manifest, csv; oracle 64/64 (Strings.csv 5722 rows); ctest green | | engine: game/sim formulas | engine | mapped | med | 80% | 2026-09-07 | economy/research/colony/movement pure fns, 356 checks; 5 low-confidence formulas fed back to open-questions | -| engine: mars/stream + rng | engine | in-progress | — | 0% | 2026-09-07 | Streamable reader/writer + MT19937; oracle = 3 real saves + reference vectors | +| engine: mars/stream + rng | engine | verified | high | 100% | 2026-09-07 | merging: 100% exact dump agreement on 3 saves; typed shapes round-trip byte-identical whole file; RNG = seed(RSeed)+2 twists confirmed; 4 tag-name fixes for SAVE_FORMAT | | engine: game/data catalogs | engine | in-progress | — | 0% | 2026-09-07 | typed WeaponDef/ShipSectionDef/TechTree/IdRegistry on mars/parse+text; oracle = data-catalogs JSON | | engine: mars/vfs (gob) | engine | verified | high | 100% | 2026-09-07 | merged: ZIP reader + native override; 8352+2035 entries = unzip -l; all 10268 files CRC-clean; byte-equal spot checks; ctest 11/11 | -| determinism oracle | verify | in-progress | — | 0% | 2026-09-07 | load same save -> End Turn -> byte-compare autosaves across runs (VM) | +| determinism oracle | verify | verified | high | 100% | 2026-09-07 | BYTE-IDENTICAL across 5 runs incl. cross-process: (Autosave).sav 978041ac…, (Autosave EndTurn).sav bb4fd9ac…; gzip MTIME=0; only loaded-post-turn re-save differs (Player.Status 4->0, Summary.Checksum). findings/subsystems/determinism-oracle.md | | engine: shim trace/compare emitter | engine | in-progress | — | 0% | 2026-09-07 | C++ port of harness emit_record/esc/fmt_num + hook modes + snapshot/diff; validated by tracecmp.py exit codes | diff --git a/campaign/open-questions.md b/campaign/open-questions.md index 09306a2..30b2578 100644 --- a/campaign/open-questions.md +++ b/campaign/open-questions.md @@ -34,3 +34,5 @@ Each links to the finding that raised it. Promoted to backlog or closed by **re- - **RESOLVED: Initialize signature** — `bool __thiscall Application::Initialize(this, AppStartup*)`, RET 4; Run = `void(this)`; OnTick = `bool(this)`. Plain thiscall wrappers are safe for Run/OnTick; Initialize needs the extra arg. (from [[loader-prototypes]]) - **PARITY BUG (fix in mars/text + Python flat_kv): duplicate config keys are FIRST-occurrence-wins in the engine** (entry erased on use; duplicates log 'multiply defined'); our readers implement last-wins. Also the brace parser DROPS a final key/value pair that lacks a trailing newline, and uses whitespace-only delimiting (braces not delimiters) with `" ' \`` quotes and no escapes — mars/parse must match these for compare mode; verify on the 12 broken sections + a no-trailing-newline fixture. (from [[loader-prototypes]]) - **Not traced end-to-end** — `Species/_NPC/weapons/*.weapon` loading and the `.effect` dictionary entry `EffectDictionary_Load` (0x008b42b0, unverified). (from [[loader-prototypes]]) +- **RESOLVED: End-Turn oracle** — deterministic and cross-process: compare autosaves by sha256, no masking. Only when diffing a re-save of a *loaded post-turn* autosave: mask `Player.Status` (→0) and recompute/ignore `Summary.Checksum` (additive). (from [[determinism-oracle]]) +- **SAVE_FORMAT tag corrections (fix Python reader + spec)** — real on-disk tags: `otnF` (not `ontF`) in Odes/Owep/Otch, `nextid` (not `nextId`) in NdGr2, Design = `FAIDes/DHide/DWep/DName`; `ords`/`wpts` are real tags. Python's positional R() matching hid these. RNG: float mapping `(float)(y*2^-32)`, `next_int` mask, and lazy-vs-eager twist at `left==0` still need binary confirmation. (from [[mars-stream]]) diff --git a/findings/subsystems/determinism-oracle.md b/findings/subsystems/determinism-oracle.md new file mode 100644 index 0000000..cfbed70 --- /dev/null +++ b/findings/subsystems/determinism-oracle.md @@ -0,0 +1,135 @@ +# End-Turn determinism oracle — verified on the real game (2026-09-07) + +Question: "load the same save, press End Turn, and the resulting autosave is byte-identical run to run" +(the Phase-2 verification oracle). Answer: **YES — byte-identical**, across five runs, in two separate +game processes, with and without an intervening manual save, and also identical to the autosave the +*original* (non-loaded, freshly created) game wrote in the earlier session. + +## Verdict + +**byte-identical.** No masking is needed for the End Turn oracle when both sides start from the same +save file. sha256 of `(Autosave).sav` after End Turn from `Autosave - turn2.sav` is +`978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921` in every run (A–E) and equals the +`Autosave - turn3.sav` captured in the first session (different process, game created live, not loaded). + +The only place bytes ever varied is **load → save round-trip of a *post-turn* autosave**: the game +resets a per-player `Status` int (4 → 0) on load, which also moves the derived top-level `Checksum`. +See "Round-trip stability" below for the canonicalization rule if a harness ever compares a loaded +`(Autosave).sav` against a re-save of it. + +## Environment + +* VM140 `sots-re-win10`, `C:\SOTS\Sword of the Stars.exe` GOG v1.8.1 (`ver.1.8.1 Wed Dec 13 03:38:31 2017`), + DXVK 3.1 on Mesa lavapipe, windowed 1024x768 (see `Z:\saves\README.md` for the launch recipe). +* Phase-2 shim (`binkw32.dll` proxy, build `4a15301-20260907T2118Z`) deployed with `shim.cfg` `hooks=trace` + — only hooks `Mars_Application_Initialize` and logs; it was loaded in both processes used here + (`shim.log`: pid 3808 then pid 4384). +* Input save: `Z:\saves\Autosave - turn2.sav` (sha256 `ab4ac2d7…`, 66,739 B) copied to + `C:\SOTS\SavedGames\ref-turn2.sav`. It is the *post-turn* autosave of turn 2 (state after End Turn #1 + completed): 2 players (`re` = Morrigi @ Gamma Cephei, `Fane Lao` = AI), 28 stars, game name `MyGameverify1`. +* The three autosaves the game had in `SavedGames\` before this work (turn 5-ish, 76 kB, from a later + session) were moved to `pre-existing\` so the Load dialog only showed known files. +* UI driven with `recipe/click_helper.ps1` via the `SOTSUI` scheduled task; screenshots via `qm screendump`. + +Click path per run (1024x768 window at 0,0): +main menu `Load Game` (512,536) → `Single Player` (512,290) → `OK` (551,523) → pick row → `OK` (682,624) +→ lobby `Launch` (511,663) → ~30 s load → strategy map "Turn 2", Imperial Savings 289,688 → +`End Turn` (100,714) → ~5 s → "Turn 3", Savings 532,369 → in-game menu (1000,714) → `Quit to Main Menu` (938,699) → `OK` (537,377). +Manual save: menu → `Save` (938,679) → type suffix into the pre-filled name → `Save` (773,234) → `OK` → `Done` (773,542). + +## Runs + +| Run | Process | Procedure | `(Autosave EndTurn).sav` | `(Autosave).sav` | +|---|---|---|---|---| +| A | pid 3808 (already running) | load `ref-turn2` → End Turn | `bb4fd9ac89f41e3b` 66,732 B | `978041acd168b56e` 67,219 B | +| B | pid 3808 (same process) | quit to menu → load `ref-turn2` → End Turn | `bb4fd9ac89f41e3b` | `978041acd168b56e` | +| C | **pid 4384 (game exited with code 0, relaunched via task `SOTS`)** | load `ref-turn2` → End Turn | `bb4fd9ac89f41e3b` | `978041acd168b56e` | +| D | pid 4384 | load `ref-turn2` → **manual save `MyGameverify1rtD`** → End Turn | `bb4fd9ac89f41e3b` | `978041acd168b56e` | +| E | pid 4384 | load `MyGameverify1verify1` (the Status=0 form) → **manual save `MyGameverify1rtE`** → End Turn | `bb4fd9ac89f41e3b` | `978041acd168b56e` | +| ref (first session, 16:03) | pid 6396-era, game created live, never loaded | End Turn #2 of the live game | `Autosave EndTurn - turn3.sav` = `bb4fd9ac89f41e3b` | `Autosave - turn3.sav` = `978041acd168b56e` | + +Full hashes (`runX/sha256.txt`): + +``` +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 (Autosave).sav [turn-3 post-turn state] runs A B C D E + first session +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 (Autosave EndTurn).sav [turn-2 pre-turn state] runs A B C D E + first session +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 MyGameverify1rtD.sav manual save after loading ref-turn2 (run D) +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 MyGameverify1rtE.sav manual save after loading MyGameverify1verify1 (run E) +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 MyGameverify1verify1.sav manual save from the first session (turn 2) +ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 ref-turn2.sav input (= Z:\saves\Autosave - turn2.sav) +``` + +Inflated (gunzipped) streams are equally identical: `(Autosave).sav` → 609,080 B `17db7ed455357c4d…`, +`(Autosave EndTurn).sav` → 603,360 B `29f194dfcd932b93…`, `ref-turn2.sav` → 603,360 B `2fcf34ce4296b160…`. +The gzip container is deterministic too (header `1f8b 0800 00000000 00 0b`: MTIME=0, XFL=0, OS=11/NTFS, +no name/comment) — the compressed bytes match, not just the payload. + +## What End Turn writes + +On the transition to the new turn the game writes, in `C:\SOTS\SavedGames\` (all at the same second): + +* `(Autosave EndTurn).sav` — the **pre-turn** state, i.e. the state at the moment End Turn was pressed. + It is byte-identical to a manual save made on that turn. +* `(Autosave).sav` — the **post-turn** state (new turn number, RNG advanced, economy applied). +* `(Autosave Backup).sav` — the previous `(Autosave).sav`, rotated (only appears from the second End + Turn in a process's SavedGames history; it is a rename, hash unchanged). + +Nothing is written on Load or on Launch from the lobby. The Load dialog lists files by filename with the +in-file `GameName`, player count and `Turn`. Files in the first-session `Z:\saves\` were renamed copies of +these (`Autosave - turnN.sav` = `(Autosave).sav`, etc.). + +No timestamp, save name, machine id or session salt is stored anywhere in the file: `MyGameverify1verify1`, +`MyGameverify1rtD`, `MyGameverify1rtE` and `(Autosave EndTurn)` — four different file names, three of +them written minutes apart in two processes — are one and the same byte sequence. + +## Round-trip stability (load → save with no End Turn) + +* Loading the **pre-turn / manual form** and saving immediately reproduces the loaded bytes exactly: + `MyGameverify1verify1.sav` (`bb4fd9ac…`) → load → Save → `MyGameverify1rtE.sav` = `bb4fd9ac…`. **Stable.** +* Loading the **post-turn `(Autosave).sav` form** and saving immediately does *not* reproduce it: + `ref-turn2.sav` (`ab4ac2d7…`, 66,739 B) → load → Save → `MyGameverify1rtD.sav` = `bb4fd9ac…` (66,732 B). + The two inflated streams (603,360 B each) differ in exactly **5 bytes**: + +``` +inflated offset (1-based) ref-turn2 re-save field (save_reader --dump path) + 89 0x64 0x54 Summary.Checksum int -1205790620 -> -1205790636 (delta 16 = 4 x 4) + 66291 (0x102e8+4) 4 0 Player[PlryName="re"].Status + 116067 (0x1c558+4) 4 0 Player[PlryName="Fane Lao"].Status + 177739 (0x2b640+4) 4 0 Player[PlryName="Singularity"].Status (1st of two Singularity records) + 235567 (0x39824+4) 4 0 Player[PlryName="Singularity"].Status (2nd) +``` + + The other four `Player.Status` fields (Alien Menace, Peacekeeper Enforcer, Von Neumann, Independent + Colony) are 0 in both. Interpretation: `Status=4` is the "turn ended / ready" flag the server leaves on the + four turn-participating player records when it writes the post-turn autosave; loading a game clears it. + `Summary.Checksum` moves by exactly the sum of the changed bytes (16), consistent with an additive + checksum over the payload — treat it as **derived**, never as state. + (`dumps/diff_ref-turn2_vs_EndTurn.txt`, `dumps/cmp_ref-turn2_vs_EndTurn.txt`.) + +## Recommended canonicalization rule for the harness + +1. **Oracle compare (End Turn):** compare `(Autosave).sav` and `(Autosave EndTurn).sav` **byte-for-byte** + (sha256 of the file is enough; no inflate, no masking). Expected for the reference run from + `Autosave - turn2.sav`: `(Autosave).sav` = `978041acd168b56e…`, `(Autosave EndTurn).sav` = `bb4fd9ac89f41e3b…`. +2. **If a re-implementation compares its output against a *loaded* post-turn autosave, or compares a + pre-turn save with a post-turn one of the same turn**, canonicalize by masking exactly: + * every `Player.Status` int (set to 0), and + * `Summary.Checksum` (recompute, or ignore). + Nothing else varies. Use `save_reader.py --dump` on the inflated stream; both fields are named as + above at fixed paths. +3. Do not mask anything in the gzip container — it is deterministic (MTIME 0), so a container-level + sha256 is a valid first-line check; inflate only to localize a mismatch. + +## Files in this directory + +* `runA/ … runE/` — the raw `SavedGames\` contents after each run, plus `sha256.txt`. +* `pre-existing/` — the three later-turn autosaves that were in `SavedGames\` before this work (moved aside). +* `dumps/` — `save_reader.py --dump` text of `ref-turn2` (input), `Autosave-EndTurn-turn2` and + `Autosave-turn3` (outputs), plus the field diff and `cmp -l` of the round-trip pair. +* `inflated/` — gunzipped streams of the same three files. +* `screenshots/` — console captures of each run's loaded state / result / save dialogs. +* `hashes.txt` — sha256 of every `.sav` across all runs. + +State left behind: the game is still running on VM140 (pid 4384, task `SOTS`, shim `hooks=trace`), sitting +on the turn-3 strategy map after run E; `C:\SOTS\SavedGames\` holds the run-E set (`ref-turn2.sav`, +`MyGameverify1*.sav`, the three `(Autosave*)` files). diff --git a/verify/results/saves/determinism-hashes.txt b/verify/results/saves/determinism-hashes.txt new file mode 100644 index 0000000..c4f326d --- /dev/null +++ b/verify/results/saves/determinism-hashes.txt @@ -0,0 +1,27 @@ +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runA/(Autosave EndTurn).sav +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runA/(Autosave).sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runA/MyGameverify1verify1.sav +ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runA/ref-turn2.sav +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runB/(Autosave Backup).sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runB/(Autosave EndTurn).sav +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runB/(Autosave).sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runB/MyGameverify1verify1.sav +ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runB/ref-turn2.sav +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runC/(Autosave Backup).sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runC/(Autosave EndTurn).sav +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runC/(Autosave).sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runC/MyGameverify1verify1.sav +ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runC/ref-turn2.sav +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runD/(Autosave Backup).sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runD/(Autosave EndTurn).sav +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runD/(Autosave).sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runD/MyGameverify1rtD.sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runD/MyGameverify1verify1.sav +ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runD/ref-turn2.sav +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runE/(Autosave Backup).sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runE/(Autosave EndTurn).sav +978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runE/(Autosave).sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runE/MyGameverify1rtD.sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runE/MyGameverify1rtE.sav +bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runE/MyGameverify1verify1.sav +ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runE/ref-turn2.sav