alex/sots-re
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

verified: End-Turn oracle byte-identical cross-process; stream tag corrections

Browse source
This commit is contained in:
alex 2026-09-07 17:46:20 -04:00
parent 078dbbbcc7
commit 1eb967756b
4 changed files with 166 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
campaign/board.md
View file

@ -43,8 +43,8 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`).
| engine: mars/parse | engine | verified | high | 100% | 2026-09-07 | sots-engine: brace-block + .effect readers; oracle 1531/1531; cross-compiles i686; wired into host+shim builds |
| engine: mars/text | engine | verified | high | 100% | 2026-09-07 | flat-kv, id-manifest, csv; oracle 64/64 (Strings.csv 5722 rows); ctest green |
| engine: game/sim formulas | engine | mapped | med | 80% | 2026-09-07 | economy/research/colony/movement pure fns, 356 checks; 5 low-confidence formulas fed back to open-questions |
| engine: mars/stream + rng | engine | in-progress | — | 0% | 2026-09-07 | Streamable reader/writer + MT19937; oracle = 3 real saves + reference vectors |
| engine: mars/stream + rng | engine | verified | high | 100% | 2026-09-07 | merging: 100% exact dump agreement on 3 saves; typed shapes round-trip byte-identical whole file; RNG = seed(RSeed)+2 twists confirmed; 4 tag-name fixes for SAVE_FORMAT |
| engine: game/data catalogs | engine | in-progress | — | 0% | 2026-09-07 | typed WeaponDef/ShipSectionDef/TechTree/IdRegistry on mars/parse+text; oracle = data-catalogs JSON |
| engine: mars/vfs (gob) | engine | verified | high | 100% | 2026-09-07 | merged: ZIP reader + native override; 8352+2035 entries = unzip -l; all 10268 files CRC-clean; byte-equal spot checks; ctest 11/11 |
| determinism oracle | verify | in-progress | — | 0% | 2026-09-07 | load same save -> End Turn -> byte-compare autosaves across runs (VM) |
| determinism oracle | verify | verified | high | 100% | 2026-09-07 | BYTE-IDENTICAL across 5 runs incl. cross-process: (Autosave).sav 978041ac…, (Autosave EndTurn).sav bb4fd9ac…; gzip MTIME=0; only loaded-post-turn re-save differs (Player.Status 4->0, Summary.Checksum). findings/subsystems/determinism-oracle.md |
| engine: shim trace/compare emitter | engine | in-progress | — | 0% | 2026-09-07 | C++ port of harness emit_record/esc/fmt_num + hook modes + snapshot/diff; validated by tracecmp.py exit codes |

2
campaign/open-questions.md
View file

@ -34,3 +34,5 @@ Each links to the finding that raised it. Promoted to backlog or closed by **re-
- **RESOLVED: Initialize signature** — `bool __thiscall Application::Initialize(this, AppStartup*)`, RET 4; Run = `void(this)`; OnTick = `bool(this)`. Plain thiscall wrappers are safe for Run/OnTick; Initialize needs the extra arg. (from [[loader-prototypes]])
- **PARITY BUG (fix in mars/text + Python flat_kv): duplicate config keys are FIRST-occurrence-wins in the engine** (entry erased on use; duplicates log 'multiply defined'); our readers implement last-wins. Also the brace parser DROPS a final key/value pair that lacks a trailing newline, and uses whitespace-only delimiting (braces not delimiters) with `" ' \`` quotes and no escapes — mars/parse must match these for compare mode; verify on the 12 broken sections + a no-trailing-newline fixture. (from [[loader-prototypes]])
- **Not traced end-to-end** — `Species/_NPC/weapons/*.weapon` loading and the `.effect` dictionary entry `EffectDictionary_Load` (0x008b42b0, unverified). (from [[loader-prototypes]])
- **RESOLVED: End-Turn oracle** — deterministic and cross-process: compare autosaves by sha256, no masking. Only when diffing a re-save of a *loaded post-turn* autosave: mask `Player.Status` (→0) and recompute/ignore `Summary.Checksum` (additive). (from [[determinism-oracle]])
- **SAVE_FORMAT tag corrections (fix Python reader + spec)** — real on-disk tags: `otnF` (not `ontF`) in Odes/Owep/Otch, `nextid` (not `nextId`) in NdGr2, Design = `FAIDes/DHide/DWep/DName`; `ords`/`wpts` are real tags. Python's positional R() matching hid these. RNG: float mapping `(float)(y*2^-32)`, `next_int` mask, and lazy-vs-eager twist at `left==0` still need binary confirmation. (from [[mars-stream]])

135
findings/subsystems/determinism-oracle.md Normal file
View file

@ -0,0 +1,135 @@
# End-Turn determinism oracle — verified on the real game (2026-09-07)
Question: "load the same save, press End Turn, and the resulting autosave is byte-identical run to run"
(the Phase-2 verification oracle). Answer: **YES — byte-identical**, across five runs, in two separate
game processes, with and without an intervening manual save, and also identical to the autosave the
*original* (non-loaded, freshly created) game wrote in the earlier session.
## Verdict
**byte-identical.** No masking is needed for the End Turn oracle when both sides start from the same
save file. sha256 of `(Autosave).sav` after End Turn from `Autosave - turn2.sav` is
`978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921` in every run (A–E) and equals the
`Autosave - turn3.sav` captured in the first session (different process, game created live, not loaded).
The only place bytes ever varied is **load → save round-trip of a *post-turn* autosave**: the game
resets a per-player `Status` int (4 → 0) on load, which also moves the derived top-level `Checksum`.
See "Round-trip stability" below for the canonicalization rule if a harness ever compares a loaded
`(Autosave).sav` against a re-save of it.
## Environment
* VM140 `sots-re-win10`, `C:\SOTS\Sword of the Stars.exe` GOG v1.8.1 (`ver.1.8.1 Wed Dec 13 03:38:31 2017`),
DXVK 3.1 on Mesa lavapipe, windowed 1024x768 (see `Z:\saves\README.md` for the launch recipe).
* Phase-2 shim (`binkw32.dll` proxy, build `4a15301-20260907T2118Z`) deployed with `shim.cfg` `hooks=trace`
— only hooks `Mars_Application_Initialize` and logs; it was loaded in both processes used here
(`shim.log`: pid 3808 then pid 4384).
* Input save: `Z:\saves\Autosave - turn2.sav` (sha256 `ab4ac2d7…`, 66,739 B) copied to
`C:\SOTS\SavedGames\ref-turn2.sav`. It is the *post-turn* autosave of turn 2 (state after End Turn #1
completed): 2 players (`re` = Morrigi @ Gamma Cephei, `Fane Lao` = AI), 28 stars, game name `MyGameverify1`.
* The three autosaves the game had in `SavedGames\` before this work (turn 5-ish, 76 kB, from a later
session) were moved to `pre-existing\` so the Load dialog only showed known files.
* UI driven with `recipe/click_helper.ps1` via the `SOTSUI` scheduled task; screenshots via `qm screendump`.
Click path per run (1024x768 window at 0,0):
main menu `Load Game` (512,536) → `Single Player` (512,290) → `OK` (551,523) → pick row → `OK` (682,624)
→ lobby `Launch` (511,663) → ~30 s load → strategy map "Turn 2", Imperial Savings 289,688 →
`End Turn` (100,714) → ~5 s → "Turn 3", Savings 532,369 → in-game menu (1000,714) → `Quit to Main Menu` (938,699) → `OK` (537,377).
Manual save: menu → `Save` (938,679) → type suffix into the pre-filled name → `Save` (773,234) → `OK` → `Done` (773,542).
## Runs
| Run | Process | Procedure | `(Autosave EndTurn).sav` | `(Autosave).sav` |
|---|---|---|---|---|
| A | pid 3808 (already running) | load `ref-turn2` → End Turn | `bb4fd9ac89f41e3b` 66,732 B | `978041acd168b56e` 67,219 B |
| B | pid 3808 (same process) | quit to menu → load `ref-turn2` → End Turn | `bb4fd9ac89f41e3b` | `978041acd168b56e` |
| C | **pid 4384 (game exited with code 0, relaunched via task `SOTS`)** | load `ref-turn2` → End Turn | `bb4fd9ac89f41e3b` | `978041acd168b56e` |
| D | pid 4384 | load `ref-turn2` → **manual save `MyGameverify1rtD`** → End Turn | `bb4fd9ac89f41e3b` | `978041acd168b56e` |
| E | pid 4384 | load `MyGameverify1verify1` (the Status=0 form) → **manual save `MyGameverify1rtE`** → End Turn | `bb4fd9ac89f41e3b` | `978041acd168b56e` |
| ref (first session, 16:03) | pid 6396-era, game created live, never loaded | End Turn #2 of the live game | `Autosave EndTurn - turn3.sav` = `bb4fd9ac89f41e3b` | `Autosave - turn3.sav` = `978041acd168b56e` |
Full hashes (`runX/sha256.txt`):
```
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 (Autosave).sav [turn-3 post-turn state] runs A B C D E + first session
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 (Autosave EndTurn).sav [turn-2 pre-turn state] runs A B C D E + first session
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 MyGameverify1rtD.sav manual save after loading ref-turn2 (run D)
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 MyGameverify1rtE.sav manual save after loading MyGameverify1verify1 (run E)
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 MyGameverify1verify1.sav manual save from the first session (turn 2)
ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 ref-turn2.sav input (= Z:\saves\Autosave - turn2.sav)
```
Inflated (gunzipped) streams are equally identical: `(Autosave).sav` → 609,080 B `17db7ed455357c4d…`,
`(Autosave EndTurn).sav` → 603,360 B `29f194dfcd932b93…`, `ref-turn2.sav` → 603,360 B `2fcf34ce4296b160…`.
The gzip container is deterministic too (header `1f8b 0800 00000000 00 0b`: MTIME=0, XFL=0, OS=11/NTFS,
no name/comment) — the compressed bytes match, not just the payload.
## What End Turn writes
On the transition to the new turn the game writes, in `C:\SOTS\SavedGames\` (all at the same second):
* `(Autosave EndTurn).sav` — the **pre-turn** state, i.e. the state at the moment End Turn was pressed.
It is byte-identical to a manual save made on that turn.
* `(Autosave).sav` — the **post-turn** state (new turn number, RNG advanced, economy applied).
* `(Autosave Backup).sav` — the previous `(Autosave).sav`, rotated (only appears from the second End
Turn in a process's SavedGames history; it is a rename, hash unchanged).
Nothing is written on Load or on Launch from the lobby. The Load dialog lists files by filename with the
in-file `GameName`, player count and `Turn`. Files in the first-session `Z:\saves\` were renamed copies of
these (`Autosave - turnN.sav` = `(Autosave).sav`, etc.).
No timestamp, save name, machine id or session salt is stored anywhere in the file: `MyGameverify1verify1`,
`MyGameverify1rtD`, `MyGameverify1rtE` and `(Autosave EndTurn)` — four different file names, three of
them written minutes apart in two processes — are one and the same byte sequence.
## Round-trip stability (load → save with no End Turn)
* Loading the **pre-turn / manual form** and saving immediately reproduces the loaded bytes exactly:
`MyGameverify1verify1.sav` (`bb4fd9ac…`) → load → Save → `MyGameverify1rtE.sav` = `bb4fd9ac…`. **Stable.**
* Loading the **post-turn `(Autosave).sav` form** and saving immediately does *not* reproduce it:
`ref-turn2.sav` (`ab4ac2d7…`, 66,739 B) → load → Save → `MyGameverify1rtD.sav` = `bb4fd9ac…` (66,732 B).
The two inflated streams (603,360 B each) differ in exactly **5 bytes**:
```
inflated offset (1-based) ref-turn2 re-save field (save_reader --dump path)
89 0x64 0x54 Summary.Checksum int -1205790620 -> -1205790636 (delta 16 = 4 x 4)
66291 (0x102e8+4) 4 0 Player[PlryName="re"].Status
116067 (0x1c558+4) 4 0 Player[PlryName="Fane Lao"].Status
177739 (0x2b640+4) 4 0 Player[PlryName="Singularity"].Status (1st of two Singularity records)
235567 (0x39824+4) 4 0 Player[PlryName="Singularity"].Status (2nd)
```
The other four `Player.Status` fields (Alien Menace, Peacekeeper Enforcer, Von Neumann, Independent
Colony) are 0 in both. Interpretation: `Status=4` is the "turn ended / ready" flag the server leaves on the
four turn-participating player records when it writes the post-turn autosave; loading a game clears it.
`Summary.Checksum` moves by exactly the sum of the changed bytes (16), consistent with an additive
checksum over the payload — treat it as **derived**, never as state.
(`dumps/diff_ref-turn2_vs_EndTurn.txt`, `dumps/cmp_ref-turn2_vs_EndTurn.txt`.)
## Recommended canonicalization rule for the harness
1. **Oracle compare (End Turn):** compare `(Autosave).sav` and `(Autosave EndTurn).sav` **byte-for-byte**
(sha256 of the file is enough; no inflate, no masking). Expected for the reference run from
`Autosave - turn2.sav`: `(Autosave).sav` = `978041acd168b56e…`, `(Autosave EndTurn).sav` = `bb4fd9ac89f41e3b…`.
2. **If a re-implementation compares its output against a *loaded* post-turn autosave, or compares a
pre-turn save with a post-turn one of the same turn**, canonicalize by masking exactly:
* every `Player.Status` int (set to 0), and
* `Summary.Checksum` (recompute, or ignore).
Nothing else varies. Use `save_reader.py --dump` on the inflated stream; both fields are named as
above at fixed paths.
3. Do not mask anything in the gzip container — it is deterministic (MTIME 0), so a container-level
sha256 is a valid first-line check; inflate only to localize a mismatch.
## Files in this directory
* `runA/ … runE/` — the raw `SavedGames\` contents after each run, plus `sha256.txt`.
* `pre-existing/` — the three later-turn autosaves that were in `SavedGames\` before this work (moved aside).
* `dumps/` — `save_reader.py --dump` text of `ref-turn2` (input), `Autosave-EndTurn-turn2` and
`Autosave-turn3` (outputs), plus the field diff and `cmp -l` of the round-trip pair.
* `inflated/` — gunzipped streams of the same three files.
* `screenshots/` — console captures of each run's loaded state / result / save dialogs.
* `hashes.txt` — sha256 of every `.sav` across all runs.
State left behind: the game is still running on VM140 (pid 4384, task `SOTS`, shim `hooks=trace`), sitting
on the turn-3 strategy map after run E; `C:\SOTS\SavedGames\` holds the run-E set (`ref-turn2.sav`,
`MyGameverify1*.sav`, the three `(Autosave*)` files).

27
verify/results/saves/determinism-hashes.txt Normal file
View file

@ -0,0 +1,27 @@
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runA/(Autosave EndTurn).sav
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runA/(Autosave).sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runA/MyGameverify1verify1.sav
ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runA/ref-turn2.sav
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runB/(Autosave Backup).sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runB/(Autosave EndTurn).sav
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runB/(Autosave).sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runB/MyGameverify1verify1.sav
ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runB/ref-turn2.sav
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runC/(Autosave Backup).sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runC/(Autosave EndTurn).sav
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runC/(Autosave).sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runC/MyGameverify1verify1.sav
ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runC/ref-turn2.sav
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runD/(Autosave Backup).sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runD/(Autosave EndTurn).sav
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runD/(Autosave).sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runD/MyGameverify1rtD.sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runD/MyGameverify1verify1.sav
ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runD/ref-turn2.sav
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runE/(Autosave Backup).sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runE/(Autosave EndTurn).sav
978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921 runs/runE/(Autosave).sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runE/MyGameverify1rtD.sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runE/MyGameverify1rtE.sav
bb4fd9ac89f41e3bc0db2af08b18ce83417521ac4bcee695fc9fa6ce16e30948 runs/runE/MyGameverify1verify1.sav
ab4ac2d7e2977260a4de916f65c7ecc6dd5b0094dac4bb51241317232ea17ab4 runs/runE/ref-turn2.sav