106 lines
6.9 KiB
Markdown
106 lines
6.9 KiB
Markdown
# M1 — flat KEY/value config loader (`Mars::GlobalConsts::LoadFile`) old-vs-new on the live game
|
|
|
|
**Result (2026-09-07/08):** the first real old-vs-new verification. Trace mode: 19 `LoadFile` calls
|
|
(1088 constant slots), `tracecmp.py` exit 0. Compare mode: **19/19 compared, 0 divergences**.
|
|
Replace mode: our loader fed the game its constants alone; loading `ref-turn2.sav` and pressing
|
|
End Turn reproduced the determinism oracle byte for byte (`(Autosave).sav` = `978041ac…`,
|
|
`(Autosave EndTurn).sav` = `bb4fd9ac…`). The offline replay test (golden trace through the
|
|
host build) also passes. Game left running on VM140 in `hooks=trace` at the main menu.
|
|
|
|
## What was hooked
|
|
|
|
`Mars::GlobalConsts::LoadFile(const char* file, GlobalConstMap* consts)` — cdecl, verified, so
|
|
the `Hook<Descriptor>` template applies directly (`src/shim/hooks/global_consts.{h,cpp}`,
|
|
installed from `src/shim/main.cpp` via `InstallTemplateHook<>`). It is called once per data file
|
|
from `LoadAll()` inside `Application::Initialize`, before the intro movie; the files it touches
|
|
(19, in map order of the file names):
|
|
|
|
```
|
|
Data/Combat/{actor,assaultshuttle,camera,drone,gravboat,ortgay,planet,sensors,ship,spyship,terrain,vnhomeworld}.txt
|
|
Data/encounters.txt Data/globals.txt Data/options.txt Data/Species.txt
|
|
Data/Strategy/AI/AIRebellion.txt Data/Strategy/starmap.txt Data/Strategy/StrategyVars.txt
|
|
```
|
|
|
|
`ctechvars.txt`, `shipai.txt`, `damfx*.txt`, `playercolors.txt`, `systemnames.txt` are *not*
|
|
GlobalConsts files (other readers).
|
|
|
|
### Region model
|
|
|
|
The map handed to `LoadFile` is an MSVC 2010 `std::map<const char*, GlobalConst*>` holding the
|
|
constants registered for that file (`key -> {storage, key, parser, file}`; layouts are now in
|
|
`sots_addresses.h`). `regions()` walks it in order before the original runs and declares **one
|
|
region per constant, named by the key**, sized and described by the parser the registering stub
|
|
pushed:
|
|
|
|
| parser (VA) | kind | width | describer |
|
|
|---|---|---|---|
|
|
| `GlobalConst_ParseInt` 0x008b7000 | `int` | 4 | `{t:"int", v:i32}` |
|
|
| `GlobalConst_ParseFloat` 0x008b7020 | `float` | 4 | `{t:"float", v:f32}` |
|
|
| `GlobalConst_ParseFloatScaled` 0x008b70e0 | `fscaled` | 4 | `{t:"fscaled", v:f32}` |
|
|
| `GlobalConst_ParseColour` 0x008b7110 | `colour` | 16 | `{t:"colour", r,g,b,a:f32}` |
|
|
| `GlobalConst_ParseRect` 0x008b7040 **(new)** | `rect` | 16 | `{t:"rect", x,y,w,h:i32}` |
|
|
| `GlobalConst_ParseVec3` 0x008b7080 **(new)** | `vec3` | 12 | `{t:"vec3", x,y,z:f32}` |
|
|
| `GlobalConst_ParseString` 0x008b7670 **(new)** | `string` | 24 | `{t:"string", v:str}` (decoded MSVC `std::string`) |
|
|
|
|
Args: `file` (str), `consts` (ptr), `nkeys` (u32) and `scale` (f64, the scaled-float multiplier)
|
|
— the last so a golden log replays offline with nothing from the exe. Return: void.
|
|
|
|
Discoveries the RE notes did not have (all fed back into `sots-re/ghidra/addresses.json`, status
|
|
`verified-by-trace`, and regenerated into `include/generated/sots_addresses.h`):
|
|
|
|
- **Three more parsers.** 146 keys (`*_NAME`, `*_SOUND`, `*_TEXTURENAME`, `DERELICT_SECTION_nn`,
|
|
`Data/options.txt` …) are `std::string` slots (0x18 bytes: 16-byte SSO buffer / heap pointer,
|
|
size @+0x10, capacity @+0x14). Five `HIVER_SPAWN_*` keys are `float[3]` (`"0 0 50.0"`). Two
|
|
`COMBATSETUP_CARD_*_RECT` keys are `int[4]`.
|
|
- **The scaled-float constant at 0x00af5210 is a `double`** (pi/180 = 0x3f91df46a2529d39), not a
|
|
float; reading it as float gives -2.85e-18. The product is computed in double and stored as
|
|
float: `(float)((double)v * scale)`.
|
|
- `Data/Strategy/StrategyVars.txt` and `Data/encounters.txt` have no trailing newline, so the
|
|
original never applies their last key (`CIVILIAN_BURDEN_RATIO`, `HERALD_SPEECH_MAX_INVERVAL`):
|
|
visible in the trace as the only unchanged regions in those files and reproduced by ours
|
|
("expected but not found", `missing 1` in both).
|
|
|
|
## `ours` — `src/game/config/config_loader.{h,cpp}` (lib `sots_game_config`)
|
|
|
|
- File bytes come from the game's own `gobio::ReadFile` (honours the mount order / mods) and are
|
|
released with `RefCounted_Release`; the host tests read plain files.
|
|
- Pairs come from `mars::text::parse_flat_kv` (engine-parity: Mars::Script tokenizer, blocks
|
|
skipped, trailing pair dropped). `apply()` implements **first-occurrence-wins** on top of the
|
|
raw pair list (a slot is consumed on first sight; repeats and unknown keys are logged
|
|
"`[file] KEY not recognized or is multiply defined.`", absent slots
|
|
"`[file] KEY expected but not found.`", to `shim.log` with a `cfg:` prefix).
|
|
- Typed writes mirror `sscanf`: `%d` (CRT wrap-around accumulator, no write when no digit),
|
|
`%f` (one rounding from the text via `strtold` → float), colour (`%d %d %d %d`, defaults 255,
|
|
`/255.0` in double, clamp, always writes all four), vec3 / rect (components until the first
|
|
scan failure), scaled float (scan then multiply, whatever the scan did).
|
|
- `String` slots are written through an `ExternWriter`: the shim hands the text to the game's
|
|
own `ParseString` (a MinGW-built `std::string` cannot be handed to MSVCR100). In compare mode
|
|
that runs on the scratch copy of the object, which starts as the image default (empty short
|
|
string), so nothing belonging to the live object is ever freed; long strings leak one heap
|
|
block per compare call (startup only).
|
|
|
|
## Runs (`/bulk-storage/re-lab/shim/traces/`, build `4f0a9db-dirty-20260908T0205Z`)
|
|
|
|
| file | mode | calls | result |
|
|
|---|---|---|---|
|
|
| `m1-trace-golden.jsonl` | trace | 19 LoadFile + 1 selftest | `tracecmp.py` exit 0, 0 invalid |
|
|
| `m1-compare.jsonl` | compare | 19 compared | **0 diverged**, 0 errors, exit 0 |
|
|
| replace (no log) | replace | 19 | End Turn from `ref-turn2.sav`: `(Autosave).sav` `978041ac…` / `(Autosave EndTurn).sav` `bb4fd9ac…` = oracle |
|
|
| `m1-shim.log` | | | all four runs' banners, `cfg:` stats per file |
|
|
| `m1-*.png` | | | main menu (trace, compare, final), turn 2 / turn 3 in replace |
|
|
|
|
Offline: `SOTS_M1_TRACE=<golden> SOTS_DATA_DIR=<extract root> build-host/tests/game_config/game_config_replay`
|
|
→ `tracecmp.py --replay` exit 0 (19 calls, 0 files missing). Host suite: `ctest` 22/22.
|
|
|
|
## Gotchas
|
|
|
|
1. `shim.cfg` per-hook line: `hook.Mars::GlobalConsts::LoadFile=compare|replace` (default `hooks=trace`).
|
|
2. `Z:` is per-logon; from SSH there is no share. Push the dist to `C:\SOTS\shimdist\` with scp
|
|
and run `deploy.ps1 -Dist C:\SOTS\shimdist`. Inline PowerShell over ssh mangles `$` and
|
|
quotes — ship `.ps1` files (`C:\SOTS\ui\{status,loginfo,lastrun,relaunch,hashes}.ps1`).
|
|
3. Local cmake is 3.22 (< 3.25 required): use `uvx --from cmake cmake` with `Unix Makefiles`.
|
|
4. The per-call slot table is kept in statics between `regions()` → `rebind()` → `ours()`; fine
|
|
because `LoadFile` is single-threaded at startup, but do not reuse the pattern for re-entrant
|
|
hooks.
|
|
5. `Data/Combat/ctechvars.txt` style files (a single `name { … }` block) would set nothing
|
|
through this loader — they are simply not GlobalConsts files, as the trace confirms.
|