sots-re/findings/objects/svsctob-variants.md
lane-w 2308b6edb9 lane W: SvSctOb variant factories; fix the four save_reader.py defects openly
Part 1 (notes side): findings/objects/svsctob-variants.md records the two maps
that are nowhere on the wire and were read out of the game -- EncID -> class
from a 23-entry dword jump table at 0x0052bf60 indexed by EncID-1 (0x0052bf00),
and xscn -> class from an exhaustive four-way _stricmp chain at 0x005a7050.
Twelve live EncIDs, four scenario names, and "indsys" =
Game::SVSOIndependentSystems whose Read and Write are both the shared `ret 4`
stub at 0x005f8ac0, so its empty frame is correct output rather than a
truncation. Also: SVSOSots::Read accepts NPCPlr and hastraps, which Write never
emits -- read-only backward compatibility, not a hole in the recovery. And a
correction to the recovery itself: SVSOCrowDefenders writes `dsys` INSIDE the
ndsys loop; layouts.json calls it a plain member and no save can settle it
because both counts are 0 everywhere. 13 addresses in ghidra/addresses.d/
lane-w.json; gen_addresses.py merges to 640 with no duplicate name.

Part 2: lane G found four defects in both readers and deliberately did not
patch the oracle mid-campaign. Fixed now, with tests, and byte-neutral.
  1. Game::SystemParams field 1 is a string, not an int (empty string == four
     zero bytes == int 0, so it round-tripped by luck).
  2. ObservedTech/ObservedWeapon odet is a bool, not an int (byte-safe only
     because a 4-char tag makes both items 12 bytes; 3 chars would not).
  3. SpeciesRatios nv is a count, not a field.
  4. ShipRecords srbd is a count, not a field -- and this one is behaviourally
     confirmed, not inferred: srbd takes 0, 1, 3 and 4 across the players and
     every non-zero count is followed by exactly srbd x 5 scalars.
Note that 3 and 4 were an ABSENCE in save_reader.py, not an error: ShipRecs and
civr were both A(..., "any"), so the fix had to add the shapes rather than
retype a field.

Byte-neutrality: every item's inflated offset is unchanged on all four saves
(38,933 / 39,843 / 40,300 / 35,771 offsets, sequences identical), so no item
boundary moved. state_checksum.py still reports coverage: PROVED on all four
with the same rebuilt byte counts. The /CreateParams and /Sim/players digests do
change, because they hash typed VALUES and two fixes change what a value is --
and the value-byte deltas balance exactly: odet items x 3 plus p1 items x 4.
--strict exit 0 on all four saves; tests 36 -> 48.

findings/objects/wire-schema-closeout.md carries the whole account, including
the proof that CD/TurnCommands_v5 cannot be typed without a save that has
issued orders.
2026-09-08 07:05:09 -04:00

7.5 KiB
Raw Blame History

SvSctOb: the script-object tree, and the two factories that key it

Lane W, 2026-09-08. Host/static only — VM140 was held by lane U and the game was never run.

Game::StrategyServer writes one member tagged SvSctOb, typed Mars::StreamableHelper<Game::SVScriptObject> — a polymorphic pointer. In every save we hold it points at a Game::SVSOSots, whose serializer (Write 0x0059ddf0, Read 0x005a7a40, vftable 0x00A063C4) emits, in disk order:

ScnID      int
ScnObj     StreamableHelper<SVSOScenarioBase>   -- conditional; NULL, so absent, in all four saves
numx       int count
  x numx { xscn   string        -- a scenario NAME
           xsc    StreamableHelper<SVScriptObject> }
NEncObjs   int count
  x NEncObjs { EncID  int       -- an encounter ID
               EncObj StreamableHelper<SVScriptObject> }

Both xsc and EncObj are polymorphic and dispatched by the key item that immediately precedes them. Neither map is anywhere on the wire: a reader that does not know them can only carry the bodies as opaque blobs, which is exactly what sots-engine was doing (147 items on turn1). Both maps were read out of the game.

1. EncID → class: a 23-entry jump table

Game::SVSOSots::Read calls a get-or-create helper at 0x005a4450 (it scans the already loaded (id, obj) pair vector at this+0x0C..0x10 and only on a miss calls the real factory). The factory is 0x0052bf00:

0052bf00  push ebp; mov ebp,esp; mov eax,[ebp+8]
0052bf06  dec eax
0052bf07  cmp eax,0x16
0052bf0a  ja  0052bf5b            ; -> xor eax,eax; ret   (NULL)
0052bf0c  jmp dword ptr [eax*4 + 0x0052bf60]

so the encoding is a dword jump table at 0x0052bf60, 23 entries, indexed by EncID - 1. Each thunk runs a ctor, and the class name comes from the mov dword ptr [obj], <vftable> store in that ctor — a demangled-RTTI name, not a guess.

EncID ctor class in our saves
1 0x0052bb90 → 0x0052b790 Game::SVSOVonNeumann yes
3 0x0051aba0 → 0x0051a870 Game::SVSOSwarm yes
4 0x0051a270 Game::SVSODerelict yes
5 0x0052ae10 Game::SVSOMonitor yes
7 0x0050e0f0 Game::SVSOSystemKiller no
8 0x0050de50 Game::SVSOPuppetMaster no
9 0x0051a820 Game::SVSOSlaversRefuel yes
10 0x0051ae20 Game::SVSOSwarmQueen yes
14 0x0052a030 → 0x00529080 Game::SVSOLocust no
17 0x00524190 Game::SVSOCrowRuins yes
20 0x005295a0 → 0x005293f0 Game::SVSORefugees yes
21 0x004f5210 → 0x004f5170 Game::SVSOOrtgay no

Ids 2, 6, 11–13, 15, 16, 18, 19, 22, 23 are dead slots that return NULL; so is anything outside 1..23 (the ja). There is no named enum — the ids are bare integers and the jump table is the only encoding of the map anywhere in the binary.

The eight ids the saves carry were independently identified by tag-sequence match against the recovered serializers before the table was read, and the two agree exactly on all eight.

2. xscn → class: a four-way _stricmp chain

0x005a7050, falling through to "Error creating extra script %s.\n". Flat if-chain, no table, no registration list, and exhaustive — there are four scenario names in the whole game.

xscn test at ctor class
traps 0x005a7085 0x0052cbb0 → 0x0052c940 Game::SVSOTraps
crowdefs 0x005a70a4 0x0052b3c0 Game::SVSOCrowDefenders
indsys 0x005a70be 0x0075b530 Game::SVSOIndependentSystems
gmtrigger 0x005a70d8 0x004f5630 Game::SVSOGrandMenaceTrigger

Corroborated by the new-game seeder at 0x005a7d70, which registers exactly traps, crowdefs, indsys, and — only when ScnObj == NULL — gmtrigger. That is the four-entry list our saves carry, in that order.

indsys serializes nothing, and that is a fact, not an absence of evidence

Game::SVSOIndependentSystems has no entry in objects/layouts.json, which on its own only says the recovery never found a serializer. Its vftable is 0x00A20314 (from the C7 06 14 03 A2 00 store at 0x0075b585). Comparing slots against Game::SVSOCrowDefenders's vftable at 0x009F5904 (slot0 dtor, slot1 Read, slot2 Write — both of which match layouts.json):

slot0 dtor  = 0x0075afb0
slot1 Read  = 0x005f8ac0
slot2 Write = 0x005f8ac0

and 0x005f8ac0 is C2 04 00 — a bare ret 4 stub shared across the binary. So the class uses the inherited no-op for both directions and the indsys frame is genuinely zero-length on disk. Its 0x1c4-byte body (two 7-element arrays of 0x20-byte records) is runtime state rebuilt each session. The empty frame in the saves is the correct output, not a truncation.

3. Two loop-nesting questions the bytes could not answer

Both counts involved are 0 in every save, so only the binary settles them.

  • Game::SVSOCrowDefenders::Write (0x004f8c90) — the recovery is wrong about dsys. Tag strings at .rdata 0x009f0270. Emission order: sys; ndsys count then a loop writing dsys; ndes count then a loop writing des; drad. objects/layouts.json records dsys as a plain member. It is a loop element. des as an ndes element was already right.
  • Game::SVSODerelict::Write (0x004fc2b0) — the recovery is right. NDsn count then a loop of (DsnID, Dwght); NAsg count then a loop of (Eflt, Esys). Two fields per iteration in each, confirmed by the 8-byte element strides.

objects/streams.json still carries the dsys misclassification. It was left alone rather than special-cased into serializers.py's loop heuristic: the conformance check aligns by tag and disk primitive, so the engine's corrected shape and the uncorrected table still agree item for item, and the correction is recorded here and in shapes.h. A future serializers.py improvement should pick it up.

4. A third trap resolved: Read accepts tags Write never emits

Game::SVSOSots::Read (0x005a7a40) reads NPCPlr (int, first) and hastraps (bool) plus a following traps object. Write emits none of them. They are read-only backward compatibility for an older save format; a writer that omits them is correct, and ours does. Worth generalising: a tag in Read with no counterpart in Write is not a hole in the recovery.

5. What the engine does with it

sots-engine wip/wire types the whole tree: ScriptObjects (= Game::SVSOSots), the four scenario bodies, and the eight encounter bodies the saves exercise. The key is applied in both directions — the reader select()s from the xscn/EncID it just read, the writer from the one it is about to write — so a body round-trips as whatever it came in as.

The four ids with a factory entry but no occurrence in any save (7 SystemKiller, 8 PuppetMaster, 14 Locust, 21 Ortgay) are deliberately not modelled. Their serializers are recovered and shapes for them would probably be right, but nothing could check them; they fall to a generic Node and round-trip verbatim. Same for ScnObj, whose pointer is NULL in every save.

SvSctOb went from 147 / 156 / 156 / 126 opaque items to 0 on all four saves, with the byte-identical round trip preserved. All 18 new shapes bind clean against the generated wire schema: 0 mismatches, 0 wire-only, 0 shape-only.

6. Addresses

ghidra/addresses.d/lane-w.json — 13 entries (both factories, the jump table, the get-or-create helper, the seeder, SVSOSots Read/Write/vftable, the ret 4 stub, the SVSOIndependentSystems vftable, and the three Writes the nesting answers came from). gen_addresses.py merges to 640 entries with no duplicate name.