lane D2: the ship-design catalogue -- how designs persist, hull size, and the 0x400 flag

Blocker #4 on lane Y's path to a byte-matching turn.

HOW DESIGNS PERSIST, correcting a published finding. Game::ShipDesign::Write is
0x008325e0 and makes four stream calls. The recorded address 0x008747a0 is in
NO vftable at all (lane V2's inversion), so "ShipDesign::Write makes no stream
call" was a misattribution, not a fact about the class. Game::ShipDesign
derives from Game::ShipDesignDef and inherits IStreamable second, so its writer
is reached through an adjustor thunk -- which is what the slot-indexed
serializer sweep found instead. A design persists as two serializers, base and
derived: ShipDesignDef::Write 0x00827390 emits FAIDes/DHide/DWep/DName then
exactly three DSec frames (+0x4c command, +0x24 mission, +0x74 engine),
ShipDesign::Write appends Dtc, the Dwgv flag and a conditional Dwg frame.

THREE sections, not five. The campaign's "slots 3-4 reserved and always empty"
was save_reader.py's trailing Rest("sections") sweeping Dtc and Dwgv into the
section list, and stock_designs.py decoding them as two empty sections -- rule 8
in its exact form, reader and consumer agreeing with each other and both wrong.
Two independent enumerations say three: the writer's straight-line body, and the
ctor's eh_vector_constructor_iterator(this+0x24, 0x28, 3) closing at
0x9c = sizeof(ShipDesignDef).

DWep and Dwgv are BOOLs, not ints -- byte-indistinguishable from ints at a
four-character tag, the same class of defect as ObservedTech.odet.

THE 0x400 FLAG IS `defence_platform`, read off the .shipsection parser's own bit
setter at 0x005749b7. NOT lane B5's 0x400: that one is a fleet flag, on the wire
as FtFlg. The full role-flag table is in the finding. HULL SIZE is section_class
through a three-name stricmp table (Destroyer/Cruiser/Dreadnought -> 0/1/2),
absent or unrecognised meaning 0 with a log line rather than an error. Both
words are recomputed from the data files by ShipDesign::UpdateDerivedStats
0x0087e7c0 and neither is on the wire. Corroborated by the default hull-health
table the same bit picks: 500/3000/15000 without it, 100/500/1000 with.

MEASURED: the census rebuilt from each save's own state matches the record the
game archived, 480 leaves / 0 mismatched over 11 saves and 503 designs, computed
independently in Python and in C++. COVERAGE REPORTED AS LOUDLY: only 32 of the
480 leaves are nonzero, and three of the six census leaves (both cruiser rows,
dreadnought platforms) are unexercised by every save in the corpus.

Closed 0 / regressed 0 against the standalone's divergence list, reported
separately: the census leaves live in src/app's turn record, which lane A2 holds
this cycle, so this lane evaluated and reported rather than writing.

Oracles fixed openly (rule 12): save_reader.py's Des shape, 49/49 with three
corrected tests and one added that pins "exactly three DSec" against real saves;
stock_designs.json regenerated, whose diff is only raw_slots 5->3 and dWep
int->bool across all 127 designs with every other field identical;
test_design_rules.py still 32/32 with the same ground truth.

19 addresses in ghidra/addresses.d/lane-d2.json, no collision; the generated
header was validated to a scratch path, never written in place.
This commit is contained in:
alex 2026-09-08 12:45:36 -04:00
parent 1754cc20ac
commit a4aba6a9fb
7 changed files with 730 additions and 1918 deletions

View file

@ -0,0 +1,326 @@
# The ship-design catalogue: how designs persist, hull size, and the `0x400` flag
Lane D2, 2026-09-08. Static reading against the exe, checked against all 11 saves.
Blocker #4 on lane Y's path to a byte-matching turn. Three lanes had brushed against the first
question and each left it open; this closes it, and the answer corrects a published finding.
---
## 1. How ship designs persist — the serializer three lanes could not find
**`Game::ShipDesign::Write` exists, is `0x008325e0`, and makes four stream calls.** The published
finding — "`Game::ShipDesign::Write` (`0x008747a0`) makes no stream call at all" — is **wrong in its
address**. `0x008747a0` is not this class's writer and is not a serializer of any kind: lane V2's
exact vtable inversion puts it in **no vftable at all** (`vtable_map.py who 0x008747a0` →
`is in no vftable`), and its body pops a parser stack. The serializer sweep attributed it to
`ShipDesign` by a slot mapping that did not hold for this class.
The class is not the shape the earlier note assumed either. RTTI `0x00a894a8`:
```
Game::ShipDesign : Game::ShipDesignDef, Mars::IStreamable, Mars::RefCounted, Mars::NetworkObject
```
`ShipDesignDef` is the **base**, and it is *not* an `IStreamable` — it declares its own three
virtuals (`Read`, `Write`, dtor, vftable `0x009fef64`) in a different order from `IStreamable`'s.
`ShipDesign` overrides all three; the `IStreamable` sub-object at `+0x9c` (vftable `0x00a32710`)
holds three adjustor thunks back to the primary vftable `0x00a32720`. That thunk layer is why a
slot-indexed sweep mis-keyed the class, and it is a **general warning for the serializer tool**: a
class that inherits an interface *second* reaches its own writer through a thunk, and the thunk is
what a slot walk finds.
So a design persists as **two serializers, a base and a derived one**, exactly the `*Impl` shape
lane G generalised — except the concrete class is a *derived class*, not a separate `Impl` class.
Lane V2's caution was right: check the shape, do not assume it.
### The full `Des` record, both writers, in disk order
| # | tag | primitive | member | writer |
|---|---|---|---|---|
| 1 | `FAIDes` | **bool** | `+0x4` | `ShipDesignDef::Write` `0x00827390` |
| 2 | `DHide` | **bool** | `+0x5` | " |
| 3 | `DWep` | **bool** | `+0x6` | " |
| 4 | `DName` | string | `+0x8` (`Mars::string`, `0x1c`) | " |
| 5 | `DSec` | frame | `+0x4c` — **command** | " (via `StreamableHelper<ShipDesignDef::Section>`) |
| 6 | `DSec` | frame | `+0x24` — **mission** | " |
| 7 | `DSec` | frame | `+0x74` — **engine** | " |
| 8 | `Dtc` | int | `+0x134` | `ShipDesign::Write` `0x008325e0` |
| 9 | `Dwgv` | **bool** | `+0x16c` | " |
| 10 | `Dwg` | frame, **only when `Dwgv`** | `+0x170`, `StreamableHelper<Game::WeaponGroups>` | " |
`ShipDesign::Read` `0x008324f0` mirrors it, with one extra: it reads an int `DRefCnt` into a **null
destination** — read and discarded — that the writer never emits. A network stream writes it; the
save stream does not, and the tag-addressed reader tolerates the absence.
### Three corrections this forces
**(a) Three sections, not five.** `SHIP_DESIGN_RULES.md` §1 said "five section slots, three used;
slots 3-4 are absent in every design (reserved)". There are **three**, and there is no reserved
slot. Two independent enumerations (rule 5):
* the writer emits exactly three `DSec` frames — a straight-line body, no loop;
* the constructor `0x00874c70` runs `eh_vector_constructor_iterator(this+0x24, stride 0x28,
count 3)`, and `0x24 + 3*0x28 = 0x9c`, which is exactly where `ShipDesign`'s own `IStreamable`
vptr sits. `sizeof(Game::ShipDesignDef) == 0x9c`.
The "two extra slots" were **`Dtc` and `Dwgv`**, swept into the section list by the reference Python
reader's trailing `Rest("sections")` and then decoded by `stock_designs.py` as two empty sections.
This is rule 8 in its exact form: the reader and its consumer agreed with each other and were both
wrong, and nothing downstream could see it because an empty slot is a no-op for every rule. Fixed
openly (§6).
**(b) `DWep` and `Dwgv` are bools, not ints.** Both writers call the bool primitive. With their
four-character tags a bool item and an int item are the same size on the wire and the values 0 and 1
are the same bytes, so **no save can distinguish them** — the same class of defect as lane G's
`ObservedTech.odet`, and byte-neutral for the same reason. Corrected in both readers; the engine's
typed round trip stays byte-identical on all 11 saves.
**(c) Offset order ≠ write order, again.** The section array is `[+0x24, +0x4c, +0x74]`; the wire is
`[+0x4c, +0x24, +0x74]`. Reading the array in memory order gives **mission, command, engine**;
reading the wire gives **command, mission, engine**. Both orders matter below — the wire order is
what a save reader needs, the memory order is what decides hull size.
---
## 2. Hull size and the `0x400` flag
Both live on `ShipDesign`, **neither is on the wire**, and both are recomputed from the data files
by `ShipDesign::UpdateDerivedStats` `0x0087e7c0` whenever a design changes or is loaded.
### `0x400` = `defence_platform`
`ShipSectionDef` carries a **64-bit role-flag word** at `+0x298`/`+0x29c`. The `.shipsection` parser
`0x005744e0` sets it one bit at a time: each boolean role key is a call to
`SetRoleFlagBit(&flags, loMask, hiMask, value)` `0x0056e780`. At `0x005749b7`, immediately after the
`strcmp` against the literal `defence_platform` at `0x009fea10`:
```
push ecx ; the parsed value
push 0x0 ; hi mask
lea edx,[esi+0x298]
push 0x400 ; lo mask
push edx
call 0x0056e780
```
**`0x400` is `defence_platform`.** The rest of the word, from the same scan: `refueling_capacity`
`0x2`, `repair_capacity` `0x4`, `refinery` `0x8`, `mining_capacity` `0x10`, `scanrange` /
`rebelai_scanrange` `0x20`, `gateship` `0x40`, `ewar` `0x800`, `ramscoop` `0x1000`, `aicontrol`
`0x2000`, `command_quota` `0x10000`, `node_bore` `0x20000`, `prisoner_capacity` `0x40000`,
`freighter` `0x80000`, `gravboat_bonus` `0x200000`, `construction_capacity` `0x400000`, `science`
`0x1000000`, `tradingpost` `0x2000000`, `freighterQ` `0x8000000`, `police` `0x10000000`, `spy`
`0x40000000`, `spytender` `0x80000000`; high dword `colony_trap` `0x1`, `mining_trap` `0x2`,
`monitor` `0x4`, `propaganda` `0x10`.
The design's word is the **OR** across its resolved sections
(`AggregateSectionStats` `0x00826af0`, `out[4] |= GetRoleFlags(section)`), so one flagged section
flags the whole design. In the shipped data the 49 sections carrying `defence_platform` are exactly
the `{DE,CR,DN}DefencePlatform`, `{CR,DN}TorpedoPlatform`, `{CR,DN}DronePlatform`,
`CRDeepScanPlatform` families and `_VNeumannSatellite` — all standalone mission sections — so no
shipped design mixes a flagged section with unflagged ones. That the rule is an OR is therefore
**inferred from the instruction and untestable on the corpus** (rule 6); it is unit-tested in the
engine on a synthetic design instead.
**This is not lane B5's `0x400`.** A *fleet* gets flag `0x400` when the retreat pipeline creates it,
and that flag reaches the wire as `FtFlg` — `turn1-state.sav`'s first fleet has `FtFlg 1024`. It is
a different word on a different object. The numeral is a coincidence of two bit layouts, and the
brief was right to say "check rather than assume".
### Hull size = `section_class`
`ShipSectionDef+0x260` is the parsed `section_class`, and the parse is a three-name table:
```
ParseShipClassName 0x0056e1c0: _stricmp against "Destroyer", "Cruiser", "Dreadnought"
stores the index it stopped at -> 0, 1, 2
```
Case-insensitive, which the data needs — the shipped catalogs write both `cruiser` and `Cruiser`.
The wrapper `0x0056e250` logs `" [%s] unrecognized ship class"` and stores **0** on no match, and
a section with no `section_class` key at all never reaches the parser and keeps the constructed 0.
So **absent or unrecognised is a destroyer, not an error**. 16 shipped sections have no
`section_class` (the `_Biomissile` / `_BoardingPod` families and three `_NPC` stems).
The design's hull size is `AggregateSectionStats`'s `out[0x1d] = sectionDef->+0x260`, copied to
`ShipDesign+0x12c`. That is an **assignment inside the per-section loop, not a max and not an OR**,
so the *last resolved section in memory slot order* wins — i.e. engine, else command, else mission.
Rule A6 makes every shipped design class-homogeneous, so first-wins and last-wins agree on **all 503
design records in the 11 saves** (measured, §4); the assignment order is implemented anyway because
it is what the original does, and it is unit-tested on a hand-built mixed-class design.
### One corroboration the census does not need
The same `0x400` bit picks the design's **default hull health** when no section overrides it at
`+0x2ac` (`0x00826af0`, `out[2]`):
| hull size | without `0x400` | with `0x400` |
|---|---|---|
| 0 destroyer | 500 | 100 |
| 1 cruiser | 3000 | 500 |
| 2 dreadnought | 15000 | 1000 |
A defence platform is a fifth to a fifteenth of the ship's hull. That is an independent reading of
the same two words and it agrees: `0x400` separates *platforms* from *ships*, and `+0x12c` is the
three-step hull-size ladder.
---
## 3. The census — what the two words feed
`ShipCensusByHullClass` `0x00818a50` takes a player and an `int[8]`:
```
for each fleet in server.fleets where fleet->owner(+0x58) == player:
for each ship in fleet->ships(+0xa4..+0xa8):
design = ship->+0x14
if (design->+0xb8 & 0x400) == 0: ++out[0]; ++out[2 + design->+0x12c]
else: ++out[1]; ++out[5 + design->+0x12c]
```
`out[0]`/`out[1]` are the two grand totals, which the caller computes and **discards**. The six
breakdowns land at `turnRecord+0x2a..+0x34` and reach the wire as the three `cls` groups' **`shpt`**
(ships) and **`satt`** (satellites — the game's own word for a defence platform).
Note what the loop does *not* do: it never looks at the ship's own `PlrID`, only at the fleet's
owner, and it never looks at the ship's health or state. Every ship in an owned fleet counts.
---
## 4. Measurement: 480 census leaves, 0 mismatched
Rebuilt from the save's own state plus the section catalog, compared against the record the game
archived for that save's own frame (the record is rebuilt at end of turn *and* on load, so it never
has to survive a round trip).
| saves | designs classified | leaves compared | **mismatched** |
|---|---|---|---|
| 11 | 503 | **480** | **0** |
Computed twice, independently: once in Python off `save_reader.py` + `design_rules.py`, once in C++
off the engine's own `shapes` reader and `game::data::Catalog`. The two disagree about how many
sections a design has (the Python reader's five vs the engine's three, §1a) and still produce the
same 480 numbers — so they are not the same check wearing two hats.
**Coverage, reported as loudly as the result (rule 15).** Only **32 of the 480 leaves are nonzero in
the archive**. A zero leaf agrees for free.
| leaf | nonzero of 80 |
|---|---|
| cls 0 `shpt` — destroyer ships | **18** |
| cls 0 `satt` — destroyer platforms | **3** |
| cls 1 `shpt` — cruiser ships | 0 |
| cls 1 `satt` — cruiser platforms | 0 |
| cls 2 `shpt` — dreadnought ships | **11** |
| cls 2 `satt` — dreadnought platforms | 0 |
**Three of the six census leaves are unexercised, not verified.** No save in the corpus contains a
cruiser of any kind or a dreadnought-class platform, so the cruiser column and the dreadnought
platform cell are hypotheses (rule 6). They are cheap to exercise — a save taken after
`IND_CruisCon` is researched and one cruiser is built would close the whole cruiser column, and
`zuul-turn23-fleet23.sav`'s 27-destroyer fleet shows the destroyer path is genuinely loaded.
Other honest residuals from the same run:
* 0 ships whose design id is missing from the save's design tables;
* 0 designs none of whose sections resolve against the catalog;
* 0 designs where first-resolved and last-resolved section disagree on hull size (rule A6 predicts
0, and this is the measurement that says so rather than the assumption).
---
## 5. What this closes, and what it does not
Against lane Y's baseline — turn1→turn2 **209→204**, turn2→turn3 **108→103**, 5 closed / 0
regressed, with the 204 broken down `/Sim/players` 82, `/Sim/systems` 80, `/Sim/turnstats` 24,
`/Sim/SvSctOb` 8, plus 10 singletons:
**closed: 0. regressed: 0.** Reported separately, and both are zero *in the standalone's divergence
list*, because this lane wrote no phase.
That is the honest number and it needs the reason spelled out. The census leaves live under
`/Sim/turnstats`, and the turn record is built by `src/app/turn_record.cpp`, which **lane A2 holds**
this cycle (the alliance mask `almem` is a field of the same record). Committing a census field into
that file now is exactly the cross-lane collision the fragment discipline exists to prevent. So this
lane did what lane S2's convention prescribes for a phase whose formula is held but whose write-out
is not this lane's: **evaluate and report, do not write**.
What is now unblocked, quantified:
* **6 of the 24 `/Sim/turnstats` leaves per pair** — `shpt` ×3 and `satt` ×3 — have a formula that
reproduces the archived value on 480 of 480 observations. Wiring them is one call to
`game::design::ShipCensus` inside `BuildTurnRecord`, and the earlier `--commit-blocked` run's
"three leaves of the ship census" among its 17 regressions were exactly these, wrong because the
record committed zeros. They should now commit clean.
* **The ship records** (`Player.ShipRecs`) and **tail phase T24** both need the same two words per
design; they are now available as `DesignStats::hull_size` / `DesignStats::defence_platform` and
as the standalone `classify_design()`.
* The turn record's other four unmodelled fields are **not** touched: `almem` is lane A2's,
`tdinc` waits on per-system money output, `bat` is a tail stub, and `sacq`/`slost` have count 0 in
every save (rule 6).
Nothing here was compared against the live game. `verified` in the phase catalog means "compared
against the running game", and this is not that — it is a comparison against bytes the original
wrote, which is the strongest instrument a lane holding no VM has.
---
## 6. Fixes to the oracles, made openly (rule 12)
Two readers carried the same wrong record shape; both are fixed, with a test per defect and
byte-neutrality shown.
**`verify/save-reader/save_reader.py`** — the `Des` shape ended in `Rest("sections")`. Now: three
`DSec` frames as a `Repeat`, then named `Dtc` (int) and `Dwgv` (bool), then the conditional `Dwg`;
`DWep` retyped int → bool. All 11 saves still read strict-clean; the reader's own suite is **49/49**
(was 48; three existing tests asserted the old shape and were corrected, one test added that pins
"exactly three `DSec`" against the real saves).
**`verify/design-rules/stock_designs.json`** regenerated from the same three saves. The diff is
*only* `raw_slots` 5 → 3 and `dWep` `0/1` → `false/true`; every other field of all 127 designs is
byte-for-byte the same, and `test_design_rules.py` is **32/32** with the same ground truth
(127/127 structural, 197/197 `DOpts`, 121/127 gating with the six Tarkas hidden-rider warns).
**`sots-engine` `src/mars/stream/shapes.h`** — `Design::dWep` and `Design::dwgv` retyped to bool,
the conditional `Dwg` node added, and the comment that said the derived writer made no stream call
replaced with the recovered layout. The typed round trip is still **byte-identical on all 11 saves**
at 100% named coverage.
The engine's `stock_designs.cpp` fixture loader now accepts 3–5 `raw_slots` so an old fixture still
loads. `kSlotCount` is deliberately **left at 5**: the extra two array entries are inert and
touching the enum reaches `rules.cpp` and another lane's tests for no behavioural gain. The comment
in `design.h` is corrected instead — the record is three sections, and the fifth-slot slack is this
engine's own, not the game's.
---
## 7. Open, and what would close it
1. **The cruiser column and the dreadnought-platform cell** are unexercised (§4). One save with a
cruiser in it and one with a DN defence platform close three leaves. Cheapest of everything on
this page.
2. **`Dtc`** (`+0x134`) is on the wire and is stamped by at least one caller immediately before the
recompute; its meaning is not read here. It is 1 on the stock designs.
3. **`Dwgv`/`Dwg`** — weapon groups — are false in every save. The conditional branch is implemented
from the instruction and is unexercised.
4. **The other 24 role-flag bits** are decoded (§2) but only `0x400` is consumed by anything this
lane read. `0x80000` (`freighter`) and `0x8000000` (`freighterQ`) drive a *second* classifier
`0x0081a430`/`0x0082c7c0` over the same two words — worth reading if a freighter count turns up
missing elsewhere.
5. **The remaining ~24 fields of the aggregator's output struct** (`+0x18`..`+0x64` →
`ShipDesign+0xc0`..`+0x118`) are the rest of the derived-stat block — mass, cost, capacities,
speeds. `SHIP_DESIGN_RULES.md` §6 derives most of them from the data with the formulas marked as
assumptions; `0x00826af0` is where every one of those assumptions can be settled, field by field,
and it is now a named function.
---
## Addresses
`ghidra/addresses.d/lane-d2.json` (19 entries, no collision with any existing fragment).
Key ones: `ShipDesignDef::Write` `0x00827390`, `ShipDesign::Write` `0x008325e0`,
`ShipDesign::UpdateDerivedStats` `0x0087e7c0`, `AggregateSectionStats` `0x00826af0`,
`ShipSectionDef::GetRoleFlags` `0x0056ee80`, `ParseShipClassName` `0x0056e1c0`,
`SetRoleFlagBit` `0x0056e780`, `ShipCensusByHullClass` `0x00818a50`;
offsets `ShipDesign+0xb8` (role flags), `+0x12c` (hull size), `ShipSectionDef+0x260`
(`section_class`), `+0x298` (role flags).

View file

@ -0,0 +1,156 @@
{
"entries": [
{
"name": "Game_ShipDesignDef_Write",
"addr": "0x00827390",
"convention": "thiscall",
"prototype": "void (Game_ShipDesignDef* this, Mars::Stream* s) // THE DESIGN SERIALIZER LANE D SAID DID NOT EXIST. Slot 1 of the ShipDesignDef vftable 0x009fef64. Writes, in DISK order: WriteBool 'FAIDes' this+0x4, WriteBool 'DHide' this+0x5, WriteBool 'DWep' this+0x6 (a BOOL, not an int -- the campaign schema had it as int; byte-neutral because a 4-char tag makes both items 12 bytes), WriteString 'DName' this+0x8, then THREE 'DSec' frames through StreamableHelper<ShipDesignDef::Section> at this+0x4c, this+0x24, this+0x74 in that order. THREE sections, not five: the ctor 0x00874c70 runs eh_vector_constructor_iterator(this+0x24, stride 0x28, count 3). MEMORY ORDER != WRITE ORDER: the array is [+0x24, +0x4c, +0x74] and the wire is [+0x4c (command), +0x24 (mission), +0x74 (engine)]",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipDesignDef_Read",
"addr": "0x00827240",
"convention": "thiscall",
"prototype": "void (Game_ShipDesignDef* this, Mars::Stream* s) // slot 0 of vftable 0x009fef64. Mirrors Write field for field, same tags, same three DSec frames in the same order",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipDesign_Write",
"addr": "0x008325e0",
"convention": "thiscall",
"prototype": "void (Game_ShipDesign* this, Mars::Stream* s) // CORRECTS 'Game::ShipDesign::Write (0x008747a0) makes no stream call at all': 0x008747a0 is in NO vftable and is not this class's writer. Game::ShipDesign derives from Game::ShipDesignDef (RTTI 0x00a894a8: ShipDesign, ShipDesignDef, IStreamable, RefCounted, NetworkObject; IStreamable sub-object at +0x9c, NetworkObject at +0xa0). Primary vftable 0x00a32720 slot 1; the +0x9c IStreamable vftable 0x00a32710 reaches it through an adjustor thunk at 0x00874de0. Body: direct call to ShipDesignDef::Write 0x00827390 (the base part), then WriteInt 'Dtc' this+0x134, WriteBool 'Dwgv' this+0x16c, and ONLY IF that flag is set a 'Dwg' frame through StreamableHelper<Game::WeaponGroups> at this+0x170. Dwgv is false in all 11 saves, so the Dwg branch is unexercised (rule 6)",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipDesign_Read",
"addr": "0x008324f0",
"convention": "thiscall",
"prototype": "void (Game_ShipDesign* this, Mars::Stream* s) // primary vftable 0x00a32720 slot 0. Calls ShipDesignDef::Read, then ReadInt 'DRefCnt' into a NULL destination (read and discarded; the WRITER never emits it, so it is a network-stream field the tag-addressed reader tolerates), then 'Dtc', 'Dwgv' and the conditional 'Dwg'",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipDesign_ctor",
"addr": "0x00874c70",
"convention": "thiscall",
"prototype": "Game_ShipDesign* (Game_ShipDesign* this) // installs ShipDesignDef::vftable at +0, then IStreamable/NetworkObject at +0x9c/+0xa0 and the three ShipDesign vftables. Constructs the DName string at +0x8 and eh_vector_constructor_iterator(this+0x24, 0x28, 3) -- the three section records. Enumerates sizeof(Game_ShipDesignDef) == 0x9c",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipDesign_UpdateDerivedStats",
"addr": "0x0087e7c0",
"convention": "fastcall",
"prototype": "void (Game_ShipDesign* this) // THE RECOMPUTE THAT PRODUCES BOTH CENSUS WORDS. 16 call sites; one of them stamps this+0x134 (Dtc) and calls straight in. (1) resolves each of the three section records this+0x24/+0x4c/+0x74 to a ShipSectionDef* through the catalog lookup 0x0056edd0 and caches them at this+0xac/+0xb0/+0xb4, IN MEMORY SLOT ORDER; (2) builds a 3-element context array (stride 0x124) and hands it to the aggregator 0x00826af0 together with this+0x130; (3) copies the aggregator's ~0x7c-byte output struct into the design: struct+0x10 -> this+0xb8 (role flags, low dword), struct+0x14 -> this+0xbc (high dword), struct+0x18..+0x64 -> this+0xc0..+0x118, struct+0x74 -> this+0x12c (HULL SIZE), struct+0x78 -> this+0x1a4, struct+0x00..+0x0c -> this+0x138..+0x144. Neither this+0xb8 nor this+0x12c is on the wire: they are rebuilt from the data files whenever a design changes or is loaded",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipDesign_AggregateSectionStats",
"addr": "0x00826af0",
"convention": "cdecl",
"prototype": "void (DesignStatBlock* out, SectionContext ctx[3], void* techCtx, char flag) // walks the three section contexts (stride 0x124, ctx[i]+0 = the ShipSectionDef*, null = empty slot). out[4]/out[5] (the 64-bit role-flag word) |= ShipSectionDef::GetRoleFlags 0x0056ee80 per section -- an OR, so ONE flagged section flags the whole design. out[0x1d] (hull size) = sectionDef+0x260 -- an ASSIGNMENT, so the LAST resolved section in memory slot order wins; every shipped design is section_class-homogeneous so first-wins and last-wins agree on all 503 design records in the corpus. Also: out[6] x5 when any section carries tech 0x2756; out[4] &= ~0x80 for tech 0x2757; out[4] |= 0x100000 when EVERY section carries tech 0x2742. Default hull health (out[2]) is chosen by the SAME 0x400 bit when no section overrides it at +0x2ac: without 0x400 hull 0/1/2 -> 500/3000/15000, with 0x400 -> 100/500/1000",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipSectionDef_GetRoleFlags",
"addr": "0x0056ee80",
"convention": "thiscall",
"prototype": "unsigned __int64 (Game_ShipSectionDef* this, void* ctx) // returns CONCAT(this+0x29c, this+0x298) -- the section's 64-bit role-flag word straight out of the parsed .shipsection -- with bit 0x20 of the low dword OR-ed in when ctx is non-null, ctx+0xfc is set and this+0x304 > 0. That conditional bit is the only part of the word that is not pure file data",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipSectionCatalog_FindByID",
"addr": "0x0056edd0",
"convention": "thiscall",
"prototype": "Game_ShipSectionDef* (SectionCatalog* this, Game_ShipSectionID* id) // linear scan of the vector at this+0x8/+0xc matching def+0x4 == id->species and def+0x8 == id->sectionId. Returns null for the (0,0) empty slot without scanning",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ParseShipClassName",
"addr": "0x0056e1c0",
"convention": "cdecl",
"prototype": "bool (int* out, const char* name) // THE HULL-SIZE DEFINITION. _stricmp against \"Destroyer\", \"Cruiser\", \"Dreadnought\" in that order and stores the index it stopped at: 0, 1, 2. Returns false without writing on no match",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipSectionDef_ParseSectionClass",
"addr": "0x0056e250",
"convention": "cdecl",
"prototype": "bool (int* out, const char* name) // wraps 0x0056e1c0 for the `section_class` key; on failure logs \" [%s] unrecognized ship class\" and stores 0, so an unknown OR ABSENT section_class is a destroyer, not an error",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_SetRoleFlagBit",
"addr": "0x0056e780",
"convention": "cdecl",
"prototype": "void (unsigned __int64* flags, unsigned int loMask, unsigned int hiMask, bool value) // the .shipsection parser's flag setter. Every boolean role key in the section parser 0x005744e0 is one call to this with its own mask pair; `defence_platform` is (lo 0x400, hi 0) at 0x005749b7, `monitor` is (lo 0, hi 0x4), `refinery` 0x8, `mining_capacity` 0x10, `scanrange`/`rebelai_scanrange` 0x20, `gateship` 0x40, `ewar` 0x800, `ramscoop` 0x1000, `aicontrol` 0x2000, `command_quota` 0x10000, `node_bore` 0x20000, `prisoner_capacity` 0x40000, `freighter` 0x80000, `gravboat_bonus` 0x200000, `construction_capacity` 0x400000, `science` 0x1000000, `tradingpost` 0x2000000, `freighterQ` 0x8000000, `police` 0x10000000, `spy` 0x40000000, `spytender` 0x80000000, `refueling_capacity` 0x2, `repair_capacity` 0x4; high dword: `colony_trap` 0x1, `mining_trap` 0x2, `monitor` 0x4, `propaganda` 0x10",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ServerPlayer_ShipCensusByHullClass",
"addr": "0x00818a50",
"convention": "cdecl",
"prototype": "void (ServerPlayer* p, int out[8]) // THE CENSUS. Zeroes out[0..7], then walks the server's fleet vector (p+0x8 -> S, S+0x60/+0x64), keeps fleets whose owner (fleet+0x58) is p, and for every ship in fleet+0xa4/+0xa8 takes design = ship+0x14. If (design+0xb8 & 0x400) == 0: ++out[0] and ++out[2 + design->hullSize(+0x12c)]; else ++out[1] and ++out[5 + hullSize]. So out[0]/out[1] are the two grand totals (computed and DISCARDED by the caller), out[2..4] are ships by hull size 0/1/2 and out[5..7] are defence platforms by hull size. The six land at turnRecord+0x2a..+0x34 and reach the wire as the three `cls` groups' `shpt` and `satt`",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "Game_ShipDesign_IsMobileWarshipClass",
"addr": "0x0081a430",
"convention": "cdecl",
"prototype": "int (Game_ShipDesign* design) // a SECOND classifier over the same two words, kept because it shows the flag word is a role set and not a single bit: returns -1 when the design lacks flag 0x80000 (`freighter`), else for hull size 1 returns 0 when 0x8000000 (`freighterQ`) is set and 1 otherwise, and 2 for any other hull size. 0x0082c7c0 is the matching counter over a fleet list. NOT the census -- neither reads 0x400",
"status": "unverified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "ShipDesign_off_RoleFlagsLow",
"offset": "0xb8",
"convention": "offset",
"prototype": "unsigned int // low dword of the design's 64-bit role-flag word, the OR of its sections'. Bit 0x400 = `defence_platform`. High dword at +0xbc. NOT on the wire; rebuilt by 0x0087e7c0",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "ShipDesign_off_HullSize",
"offset": "0x12c",
"convention": "offset",
"prototype": "int // 0 destroyer / 1 cruiser / 2 dreadnought, from the last resolved section's `section_class`. NOT on the wire; rebuilt by 0x0087e7c0",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "ShipDesign_off_Dtc",
"offset": "0x134",
"convention": "offset",
"prototype": "int // the wire field `Dtc`, written by ShipDesign::Write and stamped by at least one caller immediately before it calls the recompute 0x0087e7c0",
"status": "unverified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "ShipSectionDef_off_SectionClass",
"offset": "0x260",
"convention": "offset",
"prototype": "int // parsed `section_class`: 0 destroyer / 1 cruiser / 2 dreadnought, 0 when absent or unrecognised",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
},
{
"name": "ShipSectionDef_off_RoleFlagsLow",
"offset": "0x298",
"convention": "offset",
"prototype": "unsigned int // low dword of the section's 64-bit role-flag word (high dword at +0x29c), one bit per boolean role key in the .shipsection file",
"status": "verified",
"source": "findings/objects/ship-design-catalogue.md"
}
]
}

View file

@ -29,15 +29,31 @@ From the save (`Player.designs[]` / `Player.legacyDesigns[]`, fully decoded in
`stock_designs.py`):
```
Des { faiDes bool, dHide bool, dWep int, dName string,
sections[5] : DSec { DSec { int species_idx, int section_id },
Des { faiDes bool, dHide bool, dWep bool, dName string,
sections[3] : DSec { DSec { int species_idx, int section_id },
DGbnk2 { int n_banks, n x ( DW2 { bool bID, wid int | wfn string, int did } ) },
DOpts { int n, n x tech-name string } } }
DOpts { int n, n x tech-name string } },
dtc int, dwgv bool, [ Dwg {...} only when dwgv ] }
```
* **Five section slots, three used**: slot 0 = command, 1 = mission, 2 = engine;
slots 3-4 are absent in every design (reserved). An empty slot is
`(species 0, section 0)`; ids in `_shipsections.txt` start at 1.
> **CORRECTION (lane D2, 2026-09-08).** This section used to read `sections[5]` and
> "five section slots, three used; slots 3-4 are absent in every design
> (reserved)". **There are three sections and no reserved slots.** The two
> extra entries were `Dtc` and `Dwgv` — the derived writer's own tail — swept
> into the section list by `save_reader.py`'s trailing `Rest("sections")` and
> then decoded by `stock_designs.py` as two empty sections. Both writers are
> now recovered: `Game::ShipDesignDef::Write 0x00827390` emits the four scalars
> and exactly three `DSec` frames, `Game::ShipDesign::Write 0x008325e0` appends
> `Dtc`, `Dwgv` and the conditional `Dwg`. The constructor confirms the count
> independently (`eh_vector_constructor_iterator(this+0x24, stride 0x28,
> count 3)`, and `0x24 + 3*0x28 = 0x9c = sizeof(ShipDesignDef)`). `dWep` is a
> **bool** on the wire, not an int — byte-indistinguishable, writer-decidable.
> See `findings/objects/ship-design-catalogue.md`.
* **Three section slots**: disk order 0 = command, 1 = mission, 2 = engine. An
empty slot is `(species 0, section 0)`; ids in `_shipsections.txt` start at 1.
(In memory the array is ordered mission, command, engine — offset order is not
write order here, and the memory order is what decides hull size.)
* `species_idx` indexes the save's species table `Human, Hiver, Tarkas, Liir,
_NPC, Zuul, Morrigi` (`SPECIES_INDEX`); `section_id` is the race's
`_shipsections.txt` id. Every design in the saves uses one species for all
@ -260,6 +276,30 @@ Zuul sections) -- last value taken; engine behaviour unknown.
4x gauss + `mis`), hidden `Default Assault Shuttle`. The `Independent
Colony` NPC has a Tarkas `Defence Platform` with `las_red` x4 + `mis`.
## 7A. Hull size and the class flag (lane D2, code-settled)
Two derived words the design caches, **neither on the wire**, both rebuilt from
this catalog by `ShipDesign::UpdateDerivedStats 0x0087e7c0` whenever a design
changes or is loaded. They are what the turn record's per-hull-class ship
census counts by. Full derivation in
`findings/objects/ship-design-catalogue.md`.
| word | rule |
|---|---|
| **hull size** (`ShipDesign+0x12c`) | the `section_class` of the **last resolved section in memory slot order** (engine, else command, else mission), mapped `Destroyer 0 / Cruiser 1 / Dreadnought 2` case-insensitively. Absent or unrecognised -> **0**, with a log line, not an error — settles the 16 shipped sections with no `section_class`. Assignment, not max: A6 makes first-wins and last-wins agree on all 503 design records in the 11 saves |
| **role flags** (`ShipDesign+0xb8`/`+0xbc`, 64-bit) | the **OR** of every resolved section's own flag word. One bit per boolean role key; `defence_platform` is `0x400` of the low dword. The OR is read from the instruction and is untestable on shipped data — no design mixes a flagged with an unflagged section |
The same `0x400` bit also picks the design's default hull health when no section
overrides it: without it `500 / 3000 / 15000` by hull size, with it
`100 / 500 / 1000`. So the flag separates *platforms* from *ships*, which is
exactly how the census uses it (`shpt` vs `satt`).
This also answers §8 question 1 in part: **A6 is not enforced by this path.**
The aggregator assigns the class from whichever section it visits last and never
compares the three, so a mixed-class design would be silently classified by its
engine section rather than rejected here. Whether the *designer UI* rejects it
is still open.
## 8. Open questions for the Ghidra side
1. **Class-equality (A6)** -- enforced in `ShipDesignDef` validation, or only

File diff suppressed because it is too large Load diff

View file

@ -161,7 +161,20 @@ IsFxAv FxAv Tag Pwd(string) Team Settings{4×"." int}`.
senp seno2 n×seot2`) `trn almem inc tdinc sav col bat tch`(int) `ncls` 3 × (`cls shpt shpl shpk satt
satl satk`).
`Des` (design header, under `NumDes`/`NumLeg` n × (`DesID`, `Des{}`)): `FAIDes`(bool) `DHide`(bool)
`DWep`(int) `DName`(string), then the `DSec` section frames (kept generic).
`DWep`(**bool**) `DName`(string), then **exactly three** `DSec` section frames — command, mission,
engine in that disk order, kept generic — then `Dtc`(int) `Dwgv`(**bool**) and, only when `Dwgv` is
set, a `Dwg` weapon-group frame (`Dwgv` is false in every save available).
> **Confirmed against the binary (2026-09-08, lane D2).** A design is written by
> **two** serializers: `Game::ShipDesignDef::Write` `0x00827390` (the four scalars and the three
> `DSec` frames, in wire order `+0x4c +0x24 +0x74`) and its subclass `Game::ShipDesign::Write`
> `0x008325e0` (`Dtc`, `Dwgv`, conditional `Dwg`). This **corrects two published claims**:
> `Game::ShipDesign::Write` is not `0x008747a0` and does not "make no stream call" — that address is
> in no vftable at all; and the record has **three** sections, not five. The reader's old
> `Rest("sections")` swept `Dtc` and `Dwgv` into the section list, which is where the "slots 3-4 are
> reserved and always empty" claim came from. `DWep` and `Dwgv` are `WriteBool`; with four-character
> tags a bool item and an int item are the same size and 0/1 the same bytes, so the change is
> byte-neutral and no save could have shown it. See `findings/objects/ship-design-catalogue.md`.
Objective records (Player `odes`/`owep`/`otch`, each a framed VectorHelper of `"."` elements):
`odes` = `otnF otnL odid opid`; `owep` = `otnF otnL odet owep`(string) `owith`; `otch` = `otnF otnL odet
otch`(string) `owith`.

View file

@ -520,9 +520,28 @@ CivilianRatios = Shape("civr", [A("smx", "float"), A("spe", SpeciesRatios), Rest
# Design header: on-disk tags are FAIDes/DHide/DWep/DName (R1's camel-case names
# were wrong). The typed-dict keys keep R1's spelling because
# verify/design-rules/stock_designs.py consumes them.
#
# TWO writers, a base and a derived one (lane D2): the base emits the four
# scalars then EXACTLY THREE `DSec` frames -- command, mission, engine, in that
# order -- and the derived one appends `Dtc`, the `Dwgv` flag and, only when
# that flag is set, the `Dwg` weapon-group frame.
#
# `DWep` and `Dwgv` are BOOLs, not ints: both writers call the bool primitive.
# With four-character tags a bool item and an int item are the same size on the
# wire and the values 0/1 are the same bytes, so no save can tell them apart --
# this is the same class of defect as `ObservedTech.odet`, and it is byte-neutral.
#
# CORRECTION (lane D2): this shape used to end in `Rest("sections")`, which swept
# `Dtc` and `Dwgv` into the section list. That is where the campaign's "five
# section slots, of which slots 3 and 4 are reserved and always empty" claim came
# from -- there are three sections and no reserved slots.
Design = Shape("Des", [
A("FAIDes", "bool", key="faiDes"), A("DHide", "bool", key="dHide"),
A("DWep", "int", key="dWep"), A("DName", "string", key="dName"), Rest("sections"),
A("DWep", "bool", key="dWep"), A("DName", "string", key="dName"),
Repeat(A("DSec", "any"), key="sections"),
A("Dtc", "int", key="dtc"), A("Dwgv", "bool", key="dwgv"),
If("dwgv", A("Dwg", "any", key="weaponGroups")), # never set in any save available
Rest(),
])
ConMods = Shape("ConMods", [
A("ConMod", "float", key="ConMod0"), A("SavMod", "float", key="SavMod0"),

View file

@ -426,11 +426,16 @@ class TagNameCorrectionsTest(unittest.TestCase):
names = self.field_names(shape)
self.assertEqual(names[0], ("otnF", True)) # not R1's "ontF"
self.assertTrue(all(auth for _, auth in names), names)
# Lane D2: the derived writer's Dtc/Dwgv are named fields, not part of the
# section run; the run itself is a Repeat and holds exactly the three DSec frames.
self.assertEqual(self.field_names(sr.Design),
[("FAIDes", True), ("DHide", True), ("DWep", True), ("DName", True)])
[("FAIDes", True), ("DHide", True), ("DWep", True), ("DName", True),
("Dtc", True), ("Dwgv", True)])
# typed-dict keys keep R1's spelling (verify/design-rules/stock_designs.py reads them)
self.assertEqual([f.key for f in sr.Design.fields if isinstance(f, sr.Field)],
["faiDes", "dHide", "dWep", "dName"])
["faiDes", "dHide", "dWep", "dName", "dtc", "dwgv"])
rep = [f for f in sr.Design.fields if isinstance(f, sr.Repeat)]
self.assertEqual([(r.lead, r.key) for r in rep], [("DSec", "sections")])
self.assertEqual(self.field_names(sr.NodeGrid), [("paths", True), ("nextid", True)])
self.assertEqual(self.field_names(sr.BuildQueue), [("ords", True)])
self.assertEqual(self.field_names(sr.FlightPlan)[0], ("wpts", True))
@ -444,7 +449,8 @@ class TagNameCorrectionsTest(unittest.TestCase):
self.assertEqual(sr.GLOBAL_KINDS["nextid"], "int")
self.assertEqual((sr.GLOBAL_KINDS["FAIDes"], sr.GLOBAL_KINDS["DHide"],
sr.GLOBAL_KINDS["DWep"], sr.GLOBAL_KINDS["DName"]),
("bool", "bool", "int", "string"))
("bool", "bool", "bool", "string")) # DWep is WriteBool, not WriteInt
self.assertEqual(sr.GLOBAL_KINDS["Dwgv"], "bool") # likewise
for name in ("ords", "wpts", "paths"):
self.assertIsInstance(sr.GLOBAL_SHAPES[name], sr.CArr)
@ -479,8 +485,9 @@ class TagNameCorrectionsTest(unittest.TestCase):
def design(tags):
w = sw.SaveWriter()
w.begin("Des")
w.bool(tags[0], True); w.bool(tags[1], False); w.int(tags[2], 1); w.string(tags[3], "Honor Lance")
w.bool(tags[0], True); w.bool(tags[1], False); w.bool(tags[2], True); w.string(tags[3], "Honor Lance")
w.begin("DSec"); w.int("ga", 1); w.end() # sections stay generic
w.int("Dtc", 1); w.bool("Dwgv", False) # the derived writer's tail
w.end()
return w.bytes()
@ -491,8 +498,10 @@ class TagNameCorrectionsTest(unittest.TestCase):
self.assertTrue(all(c.hinted for c in root.children[0].children[:4])) # typed from the Des shape
res = sr.read_bytes(data, padding="joint", strict=True, schema=schema)
d = res.typed["Des"]
self.assertEqual((d["faiDes"], d["dHide"], d["dWep"], d["dName"]), (True, False, 1, "Honor Lance"))
self.assertEqual((d["faiDes"], d["dHide"], d["dWep"], d["dName"]), (True, False, True, "Honor Lance"))
# the section run stops at Dtc: the tail is NOT two more "reserved slots"
self.assertEqual([s["_name"] for s in d["sections"]], ["DSec"])
self.assertEqual((d["dtc"], d["dwgv"]), (1, False))
self.assertNotIn("FAIDes", d) # key stays R1's; only the tag changed
with self.assertRaises(sr.SaveFormatError): # camel-case R1 spelling is not on disk
sr.read_bytes(self.design(("faiDes", "dHide", "dWep", "dName")),
@ -779,6 +788,33 @@ class WireSchemaDefectsTest(unittest.TestCase):
for d in pl["shipRecs"]["designRecords"]:
self.assertEqual(set(d) - {"_off"}, {"srd", "src", "srb", "srl", "sri"})
# --- lane D2: the Des record has THREE sections, and DWep/Dwgv are bools ----
@unittest.skipUnless(os.path.exists(os.path.join(REAL, "turn3-state.sav")), "real saves not present")
def test_design_records_hold_exactly_three_sections(self):
"""The campaign's "five slots, 3 and 4 reserved" was this reader's own
`Rest()` sweeping Dtc and Dwgv into the section list. Both design
writers are recovered: the base emits three DSec frames and nothing
else, the derived one appends Dtc and the Dwgv flag."""
seen = 0
for name in ("turn1-state.sav", "turn2-state.sav", "turn3-state.sav"):
path = os.path.join(self.REAL, name)
if not os.path.exists(path):
continue
res = sr.read_save(path, padding="joint", strict=True)
for pe in res.typed["sim"]["players"]:
pl = pe["Player"]
for key in ("designs", "legacyDesigns"):
for d in pl.get(key, []):
des = d["Des"]
self.assertEqual([x["_name"] for x in des["sections"]],
["DSec", "DSec", "DSec"], des["dName"])
self.assertIsInstance(des["dWep"], bool)
self.assertIsInstance(des["dwgv"], bool)
self.assertFalse(des["dwgv"]) # never set in any save available
self.assertNotIn("weaponGroups", des)
seen += 1
self.assertGreater(seen, 100)
class CliTest(unittest.TestCase):
def setUp(self):