Blocker #4 on lane Y's path to a byte-matching turn. HOW DESIGNS PERSIST, correcting a published finding. Game::ShipDesign::Write is 0x008325e0 and makes four stream calls. The recorded address 0x008747a0 is in NO vftable at all (lane V2's inversion), so "ShipDesign::Write makes no stream call" was a misattribution, not a fact about the class. Game::ShipDesign derives from Game::ShipDesignDef and inherits IStreamable second, so its writer is reached through an adjustor thunk -- which is what the slot-indexed serializer sweep found instead. A design persists as two serializers, base and derived: ShipDesignDef::Write 0x00827390 emits FAIDes/DHide/DWep/DName then exactly three DSec frames (+0x4c command, +0x24 mission, +0x74 engine), ShipDesign::Write appends Dtc, the Dwgv flag and a conditional Dwg frame. THREE sections, not five. The campaign's "slots 3-4 reserved and always empty" was save_reader.py's trailing Rest("sections") sweeping Dtc and Dwgv into the section list, and stock_designs.py decoding them as two empty sections -- rule 8 in its exact form, reader and consumer agreeing with each other and both wrong. Two independent enumerations say three: the writer's straight-line body, and the ctor's eh_vector_constructor_iterator(this+0x24, 0x28, 3) closing at 0x9c = sizeof(ShipDesignDef). DWep and Dwgv are BOOLs, not ints -- byte-indistinguishable from ints at a four-character tag, the same class of defect as ObservedTech.odet. THE 0x400 FLAG IS `defence_platform`, read off the .shipsection parser's own bit setter at 0x005749b7. NOT lane B5's 0x400: that one is a fleet flag, on the wire as FtFlg. The full role-flag table is in the finding. HULL SIZE is section_class through a three-name stricmp table (Destroyer/Cruiser/Dreadnought -> 0/1/2), absent or unrecognised meaning 0 with a log line rather than an error. Both words are recomputed from the data files by ShipDesign::UpdateDerivedStats 0x0087e7c0 and neither is on the wire. Corroborated by the default hull-health table the same bit picks: 500/3000/15000 without it, 100/500/1000 with. MEASURED: the census rebuilt from each save's own state matches the record the game archived, 480 leaves / 0 mismatched over 11 saves and 503 designs, computed independently in Python and in C++. COVERAGE REPORTED AS LOUDLY: only 32 of the 480 leaves are nonzero, and three of the six census leaves (both cruiser rows, dreadnought platforms) are unexercised by every save in the corpus. Closed 0 / regressed 0 against the standalone's divergence list, reported separately: the census leaves live in src/app's turn record, which lane A2 holds this cycle, so this lane evaluated and reported rather than writing. Oracles fixed openly (rule 12): save_reader.py's Des shape, 49/49 with three corrected tests and one added that pins "exactly three DSec" against real saves; stock_designs.json regenerated, whose diff is only raw_slots 5->3 and dWep int->bool across all 127 designs with every other field identical; test_design_rules.py still 32/32 with the same ground truth. 19 addresses in ghidra/addresses.d/lane-d2.json, no collision; the generated header was validated to a scratch path, never written in place.
326 lines
18 KiB
Markdown
326 lines
18 KiB
Markdown
# The ship-design catalogue: how designs persist, hull size, and the `0x400` flag
|
||
|
||
Lane D2, 2026-09-08. Static reading against the exe, checked against all 11 saves.
|
||
|
||
Blocker #4 on lane Y's path to a byte-matching turn. Three lanes had brushed against the first
|
||
question and each left it open; this closes it, and the answer corrects a published finding.
|
||
|
||
---
|
||
|
||
## 1. How ship designs persist — the serializer three lanes could not find
|
||
|
||
**`Game::ShipDesign::Write` exists, is `0x008325e0`, and makes four stream calls.** The published
|
||
finding — "`Game::ShipDesign::Write` (`0x008747a0`) makes no stream call at all" — is **wrong in its
|
||
address**. `0x008747a0` is not this class's writer and is not a serializer of any kind: lane V2's
|
||
exact vtable inversion puts it in **no vftable at all** (`vtable_map.py who 0x008747a0` →
|
||
`is in no vftable`), and its body pops a parser stack. The serializer sweep attributed it to
|
||
`ShipDesign` by a slot mapping that did not hold for this class.
|
||
|
||
The class is not the shape the earlier note assumed either. RTTI `0x00a894a8`:
|
||
|
||
```
|
||
Game::ShipDesign : Game::ShipDesignDef, Mars::IStreamable, Mars::RefCounted, Mars::NetworkObject
|
||
```
|
||
|
||
`ShipDesignDef` is the **base**, and it is *not* an `IStreamable` — it declares its own three
|
||
virtuals (`Read`, `Write`, dtor, vftable `0x009fef64`) in a different order from `IStreamable`'s.
|
||
`ShipDesign` overrides all three; the `IStreamable` sub-object at `+0x9c` (vftable `0x00a32710`)
|
||
holds three adjustor thunks back to the primary vftable `0x00a32720`. That thunk layer is why a
|
||
slot-indexed sweep mis-keyed the class, and it is a **general warning for the serializer tool**: a
|
||
class that inherits an interface *second* reaches its own writer through a thunk, and the thunk is
|
||
what a slot walk finds.
|
||
|
||
So a design persists as **two serializers, a base and a derived one**, exactly the `*Impl` shape
|
||
lane G generalised — except the concrete class is a *derived class*, not a separate `Impl` class.
|
||
Lane V2's caution was right: check the shape, do not assume it.
|
||
|
||
### The full `Des` record, both writers, in disk order
|
||
|
||
| # | tag | primitive | member | writer |
|
||
|---|---|---|---|---|
|
||
| 1 | `FAIDes` | **bool** | `+0x4` | `ShipDesignDef::Write` `0x00827390` |
|
||
| 2 | `DHide` | **bool** | `+0x5` | " |
|
||
| 3 | `DWep` | **bool** | `+0x6` | " |
|
||
| 4 | `DName` | string | `+0x8` (`Mars::string`, `0x1c`) | " |
|
||
| 5 | `DSec` | frame | `+0x4c` — **command** | " (via `StreamableHelper<ShipDesignDef::Section>`) |
|
||
| 6 | `DSec` | frame | `+0x24` — **mission** | " |
|
||
| 7 | `DSec` | frame | `+0x74` — **engine** | " |
|
||
| 8 | `Dtc` | int | `+0x134` | `ShipDesign::Write` `0x008325e0` |
|
||
| 9 | `Dwgv` | **bool** | `+0x16c` | " |
|
||
| 10 | `Dwg` | frame, **only when `Dwgv`** | `+0x170`, `StreamableHelper<Game::WeaponGroups>` | " |
|
||
|
||
`ShipDesign::Read` `0x008324f0` mirrors it, with one extra: it reads an int `DRefCnt` into a **null
|
||
destination** — read and discarded — that the writer never emits. A network stream writes it; the
|
||
save stream does not, and the tag-addressed reader tolerates the absence.
|
||
|
||
### Three corrections this forces
|
||
|
||
**(a) Three sections, not five.** `SHIP_DESIGN_RULES.md` §1 said "five section slots, three used;
|
||
slots 3-4 are absent in every design (reserved)". There are **three**, and there is no reserved
|
||
slot. Two independent enumerations (rule 5):
|
||
|
||
* the writer emits exactly three `DSec` frames — a straight-line body, no loop;
|
||
* the constructor `0x00874c70` runs `eh_vector_constructor_iterator(this+0x24, stride 0x28,
|
||
count 3)`, and `0x24 + 3*0x28 = 0x9c`, which is exactly where `ShipDesign`'s own `IStreamable`
|
||
vptr sits. `sizeof(Game::ShipDesignDef) == 0x9c`.
|
||
|
||
The "two extra slots" were **`Dtc` and `Dwgv`**, swept into the section list by the reference Python
|
||
reader's trailing `Rest("sections")` and then decoded by `stock_designs.py` as two empty sections.
|
||
This is rule 8 in its exact form: the reader and its consumer agreed with each other and were both
|
||
wrong, and nothing downstream could see it because an empty slot is a no-op for every rule. Fixed
|
||
openly (§6).
|
||
|
||
**(b) `DWep` and `Dwgv` are bools, not ints.** Both writers call the bool primitive. With their
|
||
four-character tags a bool item and an int item are the same size on the wire and the values 0 and 1
|
||
are the same bytes, so **no save can distinguish them** — the same class of defect as lane G's
|
||
`ObservedTech.odet`, and byte-neutral for the same reason. Corrected in both readers; the engine's
|
||
typed round trip stays byte-identical on all 11 saves.
|
||
|
||
**(c) Offset order ≠ write order, again.** The section array is `[+0x24, +0x4c, +0x74]`; the wire is
|
||
`[+0x4c, +0x24, +0x74]`. Reading the array in memory order gives **mission, command, engine**;
|
||
reading the wire gives **command, mission, engine**. Both orders matter below — the wire order is
|
||
what a save reader needs, the memory order is what decides hull size.
|
||
|
||
---
|
||
|
||
## 2. Hull size and the `0x400` flag
|
||
|
||
Both live on `ShipDesign`, **neither is on the wire**, and both are recomputed from the data files
|
||
by `ShipDesign::UpdateDerivedStats` `0x0087e7c0` whenever a design changes or is loaded.
|
||
|
||
### `0x400` = `defence_platform`
|
||
|
||
`ShipSectionDef` carries a **64-bit role-flag word** at `+0x298`/`+0x29c`. The `.shipsection` parser
|
||
`0x005744e0` sets it one bit at a time: each boolean role key is a call to
|
||
`SetRoleFlagBit(&flags, loMask, hiMask, value)` `0x0056e780`. At `0x005749b7`, immediately after the
|
||
`strcmp` against the literal `defence_platform` at `0x009fea10`:
|
||
|
||
```
|
||
push ecx ; the parsed value
|
||
push 0x0 ; hi mask
|
||
lea edx,[esi+0x298]
|
||
push 0x400 ; lo mask
|
||
push edx
|
||
call 0x0056e780
|
||
```
|
||
|
||
**`0x400` is `defence_platform`.** The rest of the word, from the same scan: `refueling_capacity`
|
||
`0x2`, `repair_capacity` `0x4`, `refinery` `0x8`, `mining_capacity` `0x10`, `scanrange` /
|
||
`rebelai_scanrange` `0x20`, `gateship` `0x40`, `ewar` `0x800`, `ramscoop` `0x1000`, `aicontrol`
|
||
`0x2000`, `command_quota` `0x10000`, `node_bore` `0x20000`, `prisoner_capacity` `0x40000`,
|
||
`freighter` `0x80000`, `gravboat_bonus` `0x200000`, `construction_capacity` `0x400000`, `science`
|
||
`0x1000000`, `tradingpost` `0x2000000`, `freighterQ` `0x8000000`, `police` `0x10000000`, `spy`
|
||
`0x40000000`, `spytender` `0x80000000`; high dword `colony_trap` `0x1`, `mining_trap` `0x2`,
|
||
`monitor` `0x4`, `propaganda` `0x10`.
|
||
|
||
The design's word is the **OR** across its resolved sections
|
||
(`AggregateSectionStats` `0x00826af0`, `out[4] |= GetRoleFlags(section)`), so one flagged section
|
||
flags the whole design. In the shipped data the 49 sections carrying `defence_platform` are exactly
|
||
the `{DE,CR,DN}DefencePlatform`, `{CR,DN}TorpedoPlatform`, `{CR,DN}DronePlatform`,
|
||
`CRDeepScanPlatform` families and `_VNeumannSatellite` — all standalone mission sections — so no
|
||
shipped design mixes a flagged section with unflagged ones. That the rule is an OR is therefore
|
||
**inferred from the instruction and untestable on the corpus** (rule 6); it is unit-tested in the
|
||
engine on a synthetic design instead.
|
||
|
||
**This is not lane B5's `0x400`.** A *fleet* gets flag `0x400` when the retreat pipeline creates it,
|
||
and that flag reaches the wire as `FtFlg` — `turn1-state.sav`'s first fleet has `FtFlg 1024`. It is
|
||
a different word on a different object. The numeral is a coincidence of two bit layouts, and the
|
||
brief was right to say "check rather than assume".
|
||
|
||
### Hull size = `section_class`
|
||
|
||
`ShipSectionDef+0x260` is the parsed `section_class`, and the parse is a three-name table:
|
||
|
||
```
|
||
ParseShipClassName 0x0056e1c0: _stricmp against "Destroyer", "Cruiser", "Dreadnought"
|
||
stores the index it stopped at -> 0, 1, 2
|
||
```
|
||
|
||
Case-insensitive, which the data needs — the shipped catalogs write both `cruiser` and `Cruiser`.
|
||
The wrapper `0x0056e250` logs `" [%s] unrecognized ship class"` and stores **0** on no match, and
|
||
a section with no `section_class` key at all never reaches the parser and keeps the constructed 0.
|
||
So **absent or unrecognised is a destroyer, not an error**. 16 shipped sections have no
|
||
`section_class` (the `_Biomissile` / `_BoardingPod` families and three `_NPC` stems).
|
||
|
||
The design's hull size is `AggregateSectionStats`'s `out[0x1d] = sectionDef->+0x260`, copied to
|
||
`ShipDesign+0x12c`. That is an **assignment inside the per-section loop, not a max and not an OR**,
|
||
so the *last resolved section in memory slot order* wins — i.e. engine, else command, else mission.
|
||
Rule A6 makes every shipped design class-homogeneous, so first-wins and last-wins agree on **all 503
|
||
design records in the 11 saves** (measured, §4); the assignment order is implemented anyway because
|
||
it is what the original does, and it is unit-tested on a hand-built mixed-class design.
|
||
|
||
### One corroboration the census does not need
|
||
|
||
The same `0x400` bit picks the design's **default hull health** when no section overrides it at
|
||
`+0x2ac` (`0x00826af0`, `out[2]`):
|
||
|
||
| hull size | without `0x400` | with `0x400` |
|
||
|---|---|---|
|
||
| 0 destroyer | 500 | 100 |
|
||
| 1 cruiser | 3000 | 500 |
|
||
| 2 dreadnought | 15000 | 1000 |
|
||
|
||
A defence platform is a fifth to a fifteenth of the ship's hull. That is an independent reading of
|
||
the same two words and it agrees: `0x400` separates *platforms* from *ships*, and `+0x12c` is the
|
||
three-step hull-size ladder.
|
||
|
||
---
|
||
|
||
## 3. The census — what the two words feed
|
||
|
||
`ShipCensusByHullClass` `0x00818a50` takes a player and an `int[8]`:
|
||
|
||
```
|
||
for each fleet in server.fleets where fleet->owner(+0x58) == player:
|
||
for each ship in fleet->ships(+0xa4..+0xa8):
|
||
design = ship->+0x14
|
||
if (design->+0xb8 & 0x400) == 0: ++out[0]; ++out[2 + design->+0x12c]
|
||
else: ++out[1]; ++out[5 + design->+0x12c]
|
||
```
|
||
|
||
`out[0]`/`out[1]` are the two grand totals, which the caller computes and **discards**. The six
|
||
breakdowns land at `turnRecord+0x2a..+0x34` and reach the wire as the three `cls` groups' **`shpt`**
|
||
(ships) and **`satt`** (satellites — the game's own word for a defence platform).
|
||
|
||
Note what the loop does *not* do: it never looks at the ship's own `PlrID`, only at the fleet's
|
||
owner, and it never looks at the ship's health or state. Every ship in an owned fleet counts.
|
||
|
||
---
|
||
|
||
## 4. Measurement: 480 census leaves, 0 mismatched
|
||
|
||
Rebuilt from the save's own state plus the section catalog, compared against the record the game
|
||
archived for that save's own frame (the record is rebuilt at end of turn *and* on load, so it never
|
||
has to survive a round trip).
|
||
|
||
| saves | designs classified | leaves compared | **mismatched** |
|
||
|---|---|---|---|
|
||
| 11 | 503 | **480** | **0** |
|
||
|
||
Computed twice, independently: once in Python off `save_reader.py` + `design_rules.py`, once in C++
|
||
off the engine's own `shapes` reader and `game::data::Catalog`. The two disagree about how many
|
||
sections a design has (the Python reader's five vs the engine's three, §1a) and still produce the
|
||
same 480 numbers — so they are not the same check wearing two hats.
|
||
|
||
**Coverage, reported as loudly as the result (rule 15).** Only **32 of the 480 leaves are nonzero in
|
||
the archive**. A zero leaf agrees for free.
|
||
|
||
| leaf | nonzero of 80 |
|
||
|---|---|
|
||
| cls 0 `shpt` — destroyer ships | **18** |
|
||
| cls 0 `satt` — destroyer platforms | **3** |
|
||
| cls 1 `shpt` — cruiser ships | 0 |
|
||
| cls 1 `satt` — cruiser platforms | 0 |
|
||
| cls 2 `shpt` — dreadnought ships | **11** |
|
||
| cls 2 `satt` — dreadnought platforms | 0 |
|
||
|
||
**Three of the six census leaves are unexercised, not verified.** No save in the corpus contains a
|
||
cruiser of any kind or a dreadnought-class platform, so the cruiser column and the dreadnought
|
||
platform cell are hypotheses (rule 6). They are cheap to exercise — a save taken after
|
||
`IND_CruisCon` is researched and one cruiser is built would close the whole cruiser column, and
|
||
`zuul-turn23-fleet23.sav`'s 27-destroyer fleet shows the destroyer path is genuinely loaded.
|
||
|
||
Other honest residuals from the same run:
|
||
|
||
* 0 ships whose design id is missing from the save's design tables;
|
||
* 0 designs none of whose sections resolve against the catalog;
|
||
* 0 designs where first-resolved and last-resolved section disagree on hull size (rule A6 predicts
|
||
0, and this is the measurement that says so rather than the assumption).
|
||
|
||
---
|
||
|
||
## 5. What this closes, and what it does not
|
||
|
||
Against lane Y's baseline — turn1→turn2 **209→204**, turn2→turn3 **108→103**, 5 closed / 0
|
||
regressed, with the 204 broken down `/Sim/players` 82, `/Sim/systems` 80, `/Sim/turnstats` 24,
|
||
`/Sim/SvSctOb` 8, plus 10 singletons:
|
||
|
||
**closed: 0. regressed: 0.** Reported separately, and both are zero *in the standalone's divergence
|
||
list*, because this lane wrote no phase.
|
||
|
||
That is the honest number and it needs the reason spelled out. The census leaves live under
|
||
`/Sim/turnstats`, and the turn record is built by `src/app/turn_record.cpp`, which **lane A2 holds**
|
||
this cycle (the alliance mask `almem` is a field of the same record). Committing a census field into
|
||
that file now is exactly the cross-lane collision the fragment discipline exists to prevent. So this
|
||
lane did what lane S2's convention prescribes for a phase whose formula is held but whose write-out
|
||
is not this lane's: **evaluate and report, do not write**.
|
||
|
||
What is now unblocked, quantified:
|
||
|
||
* **6 of the 24 `/Sim/turnstats` leaves per pair** — `shpt` ×3 and `satt` ×3 — have a formula that
|
||
reproduces the archived value on 480 of 480 observations. Wiring them is one call to
|
||
`game::design::ShipCensus` inside `BuildTurnRecord`, and the earlier `--commit-blocked` run's
|
||
"three leaves of the ship census" among its 17 regressions were exactly these, wrong because the
|
||
record committed zeros. They should now commit clean.
|
||
* **The ship records** (`Player.ShipRecs`) and **tail phase T24** both need the same two words per
|
||
design; they are now available as `DesignStats::hull_size` / `DesignStats::defence_platform` and
|
||
as the standalone `classify_design()`.
|
||
* The turn record's other four unmodelled fields are **not** touched: `almem` is lane A2's,
|
||
`tdinc` waits on per-system money output, `bat` is a tail stub, and `sacq`/`slost` have count 0 in
|
||
every save (rule 6).
|
||
|
||
Nothing here was compared against the live game. `verified` in the phase catalog means "compared
|
||
against the running game", and this is not that — it is a comparison against bytes the original
|
||
wrote, which is the strongest instrument a lane holding no VM has.
|
||
|
||
---
|
||
|
||
## 6. Fixes to the oracles, made openly (rule 12)
|
||
|
||
Two readers carried the same wrong record shape; both are fixed, with a test per defect and
|
||
byte-neutrality shown.
|
||
|
||
**`verify/save-reader/save_reader.py`** — the `Des` shape ended in `Rest("sections")`. Now: three
|
||
`DSec` frames as a `Repeat`, then named `Dtc` (int) and `Dwgv` (bool), then the conditional `Dwg`;
|
||
`DWep` retyped int → bool. All 11 saves still read strict-clean; the reader's own suite is **49/49**
|
||
(was 48; three existing tests asserted the old shape and were corrected, one test added that pins
|
||
"exactly three `DSec`" against the real saves).
|
||
|
||
**`verify/design-rules/stock_designs.json`** regenerated from the same three saves. The diff is
|
||
*only* `raw_slots` 5 → 3 and `dWep` `0/1` → `false/true`; every other field of all 127 designs is
|
||
byte-for-byte the same, and `test_design_rules.py` is **32/32** with the same ground truth
|
||
(127/127 structural, 197/197 `DOpts`, 121/127 gating with the six Tarkas hidden-rider warns).
|
||
|
||
**`sots-engine` `src/mars/stream/shapes.h`** — `Design::dWep` and `Design::dwgv` retyped to bool,
|
||
the conditional `Dwg` node added, and the comment that said the derived writer made no stream call
|
||
replaced with the recovered layout. The typed round trip is still **byte-identical on all 11 saves**
|
||
at 100% named coverage.
|
||
|
||
The engine's `stock_designs.cpp` fixture loader now accepts 3–5 `raw_slots` so an old fixture still
|
||
loads. `kSlotCount` is deliberately **left at 5**: the extra two array entries are inert and
|
||
touching the enum reaches `rules.cpp` and another lane's tests for no behavioural gain. The comment
|
||
in `design.h` is corrected instead — the record is three sections, and the fifth-slot slack is this
|
||
engine's own, not the game's.
|
||
|
||
---
|
||
|
||
## 7. Open, and what would close it
|
||
|
||
1. **The cruiser column and the dreadnought-platform cell** are unexercised (§4). One save with a
|
||
cruiser in it and one with a DN defence platform close three leaves. Cheapest of everything on
|
||
this page.
|
||
2. **`Dtc`** (`+0x134`) is on the wire and is stamped by at least one caller immediately before the
|
||
recompute; its meaning is not read here. It is 1 on the stock designs.
|
||
3. **`Dwgv`/`Dwg`** — weapon groups — are false in every save. The conditional branch is implemented
|
||
from the instruction and is unexercised.
|
||
4. **The other 24 role-flag bits** are decoded (§2) but only `0x400` is consumed by anything this
|
||
lane read. `0x80000` (`freighter`) and `0x8000000` (`freighterQ`) drive a *second* classifier
|
||
`0x0081a430`/`0x0082c7c0` over the same two words — worth reading if a freighter count turns up
|
||
missing elsewhere.
|
||
5. **The remaining ~24 fields of the aggregator's output struct** (`+0x18`..`+0x64` →
|
||
`ShipDesign+0xc0`..`+0x118`) are the rest of the derived-stat block — mass, cost, capacities,
|
||
speeds. `SHIP_DESIGN_RULES.md` §6 derives most of them from the data with the formulas marked as
|
||
assumptions; `0x00826af0` is where every one of those assumptions can be settled, field by field,
|
||
and it is now a named function.
|
||
|
||
---
|
||
|
||
## Addresses
|
||
|
||
`ghidra/addresses.d/lane-d2.json` (19 entries, no collision with any existing fragment).
|
||
Key ones: `ShipDesignDef::Write` `0x00827390`, `ShipDesign::Write` `0x008325e0`,
|
||
`ShipDesign::UpdateDerivedStats` `0x0087e7c0`, `AggregateSectionStats` `0x00826af0`,
|
||
`ShipSectionDef::GetRoleFlags` `0x0056ee80`, `ParseShipClassName` `0x0056e1c0`,
|
||
`SetRoleFlagBit` `0x0056e780`, `ShipCensusByHullClass` `0x00818a50`;
|
||
offsets `ShipDesign+0xb8` (role flags), `+0x12c` (hull size), `ShipSectionDef+0x260`
|
||
(`section_class`), `+0x298` (role flags).
|