805409ec11
Game::StrategyAIAgent::Streamable typed end to end. The writer is entirely unconditional; the branch the decompiler shows is an inlined vector destructor whose operator delete is marked noreturn. dsh is a std::map's _Mysize, so it is the (pid, trns) count and not a scalar; lnat and lat are scalars, not elements. AISystem::Write never reads its object, so every AISys body on disk is empty regardless of game state. StreamableEnum<T> is a frame holding one int, which makes SysMem/mts/nalat arrays of frames rather than of ints. Named coverage 98.0% -> 99.9% on all four saves; state_checksum.py unchanged and still PROVED. Section 7 lists the ten element layouts no save exercises and says what workload would settle each. 29 addresses in the per-lane fragment; merges to 669 with no duplicate name.
260 lines
16 KiB
Markdown
260 lines
16 KiB
Markdown
# Typing the `AIAgent` custom-data blocks
|
|
|
|
Lane A, 2026-09-08. Host/static only — VM140 was held by lane O and the game was never run.
|
|
Picks up `findings/objects/wire-schema-closeout.md` §4b, the job lane W left deliberately.
|
|
|
|
## Headline: named coverage 98.0 % → 99.9 %
|
|
|
|
The honest metric is `CoverageArchive`'s split of items a field **names** from items a generic
|
|
`Node` merely **carries**. Byte-identical round trip held throughout, on all four saves.
|
|
|
|
| save | before | after | opaque before → after | still opaque after |
|
|
|---|---|---|---|---|
|
|
| turn1-state | 98.0 % | **99.9 %** | 748 → 37 | `<rest:CD>` 35, `RNG` 2 |
|
|
| turn2-state | 98.0 % | **99.9 %** | 748 → 37 | `<rest:CD>` 35, `RNG` 2 |
|
|
| turn3-state | 98.0 % | **99.9 %** | 748 → 37 | `<rest:CD>` 35, `RNG` 2 |
|
|
| zuul-turn5-species5 | 98.4 % | **99.9 %** | 521 → 37 | `<rest:CD>` 35, `RNG` 2 |
|
|
|
|
Conformance: **86 shapes bound, 838 items matched, 0 MISMATCH** (was 74 / 769 / 0). Twelve new
|
|
bindings, every one of them a full match with **zero wire-only and zero shape-only items** — the
|
|
recovered sequence and the shape agree item for item, which is unusual and is the point of §2.
|
|
|
|
`ctest` **35/35**. `tools/clean_room_check.sh` **OK** (run as a separate command). `test_save`
|
|
still skips cleanly with `SOTS_SAVES_DIR` unset. `verify/state-checksum/state_checksum.py --strict`
|
|
still reports **`coverage: PROVED`** on all four saves with `0 error, 0 warn` — unchanged, because
|
|
nothing on the Python side was touched (§6).
|
|
|
|
The remaining `CD` items are the one `TurnCommands_v5` body per save, which is lane W's proven
|
|
blocker and still needs a save with issued orders. `RNG` is correctly opaque. `Attrib` closed as a
|
|
side effect (§3).
|
|
|
|
## 1. The block is one unconditional sequence — there is no branch to read
|
|
|
|
`Game::StrategyAIAgent::Streamable::Write` is at **`0x006c6f00`**. The decompiler presents it with a
|
|
large `if` wrapped around everything from `lnat` onwards, which looks exactly like the conditional
|
|
the campaign method warns about. It is not one. The `if` is an **inlined `std::vector` destructor**
|
|
for the temporary vector `nalat` is written from, and this database has `operator delete` marked
|
|
noreturn, so Ghidra could not join the two paths and hoisted the continuation into the null branch.
|
|
The disassembly settles it:
|
|
|
|
```
|
|
0x006c72ce cmp [ebp-0x50],ebx ; _Myfirst == 0 ?
|
|
0x006c72d1 je 0x006c72e8 ; skip the delete
|
|
0x006c72d3 ... _Orphan_all / operator delete
|
|
0x006c72e8 ... push "lnat"; call WriteInt
|
|
```
|
|
|
|
Both paths converge at `0x006c72e8`. **Every one of the 36 items is written unconditionally.** That
|
|
is why this block, unlike `TurnCommands_v5`, aligns perfectly: the recovery's "all branches" view
|
|
and the actual single-record view are the same view. All eleven `AIAgent` blocks across the four
|
|
saves have byte-for-byte the same top-level item order.
|
|
|
|
This is the opposite outcome to lane W's §4a and for a concrete reason, not luck.
|
|
|
|
## 2. What each item is, and how the count/element structure was settled
|
|
|
|
The recovery gives tags and disk primitives; it does not give the **count/element** structure,
|
|
because a linear pass flattens loops. Each of these came from reading the loop in `Write`.
|
|
|
|
| item | what it is | evidence |
|
|
|---|---|---|
|
|
| `AIAttr` | `StreamableHelper<Game::AttribMap>` | §3 |
|
|
| `AITurnPris` | `AIWeightMap<StreamableEnum<AIPurposeID>>` | §4 |
|
|
| `AISit` | `Game::AISituation` | two weight maps, `0x0069a220` |
|
|
| `AIPlyHat` | `AIWeightMap<StreamableEnum<uint>>` | §4 |
|
|
| `prs2` | `VectorHelper<Game::AIPlayerRequestStamp>` | §5 |
|
|
| **`dsh`** | **a count, not a scalar** | §2a |
|
|
| `pid`, `trns` | the `dsh` element: an int and a `VectorHelper<int>` | §2a |
|
|
| `NBStab` | count = span **>> 3** over the vector at `+0x38..+0x3c` | loop writes `BStabPl` at elem+0, `BStabTn` at elem+4 |
|
|
| `NMBlst` | count = span **>> 3** over `+0x48..+0x4c` | loop writes `MBlstSy` at elem+0, `MBlstTn` at elem+4 |
|
|
| `AIDNG` | one nested `DesignNameGen`, **not** a loop body | single helper call at `0x006c710f` |
|
|
| `AIHivJ`, `SDFlT` | plain ints at `+0x104`, `+0x368` | — |
|
|
| `nalat` | `VectorHelper<StreamableEnum<uint>>` over a **temporary** vector built at the call site | §4a |
|
|
| **`lnat`, `lat`** | **plain scalar members** at `+0x58`, `+0x5c` | §2b |
|
|
| `AINumSys` | a **computed** count of the systems the following loop actually emits | both loops walk the same list under the same `*(x+8) == 3` test |
|
|
| `AISysID`, `AISys` | the `AINumSys` element pair | — |
|
|
| `NCmbR` | count = span >> 2 over the pointer vector at `+0x94`'s `+0x28..+0x2c` | each element writes one `CmbR` frame |
|
|
| `NumCL` | count = span **/ 0xc** over `+0x1ac..+0x1b0` | loop writes `CLTn`(+0), `CLSyID`(+4), `CLPlID`(+8) |
|
|
| `NPrv` | count = span >> 3 over `+0x32c..+0x330` | loop writes `NPrvId`(int, +0), `NPrvVa`(float, +4) |
|
|
| `NTecS` | count = span >> 2 over `+0x13c..+0x140` | loop writes `TecS` |
|
|
| `fct` | plain int at `+0x1f0` | — |
|
|
| `apr` | `VectorHelper<Game::AIAutoPeaceRun>` at `+0x1f8` | — |
|
|
|
|
Two of these are also **confirmed by the data**, not only by the disassembly: `NPrv` is 1 or 2 in
|
|
our saves and is followed by exactly that many `(NPrvId, NPrvVa)` pairs; `NTecS` is 2 or 3 and is
|
|
followed by exactly that many `TecS`. `dnnc` inside `AIDNG` is 0, 1 or 2 and is followed by exactly
|
|
that many `(dnid, dnnm)` pairs — the zuul save's two-entry case is what makes it a count rather
|
|
than a coincidence.
|
|
|
|
### 2a. `dsh` is a `std::map`'s `_Mysize`, so it is the element count
|
|
|
|
The recovery lists `dsh` as a scalar member and `pid`/`trns` as elements of nothing. Taking that at
|
|
face value would desynchronise the reader, because there is no other count before the pairs. What
|
|
`Write` actually does is:
|
|
|
|
```
|
|
local = *(agent + 0x240) ; WriteInt("dsh", &local)
|
|
node = *(agent + 0x23c) ; red-black walk until back at the head
|
|
WriteInt("pid", node + 0x0c)
|
|
VectorHelper<int>("trns", node + 0x10)
|
|
```
|
|
|
|
`+0x23c` is `_Myhead` and `+0x240` is `_Mysize` — the same head@0 / size@4 layout the campaign
|
|
already pinned on `ServerSystem::NVO` (`0x274` / `0x278`). The node's `_Isnil` is at `+0x21`, which
|
|
puts `_Myval` at `+0x0c` and makes it `pair<const int, std::vector<int>>` — 4 + 16 = 0x14 bytes,
|
|
landing `_Color` exactly on `+0x20`. So the container is `std::map<int, std::vector<int>>` and
|
|
**`dsh` is its size**. Structurally it has to be: a reader has no terminator and no other number.
|
|
|
|
### 2b. `lnat` and `lat` are scalars, and the data agrees
|
|
|
|
The recovery marks both `[element]`. They are two ordinary `WriteInt`s of `agent+0x58` and
|
|
`agent+0x5c`, immediately after the `nalat` frame. The saves confirm it independently: `nalat`'s
|
|
count is 0 in all eleven blocks and `lnat`/`lat` are present anyway, which no element reading
|
|
survives.
|
|
|
|
More generally the `[element]` flag is unreliable for this class — it is also set on `AIDNG`,
|
|
`nalat`, `AINumSys` and `apr`, which are not loop bodies. It does not affect the conformance check,
|
|
which aligns on tag and disk primitive only, so `objects/streams.json` was left alone; this is the
|
|
same class of recovery artefact as lane W's open item 4 (`SVSOCrowDefenders::dsys`).
|
|
|
|
## 3. `Game::AttribMap` — and `Attrib` fell out with it
|
|
|
|
`AttribMap::Write` (`0x0059ee60`) writes `*(this+8)` as an anonymous count, then walks the tree at
|
|
`*(this+4)` writing **two anonymous strings** per node — key at `node+0x0c`, value at `node+0x28`,
|
|
`_Isnil` at `node+0x45`. Key and value are both `Mars::String` (0x1c), so `_Myval` is 0x38 bytes and
|
|
`_Color` lands on `+0x44`: `std::map<Mars::String, Mars::String>`, exactly.
|
|
|
|
Lane W recorded `Attrib` in `Game::StrategyServer` as "an empty frame in every save; there is
|
|
nothing to type". It is not empty — it is a frame holding a count of 0, which is why it showed as
|
|
**2** opaque items. Typing it as `AttribMap` closes both, on all four saves.
|
|
|
|
The entry pair is a hypothesis in the behavioural sense: the map is empty everywhere we can look.
|
|
The *type* is certain, from the writer.
|
|
|
|
## 4. `Mars::StreamableEnum<T>` is a frame, and that corrects two of lane W's shapes
|
|
|
|
`AIWeightMap<K>` (four instantiations, one wire shape) writes an anonymous count then, per entry,
|
|
an anonymous **key frame** and an anonymous float. The key frame's body is what matters:
|
|
|
|
* `StreamableHelper<StreamableEnum<T>>::Write` is `0x0054b6e0` and writes **exactly one anonymous
|
|
int**. So a `StreamableEnum` member is a *frame containing an int*, never a bare int.
|
|
* `StreamableHelper<Game::ShipSectionID>` writes two anonymous ints (`0x0056ed50`), which is the
|
|
key for the `AIWeightMap<ShipSectionID>` instantiation and matches the `dnid` frames on disk
|
|
(`{2, 44}`, `{5, 37}`, …).
|
|
|
|
The consequence reaches outside this block. `VectorHelper<StreamableEnum<uint>>::Write`
|
|
(`0x0042f7f0`) writes a count and then **n anonymous frames**, one int each. Lane W typed
|
|
`Game::SVSOSwarmQueen::SysMem` and `Game::SVSOVonNeumann::mts` as `std::vector<int32_t>` — a framed
|
|
array of plain ints — reasoning from the decorated helper name. The name is right; the element
|
|
framing is not. Both are now `std::vector<StreamEnum>`, along with `nalat` here.
|
|
|
|
**This changes no bytes and closes no items**: all three have count 0 in every save, which is
|
|
exactly why the error was invisible. It is a correction to an unexercised layout, and it stays an
|
|
unexercised layout — see §7. Lane W's own note flagged these three as hypotheses; this is what
|
|
checking one of them looked like.
|
|
|
|
### 4a. `nalat` is computed, not stored
|
|
|
|
`nalat`'s vector is built on the stack at the call site (`FUN_006c0360` over the members at `+0x18`
|
|
and `+0x1c`) and destroyed immediately after. It is a derived list, not a member, so its content is
|
|
not recoverable from a static read of the agent's fields.
|
|
|
|
## 5. `Game::AIPlayerRequestStamp` has no wire-table entry, by construction
|
|
|
|
`prs2` is `VectorHelper<Game::AIPlayerRequestStamp>` (`0x0069b310`): a count of 8-byte elements,
|
|
each an anonymous frame. The element writer is `0x00695400` and emits two **named** ints, `pid` and
|
|
`trn`. `tools/serializers.py` reports "no serializer" for the class because it is a POD with no RTTI
|
|
class of its own — it is reached only through a specialised `StreamableHelper` thunk, one of the 38
|
|
such types §3 of `serializer-struct-recovery.md` describes. So the shape is typed from the writer
|
|
directly and is **not bound** in `test_wire_schema.cpp`: there is nothing in the table to bind it to.
|
|
|
|
## 6. `Game::AISystem::Write` never reads its object
|
|
|
|
`0x006bda30` takes only the stream. It constructs an empty `AISituation` on the stack — two
|
|
16-byte vectors, both zeroed — writes it as `AISysSit`, and destroys it. `this` is never touched.
|
|
|
|
So **every `AISys` body on disk is two empty weight maps regardless of game state**, which is what
|
|
the saves show: 28 identical `AISys` frames per block, in every block, in every save. The
|
|
`AINumSys`/`AISysID` pairing still carries real information (which systems the AI tracks); the
|
|
situation payload does not. Worth knowing before anyone tries to reconstruct AI behaviour from it.
|
|
|
|
The recovery graded `AISystem` "partial — only container elements / computed values recovered". That
|
|
grade is right, and this is why.
|
|
|
|
## 7. What is still a hypothesis, and what would settle it
|
|
|
|
Every container in the block has **count 0 in all eleven `AIAgent` blocks of all four saves**, apart
|
|
from `AIDNG`'s `dnnc`, `NPrv` and `NTecS`. So the *frames* are exercised and most *element layouts*
|
|
are not. Labelling it honestly:
|
|
|
|
| item | element layout | exercised? |
|
|
|---|---|---|
|
|
| `AIDNG` / `dnnc` | `(dnid: ShipSectionID, dnnm: string)` | **yes** — 0, 1 and 2 entries observed |
|
|
| `NPrv` | `(NPrvId: int, NPrvVa: float)` | **yes** — 1 and 2 entries observed |
|
|
| `NTecS` | `TecS: int` | **yes** — 2 and 3 entries observed |
|
|
| `AINumSys` | `(AISysID: int, AISys: frame)` | **yes** — 28 entries in every block |
|
|
| `AIAttr` entries | `(string, string)` | no — map empty everywhere |
|
|
| `AITurnPris`, `AIPlyHat`, `AISitSecs`, `AISitWeapFams` entries | `(key frame, float)` | no — count 0 everywhere |
|
|
| `prs2` | `(pid: int, trn: int)` | no |
|
|
| `dsh` | `(pid: int, trns: int[])` | no |
|
|
| `NBStab`, `NMBlst` | two ints | no |
|
|
| `nalat` | one-int frame | no |
|
|
| `NCmbR` / `CmbR` | `Game::AICombatReport` | no |
|
|
| `NumCL` | three ints | no |
|
|
| `apr` | `Game::AIAutoPeaceRun` | no |
|
|
|
|
The unexercised rows are typed from the writers, not guessed from bytes, and the conformance check
|
|
agrees with the recovery on every one. They are still hypotheses in the behavioural sense and
|
|
`tests/mars_stream/test_stream.cpp::test_aiagent_block` is the only thing that exercises them: it
|
|
populates every container, round-trips, and asserts the bytes are stable. It also asserts the trap
|
|
in §4 directly — that a two-element `nalat` produces two **complex** nodes, which a bare-int typing
|
|
would get wrong.
|
|
|
|
**What would settle them**: a save from a longer game with several AI players in contact. `NBStab`
|
|
and `NMBlst` need the AI to have border-stability and menace-blacklist entries, `NCmbR` needs the AI
|
|
to have fought a battle it kept a report of, `prs2` and `dsh` need multi-player AI diplomacy and
|
|
design sharing, and `AIAttr` needs whatever sets AI attributes (possibly a scenario). None of that
|
|
needs anything the shim cannot produce; it needs turns played, which is lane O's machine.
|
|
|
|
Two things stay carried on purpose:
|
|
|
|
* **`Game::TacReport`** (`CombatPlayerStats::TacReports`) has a computed count `TRnc` and three
|
|
trailing scalar runs (`TRships`, `TRsats`, `TRshipsL`) whose relationship the linear recovery
|
|
cannot resolve, and no save we hold contains one. Its body is carried as `Node`s rather than typed
|
|
on a guess. It sits two levels under `NCmbR`, which is 0 everywhere.
|
|
* **`TurnCommands_v5`**, unchanged from lane W §4a — a proven blocker, not a suspected one.
|
|
|
|
## 8. Artifacts
|
|
|
|
* `sots-engine` branch `wip/aiagent` (off `f4f3a7a`, never committed to main):
|
|
`src/mars/stream/shapes.h` (`StreamEnum`, `AttribMap`, `AIWeightMap<K>` + two aliases,
|
|
`AISituation`, `AISystem`, `DesignNameGen`, `AIAutoPeaceRun`, `CombatPlayerStats`,
|
|
`AICombatReport`, `AIRequestStamp`, `StrategyAIAgent`, `CustomDataBlock`; `Sim::Attrib` retyped;
|
|
`SysMem`/`mts` element type corrected; `SaveGame` now selects each `CD` body by the id at the same
|
|
ordinal in `CDT`), `tests/mars_stream/test_wire_schema.cpp` (+12 bindings),
|
|
`tests/mars_stream/test_stream.cpp` (+2 unit tests, no saves needed),
|
|
`tests/mars_stream/test_save.cpp` (ratchet 97.5 → 99.8).
|
|
* `sots-re`: `ghidra/addresses.d/lane-a.json` (29 entries, merges to 669 with no duplicate name),
|
|
this note.
|
|
* `include/generated/sots_stream_schema.h` was **not** regenerated: `tools/streams.py` and
|
|
`tools/gen_stream_schema.py` were re-run and the output is identical to the committed header apart
|
|
from the provenance line, which carries the notes repo's HEAD and so changes on every commit.
|
|
`objects/streams.json` is unchanged.
|
|
* `ghidra/generated/sots_addresses.h` was **not** regenerated either — same reasoning as lane W: the
|
|
fragment carries this lane's addresses, and regenerating in place would sweep other lanes'
|
|
in-flight fragments into this branch. The engine consumes none of these addresses.
|
|
* `verify/save-reader/save_reader.py` was **not** changed. Its `CD` handling is an *absence* (the
|
|
blocks are `any`), not a defect, so there was nothing to correct mid-campaign; `state_checksum.py`
|
|
is byte-identical and still `PROVED`. Mirroring the `AIAgent` shape into the Python oracle is
|
|
available as follow-up and would change the `/CD` subsystem digests (and only those).
|
|
|
|
## 9. Open items for the next lane
|
|
|
|
1. A save with issued turn commands, for `TurnCommands_v5` — still the one blocker of size.
|
|
2. A longer multi-AI save would exercise ten of the thirteen element layouts in §7 at once.
|
|
3. `Game::TacReport`'s `TRnc` / `TRships` / `TRsats` / `TRshipsL` structure needs a combat report on
|
|
disk before it is worth resolving.
|
|
4. `objects/streams.json`'s `[element]` flags are unreliable inside
|
|
`Game::StrategyAIAgent::Streamable` (§2b) and in `Game::SVSOCrowDefenders` (lane W's item 4). The
|
|
flag is not consumed by the conformance check, so this is a cleanliness item, not a correctness
|
|
one — but anyone reading the table as documentation should know.
|