sots-re/findings/objects/turncommands-block.md

TurnCommands_v5 — the pending order queue, and why the recovery could not be aligned to it

Lane Q, 2026-09-08. Static + host only; VM140 was not needed.

Closes the last opaque block of the save format. Named coverage on all eleven saves goes to 100.0%, with the two items of the MT19937 blob the only things left carried as Nodes.

---

1. The reconciliation

Lane W's negative result was real work and its arithmetic was right. Its conclusion — "a save with issued orders is required" — was also right. But the reason the two sequences would not align is not that the writer takes branches a no-orders turn does not (it does, but that is the smaller half). It is that three different things were being compared as if they were one.

Game::TurnCommands::Write is at 0x00842540 and is 764 bytes. Read as instructions, it has two halves and no loops of its own:

(1) A prologue of six flag-gated groups. Each group is a WriteBool on a member, followed — only when that bool is set — by that command's payload:

gate	payload	meaning
—	i32 @0x04	player id, always written
bool @0x0c	f32 @0x08	research rate (the empire savings/research slider)
bool @0x14	i32 @0x10	research target tech id
bool @0x20	i32 @0x18, f32 @0x1c	research boost: savings spent, and a fraction
bool @0x2c	bool @0x24, i32 @0x28	never observed set
bool @0x3c	f32 @0x30, f32 @0x34, f32 @0x38	never observed set
bool @0x6c	StreamableHelper<Game::CivilianRatios> frame on @0x40	never observed set

(2) Twenty-seven std::list<T> members at 0x70 … 0x1a8, stride 0x0c ({_Myhead, _Mysize, _Alval} — allocator-last, as rule 5 warns). Each is handed to its own free-function writer, helper(stream, &list), and every helper has the same shape:

WriteInt(list._Mysize)                 ; the count -- ALWAYS written, even for an empty list
for each node:  <that list's element record>

That is the whole class. Now the three ways the comparison went wrong:

1a. The sequence lane W aligned against was offset-sorted, not write-ordered

objects/layouts.md sorts fields by off_abs. Game::TurnCommands is one of the 89 of 386 classes where offset order is not write order — the writer emits 0x04, 0x0c, 0x08, 0x14, 0x10, 0x20, 0x18, 0x1c, … because each gate bool precedes its payload in the stream while sitting after it in the struct. In the sorted view, item 4 is the i32 @0x10; in the writer, item 4 is the bool @0x14. Lane W's "the save's item 4 is 8 bytes, which can only be a bool where the recovery says i32" is therefore an artifact of the sorted view, not a contradiction. The write-ordered table (objects/streams.json, which is what the engine's wire-schema channel consumes) already has a bool in that slot, and matches the save.

This is the same trap the wire-schema channel was built to avoid, hit from the other side. It is worth restating as an operational rule: never align a save against layouts.md; align against streams.json.

1b. The tail is 27 container call sites, not 27 scalars

The recovery is a linear pass. It cannot express a loop, so for each of the 27 list writers it emits one item, guessing a kind from an element field it could resolve, and drops the count word entirely. That is where the recovered 44 comes from:

44 = 17 member writes (all branches, as the recovery always presents them)
   + 27 container call sites (one item each, element kind guessed, count dropped)

44 − 17 = 27, and there are exactly 27 list members. Lane W's second objection — "27 consecutive ints on disk but only 22 i32 slots after index 15" — is that same 27 seen from both sides. The five slots that are not i32 in the table (a Game::StarSystem::OutputRates frame, a narr, two bools, a Game::Population frame, a Game::RaidTargets frame) are the recovery describing an element, while the wire at that position always shows a count. It was a kind mismatch in the table, not a structural impossibility in the save.

Two of those five are exactly right about which list they name, which is the tell: the table's 5th tail item is OutputRates and list 5 is the planetary-budget list; its 8th tail item is narr and list 8 is the only one whose element ends in a nested counted vector.

1c. The empty block is 8 + 27, and that is why every old save was bit-identical

A no-orders turn sets only the research-rate gate. So the block is

i32 playerId, bool(true), f32 rate, bool×5 (all false)   = 8 items
27 × WriteInt(0)                                         = 27 items
                                                          -- 35 items, 122 B

which is CD[0] digest 3df7d93164fb1d7d in turn1/2/3-state.sav, zuul-turn5-species5.sav, human-turn3-noderoute.sav, zuul-turn17-rollpending.sav and zuul-turn23-fleet23.sav. Nothing could be learned from those saves because 27 of their 35 items are the absence of every command list.

1d. The arithmetic checks on every save

save	prologue	list contents	total	observed
any no-orders save	8	27 counts	35	35
zuul-turn16-noderoute	11	27 counts	38	38
human-turn2-orders	8	27 + 3 (move) + 3 (list 14)	41	41
zuul-turn15-orders	11	27 + 5×4 (build) + 3 (move)	61	61
zuul-turn17-orders2	8	27 + 20×4 (build) + 2 (rates) + 3×2 (colonize)	123	123

(state_checksum --tree counts 129 leaves for the last one because the OutputRates sub-frame contributes 7 leaves in place of 1 item.)

---

2. Corrections to lane O's provisional layout

Lane O's §11 was derived from bytes and UI correlation and is right about every command it observed. Two things are corrected from the writer:

The fleet-move element is not {fleetId, 1, destSystemId, 0}. The helper at 0x0083e550 writes WriteInt(fleetId) and then a nested counted int vector: n = (v._Mylast − v._Myfirst) >> 2, then n ints. So the wire is {fleetId, hopCount, hopCount × systemId} — three items for a one-hop order, and longer for a multi-hop route. Lane O read the 1 as a constant and the next list's zero count as a trailing 0. Only the three-item reading makes the item totals close: with a four-item element human-turn2-orders needs 26 lists and zuul-turn15-orders needs 28, and the writer has 27.

This also means a route order is directly visible in the block, which is what the noderoute saves were built to catch — and worth recording, neither of them carries one (both decode to zero fleet moves), so the node-route UI does not queue through this list.

There are 27 lists, not "five plus trailing zeros". Lane O's "int x2 = 0" before the build count and its "int 0" separators are lists 1, 2, 4, 6 and 9…27 with count 0. The five observed are list 3 (build), 5 (system rates), 7 (colonize), 8 (fleet move) and 14.

verify/save-reader/SAVE_FORMAT.md §11 has been corrected in place.

---

3. The 27 lists

Element records read off each helper. RTTI resolves every nested frame's helper vftable to a class, which is where the names come from.

#	member	helper	element record	status
1	+0x70	0x0082e310	StreamableHelper<ShipDesignDef> frame, i32	hypothesis
2	+0x7c	0x00822ca0	i32	hypothesis
3	+0x88	0x00822870	i32 ordinal, i32 designId, i32 systemId, i32 w	observed
4	+0x94	0x008228f0	i32 ×3	hypothesis
5	+0xa0	0x0082e3d0	i32 systemId, StarSystem::OutputRates frame	observed
6	+0xac	0x0082e490	PlayerNotes frame	hypothesis
7	+0xb8	0x00822960	i32 shipId, i32 w	observed
8	+0xc4	0x0083e550	i32 fleetId, counted i32 route	observed
9	+0xd0	0x008229c0	i32, bool, i32, i32, f32	hypothesis
10	+0xdc	0x0083e5f0	i32, i32, counted i32	hypothesis
11	+0xe8	0x00822a50	i32, bool	hypothesis
12	+0xf4	0x0082e530	i32, FleetLayout frame	hypothesis
13	+0x100	0x00822ab0	i32, string	hypothesis
14	+0x10c	0x00822b10	i32, i32, bool	observed (see below)
15	+0x118	0x00822a50	i32, bool	hypothesis
16	+0x124	0x00822a50	i32, bool	hypothesis
17	+0x130	0x00822b80	i32 ×3	hypothesis
18	+0x13c	0x00822b80	i32 ×3	hypothesis
19	+0x148	0x00822bf0	i32	hypothesis
20	+0x154	0x00822bf0	i32	hypothesis
21	+0x160	0x0082e5f0	i32, i32, VectorHelper<StreamableEnum<uint>>	hypothesis
22	+0x16c	0x0082e6c0	i32, WeaponGroups frame	hypothesis
23	+0x178	0x0082e780	i32, Population frame	hypothesis
24	+0x184	0x00822c40	i32, f32	hypothesis
25	+0x190	0x0082e840	i32, DefenceLayout frame	hypothesis
26	+0x19c	0x0082e900	RaidTargets frame	hypothesis
27	+0x1a8	0x00822ca0	i32	hypothesis

Six member slots share a helper with an earlier slot (15/16 with 11, 18 with 17, 20 with 19, 27 with 2), so there are 22 distinct helper functions for 27 call sites.

Rule 6 applies to twenty-two of these rows. The scalar sequences are read directly out of the instruction stream, which is stronger than a guess, but no save exercises them and no element value has ever been observed. Where an element holds a nested object whose body is not otherwise modelled by the engine's shapes (lists 1, 12, 22, 25, 26), the frame is carried as an opaque Node rather than decoded — a wrong body cannot then desynchronise a reader if one of those lists ever turns up populated.

List 14 is observed but not understood. human-turn2-orders.sav — a turn whose only UI action was one fleet move — carries one list-14 element {1456, 0, true}, and 1456 is the same fleet id list 8 carries. So issuing a move queues a second, separate command against the same fleet. What it is remains open; it is named list14 in the engine rather than guessed at.

What lane O's saves could not produce, and therefore what stays a hypothesis: the three unset prologue gates (@0x2c, @0x3c, @0x6c — the last being a whole CivilianRatios frame, i.e. the empire civilian-settings command), and lists 1, 2, 4, 6, 9–13 and 15–27.

---

4. Is the block version-tagged or otherwise variable?

No version word is written. The id string carries the version instead (Player.<id>.TurnCommands_v5), and the shape is fixed: 27 lists always, six gates always. All the variability is the gates and the list lengths, so a single static shape is correct — with one guard. The engine's CustomDataBlock::select() matches the exact suffix .TurnCommands_v5; anything else, including a hypothetical _v6, falls back to the carried Node rather than being decoded with a stale layout. There is a unit test for that.

---

5. Conformance, and why the generic check could not be used

Every item in the class is written with a NULL name, so on disk every tag is .. The wire-schema test's LCS degenerates on such a sequence: with all tags equal, the gap branches are unreachable and the walk becomes a strict positional comparison in which any primitive disagreement is a fatal MISMATCH. That is correct behaviour when the two sequences describe the same thing — and here they do not, for the reason in §1b. Binding the shape with the generic check<T>() would have failed the build, and "make it pass" would have meant deforming the shape to match a table that is wrong.

So test_wire_schema.cpp gets a dedicated check that states what is actually checkable:

• the 17-item prologue, item for item and primitive for primitive — 17/17, zero mismatches. SchemaProbe takes every branch, which is the same all-branches view the recovery has, so the conditional structure read out of the instruction stream must reproduce the recovered sequence exactly. It does. That is the independent evidence for §1a, and it is not a weak test: get any gate/payload pairing wrong and the primitives desynchronise immediately.
• the tail count — the shape has exactly 27 top-level counted lists and the table has exactly 27 tail items. 27 == 27 is the entire content of that half of the table.

The 27 tail items are reported as wire-only, never claimed as matched.

Totals: 87 shapes bound, 856 items matched, 0 MISMATCH (was 86 / 838 / 0).

---

6. A second defect this work turned up: NVs element tags

Not TurnCommands, but found by running the whole corpus and worth reporting loudly.

zuul-turn23-fleet23.sav is the first save in the campaign with a non-empty NVs list on a ServerSystem. Its element's leading id had been typed positionally as R("pid") — disk tag . — on the assumption every anonymous element carries .. The real tag is PID, so the engine's typed round trip on that save differed at offset 0x89c14, at the very first element. This was already broken at engine main (c883a32), before any of lane Q's changes.

Rule 6 exactly: a path no save exercised was a hypothesis flying as a fact, and the first workload that touched it falsified it. Fixed to A("PID"). The fix is independently corroborated — the recovered wire table for Game::ServerSystem names that item PID, and the conformance row moves from 102 matched / 1 wire-only / 3 shape-only to 103 / 0 / 2.

---

7. Coverage, before and after

CoverageArchive's typed-vs-carried split (not round-trip success), all eleven saves:

save	before	after	opaque before → after
turn1-state	99.9%	100.0%	37 → 2
turn2-state	99.9%	100.0%	37 → 2
turn3-state	99.9%	100.0%	37 → 2
zuul-turn5-species5	99.9%	100.0%	37 → 2
human-turn2-orders	99.9%	100.0%	43 → 2
human-turn3-noderoute	99.9%	100.0%	37 → 2
zuul-turn15-orders	99.8%	100.0%	63 → 2
zuul-turn16-noderoute	99.9%	100.0%	40 → 2
zuul-turn17-orders2	99.7%	100.0%	132 → 2
zuul-turn17-rollpending	99.9%	100.0%	37 → 2
zuul-turn23-fleet23	99.9%	100.0%	37 → 2

The two remaining opaque items on every save are the RNG MT19937 block, which is deliberately carried. Nothing else in any save we hold is untyped. The ratchet moves 99.8 → 99.99.

Note zuul-turn17-orders2 was below the old 99.8 ratchet: lane O's largest order save had already pushed the opaque count high enough to fail the coverage gate, which is the gate working.

Round trip stayed byte-identical on all eleven saves before and after (and became byte-identical on zuul-turn23-fleet23, where it was not).

---

8. save_reader.py was deliberately left alone, and why

The Python oracle reads CD frames as Repeat(A("CD", "any")) — it types neither custom-data body, AIAgent included. Lane A judged that an absence rather than a defect and did not touch it. That judgement still holds, and the case for mirroring is weaker than it looks:

• the block is already fully accounted for by the oracle's generic walk — state_checksum --strict reports coverage: PROVED, 0 error 0 warn, on all eleven saves, before and after;
• state_checksum's digest tree is built from the generic node tree, not the typed dict, so typing CD would not put a single named leaf into a diff;
• nothing consumes the typed dict for CD;
• the dispatch would need new machinery in the oracle (selection by CDT ordinal, which the schema DSL has no construct for) — new risk in the instrument the campaign trusts most;
• and rule 8 is pointed at exactly this move. A layout mirrored into both readers from one reading does not become two checks; its failure mode is silent agreement.

So the independent evidence for this layout is not "two readers agree". It is the disassembly, the 17/17 prologue conformance against a table produced by a different tool, and the item arithmetic in §1d closing to the unit on five distinct workloads.

What was added instead, where real saves are actually available: test_save.cpp now asserts, on every save, that each TurnCommands_v5 block is consumed by the prologue plus the 27 lists with nothing left over (extra.empty()). That is the item-granular statement, and a wrong list count or a wrong element width breaks it first.

---

9. Gates

Run as separate commands, as rule 13 requires.

• tools/clean_room_check.sh — OK
• host ctest — 36/36
• test_save with SOTS_SAVES_DIR — 11 saves, 0 failures, round trip byte-identical on all
• state_checksum.py --strict — coverage: PROVED, 0 error 0 warn, on all eleven saves
• test_wire_schema — 87 shapes, 856 items matched, 0 MISMATCH

src/shim/ was not touched, so no CT111 cross-build was needed.

New addresses are in ghidra/addresses.d/lane-q.json (6 entries). The fragment was validated by generating to a scratch path, never over the tracked header.