befaf3dbfc
TacReport in particular: TRnc is a count and TRships/TRsats/TRshipsL are ONE interleaved loop body, not three trailing runs -- which is why two lanes could not settle it from a flattened linear recovery with TRnc == 0 everywhere. Also two corrections the conformance binding found: Game::CombatReport's auto and cdst are bools and dur/cdt/cdi are floats (cdt was being read as 1070805848 instead of 1.598), and Game::CombatWeaponReport's damage quartet is flat with float dami/damt -- the nested 'dams' frame the shape modelled does not exist, and modelling it also inflated the coverage denominator by one per weapon report. FTPnts stays carried and labelled: the one item the recovery itself marks unresolved, count 0 everywhere, element framing a property of the helper.
189 lines
11 KiB
Markdown
189 lines
11 KiB
Markdown
# Paying the ratchet debt: the five bodies the new saves exposed
|
||
|
||
Lane WS, 2026-09-08. Host/static only — no VM. Continues `findings/objects/wire-schema-closeout.md`
|
||
(lane W) and `findings/objects/wire-schema-channel.md` (lane G).
|
||
|
||
## Headline: 97.6 % → 99.9959 % on the worst save, and the bar did not move
|
||
|
||
The corpus went from 11 saves to 20 in one evening as lanes manufactured workloads nobody had.
|
||
The coverage ratchet in `test_save.cpp` broke on contact — named coverage fell to 97.6 % against a
|
||
99.99 bar — because four new saves carry content no shape named. **That is the ratchet working**
|
||
(rule 27). The reader was sound on every one of them the whole time: `round trip: tree identical,
|
||
typed identical`. It carried the bytes; it did not name them.
|
||
|
||
Named coverage per save, before → after. The honest metric is `CoverageArchive`'s split of items a
|
||
field **names** from items a generic `Node` merely **carries** (rule 7), never round-trip success.
|
||
|
||
| save | before | after | opaque items |
|
||
|---|---|---|---|
|
||
| human-turn5-traderoutes | 99.0003 % | **99.9951 %** | 409 → 2 |
|
||
| human-turn8-traderoutes | 98.6349 % | **99.9955 %** | 613 → 2 |
|
||
| human-turn11-spytechs | 97.6382 % | **99.9959 %** | 1166 → 2 |
|
||
| human-turn15-spyprogram | 96.9412 % | **99.9964 %** | 1706 → 2 |
|
||
| the other 16 saves | 99.9940–99.9953 % | unchanged | 2 → 2 |
|
||
| **corpus** | **99.5021 %** | **99.9949 %** | 3926 → 40 |
|
||
|
||
All 20 saves are above the ratchet. The 2 items left on every save are the MT19937 blob, which is
|
||
deliberately opaque. **No path stayed opaque that any save exercises.**
|
||
|
||
Conformance: **100 shapes bound, 966 items matched, 0 MISMATCH** (was 87 / 856 / 0). Fourteen new
|
||
bindings, every one a full match with zero wire-only and zero shape-only items. `ctest` 58/58 over
|
||
all 20 saves; `tools/clean_room_check.sh` OK; shim cross-build clean (see §6).
|
||
|
||
## 1. `<rest:prep>` — 704 items on turn11, the largest. Closed.
|
||
|
||
`Game::CombatPlayerReport`'s tail. The shape stopped at `mxmsl` and swept the rest into `ar.rest`.
|
||
The recovery had the whole thing all along — the corpus simply had no combat report until the
|
||
trade saves — and the data confirms the flattened loops item for item:
|
||
|
||
```
|
||
plr ai ally status mxeng mxcls mxmsl
|
||
ncls <- count
|
||
cls nshp nsat nshfld nshlst nsatl <- x ncls, one loop body of six
|
||
nsec <- count
|
||
ssec nshp <- x nsec, one loop body of two
|
||
ndam
|
||
srep { count, elements } <- Game::CombatShipReport
|
||
```
|
||
|
||
`Game::CombatShipReport` in turn: `name did cls caps2 nshp nfld nlst dtak dams damp dami damt`,
|
||
where `caps2` is an **i64** (the `Caps` bitfield) and `dami`/`damt` are **floats**.
|
||
|
||
## 2. `TacReports` — 369 items, and why two lanes could not close it
|
||
|
||
Lanes B5 and J both recorded `Game::TacReport` as carried-not-typed, with the same reason: the
|
||
recovery ends with a computed count `TRnc` and then three trailing scalar items —
|
||
`TRships`, `TRsats`, `TRshipsL` — and there is no way to read a flattened linear pass and tell
|
||
|
||
* **three trailing runs** (`TRships` × n, then `TRsats` × n, then `TRshipsL` × n), from
|
||
* **one loop body of three** (`TRships TRsats TRshipsL`, × n).
|
||
|
||
Both produce the same three items in the same order in the recovery, because the recovery cannot
|
||
see the loop. And `TRnc` was **0 in every save in the corpus**, two levels down under a count that
|
||
was itself 0, so the wire could not arbitrate either. Neither lane guessed, which was right.
|
||
|
||
The trade saves settle it by inspection. `TRnc == 3`, and the wire reads:
|
||
|
||
```
|
||
TRnc 3
|
||
TRships 0 TRsats 0 TRshipsL 0
|
||
TRships 0 TRsats 0 TRshipsL 0
|
||
TRships 0 TRsats 0 TRshipsL 0
|
||
```
|
||
|
||
**Interleaved.** One loop body of three, run `TRnc` times. It is a `narr`, and the whole
|
||
20-item class now binds at 20/20. `Game::TacReportEvents` (`TRby` / `TRto`) comes with it: two
|
||
ints, four floats, one bool.
|
||
|
||
This is rule 6 working as intended in both directions — the path was flagged as a hypothesis
|
||
rather than closed on a guess, and when a save finally exercised it the answer was one line of
|
||
dump.
|
||
|
||
## 3. `Lay` — 79 items, entirely new. Closed.
|
||
|
||
`Game::FleetLayout`, gated by `HLay`, which is false on every fleet in the old corpus. Three
|
||
NULL-named items: a `Game::FieldTemplate` frame, a count, then that many ship ids. The count word
|
||
is on the wire (`narr`), so nothing here is inferred.
|
||
|
||
## 4. `<rest:trdmgr>` — 9 items — is `rt`, and it has no count word
|
||
|
||
`Game::ServerTradeManagerImpl` writes `NumTradeSectors`, the sector pairs, `SctSize`, and then
|
||
`Game::TradeRoute` frames tagged `rt`. The recovery marks `rt` a loop body (`member: false`) and
|
||
there is **no count item before it in the table and none on the wire** — the trdmgr frame holds
|
||
`1 + 2×6 + 1 + 1 = 15` items on turn11, which accounts for every one. So it is an uncounted run,
|
||
read with `ar.repeat("rt", …)` keyed on the tag, the same shape as `DSec` in a design.
|
||
|
||
Both saves that have any route at all carry exactly one, so "uncounted" is the framing the
|
||
recovery supports; "exactly one" is not a claim the evidence can make.
|
||
|
||
`Game::TradeRoute` is eight ints: `tro trfow trfr trfrs trtow trto trtos trtc`.
|
||
|
||
## 5. `fwarn` — 3 items — and two more the brief did not list
|
||
|
||
* **`fwarn` = `Game::ServerTradeSector::FreighterWarning`**, `{pid, ntrns}`. The count was 0 in
|
||
every sector of every save until turn11, where a raided route produced one.
|
||
* **`spy` = `Game::SpyCraft`** (14 items on turn15, the first save with `nspy != 0`): five ints,
|
||
two floats, four ints, a float, then an `ncp` count and its `cpl` elements.
|
||
* **`Dwg` = `Game::WeaponGroups`** (17 items on turn15, the first design with `Dwgv` set):
|
||
`wgng` count, then `(wgid, wgb)` pairs where `wgb` is a `Game::GunBankSelection` frame. That
|
||
frame carries a run of NULL-named ints — three in the only record we have — with no count word,
|
||
so it too is read as a run rather than as a fixed three.
|
||
|
||
## 6. Two corrections found by binding shapes to the table (rule 11)
|
||
|
||
Neither was visible from the round trip, which is the point of the conformance test.
|
||
|
||
**`Game::CombatReport`: five items were the wrong primitive.** `auto` and `cdst` are BOOLs;
|
||
`dur`, `cdt` and `cdi` are FLOATS. All five were `int32` in the shape. Four of them are 0 or 1 on
|
||
every save, where a bool and an int are the same bytes under a four-character tag, so nothing ever
|
||
moved. `cdt` is not: it holds `0x3FCC…`, and the shape was reading **1070805848** for what is
|
||
1.598. This is the same trap as `odet` in an earlier round, one level up.
|
||
|
||
**`Game::CombatWeaponReport`: the damage quartet is flat, and the last two are floats.** The old
|
||
shape reached `dams damp dami damt` through an `obj_flex` sub-struct, i.e. it modelled a nested
|
||
`dams` frame. The binary does not write one — the recovered class lists the four as its own
|
||
scalars, grade *verified* — and no save in the corpus contains one either.
|
||
|
||
That second one also had a **measurement** consequence worth recording under rule 12.
|
||
`CoverageArchive::obj` charges one typed item for the frame it is about to descend into; under
|
||
`obj_flex` with `framed == false` there is no frame, and it charged anyway. So the coverage
|
||
denominator was inflated by exactly one per weapon report. The four affected saves have
|
||
**8, 13, 21 and 32** weapon reports, and their before/after item totals differ by exactly
|
||
**8, 13, 21 and 32**. The pre-fix coverage figures in the table above were therefore slightly
|
||
*optimistic* as well as too low; the after figures are on the corrected count.
|
||
|
||
## 7. What stays carried, and the workload that would settle it
|
||
|
||
One item from this round, deliberately:
|
||
|
||
**`FTPnts` (`Game::FieldTemplate`)** — the only item in this work that the recovery *itself* marks
|
||
`unresolved`. It names the element class from the decorated helper name
|
||
(`Game::FieldTemplate::Point`, whose own serializer is fully recovered: `FTPShID FTPDesID FTPPosX
|
||
FTPPosY FTPSqd`) but could not type the item, and the count is **0 in every `Lay` in the corpus**.
|
||
That is precisely the pair of conditions under which `SysMem`, `mts` and `nalat` were each typed
|
||
wrong and stayed wrong invisibly: the element **framing** is a property of the writer's helper,
|
||
not of the element class, and no record has ever shown one. So the elements are carried, and
|
||
`Game::FieldTemplate::Point` is deliberately **not** bound in the conformance test.
|
||
|
||
*Workload:* a save whose fleet has a stored tactical formation — set a fleet's combat layout in
|
||
the tactical setup screen, then save. One such save closes it, the same way turn15's single spy
|
||
craft closed `spy` and its single `Dwgv` design closed `Dwg`.
|
||
|
||
Everything else still carried is a container no save has ever filled — `nodePaths`, `Ojvs`,
|
||
`usnc`, the four `SpyReport` sub-lists, the `deflay`/`rtgt` pairs, and the polymorphic keys with
|
||
no factory entry — plus the RNG blob. None of them costs a single item on any save in the corpus.
|
||
|
||
## 8. The honest floor
|
||
|
||
The ratchet stands at **99.99 %** and every save clears it, so there is no new floor to declare.
|
||
The tightest save is `zuul-turn5-species5` at 99.9940 %, and its whole residual is the RNG blob's
|
||
two items against 33,316 typed. A bar of 99.99 is now equivalent to "nothing but the RNG blob",
|
||
which is the right thing for it to mean.
|
||
|
||
The number that will move next is not this one. It is the count of *hypotheses* in §7: each is
|
||
worth exactly one manufactured save, and until then each is a labelled guess, not coverage.
|
||
|
||
## 9. Gates
|
||
|
||
Run as separate commands on fresh build directories (rules 13, 24):
|
||
|
||
* `tools/clean_room_check.sh` — OK
|
||
* host `ctest` — **58/58**, `SOTS_SAVES_DIR` pointed at all 20 saves
|
||
* shim cross-build (`--preset shim`) — clean, 129/129
|
||
|
||
Caveat on the third, stated rather than assumed (rule 13): CT111 was unreachable from this lane
|
||
(`Permission denied (publickey,password)`, and ReVa was down with `CONNECTION_CLOSED` throughout),
|
||
so the cross-build was run locally against `i686-w64-mingw32-g++ (GCC) 10-posix 20220113`. That is
|
||
**not necessarily CT111's toolchain**, so the integrator should still run the CT111 shim build.
|
||
It did catch one thing worth having: `Game::TacReport`'s `TRmin` field wanted a member named
|
||
`min`, which `windows.h` defines as a macro — it is `mine` in the shape for that reason.
|
||
|
||
Note the win32-threads variant of that compiler cannot build `src/mars/vfs/zip_archive.cpp` at
|
||
all (`std::mutex` does not exist); the posix variant is required.
|
||
|
||
## Files
|
||
|
||
`sots-engine` `wip/ws`: `src/mars/stream/shapes.h`, `tests/mars_stream/test_wire_schema.cpp`,
|
||
`tests/mars_stream/test_stream.cpp`, `docs/mars-stream.md`. No generated header changed — every
|
||
fact used here was already in `objects/streams.json` and `include/generated/sots_stream_schema.h`;
|
||
what was missing was never the schema, it was a save that put something in the containers.
|