sots-re/verify/results/saves/certified-pairs.md
alex 7dcc66bc66 lane CR: our code ran instead of ProcessResearch on a real completion; the oracle missed by 16 leaves, all of them OnTechResearched's
Replace mode was tried live on a turn that actually completes a tech, with a
two-process hooks=off oracle established first on that exact (save, procedure,
route). Verdict: game/sim/research stays compared.

What displaced: all 13 tech-tree leaves the turn moves -- 2 from the pass itself
and 11 from the SetResearched cascade -- produced by our code in live game memory,
with the original's ProcessResearch never executing.

What did not: 16 leaves, every one written by ServerPlayer::OnTechResearched.
Five player tech-effect fields (OutMod, ConMod[0..2], ResTNm), one ObservedTech
element, two event records plus EvNxID, and five derived leaves behind them.

Also: ref-turn2 + one End Turn does NOT complete a tech, so every research oracle
before this one was taken on a quiet turn; and a config that names all 27 registered
hooks off and passes check_shim_configs.py still installs six detours, because the
M0 stub and the FPU module's four sampling detours have no hook. key.
2026-09-09 10:05:48 -04:00

95 lines
6.3 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Certified pairs — what has actually been shown to reproduce, and under what procedure
`determinism-hashes.txt` next to this file is a **file-hash inventory**: it says what a byte string
hashed to. It cannot say what was *done* to produce it, and after 2026-09-09 that distinction is
load-bearing. Two findings made it so:
- **A pair certifies a procedure, not a state** (rule 26(c)). A control that agrees means "no exposed
decision was reached on these turns", never "this state is deterministic".
- **The route is part of the procedure.** The same turn runs one way by *continuation* and another way
after a *load*, **both reproducibly** — because a `Colonize` task alive in the AI agent by
continuation is simply absent after a load (lane BS). A hash without its route is not a claim.
So a pair is `(input save, procedure, route, outputs, process count, masks)`. Anything missing one of
those columns is an observation, not a certification.
## Certified
| input | procedure | route | `(Autosave EndTurn)` | `(Autosave)` | processes | evidence |
|---|---|---|---|---|---:|---|
| `ref-turn2.sav` `ab4ac2d7…` | one End Turn | load | `bb4fd9ac…` | `978041ac…` | 5 + 1 | runs A–E; reproduced by lane H after everything changed |
| `ar-turn37-816raiders.sav` `b6f4e05f…` | End Turn, **auto-resolve peacefully**, End Turn | load | `15b99255…` | `7a8b3d5e…` | **4** | lane AR ×3 `hooks=off`, lane BS ×1 instrumented |
| `ar-oracle-A-pre.sav` `15b99255…` | one End Turn | **load** | `33e30092…` | `4c356f59…` | **3** | lane BQ ×2 `hooks=off`, lane BS ×1 instrumented |
| `turn3-state.sav` `978041ac…` | one End Turn | **load** | `e00eed0c…` | `79df5047…` | **3** | lane CR ×2 `hooks=off`, ×1 compare-instrumented |
**The second and third rows are the same lineage and they do not agree with each other.** Row 2's
second turn reached by continuation gives `7a8b3d5e…`; the identical turn reached by *loading* row 2's
own intermediate save gives `4c356f59…`. Both are certified; neither is wrong. This is the load
boundary, and it is why the route column exists.
**Row 3 is the standalone's first trade-raid pair** — one End Turn, no encounter, four raid rolls at
one word each.
**Row 4 is the first pair on a turn that completes a research tech** (lane CR, 2026-09-09).
`turn3-state.sav` is the *output* of row 1 — the `(Autosave).sav` that `ref-turn2` + one End Turn
produces — and the turn it starts is the one where `Player[32 "Fane Lao"]` completes tech 144
`IND_Waldo` and the cascade unlocks three nodes. Row 1's own turn does **not** complete anything
(`unlock-b3-t1.md`: `0 undeclared write(s) in 0 call(s)`), so every research oracle before this row
was taken on a turn where the completion path did not run. Exposure at entry: none of the three
players with an AI client (32, 496, 512) has `ResTNm == ''`; `NumDes` does not move; two ships
complete and join existing fleets; the four factions at 528–576 carry the empty-`ResTNm` signature
inertly (`Status 0`, no client). Two ships completing *and* three fleets changing did **not** expose
this turn — a reminder that 26(c) is a screen and the control is the decision.
Masks for row 4, measured against the input: **exactly `/Summary/Checksum` and `Player.Status 4 → 0`
on the four live players. No `/CD[1]/NPrvVa` term** — this state's `CD[1]` diplomacy block is
early-game and the leaf does not move, so the documented `--mask resave` rule holds here in its
original form.
### Masks that must be on the line
- `Player.Status` (4 → 0 on load) and the derived `Summary.Checksum`.
- **`/CD[1]/NPrvVa`, which advances by exactly +5.00 across a load** and falls on a combat turn. The
documented `--mask resave` rule said "nothing else varies"; that was true of early-game saves with
an empty `CD[1]` diplomacy block and **false on these**. Lane BQ's probe was unsatisfiable on that
leaf alone, before any AI behaviour entered the question.
- Row 3's `(Autosave EndTurn)` is a **resave** of `ar-oracle-A-pre.sav` and differs from it on that
one leaf.
## Explicitly NOT certified
| input | procedure | route | outcome set | why |
|---|---|---|---|---|
| `ad-turn27-two-raiders.sav` `1c8baa27…` | one End Turn | load | **2**: `e913ff41…` ×3, `724528ff…` ×2 | the fleet-visit-order residue |
Five processes across three instrument configurations, **no third file**, both members reproduced at
least twice. The two outputs are `bp-pinA-turn28.sav` and `bp-pinB-turn28.sav`, and lane BU showed
the difference **is** the two orderings of two heap pointers: the assignment pass walks a container
keyed on `fleet->Location`, and under LFH randomisation two same-size allocations land in either
relative order.
That is not a defect to fix and not a pair to keep hunting for. It is a **two-member outcome set**,
which is what C-set's shape needs, and `state_checksum --relabel-new-ids` compares across it:
`IDENTICAL modulo π = {1970↔1986}`. **The unpinned pair on the same save is refused by that tool and
still reports 94 leaves**, so the relabelling cannot launder a real divergence.
Do not add a row here on the strength of one agreeing run. Three processes on `as-turn15` produced
the same bytes on a turn the predicate says is *exposed*, and a fourth produced different ones —
agreement on a single pair proves nothing (lane AP).
## Standing rules for adding a row
1. **Two fresh processes minimum**, and say how many.
2. **Run the control before reading anything from an instrumented run**, so that if it is a single
outcome the 1/k coincidence caveat does not apply (lane AR).
3. **Never extend a certified pair by a turn** without its own agreement. Row 2's state enters the
following turn with `ResTNm == ''` and is *predicted* to vary.
4. **Record the exposure facts next to the hashes**: each AI player's pre-turn `ResTNm`, `NumDes`,
`NumOwn`, whether ships completed, **and which players actually have an AI client** — a save with
seven non-human players builds three clients, and the other four carry the signature inertly.
5. **When the instrument cannot be removed**, pin to a natural run's observed values and require
byte-identity with it first (rule 26(d)). That agreement is both the pin's control and a noise
mask.
Hashes here are recorded to 8 bytes for readability; the full values are in `determinism-hashes.txt`
and in each lane's findings file.