Lane P wrote the expected numbers into sots-engine docs/P-events-wiring.md §4 before
this run existed. Every one held.
First End Turn (ref-turn2 -> Launch -> End Turn, shim.cfg.recapb3 unchanged, build
eventlive-dd38117-20260908T0916Z, main dd38117 with no source change): 3 calls, 3
compared, 0 divergent, tracecmp exit 0 — where lane R's run exited 1 on
side.events.after.v.next_id orig=4 ours=3. Call 0 read turn=3,
events_turn_bucket_exists=true, events_next_id_in=3, events_in_turn_bucket=1,
events_dedup_risk=0, no events_scan_truncated, next_id 3->4 on both sides,
turns/turns_bytes 2/48 unchanged, node[144].progress 2879->5768, flag 1->2, rng
identical — the whole predicted list.
Five End Turns: exit 1 with 2 divergent calls instead of 3, each short by exactly 1
(next_id orig 7 ours 6, orig 12 ours 11) = the deliberately unmodelled
EVENT_TECHS_UNLOCKED. A 0-divergence result there would have been suspicious.
sizeof(Game::ObservedTech) = 44 measured live: observed_techs.bytes grew by exactly
44 on both completion calls (440->484, 484->528), confirming lane X's static pin
behaviourally. Non-researching players measured 880 = 20 x 44 and never moved.
One deviation from lane R, and it is a workload effect, not a defect: rng diverged on
call 9 because the completed tech had research_roll_pending set and the
OnTechResearched callback drew one word ours does not (left 374 vs 375). Lane R's
"RNG matched 15 of 15" was workload luck; the honest statement is that the rng region
matches on every call that does not complete a roll-triggering tech.
Zuul double roll CLOSED. No species-5 save existed, so this lane made one: a custom
game with only Zuul in the Available Species pool, 4 End Turns under the same compare
config. 8 calls, 8 compared, 0 divergent, exit 0; all four researching calls have
species=5 and advance the generator by two, not one (left 540->538, 522->520,
504->502, 485->483), and ours reproduced every post-state bit-for-bit. Save added as
verify/results/saves/zuul-turn5-species5.sav.
End-Turn oracle byte-identical to lane R's on the first turn, so none of this
perturbs the game. clean_room_check OK and host ctest 33/33, run as separate commands.
VM140 restored to the recap build at the main menu and released.
Every serializable class carries an enumeration of its own fields -- its
Write(Stream&), walking the members in order with a 4-char tag. This decodes
that idiom mechanically for the whole binary in 0.35 s.
Validation first (tools/serializers.py validate), against answers the campaign
already had before the tool existed:
A 305/307 field offsets+kinds exact across 17 classes, 0 WRONG, vs
struct-recovery.md 1-4 and observedtech-append.md
B sizeof from the container-stride divides: ObservedTech 0x2c, MoraleEvent
0x50, PlayerReport 0x30, DiplomacyStats 0x24 -- all matching
C 22 of save_reader.py's shapes, tag order identical (Sys 78 tags,
Player 104, CreateParams 25, Ship 22): 22 agree, 0 disagree
D Read/Write cross-check on every class: 437/437 field offsets agree
At scale: 386 classes with a Write, 1,682 member fields.
verified 87 (542 fields) | clean 77 (328) | unnamed 176 (471)
partial 31 (341) | empty 15
58 classes with a sizeof corroborated by a second line of evidence
(45 container stride, 13 enumeration meeting the embedding bound); the rest
report a lower bound and say so.
Four things each worth 10-170 classes: the RTTI class hierarchy descriptor as
the only honest "is this an IStreamable" test (a 3-slot vftable also matches
TacAISquadRule_* and the row parsers); mod=0 memory operands, which x86disp.py
cannot index and which hide every field at offset 0; the member->id pointer
idiom behind every handle field; and sub-writers, both base-class and private
(StrategyServer's six id lists live in FUN_00794cd0).
Failure classes are enumerated in the finding -- 176 anonymous-tag classes are
a hard limit on names but not on layout, and the other 64 are bounded
mechanical fixes. Two fields lost to a value assembled across a branch were
left unrecovered rather than patched with an unverifiable heuristic.
Write-back: 288 structures + 328 labels into Ghidra (0 failures), +201
addresses.json entries, header regenerated with tools/gen_addresses.py.
Note: ghidra/addresses.json also carries lane V's already-written live
confirmation text on ObservedTech_sizeof and ServerPlayer_off_ObservedTechs --
their edit, swept in only because we share the file.
Regenerated after the lane S integration; it was last written at fdd0b72 and had
drifted 12 board rows. Drops the now-resolved 'notes disagree on std::string
layout' open question.
Settles the 0x18-vs-0x1c contradiction lane X raised. 0x1c is right, everywhere,
and there is exactly one std::string instantiation in this binary:
_Bx@0, _Mysize@0x10, _Myres@0x14, _Alval@0x18.
ObservedTech+0x24 is that string's trailing empty-allocator word, not the
unaccounted data field it was read as. Three complete enumerations of the element
each skip it: ObservedTech::Write 0x00817cf0, the ctor 0x008562a0, and the copy
ctor inlined at 0x0079a184. Generalised with a new scanner, tools/strfootprint.py,
which recovers every (base, disp, tag) handed to the Mars::Stream string helpers:
65 std::string members off a non-stack base across every serializer in the exe,
ZERO with a sibling member inside the 0x1c span, and 51 of the 52 measurable
inter-member gaps exactly 0x1c. Corroborated by the vector<string> walk stride
(add esi,0x1c @0x00699c29), PostEvent's by-value strings at [ebp+8]/[ebp+0x24]
with RET 0x4c, and MoraleEvent 0x50 = name@0x34 + 0x1c.
Blast radius: zero recovered struct tables were wrong. Every string-bearing layout
already used 0x1c spans and 0x1c gaps -- ServerPlayer::pswd @0x2dc..0x2f7, the row
flagged for re-checking, included. Only prose carried the 0x18 number: the
loader-prototypes conventions line, the GlobalConst_ParseString prototype, and the
ObservedTech element table. struct-recovery S0 additionally had _Mysize/_Myres
transposed (size@0x14, res@0x18) while every table in the same file used the
correct offsets; fixed.
ObservedTech's four on-disk fields are now mapped rather than guessed, by reading
the serializer as lane X suggested: +0x04 uint16 otnF, +0x06 uint16 otnL, +0x08
bool odet (ONE BYTE, WriteBool), +0x0c std::string otch (0x1c), +0x28 int owith
= 0x2c exactly. That matches save_reader.py's on-disk order already. Game::
ObservedWeapon (0x00817bc0/0x00817b10) is the same element with tag owep.
Oracles unaffected and re-run: save_reader 36/36 and --strict exit 0 on all three
real saves; state_checksum 38 tests OK, coverage PROVED byte-for-byte on turn1 and
turn3. sots-engine wip/strings 32d3e36 syncs the header and corrects two stale
"unpinned" comments: clean_room_check OK, host ctest 33/33.
Standing rule this produced: never size a struct member from the offsets the code
touches. This build's STL puts the empty allocator LAST in both string (0x1c) and
vector (0x10), and an empty allocator is never loaded or stored, so a touch-scan
undercounts by exactly 4 every time. Size from an enumeration instead.
Marks the fpu_cw sensitivity row verified with the result and the correction to
the briefed control-word values, annotates the state-checksum and determinism-
oracle rows, and sets VM140 exclusivity back to FREE with the lane-F gotchas
(the >60 s startup is real and cost a wasted run; pin the SavedGames file set so
the Load dialog rows do not move; PowerShell-over-SSH quoting).
NOTE: campaign/board.md is shared and was already modified in the working tree
when lane F started, so this commit also carries another lane's in-flight rows
for ObservedTech / std::string. Those are not lane F's edits.
Seven End Turns from ref-turn2.sav on VM140, six control words, whole-state
checksum on every post-turn autosave.
53-bit and 64-bit x87 give byte-identical state across all 35,394 leaves, so
an x64/SSE port computing in IEEE double has NO double-rounding budget to
preserve and floats=bits is free. Two settings do move state, each reproduced
on a repeat run:
0x007f (24-bit) Sys[112 "Gamma Cephei"]/Pop2/PopG/PopC 540000000 -> 540000002
0x1a7f (round-up) Flt[34 "Beta Fleet"]/Pos/.[0] and /Pos/.[2], 1 ULP each
So the port must hold intermediates at 53 bits and use round-to-nearest --
both SSE defaults, now measured rather than assumed, each with a named
regression witness.
The briefed triple was under-powered: 0x027f is 53-bit (it differs from 0x127f
only in bit 12, infinity control, ignored since the 387) and 0x137f is 64-bit,
not a rounding change. Run as written all three come back identical, and that
would have "proved" something false on both axes that matter.
Evidence the forced word actually held: read-back at each force site plus 38
independent in-pipeline hook samples per run spanning turn phases 4, 6 and 8,
all reading the forced value. Mars::Application::Run calls
_controlfp(0x50000,0x3070300) at 0x0089f606 every frame, which is 0x127f, so
forcing at StrategyClient::EndTurn is wiped before the turn runs;
StrategyServer::BeginProcessTurn is the point that works.
Also re-confirms the End-Turn determinism oracle on engine cef889e:
bb4fd9ac... / 978041ac... unchanged.
New tools: verify/fpu-cw/cw_census.py, verify/fpu-cw/trace_bitdiff.py (the
latter exists because under a forced 24-bit word the CRT's own %g rendering
degrades, so trace text is not a valid comparison surface).
Ghidra does not index ModRM displacements, so `lea reg,[reg+disp]` -- the MSVC
idiom for taking a member's address -- is invisible to find-constant-uses. That
blind spot parked ServerPlayer+0x274 and covers every non-trivial member of the
~1,600 classes still to map.
tools/x86disp.py: full x86-32 length decoder (prefixes, 1/2/3-byte opcodes,
ModRM, SIB, sign-extended disp8, disp32, every immediate form) swept from
Ghidra's 41,089 function starts so decodes begin on real instruction boundaries.
2,174,504 instructions, 612,166 displacement sites, 100.0% code coverage, 70
desyncs (0.17%), zero unknown opcodes. Excludes no-base disp32 forms
(mod=0/rm=5, sib.base=5) which are absolute globals, not member offsets.
Commands: build/query/cohort/func/dis/stats/brute. Works off a gitignored local
cache in dumps/ rather than hammering CT111.
Validated before use: re-finds lea eax,[ecx+0x29c] in ServerPlayer::GetEventStorage
(0x0080db00) and both known OnTechResearched +0x29c sites, plus a new one in
ProcessTurn. Positive control: the ServerPlayer serializer scores 50/50 known
offsets.
sizeof(Game::ObservedTech) = 0x2c (44), proven three ways: the exact magic
divide 0x2e8ba2e9 sar 3 at 0x0087239f, imul reg,reg,0x2c at 0x0087243a and
0x007b735b, and the search stride add edi,0x2c at 0x007ba257.
Append site: RecordObservedTech+0xdf (0x007ba27f) --
lea ecx,[player+0x274]; call vector_ObservedTech_push_back 0x007b7320
RecordObservedTech (0x007ba1a0) is a direct callee of OnTechResearched and
de-duplicates by tech name before appending. The realloc through 0x007b5820 is
why lane R's guard saw all three vector words move. Element carries a vptr
(RTTI .?AVObservedTech@Game@@) at +0 and a 0x18-byte std::string at +0x0c; the
four on-disk ints map onto +0x04/+0x06/+0x08/+0x24/+0x28 in an order this read
does NOT determine, and is not guessed.
Also corrects harness-audit row 11: ComputeBudget has no store to Budget+0x64
(its only +0x64 accesses are loads off a different base), and ProcessResearch's
int* overbudget arg is a ProcessTurn stack local, not Budget+0x64. Agrees with
lane R's guard seeing 0 changes in 4284 calls.
Honest limits are recorded in the note and the board: this is a recall tool, not
an oracle. Class-level precision at 0x274 is ~13% by function, i.e. a ~900x
search-space cut that still needs one call-graph check. Cohort ranking must not
be used as a hard filter -- it would have discarded the correct answer here.
Ghidra writeback: labels + plate comments on RecordObservedTech,
vector_ObservedTech_push_back, ObservedTech_ctor, vector_ObservedTech_assign,
vector_44B_grow, vftable_ObservedTech.
Read TechTree::SetResearched (0x00581e10) end to end while wiring the research event
posts into ours (sots-engine wip/post-events). Two things it settles:
- SetResearched invokes the owner callback as vft+0x10(def, (flags>>2)&1), so bit 2 of
flags IS OnTechResearched's "silent" argument. ProcessResearch passes flags = 2, so
silent is FALSE and the completion event IS posted. Previously this was only
inferable from lane R's observation that a completion call moved EvNxID by two.
- The unlock cascade in full: the node's child edges set child state 0->1 and
child.costRP = min(child.costRP, edge.costRP); then a whole-tree sweep sets state 2
and stamps turnAvailable from ModCount ONLY when it currently reads -1 (so it is
sticky, and a node re-entering state 2 is not re-announced), and recurses into any
zero-cost available node. Both loops test the node itself, not a parent -- there is
no parent clause anywhere in the function, confirming lane E's correction.
Also records what is and is not known about vector<ObservedTech> at ServerPlayer+0x274:
the on-disk element shape is confirmed against turn3-state.sav (its string holds a tech
name), while sizeof(ObservedTech) and the append call site are still unpinned. The
cheapest route to the stride is now a measurement from the new observed_techs region,
not a search -- find-constant-uses does not index lea displacements.
New entries: ServerPlayer_off_ObservedTechs, TechNode_off_TurnAvailable /
TurnResearched / Order / Children, TechEdge_off_CostRP / ChildDef,
TechTree_off_OrderCounter, TechTree_SetResearched_flag_Force / _flag_Silent.
Generated header regenerated with tools/gen_addresses.py (386 -> 396 entries).
The mechanism, read off the instruction stream rather than fitted: Mars_Vec3_Normalize
(0x00422520, 123 callers) narrows to float32 four separate times, and MoveFleet stores
each dest.c - pos.c back to a float32 slot before calling it and takes the leg distance
from that same call's return value. Live 8/45 -> 0/45 with a control run of the unchanged
build, identical inputs and identical original outputs on all 45 calls.
Board: the two MoveFleet rows go verified; VM140 exclusivity back to FREE with two new
click-path gotchas; a new backlog row for waypoint types 2-5, which this lane attempted
and could not reach - ref-turn2 structurally cannot produce a node-line move because the
only player that would travel one has no ships.
(Most of this lane's files were swept into 9d385a7 by another lane's `git add -A` on the
shared repo; this commit carries what was left.)
The complement to the per-function compare harness. Instead of "did this
function's declared outputs match", it asks "is the entire simulation state
still identical" -- so no region-declaration mistake can hide from it.
Coverage is PROVED, not declared: the digest tree is re-serialised and compared
byte-for-byte against the inflated save on every run. When that reconstruction
reproduces the stream, the whole file is a function of the digest's inputs. A
run that cannot account for the file says so and exits non-zero. This is the
direct answer to B4's three hooks that printed "0 diverged" over an empty
region set.
It localises. The root is the fold of a per-subsystem / per-object tree with
named objects, so the known load->re-save delta reports as exactly five leaves
-- /Summary/Checksum and four /Sim/players/Player[...]/Status 4->0 -- naming the
two Singularity players by id where the raw byte diff could only say "1st of
two". One real End Turn reports as 108 fully attributed differences.
Float-parity policy is explicit and strict by default (STATE_CHECKSUM.md 3):
raw IEEE-754 bits; a `canonical` policy for signed zero and NaN payloads only;
and deliberately NO tolerant hashing mode, because quantisation moves the cliff
rather than removing it and destroys the roll-up. Tolerance lives in the differ
as --ulps, applied after localisation. Corpus census: 0 NaN, 0 -0.0, 0
subnormals across 4,474 float leaves, so the strict default costs nothing today
and a test fails the day that changes.
Validated on the real saves (verify/results/state-checksum/): 10 files, 4
distinct contents, all STABLE + COVERED; chain record/verify works on the real
turn1-3 saves. The VM-driven replay loop is designed (section 5) but UNRUN.
Section 3.5 names the one question the host side cannot settle -- whether the
turn pipeline depends on x87 intermediate precision -- and the experiment that
would: force fpu_cw to 0x027f / 0x127f / 0x137f across End Turn and checksum
the three autosaves.
Also recorded: Summary.Checksum is NOT a byte sum over the inflated stream nor
a sum over the int leaves (both ruled out), so nobody repeats those two.
38 tests; sots-engine untouched, clean_room_check.sh OK.
Five hooks recaptured, build recap-7584bad-20260908T0615Z, no source change.
- ProcessResearch: the over-budget event is now a compare divergence
(side.events.after.v.next_id orig=4 ours=3) instead of a 609 KB save diff.
15 calls over 5 turns, 3 diverged, RNG matched 15/15.
- ComputeBudget: 4284 compared, 0 diverged, guard quiet - but only 20 distinct
states and 13 of 22 slots always zero.
- MoveFleet: 8 of 45 calls diverge by 1 ULP of position. B4's clean verdict was
a one-sample verdict.
- First guarded captures for OnTechResearched (RollResearchEvent draw fired and
matched), the colony turn and MoveFleet.
- Guards mapped SetResearched live and found an undeclared ObservedTech append.
VM140 released at the main menu, hooks=trace.
Closes B3's oracle gap and B2's known gap. Both milestones write into the
owner's event list; nothing modelled it, so their clean compares bounded the
economy fields only.
Container: EventStorage embedded at ServerPlayer+0x29c (0x1c bytes), verified by
ServerPlayer::GetEventStorage 0x0080db00 whose whole body is 'lea eax,[ecx+0x29c];
ret'. EvNxID at +0x14 == player+0x2b0, exactly the byte run the harness guard
reports. The list is a vector<TurnEvents{int EvTurn; vector<PlayerEvent>}> --
bucketed by turn, not flat, which the save-editor struct note had wrong.
Entry point: int __thiscall EventStorage::PostEvent(this, std::string BY VALUE,
std::string BY VALUE, void* obj, Vector3* pos, int turn, const char* img,
int act), 0x008862b0, RET 0x4c. 161 call sites in 110 functions: this is the
whole simulation's event API, not a research helper. Dedup is per turn bucket on
message/image/location/position/action but NOT summary; act 0 with no subject and
no position is stored as 2; EvNxID starts at 0 and is promoted to 1 on first post;
PruneOldTurns drops buckets older than turn-50 with an off-by-one that always
leaves one stale bucket behind.
Record: 0x74 bytes, EvEID(+4) EvDsc(+8) EvMsg(+0x24) EvLoc(+0x40) EvPos(+0x44)
EvImg(+0x50) EvAct(+0x6c) EvCID(+0x70) -- confirmed field by field against
turn3-state.sav, which contains the EVENT_RESEARCH_OVERBUDGET record itself.
B3's defect is fully explained: 0x00587b97, reached only from the
completion-roll-FAILED branch, under !wasDone && nowDone && owner.
Three note corrections: EvPos defaults to FLT_MAX, not infinity (writing +inf
changes the save bytes); the save array is turn-bucketed, not flat;
EVENT_TECHS_UNLOCKED has no 'parent researched' clause.
56 entries added to ghidra/addresses.json (header regenerated); 11 prototypes,
13 labels, 12 comments and 2 structs written back to the Ghidra project.
