findings/subsystems/output-term.md is the whole reading: the call chain with
real function boundaries, the formula, the closed list of nine values the data
files supply (down from an unbounded fear), the advance prediction with its
falsification table, and the live result.
Corrects strategic-turn-internals.md 3.3 in place. That block had the SHAPE
wrong, not just the detail: a system's output is a sum of three terms, and the
function it named as the population base-output term is the over-harvest
resource demand.
Live on VM140, two builds, two species, both hooks in compare mode:
GroupOutput 13,105 calls / 0 divergences, ComputeTotalOutput 11,252 / 1 (one
ulp), 0 undeclared writes in 24,357 guarded calls. Thirteen distinct system
states, and every unexercised branch is listed rather than counted as covered.
tools/max_income_predict.py is the other half: it computes lane Y's
bankruptcy-limit oracle from colony state and reports 6 of 25 player-records
matching exactly, with the misses all AI-owned and the single-system ones short
by exactly the 1.1 difficulty income multiplier.
Every call-graph result in this repo was computed over direct (E8) edges.
5,045 of the 5,207 functions named by a vftable slot have zero direct call
sites, so all of those results were lower bounds. Lane Z's dominant RNG
consumer hung off exactly such an edge.
tools/vtable_map.py builds, from the RTTI walk plus a full sweep to the next
function start (rule 17):
* vftable -> class -> sub-object offset -> slot -> target, and its inverse
* the class hierarchy from the RTTI base lists, so an abstract interface
with one concrete override resolves uniquely
* constructor-derived member typing (ctor result -> [this+d])
* the slot index at every indirect call site, with a backward register
resolver that refuses to cross a branch target rather than guess
* `this`-carrier spans and this/member call-graph propagation of class
Validation (12/12): rediscovers ServerTradeManagerImpl slot 10 ->
GenerateTradeRaidEncounters from the dispatch at 0x007d8469 with nothing
hand-fed, and re-derives the *Impl rule for both managers. Receiver-class
pinning reaches only 2.5% of the 6,398 virtual sites, at 0.6% out-of-range
against a 70% chance baseline; the displacement-only route measured worse
than random (81% vs 58%) and is rejected outright.
Closes lane K's tier-4 blind spot: all nine phase-23 calls and both phase-33
calls named. Four of the eleven reach a draw on the strategic generator
(StrategyServer+0x16c) at eight instruction-verified sites, none ever
observed firing — so "the tail draws nothing" is a property of eight turns,
not of the code. Also resolves the nine parked inlined-draw functions to
their vtable roots (correcting how that was recorded: none is itself in a
vftable; their topmost ancestors are), and finds 14,958 inter-function tail
jump edges without which three of them look like dead code.
The standalone now models the turn's dominant generator cost -- 16 of a measured 18-22
words -- and lands 4 and 2 short of the two calibrated pairs, which is exactly the
per-call-site ledger's split for those turns. The state block is byte-identical; only
left differs. The answer to 'does it match the oracle' is no, by a stated amount, and
tools/rng_oracle_check.py is the instrument that says so.
The tail's last phase is modelled for the six turn-record fields recoverable from the
wire and checked against the record the game itself archived: 480 fields over 80
player-records, 0 mismatches. It stays blocked; --commit-blocked shows exactly which
five fields are missing and what they cost.
By-product, and probably worth more than the phase: the stored bankruptcy elimination
limit is injective in the maximum-income sum it is built from, so every save states the
per-system output term that blocks ComputeBudget. tools/max_income_oracle.py inverts it
-- 25 player-records over the corpus -- and recovers the protection factor as 3.3 from
the saves rather than from the data files. It also shows the engine's -0.15 divisor
disagrees with the game on 6 of those 25.
divergence unchanged: 209->204 and 108->103, 5 closed / 0 regressed on both pairs.
Seven entry points detoured, each call keyed by __builtin_return_address(0)
with the word cost from left before/after. Three consecutive turns on
ref-turn2: site sums 19/18/20 against independently measured ProcessTurn totals
of 19/18/20, residual 0 every time. The 18-20 spread is now explained rather
than reported -- it is the two gated research draws.
The dominant consumer is FUN_00893290: two Chance calls per player across all
eight player-vector entries, 16 of every turn's 18-20 words, and it is NOT one
of lane I's 22 sites. Lane I said its closure covered direct edges only and
that indirect reachability was unsettled; this is that gap, measured. The
function is unidentified and is the highest-value target left.
Two bookkeeping corrections are in the report tool, not the shim: helper-
internal rows (Chance's own NextFloat) double-count, and 8 calls per turn are
on the StrategyClient's generator, not the strategic one. The first build did
not distinguish generators and reported 44 words against a bracket of 18 --
which is what caught it. A per-site ledger that cannot say which generator a
draw came from is not a ledger.
tools/standalone_report.py drives sots-engine's sots_turn over each
consecutive-turn save pair and diffs the result against the game's own
post-turn save with state_checksum.py, which localises to named leaves and
proves its own coverage by re-serialisation.
turn1-state -> turn2-state baseline 209 diverging, after 204, closed 5
turn2-state -> turn3-state baseline 108 diverging, after 103, closed 5
regressed 0 on both
`regressed` is reported next to `closed` and never netted off. It earned its
place immediately: committing the phase-31 player-status restore turned two
agreeing leaves into disagreeing ones, because the phase writes 1 and the file
carries 4.
The stable-system stand-in feeding the colony pass is a labelled hypothesis and
it survived a changed workload -- the same 3 ntdev leaves closed on both pairs,
six agreements, zero disagreements.
Two things deliberately NOT implemented: the TShn/ltis counters (18 leaves, a
`+1` would close them, but "+1 across one observed turn" is a hypothesis, not a
reading), and the RNG state write-back (an advanced-but-incomplete generator is
wrong in a different way from an untouched one).
dashboard.py gains section 6, reading verify/results/standalone/status.json:
phases modelled/committed per driver, baseline vs after, closed vs regressed,
the subsystem breakdown of what still differs, and the RNG gap. Sections 6-8
renumbered to 7-9; the delta footer tracks the two new counts.
DASHBOARD_README.md documents every number.
findings/control-flow/standalone-scaffold.md has the ranked blocker list.
We consume 18-22 generator words per turn and model none of them as a count.
All of it is inside StrategyServer::ProcessTurn; OnAllCombatDone_Tail costs 0
on every turn observed; the residual outside the two drivers is exactly 0. The
generator does not move between turns at all, so the interval a standalone has
to reproduce is closed at both ends.
The instrument reads generator STATE, not calls, and that choice paid: the
image has four draw entry points, not three (NextUInt 0x004f7670 is in no
lane's primitive set) plus inlined draws in twelve functions, two reachable
from the turn roots. A primitive-counting hook would have undercounted
silently.
Checked against the save files independently: the turn-6 autosave pair gives
18 words read from the two Sim.RNG blobs, and with twists == 0 that number
never passes through a twist implementation -- so the two instruments do not
share the hidden assumption they could have.
Corrections to combat-done-tail.md, in place:
* the node-line 0x20000-fleet check runs AFTER the Chance(0.5f) call and
cannot gate the draw; the expiry test is NodePath::RemainingLife 0x006e2130
and is now a formula rather than a description
* StrategyHost::Autosave is ret 8 and returns the std::string* in EAX
* SNMAllCombatDone IS delivered every End Turn (8 of 8) -- lane K's inference
was right; the stronger no-encounter reading is narrowed, not closed
* S+0x8 advances 12-14 times per turn, not twice
Node-line decay still has not fired. The hook reports the distance instead of
the absence: 51 of 53 lines are permanent, the mortal ones are dug ~1/turn by
the Zuul, each ~40 turns from expiry. It stays a labelled hypothesis.
objects/layouts.json is a memory-layout projection: build() sorts fields by
off_abs (89 of 386 classes have offset order != write order) and merges
duplicate offsets into alt_tags, which is exactly the JewelsOfTheCrown
double-tag trap. Both losses are the substance of the on-disk format.
tools/streams.py is a second projection of the same recovery that keeps the
program order Lab.layout() already computes and the repeated tags, and drops
every memory fact — no off, size, sizeof, gaps or strides. The engine must read
and write the format, not inherit the original's ABI.
tools/gen_stream_schema.py emits sots-engine's include/generated/sots_stream_schema.h
under the same discipline as gen_addresses.py: generated, provenance header,
never hand-edited.
386 classes, 2042 wire items.
Every serializable class carries an enumeration of its own fields -- its
Write(Stream&), walking the members in order with a 4-char tag. This decodes
that idiom mechanically for the whole binary in 0.35 s.
Validation first (tools/serializers.py validate), against answers the campaign
already had before the tool existed:
A 305/307 field offsets+kinds exact across 17 classes, 0 WRONG, vs
struct-recovery.md 1-4 and observedtech-append.md
B sizeof from the container-stride divides: ObservedTech 0x2c, MoraleEvent
0x50, PlayerReport 0x30, DiplomacyStats 0x24 -- all matching
C 22 of save_reader.py's shapes, tag order identical (Sys 78 tags,
Player 104, CreateParams 25, Ship 22): 22 agree, 0 disagree
D Read/Write cross-check on every class: 437/437 field offsets agree
At scale: 386 classes with a Write, 1,682 member fields.
verified 87 (542 fields) | clean 77 (328) | unnamed 176 (471)
partial 31 (341) | empty 15
58 classes with a sizeof corroborated by a second line of evidence
(45 container stride, 13 enumeration meeting the embedding bound); the rest
report a lower bound and say so.
Four things each worth 10-170 classes: the RTTI class hierarchy descriptor as
the only honest "is this an IStreamable" test (a 3-slot vftable also matches
TacAISquadRule_* and the row parsers); mod=0 memory operands, which x86disp.py
cannot index and which hide every field at offset 0; the member->id pointer
idiom behind every handle field; and sub-writers, both base-class and private
(StrategyServer's six id lists live in FUN_00794cd0).
Failure classes are enumerated in the finding -- 176 anonymous-tag classes are
a hard limit on names but not on layout, and the other 64 are bounded
mechanical fixes. Two fields lost to a value assembled across a branch were
left unrecovered rather than patched with an unverifiable heuristic.
Write-back: 288 structures + 328 labels into Ghidra (0 failures), +201
addresses.json entries, header regenerated with tools/gen_addresses.py.
Note: ghidra/addresses.json also carries lane V's already-written live
confirmation text on ObservedTech_sizeof and ServerPlayer_off_ObservedTechs --
their edit, swept in only because we share the file.
Settles the 0x18-vs-0x1c contradiction lane X raised. 0x1c is right, everywhere,
and there is exactly one std::string instantiation in this binary:
_Bx@0, _Mysize@0x10, _Myres@0x14, _Alval@0x18.
ObservedTech+0x24 is that string's trailing empty-allocator word, not the
unaccounted data field it was read as. Three complete enumerations of the element
each skip it: ObservedTech::Write 0x00817cf0, the ctor 0x008562a0, and the copy
ctor inlined at 0x0079a184. Generalised with a new scanner, tools/strfootprint.py,
which recovers every (base, disp, tag) handed to the Mars::Stream string helpers:
65 std::string members off a non-stack base across every serializer in the exe,
ZERO with a sibling member inside the 0x1c span, and 51 of the 52 measurable
inter-member gaps exactly 0x1c. Corroborated by the vector<string> walk stride
(add esi,0x1c @0x00699c29), PostEvent's by-value strings at [ebp+8]/[ebp+0x24]
with RET 0x4c, and MoraleEvent 0x50 = name@0x34 + 0x1c.
Blast radius: zero recovered struct tables were wrong. Every string-bearing layout
already used 0x1c spans and 0x1c gaps -- ServerPlayer::pswd @0x2dc..0x2f7, the row
flagged for re-checking, included. Only prose carried the 0x18 number: the
loader-prototypes conventions line, the GlobalConst_ParseString prototype, and the
ObservedTech element table. struct-recovery S0 additionally had _Mysize/_Myres
transposed (size@0x14, res@0x18) while every table in the same file used the
correct offsets; fixed.
ObservedTech's four on-disk fields are now mapped rather than guessed, by reading
the serializer as lane X suggested: +0x04 uint16 otnF, +0x06 uint16 otnL, +0x08
bool odet (ONE BYTE, WriteBool), +0x0c std::string otch (0x1c), +0x28 int owith
= 0x2c exactly. That matches save_reader.py's on-disk order already. Game::
ObservedWeapon (0x00817bc0/0x00817b10) is the same element with tag owep.
Oracles unaffected and re-run: save_reader 36/36 and --strict exit 0 on all three
real saves; state_checksum 38 tests OK, coverage PROVED byte-for-byte on turn1 and
turn3. sots-engine wip/strings 32d3e36 syncs the header and corrects two stale
"unpinned" comments: clean_room_check OK, host ctest 33/33.
Standing rule this produced: never size a struct member from the offsets the code
touches. This build's STL puts the empty allocator LAST in both string (0x1c) and
vector (0x10), and an empty allocator is never loaded or stored, so a touch-scan
undercounts by exactly 4 every time. Size from an enumeration instead.
Ghidra does not index ModRM displacements, so `lea reg,[reg+disp]` -- the MSVC
idiom for taking a member's address -- is invisible to find-constant-uses. That
blind spot parked ServerPlayer+0x274 and covers every non-trivial member of the
~1,600 classes still to map.
tools/x86disp.py: full x86-32 length decoder (prefixes, 1/2/3-byte opcodes,
ModRM, SIB, sign-extended disp8, disp32, every immediate form) swept from
Ghidra's 41,089 function starts so decodes begin on real instruction boundaries.
2,174,504 instructions, 612,166 displacement sites, 100.0% code coverage, 70
desyncs (0.17%), zero unknown opcodes. Excludes no-base disp32 forms
(mod=0/rm=5, sib.base=5) which are absolute globals, not member offsets.
Commands: build/query/cohort/func/dis/stats/brute. Works off a gitignored local
cache in dumps/ rather than hammering CT111.
Validated before use: re-finds lea eax,[ecx+0x29c] in ServerPlayer::GetEventStorage
(0x0080db00) and both known OnTechResearched +0x29c sites, plus a new one in
ProcessTurn. Positive control: the ServerPlayer serializer scores 50/50 known
offsets.
sizeof(Game::ObservedTech) = 0x2c (44), proven three ways: the exact magic
divide 0x2e8ba2e9 sar 3 at 0x0087239f, imul reg,reg,0x2c at 0x0087243a and
0x007b735b, and the search stride add edi,0x2c at 0x007ba257.
Append site: RecordObservedTech+0xdf (0x007ba27f) --
lea ecx,[player+0x274]; call vector_ObservedTech_push_back 0x007b7320
RecordObservedTech (0x007ba1a0) is a direct callee of OnTechResearched and
de-duplicates by tech name before appending. The realloc through 0x007b5820 is
why lane R's guard saw all three vector words move. Element carries a vptr
(RTTI .?AVObservedTech@Game@@) at +0 and a 0x18-byte std::string at +0x0c; the
four on-disk ints map onto +0x04/+0x06/+0x08/+0x24/+0x28 in an order this read
does NOT determine, and is not guessed.
Also corrects harness-audit row 11: ComputeBudget has no store to Budget+0x64
(its only +0x64 accesses are loads off a different base), and ProcessResearch's
int* overbudget arg is a ProcessTurn stack local, not Budget+0x64. Agrees with
lane R's guard seeing 0 changes in 4284 calls.
Honest limits are recorded in the note and the board: this is a recall tool, not
an oracle. Class-level precision at 0x274 is ~13% by function, i.e. a ~900x
search-space cut that still needs one call-graph check. Cohort ranking must not
be used as a hard filter -- it would have discarded the correct answer here.
Ghidra writeback: labels + plate comments on RecordObservedTech,
vector_ObservedTech_push_back, ObservedTech_ctor, vector_ObservedTech_assign,
vector_44B_grow, vftable_ObservedTech.
