sots-re/campaign/research/research-live-record-bridge.md

229 lines
17 KiB
Markdown

# Live-record bridge readiness architecture
Status: **planning draft; contract remains needs-revision**. This is acceptance-before-implementation
for `research-live-record-bridge`, not live-safety evidence. No game, VM, allocator, constructor,
or bridge code was executed while producing it.
## 1. Bound facts and reconciliation result
The accepted dependency is the integrated static package
`verify/results/research-completion-abi-independent/integration-lead-20260910/manifest.json`
(binary `dumps/sots.exe`, SHA-256
`970b7de729956a53094c7eb98aba4270aee98e2fed5daf0d39e290013c90c841`). Its scope is static
reproduction plus archived-save inspection, not live allocator safety.
Two current generated-header statements are unsafe to consume and must be replaced by a dedicated
bridge fact channel before implementation:
* `include/generated/sots_addresses.h` says `EvDsc` is omitted from duplicate equality. The accepted
repaired windows establish the opposite: after action, location, three floats, message and image,
`0x00825d40` compares both descriptions through `0x0046f8c0`; a description-only difference is
therefore **not** a duplicate.
* The same header names `ObservedTech_copy_ctor = RVA 0x0039a184`, an interior inlined-copy site.
Fresh pinned-binary reconciliation in
`campaign/research/research-live-record-addresses.json` confirms callable helper entry VA
`0x0079a150` (RVA `0x0039a150`). It is a three-stack-argument cdecl-style helper: unused
allocator-shaped argument, destination, source; the caller removes 12 bytes after its plain
`ret`. It has no supported return-value contract. Full-image linear disassembly found exactly two
direct calls, both to `0x0079a150`; neither targets `0x0079a184`. The interior site depends on the
entry prologue's EBP, SEH, ESI and construction-state setup and is therefore an executable negative
control, never a fallback.
The dedicated package now records the ObservedTech default constructor `0x008562a0`
(`ECX=this`, no stack words, `EAX=this`, plain `ret`) and scalar-deleting destructor `0x00793610`
(`ECX=this`, one flags word, `EAX=this`, `ret 4`). Fresh raw captures and a capture manifest are at
`verify/results/research-live-record-bridge/run-a247a9d6650d9e0954596cc0/`. The constructor creates
the valid empty name string and all scalar defaults. The destructor frees a long name, resets that
string, restores base vptr `0x009e22bc`, and frees object storage only when flags bit 0 is set. Vtable
bytes at `0x00a2439c` put this destructor in slot zero; the accepted reallocation capture positively
shows old embedded elements dispatched through slot zero with a pushed zero, so bridge calls must use
flags=0 and must never repeat destruction.
The dedicated package now also records the complete PlayerEvent default constructor `0x0084ee30`
(`ECX=this`, no stack words, `EAX=this`, plain `ret`) in
`verify/results/research-live-record-bridge/run-c1238a40bfc3bfa7c2ff9ba6/`. It establishes every
logical field of the 0x74-byte object: vptr, zero ID/location/action/CID, three independent valid
empty strings, and three `FLT_MAX` position words copied from `0x00af0dc8`; inactive SSO/allocator
storage bytes are not overwritten. The fixed empty-literal/count-zero helper path allocates and
frees nothing. Direct PE decoding of the constructor's
MSVC FuncInfo records three unwind states: restore the base vptr, then destroy completed EvDsc,
then destroy completed EvMsg. The current string is not treated as completed if its constructor
helper throws. This establishes static partial-construction ownership only; no throw, allocation or
live destructor was executed.
The dedicated package now also records the complete PlayerEvent copy constructor `0x007693f0`
(`ECX=destination`, stack source, `EAX=destination`, `ret 4`) in
`verify/results/research-live-record-bridge/run-36c401e90eb0ec1db4d35865/`. It writes the vptr and
EvEID, independently empty-initializes and deep-copies EvDsc then EvMsg, copies EvLoc and all three
EvPos words, independently initializes and deep-copies EvImg, then copies EvAct and EvCID. Every
string copy uses `0x00425430` with offset zero/count `0xffffffff`; no source owning header is
transferred. Direct PE decoding binds three MSVC unwind states: base-vptr restore, completed EvDsc
destruction, then completed EvMsg destruction. Construction state advances before each potentially
allocating string assignment, so the currently failing string is not treated as complete. This is
static partial-copy ownership only; no allocation failure, throw or live destructor was executed.
The dedicated package now also records the complete PlayerEvent destructor `0x0061ae90`
(`ECX=this`, no stack words, plain `ret`, no supported return value) in
`verify/results/research-live-record-bridge/run-a4f6a9922bf5800747bfdfc6/`. It destroys the three
independent strings in reverse construction order: EvImg, EvMsg, then EvDsc. Each capacity-at-least-
`0x10` branch frees that string's owned buffer through scalar-delete thunk `0x00924faa`, and every
path resets the string to empty SSO before proceeding. Only after all three strings are empty does
the helper replace PlayerEvent vptr `0x00a21958` with base vptr `0x009e22bc`. It takes no deleting
flags and never frees enclosing PlayerEvent storage. The exact capture byte-matches both accepted
dependency and integrated independent captures. This remains static ownership evidence; no live
destruction, allocator or exception behavior was executed.
The dedicated package now also records PlayerEvent vector append `0x0086c580`
(`ECX=vector`, stack source, `ret 4`, no supported return) and reserve/growth helper
`0x00869500` in
`verify/results/research-live-record-bridge/run-e21ce0d2909d8647e6a99a7b/`. An aligned source in
the live range is reduced to an element index before growth and recomputed afterward; an external
source pointer is retained. Full capacity reserves one element, selecting at least `size+1` and
normally 1.5x capacity. Reallocator `0x00865ee0` allocates through `0x0078af40`, deep-copies the
old range through `0x00772430`, destroys old elements with deleting flags zero, frees old storage,
and only then publishes end, last and first. Append deep-copies the new element through
`0x007693f0` and publishes last only after success. Bound MSVC metadata shows that partial element
cleanup remains with the copy/range helper and the reallocator frees an unpublished new block before
continuing propagation. This is static ownership evidence; no live allocation, throw, copy or
destruction was executed.
The dedicated package now also records ObservedTech vector append `0x007b7320`
(`ECX=vector`, stack source, `ret 4`, no supported return). It distinguishes source inside the live
range from source outside it, reserves one slot through `0x007b5820` only when full, deep-copies at
the old/new last, and advances last only after normal return. Growth chooses at least
`size+1` and normally 1.5x capacity, then follows `0x007b34e0` -> `0x0057e590` -> MSVCR100 new,
deep-copies all old values, destroys the old range with flags zero, frees the old block through the
matching thunk, and publishes the new three-pointer header. Static SEH state exposes partial-copy
cleanup, but an actual throw remains unexecuted and cannot support live-safety acceptance.
The reserve window is now bound to the direct raw capture and manifest at
`verify/results/research-live-record-bridge/run-a52f7d5ebec657d8923d8402/`. Its full comparison records
that the invalidated manual transcription omitted exactly eight presentation spaces while all 49
address/opcode/mnemonic/operand rows match. Only the new 2472-byte stream is the current byte-exact
capture; the old 2464-byte stream remains invalidated and must not be cited as raw evidence.
The following accepted boundaries may seed the dedicated package, but each callable row still needs
its raw-window artifact and exact prototype in that package: PlayerEvent destructor `0x0061ae90`
(`ECX=this`, plain `ret`); get/create TurnEvents `0x00885380` (`ECX=EventStorage`, stack turn,
`EAX=bucket`, `ret 4`); TurnEvents append `0x00884cb0` (`ECX=outer vector`, stack source,
`ret 4`); TurnEvents slot-zero destructor `0x0062e120`; string assignment entry `0x00425430`
with allocating worker `0x004249a0` (`ret 8`); MSVCR100 scalar delete/new import thunks
`0x00924faa`/`0x00924fb6`. Original `RecordObservedTech`, `EventStorage::PostEvent`, and every
research completion root are forbidden.
## 2. Exclusive write set
One implementation lane owns exactly these new or modified paths in the assigned engine worktree:
* `include/generated/sots_live_record_addresses.h` (generated; never hand-maintained)
* `src/shim/live_record/{abi.h,bridge.h,bridge.cpp,fixture_entry.cpp,CMakeLists.txt}`
* top-level `CMakeLists.txt` only to add the isolated live-record targets
* `tests/shim_live_record/{CMakeLists.txt,unit_tests.cpp}`
* `tools/build-live-record-fixture.ps1`
It must not edit or link `src/shim/main.cpp`, `src/shim/hooks/research.cpp`, any research hook,
or the standalone game model. The architecture/acceptance lane owns exactly:
* `campaign/research/research-live-record-bridge.md`
* `campaign/research/research-live-record-addresses.json`
* `tools/generate_live_record_addresses.py`
* `verify/live-record-bridge/{check_package.py,expected-records.json,forbidden-symbols.txt}`
* immutable run directories below `verify/results/research-live-record-bridge/`
Contract/checkpoint mutations remain canonical campaign transactions. Any expansion of either set
requires contract revision before code changes.
## 3. Bridge-only invocation and ownership
Build a **32-bit MSVC-2010-compatible** `sots_live_record_fixture.dll`, separate from `binkw32.dll`.
A PowerShell controller starts a disposable game process without advancing a turn, loads only this
fixture DLL, invokes exported `DWORD WINAPI RunLiveRecordBridgeFixture(void*)`, and exchanges a
versioned request/result through a named file mapping. The export validates PE fingerprint/module
base and resolves only generated RVAs. The controller records loaded modules and rejects any run
where `binkw32.dll` is the campaign proxy or any forbidden decision-root address appears in the
fixture import/call audit. This route neither links nor initializes the normal shim entry point.
All owning objects stay inside the original process and one compiler/runtime family. The bridge
never transfers a `std::string` or vector header across the mapping. Requests contain scalar fields
and counted UTF-8 bytes; results contain scalar fields, copied string bytes, vector sizes/capacities,
and operation counters. Construction is field-wise through accepted constructors/assignment/copy
helpers. Append delegates to the accepted vector helper. Destruction is reverse-order, exactly once,
with scalar-delete flags zero for embedded values; only array blocks created by the compatible
original runtime are released through its matching service.
Each operation owns a journal state (`empty`, `object-constructed`, each string assigned,
`element-appended`, `result-copied`, `destroyed`). A deterministic failpoint fires **before** each
original call and unwinds only completed states. Actual MSVC allocation exceptions are caught inside
the MSVC-built DLL and converted to a result code; no C++ exception crosses the exported WINAPI
boundary. The contained-failure case is accepted only when counters show no accepted partial record,
no outstanding allocation, no mismatched family, and one destruction per completed owned value.
## 4. Required cases and accounting
The fixture package must predeclare cases for empty, spare-capacity and full-capacity vectors; SSO
and heap strings for every string field; repeated ObservedTech name update; exact event duplicate;
description-only-different event; normal destruction; and one failpoint on a long-string/growth path.
Every case records complete resulting ObservedTech, TurnEvents and PlayerEvent fields, first/last/end
offsets, event ID/order, helper call counts, allocations by family and size, destructions/frees by
object identity, failpoint, return status, forbidden-call count, and execution count. Zero cases,
missing records, or unbalanced identities fail rather than skip.
## 5. Resources, manifests, and executable gates
No resource is currently leased. Host generation/tests use the assigned paired worktrees and a
unique output directory. The 32-bit package requires an immutable compiler/linker/SDK manifest
(exact VS2010 tool binaries and hashes), generated-address JSON/header hashes, source bindings,
fixture DLL/PDB/controller hashes, original EXE/MSVCR100 hashes, expected-record fixture hash, and
command/environment manifest. Runtime uses **VM144 only** after verifying MAC/IP, console/admin
access, game/session/process state and housekeeping, then acquiring canonical lease `vm144`.
VM140 is excluded. Building on CT111 or another shared host also requires its named campaign lease.
The eventual package must make these commands literal and zero-exit (output directory replaced by a
new unique path each run):
```text
python3 tools/generate_live_record_addresses.py --input campaign/research/research-live-record-addresses.json --output <engine>/include/generated/sots_live_record_addresses.h --check
cmake -S <engine> -B <host-build> -DSOTS_LIVE_RECORD_TESTS=ON
cmake --build <host-build> --target shim_live_record_unit_tests
ctest --test-dir <host-build> -R '^shim_live_record_' --output-on-failure
powershell -NoProfile -File <engine>/tools/build-live-record-fixture.ps1 -Source <engine> -Out <win-build> -Manifest <toolchain-manifest>
powershell -NoProfile -File <run-controller> -Guest VM144 -Fixture <fixture.dll> -Cases <expected-records.json> -Out <vm-run>
python3 verify/live-record-bridge/check_package.py --engine <engine> --re <re> --package <vm-run> --require-source-match --require-binary-match --require-fixture-match --require-all-cases --require-balanced-lifetimes --require-negative-controls
```
`check_package.py` must include executable mutations for raw owning-header transfer, interior/wrong
entry, wrong convention, allocator-family mismatch, omitted cleanup, double free, incomplete fields,
forbidden roots, zero execution, missing artifacts and source/binary/fixture drift. Host success is
not live acceptance. A different verifier session must reproduce the integrated package and at least
one mutation after all evidence is bound to one integrated source manifest.
## 6. Readiness blockers and exact next probe
The contract remains needs-revision. Missing items are the generated address header and exact
prototypes for PlayerEvent append, remaining allocation, and TurnEvents calls;
verified VS2010 toolchain identity; controller and failure-containment implementation; literal VM144
identity/preflight; expected records; checker; and immutable manifests.
The `0x0079a150` versus `0x0079a184` check is complete and recorded in
`research-live-record-addresses.json`; it qualifies only the ObservedTech copy-construction row.
The shared string-assignment check is also complete in the JSON: `0x00425430` takes ECX destination
plus source/offset/count stack arguments, returns the destination in EAX, and removes 12 bytes with
`ret 0x0c`. Bridge copy sites use offset zero and count `0xffffffff`; allocation exceptions may
propagate, so the fixture boundary must contain them. The ObservedTech constructor/destructor pair is
now complete in the JSON, including virtual-slot provenance and the flags-zero embedded-object rule.
The ObservedTech vector append/growth boundary is now encoded, including both source-location
branches, growth allocation/copy/destruction, unsupported return value, and the static-only
exceptional ownership qualification. PlayerEvent default construction at `0x0084ee30` is now
complete, including exact writes across the 0x74-byte layout, three string initialization states,
ABI/return and static exceptional partial-construction cleanup. PlayerEvent copy construction at
`0x007693f0` is now complete, including the full ordered write set,
three independent deep-copy states, ABI/return and static partial-copy unwind. PlayerEvent destruction
at `0x0061ae90` is now complete, including reverse EvImg/EvMsg/EvDsc cleanup, conditional matching-
family frees, empty-SSO resets, final base-vptr transition, ABI/no-return contract and static-only
exception qualification. PlayerEvent vector append at `0x0086c580` is now complete in the JSON,
including source-location branches, reserve capacity selection, range relocation, copy/destruction
order, header publication, ABI/no-return behavior and static-only partial-copy unwind. The next
discriminating static check is EventStorage get/create TurnEvents at `0x00885380`: bind its complete
callable and helper windows and encode last-match lookup, empty-bucket construction, append lifetime,
returned pointer stability and turn publication order.
Do not begin bridge implementation until every exposed ABI row is encoded in the dedicated
generated-address package.