alex/sots-re
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
sots-re/campaign/pilots/research-replacement.md

21 KiB
Raw Blame History

Proposed pilot: completion-bearing research replacement

Status: proposed. This is a contract design, not replacement acceptance. Current architecture owner is Astra pilot architect in the architecture-review role on openai/gpt-5.6-sol; subsequent implementation and independent verification use distinct campaign executions. The target is a bounded replacement of Game::TechTree::ProcessResearch, its SetResearched cascade and required completion callback writes on explicitly certified workloads. Broad research displacement remains unproved. No new live address, remote deployment or dependency policy is authorized by this document.

1. Evidence inspected independently (2026-09-09)

Sources: findings/subsystems/research-replace.md; archived CR compare JSON, raw compressed traces, shim logs and saves; current engine research, tech-effect and event interfaces. Archived engine build is cr-618ccb1-20260909T131556Z, not the rollout's current source snapshot.

Recomputed SHA-256:

RE-relative file bytes SHA-256
verify/results/saves/turn3-state.sav 67219 978041acd168b56ed8eb3f5e42e78d5e70eae6e6517d75e659a5eb7ca3d60921
verify/results/saves/cr/cr-oracle-endturn.sav 67212 e00eed0c03a31d27a81b7470a9dcc9ba08a2ac48c20baeee4f34749164743e3f
verify/results/saves/cr/cr-oracle-autosave.sav 67811 79df50475a7b83afa927d992b9f030dcf45710f4bda0133b8b1fa4800a72e420
verify/results/saves/cr/cr-replace0-autosave.sav 67511 6b51db992b158caa5424d71b2dccf72924af7198b4165bfcfc870e0d438fb6d5
verify/results/saves/cr/cr-replace1-autosave.sav 67537 8a4309ee4fe0b3177a2820600b5016c7c256d0f51b065df469ecd0b4f2342235

Local reproduction (diagnostic tool exit alone is not an equality verdict):

python3 verify/state-checksum/state_checksum.py verify/results/saves/cr/cr-oracle-autosave.sav verify/results/saves/cr/cr-replace1-autosave.sav --limit 40

This reports 16 exact-bit, unmasked leaf differences. Oracle -> R1: OutMod 1.25 -> 1.1; ConMod[0..2] 0.9 -> 1; ResTNm empty -> IND_Waldo; EvNxID 7 -> 5; events 5/6 absent and turn-bucket count 3 -> 1; observed-tech element 11 absent and count 11 -> 10; BnkPr/BnkEl, RepCur/RepMax and Summary.Checksum differ. These are five primary scalar fields, collection writes/counts and five derived leaves. They are not permission to patch five downstream values.

verify/results/compare/cr-compare.json:289-357 reports partial coverage: three comparisons, six undeclared spans in one call and eight unmodelled notes. In verify/traces/cr-R1.jsonl.gz, call 1 is replace, allocation tech 144 / 2898, input turn 4, order counter 22, observed count 10, roll_pending false, RNG left 413 and CW 4735 (0x127f). Its RNG digest/index/left are unchanged. Logs verify/results/shim/cr/cr-R1.log:116-118 count one completion and three unlocks, then two zero-point calls. otch_appends=1 counts a decision; it does not mean an element was constructed live.

Corrections to inherited narrative

  1. R1 log lines 75-94 show seven drawsite detours and twelve probes in addition to the six sites enumerated in CR findings. Instrumentation is broader than the template-hook config. Every fresh run needs a complete installed-site manifest, not an “exhaustive config” assertion.
  2. R1 runtime CW is 0x127f, including the research trace argument; initialization prints 0x027f. Preserve the full measured word and precision/rounding separately.
  3. Runtime string lookup is already an engine interface (research_events.h:95-105, app/event_phase.cpp:10-14). The absence of bundled game prose does not establish that original PostEvent is mandatory. Runtime asset loading is a concrete route to investigate.
  4. shim/hooks/tech_effects.cpp:354-365 already writes player state/design masks and delegates node-bore updates in replace mode. The research hook does not integrate that complete callback; calling the helper “host-tested” is not evidence its full live boundary is implemented.

2. Workloads and access paths

W1 — primary positive workload, exact LOAD route

On a future explicitly leased lab with operator-supplied executable/data paths: restore the hashed turn3 input as the only selected input save; start a fresh process; use Load Game, select the input, enter its lobby/map at Turn 3, then one End Turn to Turn 4. Verify UI/state transitions rather than sleep-and-assume. Preserve both output saves before any reset. Bind save selection, route, process identity, executable hash, assets, shim hash, config and instrumentation to evidence.

Historical predictions to re-certify before measuring candidate results:

  • Exactly three research calls, owners 32/496/512 in order, allocations 144:2898, 90:0, 9:0.
  • One IND_Waldo completion: progress 5768 -> 7500, state 3 -> 4, turn_researched -1 -> 4, order -1 -> 22, counter 22 -> 23, refund 1166. Flag unchanged.
  • Nodes 132/136/142 unlock, costs 10000/16000/8000, available turn 4. Save tree slots are 94/98/104; tech 144 is slot 106. Never equate tech id with serialized slot.
  • One full ObservedTech record, two full events (completion followed by unlocked-techs), event next id 5 -> 7; primary effect fields reach oracle values; target clears.
  • Zero RNG words in this workload. This is a negative RNG control, not evidence for draws.

W0 — regression control, insufficient for completion acceptance

verify/results/saves/turn2-state.sav, one End Turn via LOAD, reaches the turn3 input above. This exercises the research pass/overbudget event and a RNG word, but zero completions. Re-certify its two-process oracle independently and pin its input hash before use. Do not substitute W0 for W1, or use zero-allocation calls as the positive execution threshold.

W2 — path to broader completion evidence (not yet certified)

Select a different, archived completion-bearing input from the unlock continuations, or generate one under an approved lab contract; record the input hash and exact LOAD/continuation route before prediction. Require a different tech/effect family, a positive RNG draw and a second completion that observes the advanced order counter. The existing U/V evidence motivates selection but is not a ready executable workload contract. Separate controls are required for that exact route. Until W2 exists, any accepted pilot claim is restricted to W1 plus its declared regression checks.

3. Full write boundary and responsibilities

Boundary Required state / behavior Existing interface and current gap
Research pass all node progress/state/flag writes; signed capped spend and refund accumulator; decay of other Available nodes; RNG state, left and next index sim::ProcessResearchTurn, ResearchCompletionHook; current CR pass writes live
Unlock cascade node costRP, availability/researched turn, order; tree order counter; child/prerequisite availability; recursive zero-cost completions and ordering sim::TechGraph / SetResearched; replace cascade opt-in exists, transcription failures can currently skip it
Completion callback target pointer/ResTNm; pending-roll flags; every applicable player economy float, masks, boolean/species flags, design-option masks; node-bore storage/lifetime ApplyTechCompletion, RunCompletionTail, tfx::ReadPlayerState/WritePlayerState; integrate with research on pre-call inputs, avoid already-researched guard
Observed technology dedup/update semantics, entire 0x2c record (name, first/last turns, detected, with), vector content/count/capacity and allocator ownership research hook currently alters scratch span only; full live construction and update semantics need evidence
Events complete turn buckets and ordered records: id, summary, message, location, position, image, action, chainId; next id; dedup and pruning; all owned allocations events::EventStorage, PostResearchPassEvents; research hook is count-only/compare-only; runtime text and live ABI adapter missing
Callback outcomes beyond player owned-system AI flags/caps/addiction/plague; ships' plague state; recursive Zuul tech grants; pending research-event RNG and plague/rebellion consequences including allocations/cancel research TechApplyOutcome reports work, does not execute it; broader closure remains incomplete
Downstream observation budget/bankruptcy/repair/checksum consequences of primary effects let ordinary turn processing derive them; full oracle must expose them, never harvest original post-state or hardcode expected leaves

The W1 observed write set is a subset of this boundary. The complete callback has conditional system/ship/object writes absent on W1. A narrow pilot must preflight its supported class and fail closed before mutating on unsupported branches. Suppressed writes, unavailable graph/text, unknown pointers, unreadable names, failed allocation, or unmodelled fired rolls are blockers, not successful no-ops. General displacement requires separate evidence for all reachable effects.

Events require exact formatting/dedup order, FLT_MAX no-position values, action conversion, the original prune-window behavior and preexisting event contents. KeylessEventText substitutes tokens for prose and must never be serialized as a passing oracle. Completion message length above 255 has an explicitly documented divergence/undefined-original boundary; exclude or resolve it explicitly. Seed comparisons from pre-call state, not the original callback's output.

4. Inputs and original dependencies

Required immutable input manifest before ready:

  • Exact engine and RE baseline commits plus actual source-content hashes; paired worktrees; compiler/toolchain/build flags, host and shim binaries, generated-address provenance.
  • The hashed W1 input and oracle archive above, plus selected W0/W2 inputs as applicable.
  • Operator-owned game executable and proxy originals with hashes. The archived trace reports exe SHA-256 970b7de729956a53094c7eb98aba4270aee98e2fed5daf0d39e290013c90c841; remeasure it.
  • Explicit runtime data root, manifest of tech definitions/graph, tuning/constants/species inputs, localized string-table files and locale/encoding/load order. Existing parser data must come from user assets, not copies of game prose embedded in source. Missing table/key is not empty text for acceptance. Asset paths and hashes are not yet supplied for this pilot.
  • Full configs and actually installed interceptors (template hooks, init, FPU sampling, probes, drawsites), FPU CW per call, input route/process-reset record, output/log/trace file hashes.

Original dependency ledger (each needs a lead decision and explicit qualification):

  1. Existing: research calls original TechTree::Cost for effective cost and cascade cost lookup (shim/hooks/research.cpp:568-573,916-919). Formula and applicable bonus-tech discovery are not displaced by this pilot merely because a host formula exists.
  2. Existing separate B2 hook: original has-researched lookup and node-bore updater (tech_effects.cpp:325-328,358-365). Node-bore updater allocates/frees state; it is not read-only despite the source comment's analogy. Reuse requires disclosure and effect accounting.
  3. Proposed, undecided: original string lookup / allocator / event posting helpers versus a runtime-asset text adapter and compatible live allocation. No selected policy or guessed address.
  4. Outside replaced root: original turn driver, budget allocator and remaining simulation, asset loading and save serializer still run in the shim experiment. Whole standalone turn equality is a different acceptance axis and cannot be inferred from this partial replacement.

5. Executable acceptance to provide before ready

The contract must eventually name real, versioned commands and dependencies for this sequence; the following are precise requirements, not a presently implemented runner:

  1. Fresh full gate on hashed source/inputs, expected test identities, positive corpus execution, asset-dependent tests and shim build. A host-only pass is insufficient. Bind full gate manifest and resulting binary to every experiment and reject source/input changes.
  2. Two fresh-process C1/C2 controls with hooks off, same proxy, input, assets and exact W1 route; both output pairs equal byte-for-byte. Archive independent process logs; one saved oracle copy is not independent proof that two processes agreed.
  3. Compare-neutral N with the candidate binary and complete instrumentation manifest; both output saves byte-equal controls; positive W1 counters and decoded state. Zero compared calls, partial required-region coverage or undeclared required writes cannot pass.
  4. Replace R, same baseline and binary, bypass original ProcessResearch and required displaced callback implementation (count explicit original helper calls separately). Require exactly one W1 completion/three unlocks, real element/event writes, zero failures/unreadable inputs, and full declared state validation including nonserialized order counter and RNG state.
  5. Require post-turn autosave byte equality, inflated equality and exact-bit unmasked state equality independently, with reader reconstruction coverage. Pre-turn EndTurn equality alone cannot satisfy acceptance. No masks for event text, callback fields or derived discrepancies.
  6. Negative controls must fail: archived R0 (27 differences) and R1 (16); zero-completion W0 cannot satisfy W1 threshold; omit callback/event/ObservedTech write or use wrong input hash; remove a trace/log/output; mark unknown RNG as zero. Checker rejects all.
  7. Independently assigned verifier re-runs the checks and ties verdict to contract, source, binary, asset and evidence digests. Implementation worker cannot self-certify acceptance. Integration requires revalidation of integrated source; changed inputs invalidate evidence.

W2 and branch-specific tests must be added before expanding the claim to RNG-consuming, Zuul/early-completion, nonzero-decay, zero-cost recursion, prerequisite/no-auto-available, re-observed/deduplicated, temperance, plague or rebellion cases. Their absence is visible coverage debt, not satisfied execution. Local simulation tests alone do not establish live allocator safety.

5.1 Acceptance interface fixed before implementation (2026-09-10)

The accepted research-completion-abi contract closes the static record/container dependency only. It does not close live allocation safety or any replacement criterion. The pilot checker and run driver do not yet exist, so the commands below are required interfaces and remain blockers, not executable claims:

python3 tools/research_replacement.py preflight --package MANIFEST.json --workload W1
python3 tools/gate.py --engine ENGINE --corpus CORPUS --out NEW_GATE_DIR \
  --profile full --data DATA --shim --sots-input SOTS_EXE=EXE --sots-input SOTS_PROXY=PROXY
python3 tools/research_replacement.py certify-controls --package MANIFEST.json --workload W1 \
  --c1 C1_DIR --c2 C2_DIR --compare N_DIR
python3 tools/research_replacement.py check-replace --package MANIFEST.json --workload W1 \
  --replace R_DIR --oracle C1_DIR
python3 tools/research_replacement.py negative-controls --package MANIFEST.json \
  --replace0 verify/results/saves/cr/cr-replace0-autosave.sav \
  --replace1 verify/results/saves/cr/cr-replace1-autosave.sav --w0 W0_DIR

MANIFEST.json must bind paired source manifests, the full-gate manifest and shim binary, owner executable/proxy, generated addresses, data root and every consumed tech/tuning/species/string file, locale/encoding/load order, exact configs and installed-site manifest, W1/W0 saves, process route, and output/log/trace hashes. Each command must fail on an absent or extra required artifact, hash or source drift, stale output directory, zero required execution, unexpected skip, unreadable input, unsupported branch, or unknown RNG. check-replace must separately report file bytes, inflated bytes, exact-bit typed state, reader coverage, nonserialized order counter, complete RNG state and actual words, callback/helper counters, allocations and complete event/ObservedTech elements.

Before any implementation contract can become ready, an independent verifier must turn these interfaces into an executable falsifier plan covering archived R0 and R1, a wrong W1 hash, each missing callback/event/ObservedTech write, an omitted output/log/trace/site manifest, W0 presented as the positive, unknown RNG presented as zero, original displaced-root fallback, and same-HEAD source-byte drift. Verifier and implementer executions must be different sessions.

5.2 Bounded W1 architecture and dependency decisions

The replacement is one ordered coordinator with a dry-run preflight, not three independently acceptable leaf patches. Preflight reads all three owner calls and computes the complete W1 plan on scratch state before the first live write. It must positively identify the exact owner/allocation sequence, one tech-144 IND_Waldo completion, the three unlocks, no pending research roll, no unmodelled conditional system/ship/Zuul/plague/temperance/rebellion effect, and an explicit outcome for every recursive-completion and decay branch. The eventual immutable W1 manifest must state the certified expected outcomes; any mismatch is unsupported and blocks before mutation. This is a W1 classifier, not a general research implementation.

The coordinator then preserves original order: research arithmetic; SetResearched node/order writes; full ObservedTech record operation; matched-target handling; full completion event; W1 player effects and common tail; unlocked-tech event; remaining pass writes. It must use pre-call state for all decisions. Allocation failure after mutation is an observed failed run and cannot pass; this plan does not claim transactional rollback that the original does not provide.

Original-helper policy for this bounded pilot is explicit:

  1. Retain and count TechTree::Cost for effective and cascade costs. Independent cost replacement is excluded from the claim.
  2. Retain and count TechTree::HasResearched for the common callback tail and design masks. The replacement owns the resulting mask and field writes; the helper is lookup only.
  3. Retain and count ServerPlayer::UpdateNodeBoreParams at its original common-tail position. It may allocate/free through the game's CRT even when W1 does not select a bore tech. Independent node-bore maintenance is excluded.
  4. Use the original MSVCR100 scalar new/delete family and accepted record/container ABI as a live memory bridge, with every allocation/copy/destruction operation counted. This is original-runtime allocation assistance, not standalone allocator replacement.
  5. Do not call original ProcessResearch, SetResearched, OnTechResearched, RecordObservedTech, EventStorage::PostEvent, or original string lookup in replace mode. Runtime owner assets must supply localized text to an engine-side lookup; replacement code owns record contents, event IDs/order/dedup/pruning and callback effects. Keyless text is forbidden.
  6. The original turn driver, budget allocation, asset loading and save serializer remain outside the replaced root and must be disclosed separately from retained helpers.

Static ABI acceptance unlocks exactly one preparatory dependency: a live record-bridge contract must exercise ObservedTech, TurnEvents and PlayerEvent operations with empty/spare/full capacity, short/long strings, repeated observed names, exact duplicate and description-only-different events, normal destruction and a recorded failure path, under a lease. It owns only the bridge and fixtures and may not wire research replacement. Only after independent acceptance of that bridge may one complete W1 coordinator implementation own src/shim/hooks/research.{cpp,h} plus narrowly named bridge/effect/event files and tests. No two workers may concurrently own those files or the same VM/build directory.

6. Current blockers / stop conditions

  • No executable pilot-specific acceptance runner or full input/asset manifest. W2 is not required for the W1-only pilot but remains required before any broader claim.
  • Live research callback integration, complete ObservedTech/event construction, broader callback outcomes and unsupported-path preflight remain unimplemented/unverified.
  • Static record/container ABI is accepted, but the live record bridge and its independent runtime safety evidence are missing. Retained Cost/HasResearched/node-bore/CRT helpers and forbidden displaced roots are fixed above; runtime asset paths and adapter execution remain unresolved.
  • Archived C1/C2 equality is documented but independent raw control-run bundles are not established by the single oracle pair inspected here. Fresh controls are mandatory.
  • Instrumentation and runtime FPU narrative need correction in acceptance inputs; archives are historical observations, not the current candidate's attestation.
  • Stop on control disagreement, silent fallback, missing data, branch outside approved class, unexpected writes/draws, stale evidence, source drift, or scope/architecture surprises; persist checkpoint and escalate to Astra before changing the contract.

This proposal authorizes planning only. Keep lifecycle proposed until required dependencies, input manifests, bounded implementation scope and executable acceptance exist.