diff --git a/docs/B3.md b/docs/B3.md index 741ab68..c02bae8 100644 --- a/docs/B3.md +++ b/docs/B3.md @@ -1,11 +1,16 @@ # B3 — `TechTree::ProcessResearch` old-vs-new, with the RNG state as a declared region -**Status (2026-09-08): code complete, cross-built and staged; every VM step still owed.** -VM140 is held by another lane for the whole of this milestone, so nothing was deployed, the -game was not stopped or relaunched, and `C:\SOTS\binkw32.dll` / `C:\SOTS\shimdist` were not -touched. Everything below is offline work plus what the *binary* says; the run list is at the -end. The build lives in its own tree (`/srv/re-lab/build/sots-engine-b3`) and its own dist -(`/srv/re-lab/shim/dist-b3`), not the shared ones. +**Result (2026-09-08): verified on the live game.** Trace: 3 calls, `tracecmp.py` exit 0. +Compare: **15 calls, 13 with zero divergence**; the two that diverged are both completion turns +and every diff is in a field `TechTree::SetResearched` owns — the documented scope boundary — +plus one extra RNG draw on one of them (below). The RNG post-state matched on 14 of 15 calls, +including every roll. Replace: our research pass fed the game for a whole turn from +`ref-turn2.sav` and the resulting 609 KB autosave is **item-for-item identical to the oracle +except for one missing event** (`EVENT_RESEARCH_OVERBUDGET`), so the End-Turn oracle does **not** +pass yet — see "The replace-mode gap". Game left at the main menu in `hooks=trace`. + +Build `b3-b791392-20260908T0353Z`, staged in `/srv/re-lab/shim/dist-b3`, deployed to the VM from +`C:\SOTS\shimdist-b3`. Evidence in `/srv/re-lab/shim/traces/b3-*`. The point of the target: one call exercises the MT19937, the completion-odds formula and the Zuul double roll at once, so a match validates all three together — and its RNG consumption is @@ -85,10 +90,18 @@ its own milestone, and the callback writes live player state that compare mode m touch. So: > **A turn on which a tech completes is expected to diverge**, in the completing node's -> `turn_researched` / `order` and in the child nodes `SetResearched` unlocks — and in nothing -> else, including the RNG (the cascade makes no draw; verified statically, though the owner -> callback itself was not audited). A turn on which nothing completes — the overwhelmingly -> common case — must match everywhere. +> `turn_researched` / `order` and in the child nodes `SetResearched` unlocks. A turn on which +> nothing completes — the overwhelmingly common case — must match everywhere. +> *(Borne out: the two divergent calls of the live compare carry exactly those fields and +> nothing else. The prediction that the cascade also makes no RNG draw was **half wrong** — one +> of the two completions consumed an extra word inside the owner's tech-effect callback; see +> "The extra draw" below.)* + +It also does **not** post the events the pass raises — and that turned out to matter: the +over-budget branch sets `node.flag = 2` *and* pushes an `EVENT_RESEARCH_OVERBUDGET` onto the +owner's event list, and only the flag is modelled. The event list was never a declared region, +so compare mode is blind to it; the End-Turn oracle is what caught it. See "The replace-mode +gap". The effective cost of a node comes from the game's own `TechTree::Cost` (read-only: it only reads `costRP`, the def and the owner, and calls the read-only cost-multiplier helper). The @@ -197,37 +210,118 @@ staged in `/srv/re-lab/shim/dist-b3`. `tools/clean_room_check.sh` OK. 6. `TechDef`'s first word is the tech id and it indexes `TechTree+0x10`; the allocation vector's stride is 8 (`{TechDef*, int}`). Both are in the address contract now rather than inferred. -## What remains (needs the VM) +## Runs (`/srv/re-lab/shim/traces/`, build `b3-b791392-20260908T0353Z`) -The lane holding VM140 must be finished first; then, in this order: +| file | mode | calls | result | +|---|---|---|---| +| `b3-trace-golden.jsonl` | trace | 3 | `tracecmp.py` exit 0, 0 invalid | +| `b3-compare.jsonl` | compare | 15 | **13 clean, 2 diverged** (both completion turns; see below) | +| `b3-replace-autosave.sav` vs `b3-orig-autosave.sav` | replace | 3 | `(Autosave EndTurn).sav` = `bb4fd9ac…` (oracle); `(Autosave).sav` = `fd071d18…` ≠ oracle `978041ac…` | +| `b3-replace-save-diff.txt` | | | the whole difference: one missing event | +| `b3-shim.log`, `b3-final-menu.png`, `b3-compare-turn7.png` | | | banners, final state | -1. Deploy `/srv/re-lab/shim/dist-b3` (build `b3-81218c7-dirty-20260908T0311Z`): `scp` it to - `C:\SOTS\shimdist-b3\` and run `deploy.ps1 -Dist C:\SOTS\shimdist-b3` — **a separate staging - directory from the shared `C:\SOTS\shimdist`**, so the other lane's dist is not overwritten. -2. Copy `shim.cfg.b3trace` over `C:\SOTS\shim.cfg`, relaunch, load `ref-turn2.sav`, press End - Turn once, and pull `C:\SOTS\shim.trace.jsonl` → `b3-trace-golden.jsonl`. - `tracecmp.py` must exit 0 with 0 invalid records. Expect one record per player that both - has a research target and did not suffer a lab accident that turn — the caller gates the - call on `if (ResT && !RollResearchAccident())` — so a turn with no records at all is a - setup problem, not a pass. - **Read off this trace before going further:** `fpu_cw` (expect `0x027f`; `0x007f`/`0x003f` - means 24-bit precision and the `float_from_pc24` route), and, per record, `rng_left_in` - minus the after-state's `left` — 1 for a non-Zuul player who rolled, 2 for a Zuul, 0 when - the spend was zero or the tech was already at the 150 % ceiling. -3. Copy `shim.cfg.b3compare`, relaunch, load `ref-turn2.sav`, End Turn → `b3-compare.jsonl`. - Expect **0 divergences on every record where no tech completed**. On a record whose - `ours` shows a completion, the only permitted diffs are `turn_researched` / `order` on that - node and the child nodes `SetResearched` unlocked; the `rng` region must still match. Any - other diff is a real finding — report it, do not tune the formula. - To get more turns, keep pressing End Turn; each turn adds one record per player. -4. If the compare is clean: copy `shim.cfg.b3replace`, relaunch, load `ref-turn2.sav`, End Turn, - and check the determinism oracle — `(Autosave).sav` = `978041ac…`, `(Autosave EndTurn).sav` - = `bb4fd9ac…`. This is the strongest single result available: replace mode means our research - pass fed the game, and the save hash means the whole turn still landed byte for byte. -5. Restore the previous `shim.cfg` (`hooks=trace`) and leave the game at the main menu, as M1/M2 - left it. +Procedure: `ref-turn2.sav` → End Turn ×1 (trace), ×5 (compare), ×1 (replace and the oracle +re-check). Each End Turn produces three calls: the AI Tarkas player (which actually researches), +the human, and a second Tarkas tree, the last two with a zero-point allocation. -Not done, and worth saying: the compare above cannot distinguish the two unit divisors (see -"Float mapping"), and it will not exercise the Zuul path at all unless a Zuul player is in the -reference save — check `species` in the trace args, and if none is 5, run one more compare from -a Zuul save before calling the double roll verified by behaviour rather than by disassembly. +### What the compare proves + +Per call, the trace records the draw count as `left` before minus after: + +| call | species | alloc `{tech, points}` | draws (orig / ours) | outcome | +|---|---|---|---|---| +| 0 | Tarkas | {144, 2889} | 1 / 1 | over-budget event raised, flag 1 → 2 | +| 1, 2 | Human, Tarkas | {90, 0}, {9, 0} | 0 / 0 | zero spend → odds 0, roll 1, no draw | +| 3 | Tarkas | {144, 2898} | 0 / 0 | reached the 150 % ceiling → guaranteed, **no draw**, completed | +| 6 | Tarkas | {142, 3064} | 1 / 1 | rolled and failed (odds −0.078, below 50 % of cost) | +| 9 | Tarkas | {142, 3074} | **2 / 1** | rolled at odds 0.178 and **completed early** (6138/8000 = 0.767) | +| 12 | Tarkas | {9, 3086} | 1 / 1 | rolled and failed | + +Every integer the pass computes matched: progress, the spend cap, `*overbudget`, and the flag on +every node, including the three interesting branches — the over-budget event flag (call 0), the +guaranteed completion at the ceiling with no draw at all (call 3), and the "completed early" +flag 1 → 0 below 80 % of cost (call 9). On call 9 ours drew the same word as the original (the +completion decision and the early flag both match), so the extra draw is the original's, not a +missing one of ours. + +**The two divergences.** Both are completion turns, and both carry only `SetResearched`-owned +fields: `order` and `turn_researched` on the completing node, and `cost_rp` / `state` / +`turn_available` on the children it unlocks (three children on call 3, one on call 9). That is +exactly the set this milestone declared out of scope in advance. + +**The extra draw (call 9).** The original consumed one word more than ours *after* the completion +roll. Call 3 also completed a tech and consumed no extra word, so this is not "SetResearched +always draws" — it is the owner's tech-effect callback for that particular tech. That callback is +`ServerPlayer::OnTechResearched`, which is the B2 lane's target; B2 should expect to find at +least one effect that rolls. + +### The replace-mode gap — one missing event + +Replace mode ran our pass for a whole turn. `(Autosave EndTurn).sav` (the pre-turn state) is the +oracle's `bb4fd9ac…`, but the post-turn `(Autosave).sav` is `fd071d18…` and not the oracle's +`978041ac…`. Dumping both trees and normalising the offsets, the **entire** difference in 40,300 +items is: + +``` +Player > Events > Events > . > Events: one entry present in the original, absent in ours + EvEID 3 EvDsc "Research Over Budget" + EvMsg "Research for Waldo Units has gone overbudget." + EvImg "EVENT_RESEARCH_OVERBUDGET" EvLoc 0 EvPos {inf,inf,inf} EvAct 1 EvCID 0 + + the two counters that index it: EvNxID 4 → 3 and the list length 2 → 1 +``` + +Nothing else — not one number the research pass computes — differs. So the formulas are right and +the milestone under-scoped itself: `docs/B3.md` promised the over-budget *flag* and treated the +event beside it as part of the same branch without saying it was excluded. It is not modelled, +`ours` writes `node.flag = 2` and stops there, and compare mode could not see it because the +player's event list was never a declared region. + +Two consequences worth carrying forward: + +* **The oracle catches what a region-scoped compare cannot.** Compare mode was clean on exactly + the call that produced the missing event, because the event lands somewhere we did not declare. + A clean compare bounds the state you declared, nothing more. +* To close this, `ours` needs to post the event through the game's own event API (the same call + the original makes right after setting the flag), the way M2 delegates to `LoadWeapon` — the + message text is composed from the tech name, so it cannot be synthesised from our side alone. + Until then, **replace mode is not oracle-clean** and should not be used as a determinism check + for this hook. + +## Corrections this run made to the notes + +1. **The current research target has state 3, not 2 — so it does not decay.** Every funded node + in the trace is state 3 (`node[144]`, `node[142]`, `node[9]`), and the decay loop's guard is + `state == 2`. `formula-gaps.md` Q8 read the loop correctly but concluded the funded node + decays; it does not, because the tree marks the selected target with a different state. Our + code was already right (it compares against `Available == 2`); the prose was wrong. The state + set observed in one tree: 164 hidden (0), 7 parent-researched (1), 23 available (2), 1 current + target (3), 22 researched (4). +2. **`fpu_cw` = `0x127f` at call time** (`0x027f` at DLL init): precision control = 10b = **53-bit + double**, rounding = nearest. Bit 12 is the legacy infinity-control flag and has no effect. So + the MSVC default path is the right one and `next_float()` — double product, then the caller's + narrowing to float32 — models the original exactly. `float_from_pc24()` stays as an unused + contingency; nothing needs to change. +3. **Nodes per tree = 293**, not the 196 the tech-name table suggested. + +## Coverage this run did not reach + +* **No Zuul.** `ref-turn2.sav` has two players, `re` (human) and `Fane Lao` (AI), and the trees + seen are species 0 (Human) and 2 (Tarkas). The double roll is therefore still verified by + disassembly and host tests only, never by behaviour. A compare from a save with a species-5 + player would close it; the check is simply that `left` drops by 2 on a roll instead of 1. +* **The decay sweep never fired.** No node in any traced tree was state 2 with non-zero progress + — idle partially-researched techs do not occur in this save, and the current target is state 3. + `DecayAllResearch` is covered by host tests only. +* Only one lab-accident-free, single-target allocation shape was seen (one entry per call). + +## If the replace-mode gap is closed + +1. Add the event post to `ours` (see above), rebuild, restage `/srv/re-lab/shim/dist-b3`. +2. Deploy from `C:\SOTS\shimdist-b3`, copy `shim.cfg.b3replace` over `C:\SOTS\shim.cfg`, + relaunch, load `ref-turn2.sav`, End Turn once, and re-check the oracle: + `(Autosave).sav` = `978041ac…`, `(Autosave EndTurn).sav` = `bb4fd9ac…`. +3. Restore the default `shim.cfg` (`hooks=trace`) and leave the game at the main menu. + +Note that replace mode also stops modelling completions the moment one happens on a later turn: +`ours` does not run the unlock cascade, so a replace run must not be pushed past the first +completion until `SetResearched` is a milestone of its own. diff --git a/docs/game-sim.md b/docs/game-sim.md index 133e8be..ba26e3b 100644 --- a/docs/game-sim.md +++ b/docs/game-sim.md @@ -56,7 +56,7 @@ Confidence legend — **high**: formula verified in the RE notes against the cod | `ResearchCompletionOdds` | `(progress − lo) / hi` evaluated in double and **narrowed to float32**: the original stores it in a 4-byte float slot before comparing | high | | `ApplyResearchPoints` | `spend = min(points, hi − progress)` — a signed min with **no floor at zero**; `progress += spend`; below `hi`: odds as above, `roll = rand01()` (also narrowed to float32), Zuul keep the lower of two rolls, zero spend → odds 0 / roll 1; at `hi`: odds 1 / roll 0, no draw; complete iff `!(odds < roll)` compared as float32; crossing 100 % without completing → over-budget event (flag 2); completing below `(double)0.8f` of cost, where the ratio is itself a float32, → "completed early" (flag 0). Only the words the original function writes itself: the unlock cascade is `SetResearched`'s | high | | `DecayResearchProgress` | `max(0, progress − ftol(cost x (double)0.05f))` — the image holds the widened float literal `0.05000000074505806`, and there is no special case for a node whose cost is still `INT_MAX` | high | -| `DecayAllResearch` | applies to every Available node whose progress is **non-zero** (not "positive"), after the target was processed; the just-funded target is **not** excluded and only escapes by completing in the same pass (net gain of the current tech = spend − 5 % of cost) | high — confirmed against the loop | +| `DecayAllResearch` | applies to every node in state **Available (2)** whose progress is **non-zero** (not "positive"), after the allocation pass. The *current* research target is **not** among them: the tree marks the selected target with state **3**, so the funded node keeps its full gain and only idle partially-researched techs decay (live trace, `docs/B3.md`) | high — confirmed against the loop and the live trace | | `ProcessResearchTurn` | the whole per-turn pass: every allocation entry in order, then the decay sweep. This is the shape the shim hooks; see `docs/B3.md` | high | | `RollLabAccident` | `randint(100) < odds`, where `randint` is uniform on `[0, 100]` **inclusive** (see `docs/mars-rng.md`) | medium — odds-from-boost function unresolved (caller supplies odds) | | `LabAccidentLossPercent` | `ceil(clamp01(rand01() x (max − min) + min) x 100)` | high | diff --git a/include/generated/sots_addresses.h b/include/generated/sots_addresses.h index 49e54a3..7ef76c0 100644 --- a/include/generated/sots_addresses.h +++ b/include/generated/sots_addresses.h @@ -1,5 +1,5 @@ // GENERATED — do not edit. Facts about Sword of the Stars.exe (GOG 1.8.1). -// Source: sots-re ghidra/addresses.json @ ead46e2, generated 2026-09-08 by tools/gen_addresses.py +// Source: sots-re ghidra/addresses.json @ 8a6c0e8, generated 2026-09-08 by tools/gen_addresses.py // Runtime address = (uintptr_t)GetModuleHandle(NULL) + RVA (the exe is ASLR-relocated). #pragma once #include @@ -253,7 +253,7 @@ constexpr uint32_t TechTree_off_Nodes = 0x00000010; constexpr uint32_t TechNode_size = 0x00000034; // offset TechDef* def; *(int*)def is the tech id used to index TechTree_off_Nodes [verified] constexpr uint32_t TechNode_off_Def = 0x00000000; -// offset int state (0 hidden, 1 parent researched, 2 available, 4 researched); ctor writes 0 [verified] +// offset int state (0 hidden, 1 parent researched, 2 available, 3 available AND selected as the current research target, 4 researched); ctor writes 0. The decay sweep tests == 2, so a state-3 node never decays -- confirmed by the B3 live trace [verified] constexpr uint32_t TechNode_off_State = 0x00000014; // offset int costRP, INT_MAX = no researched parent yet; ctor writes 0x7fffffff [verified] constexpr uint32_t TechNode_off_CostRP = 0x00000018; diff --git a/src/game/sim/research.h b/src/game/sim/research.h index 3e41a75..2b9f86f 100644 --- a/src/game/sim/research.h +++ b/src/game/sim/research.h @@ -56,7 +56,16 @@ int TechCost(int baseCost, double multiplier); // Per-turn progress // --------------------------------------------------------------------------------------- -enum class TechState : int { Hidden = 0, ParentResearched = 1, Available = 2, Researched = 4 }; +// Node states. 3 is the selected research target: it behaves as Available everywhere except +// the decay sweep, whose guard is an equality test against Available, so the funded tech does +// not lose 5 % of its cost the turn it is funded. Confirmed on the live game (docs/B3.md). +enum class TechState : int { + Hidden = 0, + ParentResearched = 1, + Available = 2, + CurrentTarget = 3, + Researched = 4, +}; // Completion-flag values the game stamps on a node. enum class TechFlag : int { CompletedEarly = 0, Default = 1, OverBudgetNotified = 2 }; @@ -114,9 +123,9 @@ ResearchStepResult ApplyResearchPoints(ResearchNode& node, int points, Species o // CONFIDENCE: high. int DecayResearchProgress(int progress, int cost); -// Apply the decay to every Available node whose progress is non-zero. This runs after the -// current target has been processed, so the current target decays too (net gain = spend - -// 5 %). CONFIDENCE: high. +// Apply the decay to every node in state Available whose progress is non-zero. This runs after +// the allocation pass. The current target is state CurrentTarget, not Available, so it keeps +// its whole gain; only idle partially-researched techs decay. CONFIDENCE: high. void DecayAllResearch(std::vector& nodes); // ---------------------------------------------------------------------------------------