merge b3 live verification; regenerate header
This commit is contained in:
alex
commit
646e4e8bb2
4 changed files with 151 additions and 48 deletions
176
docs/B3.md
176
docs/B3.md
|
|
@ -1,11 +1,16 @@
|
|||
# B3 — `TechTree::ProcessResearch` old-vs-new, with the RNG state as a declared region
|
||||
|
||||
**Status (2026-09-08): code complete, cross-built and staged; every VM step still owed.**
|
||||
VM140 is held by another lane for the whole of this milestone, so nothing was deployed, the
|
||||
game was not stopped or relaunched, and `C:\SOTS\binkw32.dll` / `C:\SOTS\shimdist` were not
|
||||
touched. Everything below is offline work plus what the *binary* says; the run list is at the
|
||||
end. The build lives in its own tree (`/srv/re-lab/build/sots-engine-b3`) and its own dist
|
||||
(`/srv/re-lab/shim/dist-b3`), not the shared ones.
|
||||
**Result (2026-09-08): verified on the live game.** Trace: 3 calls, `tracecmp.py` exit 0.
|
||||
Compare: **15 calls, 13 with zero divergence**; the two that diverged are both completion turns
|
||||
and every diff is in a field `TechTree::SetResearched` owns — the documented scope boundary —
|
||||
plus one extra RNG draw on one of them (below). The RNG post-state matched on 14 of 15 calls,
|
||||
including every roll. Replace: our research pass fed the game for a whole turn from
|
||||
`ref-turn2.sav` and the resulting 609 KB autosave is **item-for-item identical to the oracle
|
||||
except for one missing event** (`EVENT_RESEARCH_OVERBUDGET`), so the End-Turn oracle does **not**
|
||||
pass yet — see "The replace-mode gap". Game left at the main menu in `hooks=trace`.
|
||||
|
||||
Build `b3-b791392-20260908T0353Z`, staged in `/srv/re-lab/shim/dist-b3`, deployed to the VM from
|
||||
`C:\SOTS\shimdist-b3`. Evidence in `/srv/re-lab/shim/traces/b3-*`.
|
||||
|
||||
The point of the target: one call exercises the MT19937, the completion-odds formula and the
|
||||
Zuul double roll at once, so a match validates all three together — and its RNG consumption is
|
||||
|
|
@ -85,10 +90,18 @@ its own milestone, and the callback writes live player state that compare mode m
|
|||
touch. So:
|
||||
|
||||
> **A turn on which a tech completes is expected to diverge**, in the completing node's
|
||||
> `turn_researched` / `order` and in the child nodes `SetResearched` unlocks — and in nothing
|
||||
> else, including the RNG (the cascade makes no draw; verified statically, though the owner
|
||||
> callback itself was not audited). A turn on which nothing completes — the overwhelmingly
|
||||
> common case — must match everywhere.
|
||||
> `turn_researched` / `order` and in the child nodes `SetResearched` unlocks. A turn on which
|
||||
> nothing completes — the overwhelmingly common case — must match everywhere.
|
||||
> *(Borne out: the two divergent calls of the live compare carry exactly those fields and
|
||||
> nothing else. The prediction that the cascade also makes no RNG draw was **half wrong** — one
|
||||
> of the two completions consumed an extra word inside the owner's tech-effect callback; see
|
||||
> "The extra draw" below.)*
|
||||
|
||||
It also does **not** post the events the pass raises — and that turned out to matter: the
|
||||
over-budget branch sets `node.flag = 2` *and* pushes an `EVENT_RESEARCH_OVERBUDGET` onto the
|
||||
owner's event list, and only the flag is modelled. The event list was never a declared region,
|
||||
so compare mode is blind to it; the End-Turn oracle is what caught it. See "The replace-mode
|
||||
gap".
|
||||
|
||||
The effective cost of a node comes from the game's own `TechTree::Cost` (read-only: it only
|
||||
reads `costRP`, the def and the owner, and calls the read-only cost-multiplier helper). The
|
||||
|
|
@ -197,37 +210,118 @@ staged in `/srv/re-lab/shim/dist-b3`. `tools/clean_room_check.sh` OK.
|
|||
6. `TechDef`'s first word is the tech id and it indexes `TechTree+0x10`; the allocation vector's
|
||||
stride is 8 (`{TechDef*, int}`). Both are in the address contract now rather than inferred.
|
||||
|
||||
## What remains (needs the VM)
|
||||
## Runs (`/srv/re-lab/shim/traces/`, build `b3-b791392-20260908T0353Z`)
|
||||
|
||||
The lane holding VM140 must be finished first; then, in this order:
|
||||
| file | mode | calls | result |
|
||||
|---|---|---|---|
|
||||
| `b3-trace-golden.jsonl` | trace | 3 | `tracecmp.py` exit 0, 0 invalid |
|
||||
| `b3-compare.jsonl` | compare | 15 | **13 clean, 2 diverged** (both completion turns; see below) |
|
||||
| `b3-replace-autosave.sav` vs `b3-orig-autosave.sav` | replace | 3 | `(Autosave EndTurn).sav` = `bb4fd9ac…` (oracle); `(Autosave).sav` = `fd071d18…` ≠ oracle `978041ac…` |
|
||||
| `b3-replace-save-diff.txt` | | | the whole difference: one missing event |
|
||||
| `b3-shim.log`, `b3-final-menu.png`, `b3-compare-turn7.png` | | | banners, final state |
|
||||
|
||||
1. Deploy `/srv/re-lab/shim/dist-b3` (build `b3-81218c7-dirty-20260908T0311Z`): `scp` it to
|
||||
`C:\SOTS\shimdist-b3\` and run `deploy.ps1 -Dist C:\SOTS\shimdist-b3` — **a separate staging
|
||||
directory from the shared `C:\SOTS\shimdist`**, so the other lane's dist is not overwritten.
|
||||
2. Copy `shim.cfg.b3trace` over `C:\SOTS\shim.cfg`, relaunch, load `ref-turn2.sav`, press End
|
||||
Turn once, and pull `C:\SOTS\shim.trace.jsonl` → `b3-trace-golden.jsonl`.
|
||||
`tracecmp.py` must exit 0 with 0 invalid records. Expect one record per player that both
|
||||
has a research target and did not suffer a lab accident that turn — the caller gates the
|
||||
call on `if (ResT && !RollResearchAccident())` — so a turn with no records at all is a
|
||||
setup problem, not a pass.
|
||||
**Read off this trace before going further:** `fpu_cw` (expect `0x027f`; `0x007f`/`0x003f`
|
||||
means 24-bit precision and the `float_from_pc24` route), and, per record, `rng_left_in`
|
||||
minus the after-state's `left` — 1 for a non-Zuul player who rolled, 2 for a Zuul, 0 when
|
||||
the spend was zero or the tech was already at the 150 % ceiling.
|
||||
3. Copy `shim.cfg.b3compare`, relaunch, load `ref-turn2.sav`, End Turn → `b3-compare.jsonl`.
|
||||
Expect **0 divergences on every record where no tech completed**. On a record whose
|
||||
`ours` shows a completion, the only permitted diffs are `turn_researched` / `order` on that
|
||||
node and the child nodes `SetResearched` unlocked; the `rng` region must still match. Any
|
||||
other diff is a real finding — report it, do not tune the formula.
|
||||
To get more turns, keep pressing End Turn; each turn adds one record per player.
|
||||
4. If the compare is clean: copy `shim.cfg.b3replace`, relaunch, load `ref-turn2.sav`, End Turn,
|
||||
and check the determinism oracle — `(Autosave).sav` = `978041ac…`, `(Autosave EndTurn).sav`
|
||||
= `bb4fd9ac…`. This is the strongest single result available: replace mode means our research
|
||||
pass fed the game, and the save hash means the whole turn still landed byte for byte.
|
||||
5. Restore the previous `shim.cfg` (`hooks=trace`) and leave the game at the main menu, as M1/M2
|
||||
left it.
|
||||
Procedure: `ref-turn2.sav` → End Turn ×1 (trace), ×5 (compare), ×1 (replace and the oracle
|
||||
re-check). Each End Turn produces three calls: the AI Tarkas player (which actually researches),
|
||||
the human, and a second Tarkas tree, the last two with a zero-point allocation.
|
||||
|
||||
Not done, and worth saying: the compare above cannot distinguish the two unit divisors (see
|
||||
"Float mapping"), and it will not exercise the Zuul path at all unless a Zuul player is in the
|
||||
reference save — check `species` in the trace args, and if none is 5, run one more compare from
|
||||
a Zuul save before calling the double roll verified by behaviour rather than by disassembly.
|
||||
### What the compare proves
|
||||
|
||||
Per call, the trace records the draw count as `left` before minus after:
|
||||
|
||||
| call | species | alloc `{tech, points}` | draws (orig / ours) | outcome |
|
||||
|---|---|---|---|---|
|
||||
| 0 | Tarkas | {144, 2889} | 1 / 1 | over-budget event raised, flag 1 → 2 |
|
||||
| 1, 2 | Human, Tarkas | {90, 0}, {9, 0} | 0 / 0 | zero spend → odds 0, roll 1, no draw |
|
||||
| 3 | Tarkas | {144, 2898} | 0 / 0 | reached the 150 % ceiling → guaranteed, **no draw**, completed |
|
||||
| 6 | Tarkas | {142, 3064} | 1 / 1 | rolled and failed (odds −0.078, below 50 % of cost) |
|
||||
| 9 | Tarkas | {142, 3074} | **2 / 1** | rolled at odds 0.178 and **completed early** (6138/8000 = 0.767) |
|
||||
| 12 | Tarkas | {9, 3086} | 1 / 1 | rolled and failed |
|
||||
|
||||
Every integer the pass computes matched: progress, the spend cap, `*overbudget`, and the flag on
|
||||
every node, including the three interesting branches — the over-budget event flag (call 0), the
|
||||
guaranteed completion at the ceiling with no draw at all (call 3), and the "completed early"
|
||||
flag 1 → 0 below 80 % of cost (call 9). On call 9 ours drew the same word as the original (the
|
||||
completion decision and the early flag both match), so the extra draw is the original's, not a
|
||||
missing one of ours.
|
||||
|
||||
**The two divergences.** Both are completion turns, and both carry only `SetResearched`-owned
|
||||
fields: `order` and `turn_researched` on the completing node, and `cost_rp` / `state` /
|
||||
`turn_available` on the children it unlocks (three children on call 3, one on call 9). That is
|
||||
exactly the set this milestone declared out of scope in advance.
|
||||
|
||||
**The extra draw (call 9).** The original consumed one word more than ours *after* the completion
|
||||
roll. Call 3 also completed a tech and consumed no extra word, so this is not "SetResearched
|
||||
always draws" — it is the owner's tech-effect callback for that particular tech. That callback is
|
||||
`ServerPlayer::OnTechResearched`, which is the B2 lane's target; B2 should expect to find at
|
||||
least one effect that rolls.
|
||||
|
||||
### The replace-mode gap — one missing event
|
||||
|
||||
Replace mode ran our pass for a whole turn. `(Autosave EndTurn).sav` (the pre-turn state) is the
|
||||
oracle's `bb4fd9ac…`, but the post-turn `(Autosave).sav` is `fd071d18…` and not the oracle's
|
||||
`978041ac…`. Dumping both trees and normalising the offsets, the **entire** difference in 40,300
|
||||
items is:
|
||||
|
||||
```
|
||||
Player > Events > Events > . > Events: one entry present in the original, absent in ours
|
||||
EvEID 3 EvDsc "Research Over Budget"
|
||||
EvMsg "Research for Waldo Units has gone overbudget."
|
||||
EvImg "EVENT_RESEARCH_OVERBUDGET" EvLoc 0 EvPos {inf,inf,inf} EvAct 1 EvCID 0
|
||||
+ the two counters that index it: EvNxID 4 → 3 and the list length 2 → 1
|
||||
```
|
||||
|
||||
Nothing else — not one number the research pass computes — differs. So the formulas are right and
|
||||
the milestone under-scoped itself: `docs/B3.md` promised the over-budget *flag* and treated the
|
||||
event beside it as part of the same branch without saying it was excluded. It is not modelled,
|
||||
`ours` writes `node.flag = 2` and stops there, and compare mode could not see it because the
|
||||
player's event list was never a declared region.
|
||||
|
||||
Two consequences worth carrying forward:
|
||||
|
||||
* **The oracle catches what a region-scoped compare cannot.** Compare mode was clean on exactly
|
||||
the call that produced the missing event, because the event lands somewhere we did not declare.
|
||||
A clean compare bounds the state you declared, nothing more.
|
||||
* To close this, `ours` needs to post the event through the game's own event API (the same call
|
||||
the original makes right after setting the flag), the way M2 delegates to `LoadWeapon` — the
|
||||
message text is composed from the tech name, so it cannot be synthesised from our side alone.
|
||||
Until then, **replace mode is not oracle-clean** and should not be used as a determinism check
|
||||
for this hook.
|
||||
|
||||
## Corrections this run made to the notes
|
||||
|
||||
1. **The current research target has state 3, not 2 — so it does not decay.** Every funded node
|
||||
in the trace is state 3 (`node[144]`, `node[142]`, `node[9]`), and the decay loop's guard is
|
||||
`state == 2`. `formula-gaps.md` Q8 read the loop correctly but concluded the funded node
|
||||
decays; it does not, because the tree marks the selected target with a different state. Our
|
||||
code was already right (it compares against `Available == 2`); the prose was wrong. The state
|
||||
set observed in one tree: 164 hidden (0), 7 parent-researched (1), 23 available (2), 1 current
|
||||
target (3), 22 researched (4).
|
||||
2. **`fpu_cw` = `0x127f` at call time** (`0x027f` at DLL init): precision control = 10b = **53-bit
|
||||
double**, rounding = nearest. Bit 12 is the legacy infinity-control flag and has no effect. So
|
||||
the MSVC default path is the right one and `next_float()` — double product, then the caller's
|
||||
narrowing to float32 — models the original exactly. `float_from_pc24()` stays as an unused
|
||||
contingency; nothing needs to change.
|
||||
3. **Nodes per tree = 293**, not the 196 the tech-name table suggested.
|
||||
|
||||
## Coverage this run did not reach
|
||||
|
||||
* **No Zuul.** `ref-turn2.sav` has two players, `re` (human) and `Fane Lao` (AI), and the trees
|
||||
seen are species 0 (Human) and 2 (Tarkas). The double roll is therefore still verified by
|
||||
disassembly and host tests only, never by behaviour. A compare from a save with a species-5
|
||||
player would close it; the check is simply that `left` drops by 2 on a roll instead of 1.
|
||||
* **The decay sweep never fired.** No node in any traced tree was state 2 with non-zero progress
|
||||
— idle partially-researched techs do not occur in this save, and the current target is state 3.
|
||||
`DecayAllResearch` is covered by host tests only.
|
||||
* Only one lab-accident-free, single-target allocation shape was seen (one entry per call).
|
||||
|
||||
## If the replace-mode gap is closed
|
||||
|
||||
1. Add the event post to `ours` (see above), rebuild, restage `/srv/re-lab/shim/dist-b3`.
|
||||
2. Deploy from `C:\SOTS\shimdist-b3`, copy `shim.cfg.b3replace` over `C:\SOTS\shim.cfg`,
|
||||
relaunch, load `ref-turn2.sav`, End Turn once, and re-check the oracle:
|
||||
`(Autosave).sav` = `978041ac…`, `(Autosave EndTurn).sav` = `bb4fd9ac…`.
|
||||
3. Restore the default `shim.cfg` (`hooks=trace`) and leave the game at the main menu.
|
||||
|
||||
Note that replace mode also stops modelling completions the moment one happens on a later turn:
|
||||
`ours` does not run the unlock cascade, so a replace run must not be pushed past the first
|
||||
completion until `SetResearched` is a milestone of its own.
|
||||
|
|
|
|||
|
|
@ -56,7 +56,7 @@ Confidence legend — **high**: formula verified in the RE notes against the cod
|
|||
| `ResearchCompletionOdds` | `(progress − lo) / hi` evaluated in double and **narrowed to float32**: the original stores it in a 4-byte float slot before comparing | high |
|
||||
| `ApplyResearchPoints` | `spend = min(points, hi − progress)` — a signed min with **no floor at zero**; `progress += spend`; below `hi`: odds as above, `roll = rand01()` (also narrowed to float32), Zuul keep the lower of two rolls, zero spend → odds 0 / roll 1; at `hi`: odds 1 / roll 0, no draw; complete iff `!(odds < roll)` compared as float32; crossing 100 % without completing → over-budget event (flag 2); completing below `(double)0.8f` of cost, where the ratio is itself a float32, → "completed early" (flag 0). Only the words the original function writes itself: the unlock cascade is `SetResearched`'s | high |
|
||||
| `DecayResearchProgress` | `max(0, progress − ftol(cost x (double)0.05f))` — the image holds the widened float literal `0.05000000074505806`, and there is no special case for a node whose cost is still `INT_MAX` | high |
|
||||
| `DecayAllResearch` | applies to every Available node whose progress is **non-zero** (not "positive"), after the target was processed; the just-funded target is **not** excluded and only escapes by completing in the same pass (net gain of the current tech = spend − 5 % of cost) | high — confirmed against the loop |
|
||||
| `DecayAllResearch` | applies to every node in state **Available (2)** whose progress is **non-zero** (not "positive"), after the allocation pass. The *current* research target is **not** among them: the tree marks the selected target with state **3**, so the funded node keeps its full gain and only idle partially-researched techs decay (live trace, `docs/B3.md`) | high — confirmed against the loop and the live trace |
|
||||
| `ProcessResearchTurn` | the whole per-turn pass: every allocation entry in order, then the decay sweep. This is the shape the shim hooks; see `docs/B3.md` | high |
|
||||
| `RollLabAccident` | `randint(100) < odds`, where `randint` is uniform on `[0, 100]` **inclusive** (see `docs/mars-rng.md`) | medium — odds-from-boost function unresolved (caller supplies odds) |
|
||||
| `LabAccidentLossPercent` | `ceil(clamp01(rand01() x (max − min) + min) x 100)` | high |
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
// GENERATED — do not edit. Facts about Sword of the Stars.exe (GOG 1.8.1).
|
||||
// Source: sots-re ghidra/addresses.json @ ead46e2, generated 2026-09-08 by tools/gen_addresses.py
|
||||
// Source: sots-re ghidra/addresses.json @ 8a6c0e8, generated 2026-09-08 by tools/gen_addresses.py
|
||||
// Runtime address = (uintptr_t)GetModuleHandle(NULL) + RVA (the exe is ASLR-relocated).
|
||||
#pragma once
|
||||
#include <cstdint>
|
||||
|
|
@ -253,7 +253,7 @@ constexpr uint32_t TechTree_off_Nodes = 0x00000010;
|
|||
constexpr uint32_t TechNode_size = 0x00000034;
|
||||
// offset TechDef* def; *(int*)def is the tech id used to index TechTree_off_Nodes [verified]
|
||||
constexpr uint32_t TechNode_off_Def = 0x00000000;
|
||||
// offset int state (0 hidden, 1 parent researched, 2 available, 4 researched); ctor writes 0 [verified]
|
||||
// offset int state (0 hidden, 1 parent researched, 2 available, 3 available AND selected as the current research target, 4 researched); ctor writes 0. The decay sweep tests == 2, so a state-3 node never decays -- confirmed by the B3 live trace [verified]
|
||||
constexpr uint32_t TechNode_off_State = 0x00000014;
|
||||
// offset int costRP, INT_MAX = no researched parent yet; ctor writes 0x7fffffff [verified]
|
||||
constexpr uint32_t TechNode_off_CostRP = 0x00000018;
|
||||
|
|
|
|||
|
|
@ -56,7 +56,16 @@ int TechCost(int baseCost, double multiplier);
|
|||
// Per-turn progress
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
enum class TechState : int { Hidden = 0, ParentResearched = 1, Available = 2, Researched = 4 };
|
||||
// Node states. 3 is the selected research target: it behaves as Available everywhere except
|
||||
// the decay sweep, whose guard is an equality test against Available, so the funded tech does
|
||||
// not lose 5 % of its cost the turn it is funded. Confirmed on the live game (docs/B3.md).
|
||||
enum class TechState : int {
|
||||
Hidden = 0,
|
||||
ParentResearched = 1,
|
||||
Available = 2,
|
||||
CurrentTarget = 3,
|
||||
Researched = 4,
|
||||
};
|
||||
|
||||
// Completion-flag values the game stamps on a node.
|
||||
enum class TechFlag : int { CompletedEarly = 0, Default = 1, OverBudgetNotified = 2 };
|
||||
|
|
@ -114,9 +123,9 @@ ResearchStepResult ApplyResearchPoints(ResearchNode& node, int points, Species o
|
|||
// CONFIDENCE: high.
|
||||
int DecayResearchProgress(int progress, int cost);
|
||||
|
||||
// Apply the decay to every Available node whose progress is non-zero. This runs after the
|
||||
// current target has been processed, so the current target decays too (net gain = spend -
|
||||
// 5 %). CONFIDENCE: high.
|
||||
// Apply the decay to every node in state Available whose progress is non-zero. This runs after
|
||||
// the allocation pass. The current target is state CurrentTarget, not Available, so it keeps
|
||||
// its whole gain; only idle partially-researched techs decay. CONFIDENCE: high.
|
||||
void DecayAllResearch(std::vector<ResearchNode>& nodes);
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue