W3: watch.mode=tshn -- arm the NVO/TShn record and the trade+spy containers from the same S

Second mode in lane W2's watchpoint module. No second hook: only different arithmetic on the S the
ApplyAllTurnCommands detour already holds.

- picks the target system by predicate at arm time (AFlags == 0, NVO non-empty) and logs all 28
  systems, so the choice is auditable rather than a hard-coded pointer;
- probes both candidate ServerSystem bases and logs how many systems validate under each, which
  settled a documentation dispute (+0x274: 9, +0x26c: 0) by measurement;
- prints the trade-route and spy-program vector triples, which is the workload-confirmation
  instrument two earlier lanes lacked.

Rule 19 control passed: the armed run reproduced the determinism oracle byte for byte.
This commit is contained in:
alex 2026-09-08 16:38:55 -04:00
parent bb81d3db2d
commit 9e914ef358
6 changed files with 376 additions and 12 deletions

70
docs/W3-watchpoints.md Normal file
View file

@ -0,0 +1,70 @@
# W3 — a second watch mode, and what the two runs answered
Companion to `docs/W3-predictions.md` (written and committed before the build) and to
`sots-re/findings/subsystems/nvo-tshn-visible-owner.md` +
`sots-re/findings/control-flow/w3-containers-status-handlers.md` (the full reports).
Builds on lane W2's module without changing how it works.
## What was added
`watch.mode=modcount|tshn` in `src/shim/hooks/watchpoints.{h,cpp}`. **No second hook, no second
detour, no change to the VEH or to the arming mechanism** — only different arithmetic on the `S`
the existing `ApplyAllTurnCommands` detour already holds. `modcount` is W2's set and stays the
default so its run remains reproducible; `tshn` arms:
| slot | target |
|---|---|
| 0 | the `TShn` word of an `NVO` map record on a system with `AFlags == 0` |
| 1 | that map's `_Mysize` |
| 2 | trade-route vector `_Mylast` (`*(S+4+0x154) + 0x40`) |
| 3 | spy-program vector `_Mylast` (`*(S+4+0x158) + 0x14`) |
Two things about the `tshn` arming are worth keeping if the module is extended again:
- **The target is chosen by predicate at arm time, not hard-coded**, and all 28 systems are logged
with name, `AFlags` and both map sizes. A hard-coded pointer that lands on the wrong object
produces the same empty hit list as "nothing writes this" (method rule 1).
- **The `ServerSystem` base is probed, not assumed.** Two published accounts disagreed by 8 about
where the `NVO` map lives, because `ServerSystem::Write` runs on the IStreamable subobject at
`+0x8`. The code tries both candidate bases across every system and logs how many validated under
each (`+0x274`: 9 systems; `+0x26c`: 0). That turned a documentation dispute into a measurement.
Configs: `shim.cfg.w3tshn`, `shim.cfg.w3mod`, `shim.cfg.w3control`. `w3tshn` and `w3control` differ
in exactly one key (`watch=`), which is what makes the rule-19 control real.
## Rule 19: the control was taken again, and it passed again
One End Turn from `ref-turn2.sav` with the four `tshn` watchpoints armed reproduced the determinism
oracle byte for byte — `(Autosave EndTurn).sav` `bb4fd9ac…`, `(Autosave).sav` `978041ac…`, both
identical to the pre-run files. So the four *new* addresses are as neutral as W2's four were. That
mattered: W2 proved a data breakpoint is neutral for one address set, not for all of them, and this
lane's set includes a heap node that the game reallocates freely.
The second run (turn-1 workload, `watch.mode=modcount`) has **no oracle of its own** and says so.
## What the two runs answered
1. **`NVO.TShn`'s writer, trapped live**, with its whole call chain confirmed frame by frame from
the recorded `ebp` chain: `RefreshVisibleOwnerIfKnown 0x0075bd70` → `RecordVisibleOwner
0x0075bca0` → `SetVisibleOwner 0x0075b880` → `NVO::operator[] 0x0075a890` → the store
`0x0075b961`. The gate is `ServerSystem::IsKnownTo 0x00746390`, which is `IsVisibleTo` **or**
(sensor contact ∧ `CCC_AdvSens`) — not `AFlags` alone, which is why lane E3 could not fit it.
The resulting model predicts all 158 `NVO` records in the 11-save corpus with zero mismatches,
including the single frozen one.
2. **Four traps, not one, and the extra pair is a result**: the refresh runs twice per End Turn, in
driver phase 24 and again in combat-done phase 25. Both phase attributions confirmed for free.
3. **The trade and spy containers, read out of a live game for the first time.** Both managers
non-null, both vectors default-constructed with all three pointers zero, zero traps on either
`_Mylast` across the turn. The workload is *absent*, measured — which is the confirmation
instrument two earlier lanes needed and did not have.
4. **All ten command `ModCount` handlers are now named**, including two that only exist on a turn-1
workload (`OnCommand_SetResearchProject`, inlined at `0x0088fe0a`, and
`OnCommand_CreateDesign 0x00882910`).
## Reusing it
`watch.mode` is the extension point. To watch something else, add a mode and one arming function;
everything else — the canary, the VEH, the flusher, the hit format — is unchanged. Keep the canary
self-test and keep the `ref-turn2` oracle control: together they cost about two minutes and they
are what makes the numbers evidence rather than output.

View file

@ -19,16 +19,19 @@ bool g_enabled = false;
int g_players = 2;
char g_outPath[MAX_PATH] = {};
// Which four addresses the single arming point computes. `modcount` is lane W2's set and is the
// default so its run stays reproducible byte for byte; `tshn` is lane W3's.
enum class Mode { ModCount, Tshn };
Mode g_mode = Mode::ModCount;
// ---- what is being watched ------------------------------------------------------------------
//
// Slot 0 and 1 are the two words the naming disagreement is about; slots 2 and 3 are the first two
// players' Status. Four slots is the hardware limit and it is exactly enough.
const char* const kSlotName[4] = {
"S+0x8 (A2:ModCount / T:PhaseCounter)",
"S+0xc (A2:Frame / addresses.json:ModCount)",
"player[0]+0x164 Status",
"player[1]+0x164 Status",
};
// Filled in by the arming function, because in `tshn` mode two of the four slots are only nameable
// once the target object has been found. Printed with the address, so a reader of the log never
// has to trust that slot N is what the brief said it would be.
char g_slotName[4][96] = {};
void SetSlotName(int i, const char* s) { std::snprintf(g_slotName[i], sizeof g_slotName[i], "%s", s); }
// ---- hit records -----------------------------------------------------------------------------
//
@ -180,6 +183,206 @@ void LogF(const char* fmt, ...) {
bool g_armed = false;
// ---- lane W3: reaching the NVO map, the trade vector and the spy vector -----------------------
//
// All three hang off the same `S` the arming detour already has, so this adds no second hook.
//
// The offsets below were re-derived from the instruction stream of `ServerSystem::Write`
// 0x00749630 rather than taken from a table, because two published accounts disagreed by 8. The
// reason for the disagreement: `Write` is entered on the **IStreamable subobject at
// ServerSystem+0x8** (the RTTI COL offset for its vftable is +0x8), so every displacement in its
// body is 8 less than the ServerSystem-relative offset. `[edi+0x26c]`/`[edi+0x270]` at
// 0x0074a195/0x0074a07d are therefore `NVO._Myhead`/`_Mysize` at ServerSystem **+0x274/+0x278**.
//
// This code does not *rely* on that reconciliation. It probes both candidate bases on every
// system, counts how many validate under each, and logs both counts -- rule 1: an arm that landed
// on the wrong object is indistinguishable from "nothing writes this".
constexpr std::uint32_t kSysNvoHead = 0x274; // ServerSystem-relative
constexpr std::uint32_t kSysAFlags = 0xd4;
constexpr std::uint32_t kSysName = 0xa8; // std::string, 0x1c bytes
constexpr std::uint32_t kSysNveSize = 0x288;
constexpr std::uint32_t kNodeIsNil = 0x8d; // an NVO node is ~0x90 bytes; NVE's is 0x20
constexpr std::uint32_t kNodeValue = 0x10; // {int16 touched, int16 TShn, int32 OID, ...}
inline bool Readable(std::uintptr_t p, std::size_t n) {
return p != 0 && !IsBadReadPtr(reinterpret_cast<void*>(p), n);
}
inline std::uint32_t U32(std::uintptr_t p) { return *reinterpret_cast<volatile std::uint32_t*>(p); }
inline std::uint8_t U8(std::uintptr_t p) { return *reinterpret_cast<volatile std::uint8_t*>(p); }
struct NvoProbe {
bool ok = false;
std::uintptr_t head = 0;
std::uint32_t size = 0;
std::uintptr_t root = 0; // == _Myhead->_Parent
std::uint32_t key = 0; // the root node's key: a player INDEX (Write reads players[key])
};
// Validate `sys + headOff` as an MSVC `_Tree` header {_Myhead, _Mysize}. The head node is the nil
// sentinel, so its _Isnil byte must be 1 and the root it parents must have _Isnil 0.
NvoProbe ProbeNvo(std::uintptr_t sys, std::uint32_t headOff) {
NvoProbe r;
const std::uintptr_t hp = sys + headOff;
if (!Readable(hp, 8)) return r;
r.head = U32(hp);
r.size = U32(hp + 4);
if (!Readable(r.head, 0x90)) return r;
if (U8(r.head + kNodeIsNil) != 1) return r;
if (r.size == 0 || r.size > 64) return r;
const std::uintptr_t root = U32(r.head + 4);
if (!Readable(root, 0x90)) return r;
if (U8(root + kNodeIsNil) != 0) return r;
const std::uint32_t key = U32(root + 0xc);
if (key > 63) return r; // player index, not a handle
r.root = root;
r.key = key;
r.ok = true;
return r;
}
// MSVC std::string (0x1c): union _Bx at +0, _Mysize +0x10, _Myres +0x14; short strings live in the
// union. Copies at most `cap-1` bytes and always NUL-terminates.
void ReadStdString(std::uintptr_t s, char* out, std::size_t cap) {
out[0] = '\0';
if (!Readable(s, 0x18)) return;
const std::uint32_t len = U32(s + 0x10);
const std::uint32_t res = U32(s + 0x14);
const std::uintptr_t p = (res < 16) ? s : static_cast<std::uintptr_t>(U32(s));
if (len == 0 || len > 0x100 || !Readable(p, len)) return;
std::size_t n = len < cap - 1 ? len : cap - 1;
for (std::size_t i = 0; i < n; ++i) {
const char c = static_cast<char>(U8(p + i));
out[i] = (c >= 32 && static_cast<unsigned char>(c) < 127) ? c : '?';
}
out[n] = '\0';
}
// A std::vector here is {_Myfirst,_Mylast,_Myend,_Alval} = 0x10, allocator LAST (method rule 5).
// Returns the element count and reports the raw triple, because "the offset names something else"
// and "the container is empty" have to be separable from the log alone.
int VectorCount(std::uintptr_t vec, std::uint32_t* first, std::uint32_t* last, std::uint32_t* end,
std::uint32_t stride) {
*first = *last = *end = 0;
if (!Readable(vec, 12)) return -1;
*first = U32(vec);
*last = U32(vec + 4);
*end = U32(vec + 8);
if (*last < *first || (*last - *first) % stride) return -1;
return static_cast<int>((*last - *first) / stride);
}
// The systems vector in the S+4 frame. Lane E3 decoded tail phase 17 as `mov edx,[esi+0x44]` =
// systems.begin with esi = S, i.e. (S+4)+0x40 -- the same frame `StrategyServer_off_Players`
// (0x50) is expressed in. Recorded in ghidra/addresses.d/lane-w3.json; kept local rather than
// pulled from the generated header because a concurrent lane owns that header this session.
constexpr std::uint32_t kServerOffSystems = 0x40;
// The two containers lane W2 §8.3 located but did not arm. Both are one add from `S`.
constexpr std::uint32_t kServerOffTradeMgr = 0x154; // -> ServerTradeManagerImpl*
constexpr std::uint32_t kTradeMgrOffVec = 0x3c; // _Myfirst; _Mylast at +0x40
constexpr std::uint32_t kServerOffSpyMgr = 0x158; // -> ServerSpyManager*
constexpr std::uint32_t kSpyMgrOffVec = 0x10; // _Myfirst; _Mylast at +0x14
// Lane W3's four slots. Slot 0 is the whole point: the `TShn` word of an NVO record on a system
// whose AFlags is zero. Slot 1 is its map's _Mysize, which is what makes slot 0's silence mean
// something -- if the node were freed and reallocated, slot 0 would be watching dead memory and
// would report a confident nothing (rule 20's distinction, one level down).
void ArmTshnSlots(std::uintptr_t S) {
const std::uintptr_t sysVec = S + 4 + kServerOffSystems;
std::uint32_t f = 0, l = 0, e = 0;
const int nSys = VectorCount(sysVec, &f, &l, &e, 4);
LogF("watch: systems vector @0x%08x first=0x%08x last=0x%08x end=0x%08x count=%d "
"(expect 28 on the reference game)",
static_cast<unsigned>(sysVec), f, l, e, nSys);
// Does the vector hold the ServerSystem base, or its IStreamable subobject 8 bytes in? Probe
// both and let the systems themselves decide, rather than trusting either published table.
const std::uint32_t headOff[2] = {kSysNvoHead, kSysNvoHead - 8};
int okAt[2] = {0, 0};
for (int a = 0; a < 2; ++a) {
for (int i = 0; i < nSys && i < 256; ++i) {
const std::uintptr_t sys = U32(f + i * 4);
if (!Readable(sys, 0x300)) continue;
if (ProbeNvo(sys, headOff[a]).ok) ++okAt[a];
}
}
const int adj = (okAt[0] >= okAt[1]) ? 0 : 1;
const std::uint32_t d = adj ? 8 : 0;
LogF("watch: NVO header probe -- ServerSystem+0x%x validated on %d systems, +0x%x on %d; "
"using +0x%x (pointer delta %u)",
kSysNvoHead, okAt[0], kSysNvoHead - 8, okAt[1], headOff[adj], d);
std::uintptr_t target = 0;
NvoProbe tp;
char tname[40] = {};
for (int i = 0; i < nSys && i < 256; ++i) {
const std::uintptr_t sys = U32(f + i * 4);
if (!Readable(sys, 0x300)) continue;
const NvoProbe p = ProbeNvo(sys, headOff[adj]);
const std::uint32_t af =
Readable(sys + kSysAFlags - d, 4) ? U32(sys + kSysAFlags - d) : 0xffffffffu;
const std::uint32_t nve =
Readable(sys + kSysNveSize - d, 4) ? U32(sys + kSysNveSize - d) : 0xffffffffu;
char nm[40];
ReadStdString(sys + kSysName - d, nm, sizeof nm);
LogF("watch: sys[%d] @0x%08x '%s' AFlags=0x%x NVO=%u NVE=%u root=0x%08x key=%u ok=%d", i,
static_cast<unsigned>(sys), nm, af, p.size, nve, static_cast<unsigned>(p.root), p.key,
p.ok ? 1 : 0);
if (!target && af == 0 && p.ok) {
target = sys;
tp = p;
std::snprintf(tname, sizeof tname, "%s", nm);
}
}
if (target) {
g_watchAddr[0] = tp.root + kNodeValue;
g_watchAddr[1] = target + kSysNvoHead + 4 - d;
std::snprintf(g_slotName[0], sizeof g_slotName[0],
"'%s' NVO root+0x10 {touched:i16,TShn:i16} key=player %u", tname, tp.key);
std::snprintf(g_slotName[1], sizeof g_slotName[1], "'%s' NVO._Mysize (=%u at arm)", tname,
tp.size);
} else {
LogF("watch: NO system has AFlags==0 with a non-empty NVO -- slots 0/1 UNSET. That is a "
"FAILED TARGET SELECTION, not a measurement; every zero below is unmeasured.");
SetSlotName(0, "(no target found)");
SetSlotName(1, "(no target found)");
}
// The workload confirmation. Printing these two counts is the cheap half of the answer: two
// lanes have failed to build a trade/spy workload, and nobody has yet printed the containers
// from a live game to say whether a workload took.
const struct {
std::uint32_t mgrOff, vecOff;
const char* what;
} cont[2] = {
{kServerOffTradeMgr, kTradeMgrOffVec, "trade routes"},
{kServerOffSpyMgr, kSpyMgrOffVec, "spy programs"},
};
for (int i = 0; i < 2; ++i) {
const std::uintptr_t slot = S + 4 + cont[i].mgrOff;
const std::uintptr_t mgr = Readable(slot, 4) ? U32(slot) : 0;
std::uint32_t vf = 0, vl = 0, ve = 0;
int n = -1;
if (Readable(mgr, cont[i].vecOff + 12))
n = VectorCount(mgr + cont[i].vecOff, &vf, &vl, &ve, 4);
LogF("watch: %s -- manager=0x%08x (S+4+0x%x) vector@mgr+0x%x first=0x%08x last=0x%08x "
"end=0x%08x count=%d",
cont[i].what, static_cast<unsigned>(mgr), cont[i].mgrOff, cont[i].vecOff, vf, vl, ve,
n);
if (mgr) {
g_watchAddr[2 + i] = mgr + cont[i].vecOff + 4;
std::snprintf(g_slotName[2 + i], sizeof g_slotName[2 + i],
"%s vector _Mylast (mgr+0x%x, count %d at arm)", cont[i].what,
cont[i].vecOff + 4, n);
} else {
std::snprintf(g_slotName[2 + i], sizeof g_slotName[2 + i], "%s (manager null)",
cont[i].what);
}
}
}
DWORD WINAPI FlusherThread(LPVOID) {
for (;;) {
Sleep(2000);
@ -203,11 +406,22 @@ extern "C" void WatchOnApplyAll(void* self) {
return;
}
const std::uintptr_t S = reinterpret_cast<std::uintptr_t>(self);
g_watchAddr[0] = S + 0x8;
g_watchAddr[1] = S + 0xc;
g_watchAddr[0] = 0;
g_watchAddr[1] = 0;
g_watchAddr[2] = 0;
g_watchAddr[3] = 0;
if (g_mode == Mode::Tshn) {
ArmTshnSlots(S);
} else {
g_watchAddr[0] = S + 0x8;
g_watchAddr[1] = S + 0xc;
SetSlotName(0, "S+0x8 (A2:ModCount / T:PhaseCounter)");
SetSlotName(1, "S+0xc (A2:Frame / addresses.json:ModCount)");
SetSlotName(2, "player[0]+0x164 Status");
SetSlotName(3, "player[1]+0x164 Status");
// players vector lives at S+0x54 (raw +0x50 in the S+4 frame -- see the ctor enumeration in
// StrategyServer_base_delta). Read defensively: a wrong pointer here must not fault.
if (g_players > 0) {
@ -228,6 +442,8 @@ extern "C" void WatchOnApplyAll(void* self) {
}
}
} // end Mode::ModCount
// Instrument self-test: put slot 3 on a word we own, write it, and require exactly one trap
// before any game number is trusted.
const std::uintptr_t saved3 = g_watchAddr[3];
@ -255,7 +471,7 @@ extern "C" void WatchOnApplyAll(void* self) {
g_armedTid = GetCurrentThreadId();
g_armed = true;
for (int i = 0; i < 4; ++i)
LogF("watch: slot %d -> %s = 0x%08x%s", i, kSlotName[i],
LogF("watch: slot %d -> %s = 0x%08x%s", i, g_slotName[i],
static_cast<unsigned>(g_watchAddr[i]), g_watchAddr[i] ? "" : " (unset)");
LogF("watch: ARMED on tid %lu dr7=0x%08x, S=%p (ApplyAllTurnCommands this)", g_armedTid, dr7,
self);
@ -282,6 +498,12 @@ bool watch_apply_config(const char* key, const char* value, std::string* err) {
else if (err) *err = "expected on|off";
return true;
}
if (std::strcmp(key, "watch.mode") == 0) {
if (std::strcmp(value, "modcount") == 0) g_mode = Mode::ModCount;
else if (std::strcmp(value, "tshn") == 0) g_mode = Mode::Tshn;
else if (err) *err = "expected modcount|tshn";
return true;
}
if (std::strcmp(key, "watch.out") == 0) {
std::snprintf(g_outPath, sizeof g_outPath, "%s", value);
return true;

View file

@ -34,7 +34,16 @@ namespace shim::hooks {
// shim.cfg keys owned here. Returns true if `key` was ours (whether or not the value parsed).
// watch=off|on install the ApplyAllTurnCommands arming detour (default off)
// watch.out=<path> hit log (default <gamedir>\shim.watch.txt)
// watch.players=<n> how many player Status words to watch (0..2, default 2)
// watch.players=<n> how many player Status words to watch (0..2, default 2; modcount mode)
// watch.mode=modcount|tshn which four addresses the arming point computes (default modcount)
//
// `modcount` is lane W2's set: S+0x8, S+0xc and two players' Status.
// `tshn` is lane W3's: the `TShn` word of an NVO map record on a system with AFlags == 0, that
// map's _Mysize, and the `_Mylast` of the trade-route and spy-program vectors. All four are
// reachable from the same `S` the arming detour already holds, so the mode adds no second hook --
// only different arithmetic. The target system is chosen by predicate at arm time and its name,
// AFlags and map sizes are logged, because "the arm landed on the wrong object" and "nothing
// writes this" produce the same empty hit list (method rule 1).
bool watch_apply_config(const char* key, const char* value, std::string* err);
bool watch_enabled();

View file

@ -0,0 +1,21 @@
# Lane W3: the same watchpoint module, pointed at the NVO/TShn record and the trade + spy
# containers. Identical to shim.cfg.w3control except for the single key `watch=`, so the pair is a
# real rule-19 control: the armed run and the control run differ in nothing else.
hooks=trace
hook.Shim::SelfTest::Fill=off
hook.Mars::GlobalConsts::LoadFile=off
hook.Game::WeaponDictionary::Init=off
hook.Game::SectionDictionary::SectionDictionary=off
hook.Game::StrategyServer::ProcessFleetMovement=off
hook.Game::TechTree::ProcessResearch=trace
hook.Game::ServerPlayer::ComputeBudget=trace
hook.Game::ServerPlayer::OnTechResearched=trace
hook.Game::ServerSystem::ProcessTurn=trace
hook.Game::StrategyServer::MoveFleet=trace
trace.path=C:\SOTS\shim.trace.jsonl
trace.inline_max=256
trace.flush=always
watch=off
watch.players=2
watch.mode=tshn
watch.out=C:\SOTS\shim.watch.txt

21
src/shim/shim.cfg.w3mod Normal file
View file

@ -0,0 +1,21 @@
# Lane W3 run 2 (lane AI4 probe): the SAME binary as shim.cfg.w3tshn with ONE key changed --
# watch.mode -- so the module arms lane W2's ModCount set instead. Paired with shim.cfg.w3control
# for rule 19; byte-neutrality of this build was established on ref-turn2 with mode=tshn.
hooks=trace
hook.Shim::SelfTest::Fill=off
hook.Mars::GlobalConsts::LoadFile=off
hook.Game::WeaponDictionary::Init=off
hook.Game::SectionDictionary::SectionDictionary=off
hook.Game::StrategyServer::ProcessFleetMovement=off
hook.Game::TechTree::ProcessResearch=trace
hook.Game::ServerPlayer::ComputeBudget=trace
hook.Game::ServerPlayer::OnTechResearched=trace
hook.Game::ServerSystem::ProcessTurn=trace
hook.Game::StrategyServer::MoveFleet=trace
trace.path=C:\SOTS\shim.trace.jsonl
trace.inline_max=256
trace.flush=always
watch=on
watch.players=2
watch.mode=modcount
watch.out=C:\SOTS\shim.watch.txt

21
src/shim/shim.cfg.w3tshn Normal file
View file

@ -0,0 +1,21 @@
# Lane W3: the same watchpoint module, pointed at the NVO/TShn record and the trade + spy
# containers. Identical to shim.cfg.w3control except for the single key `watch=`, so the pair is a
# real rule-19 control: the armed run and the control run differ in nothing else.
hooks=trace
hook.Shim::SelfTest::Fill=off
hook.Mars::GlobalConsts::LoadFile=off
hook.Game::WeaponDictionary::Init=off
hook.Game::SectionDictionary::SectionDictionary=off
hook.Game::StrategyServer::ProcessFleetMovement=off
hook.Game::TechTree::ProcessResearch=trace
hook.Game::ServerPlayer::ComputeBudget=trace
hook.Game::ServerPlayer::OnTechResearched=trace
hook.Game::ServerSystem::ProcessTurn=trace
hook.Game::StrategyServer::MoveFleet=trace
trace.path=C:\SOTS\shim.trace.jsonl
trace.inline_max=256
trace.flush=always
watch=on
watch.players=2
watch.mode=tshn
watch.out=C:\SOTS\shim.watch.txt