From 136f98b564f64e25cc1dfd061bb4335d1a43ca0c Mon Sep 17 00:00:00 2001 From: alex Date: Tue, 8 Sep 2026 08:29:20 -0400 Subject: [PATCH] guides: the 15 earned method rules, each traced to the finding that produced it --- campaign/board.md | 1 + guides/method-rules.md | 162 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 163 insertions(+) create mode 100644 guides/method-rules.md diff --git a/campaign/board.md b/campaign/board.md index f67b4c9..52b0836 100644 --- a/campaign/board.md +++ b/campaign/board.md @@ -143,3 +143,4 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`). | unread driver: OnAllCombatDone_Tail | control-flow | backlog | — | 0% | 2026-09-08 | THE BIGGEST UNREAD BLOCK IS NOT IN ProcessTurn AT ALL. No bankruptcy, no turn results, no turn events, no autosave run in the turn driver - they are all in StrategyServer::OnAllCombatDone_Tail 0x007d92a0, a SECOND driver no lane has read. Highest-value next control-flow target | | player_turn hook prepared (boundary narrowed) | phase2 | mapped | high | 70% | 2026-09-08 | Lane T built the descriptor with compile-time coverage() (9 notes), Result regions for the three phases needing nothing but the player, roll_flags + rng as observations ours never writes, guards over the whole ServerPlayer and the TechTree header. DELIBERATELY NARROWED: phases 2/3/6 (savings, aid records, refund) depend on ComputeBudget's slots and ProcessResearch's overBudget, both STACK LOCALS of the original. The three ways to reach them are calling ComputeBudget ourselves (repairs ships - audit #6), reading the nested hooks (audit #5, SELF-FULFILLING), or inferring from the Sav delta. They are GUARDED, NOT CHECKED; formulas written and unit-tested but not in the verdict. docs/T-turn-driver.md carries the prediction incl. which checks are WEAK BY CONSTRUCTION on the reference save | | INTEGRATOR NOTE: two gate failures caught on lane T merge | meta | verified | high | 100% | 2026-09-08 | Both gates failed on merge and both were real, which is the argument for running them SEPARATELY rather than &&-chained: (1) clean-room exit 1 - a raw FUN_00889d60 identifier had reached docs/ in the public-capable engine repo; fixed by using the name the lane itself had already put in addresses.json (ServerPlayer_OnResearchRollSucceeded). (2) The shim CROSS-BUILD failed -Werror=unused-function on player_turn.cpp - the file is WIN32-only and the lane could not build it on the notes host, exactly as it flagged. Host ctest passed both times and would have hidden both. A lane that cannot cross-build its own shim TU must say so, and the integrator must run the CT111 shim build before pushing | +| guides/method-rules.md | meta | verified | high | 100% | 2026-09-08 | The 15 rules this campaign PAID FOR, each traced to the finding that produced it, ranked by damage caused. Written because the integrator was pasting the same hard-won constraints into every lane brief, which invites drift and omission. Lane briefs now REFERENCE it instead of restating it. Covers: green verdict != evidence; predict before the run; static vs behavioural find different bugs; inlined dtor looks like a branch; size by enumeration; unexercised path = hypothesis; round-trip != coverage; two checks sharing an assumption are one check; rankers never filter; derive experiment params from the ISA; correct the record; fix the oracle openly; separate gates; never hand-resolve a generated header; report thin coverage as loudly as divergences | diff --git a/guides/method-rules.md b/guides/method-rules.md new file mode 100644 index 0000000..9a358d2 --- /dev/null +++ b/guides/method-rules.md @@ -0,0 +1,162 @@ +# Earned rules + +Rules this campaign paid for. Each one exists because it was violated first and something wrong got +published, or nearly did. Lane briefs should reference this file rather than restating it. + +Ordered by how much damage the violation caused. + +--- + +## 1. A green verdict is not evidence. Coverage is. + +`tracecmp` exit 0 means "the declared regions agreed". It does **not** mean the hook compared +anything. B4 found three hooks that each printed `0 diverged` while comparing nothing: + +- `describe_args` ran **before** `regions()`, so every logged argument was one call stale; +- the `StrategyServer` has **two bases 4 bytes apart** — reading the wrong one yields an empty + player vector, zero declared regions, and a confident "1 call, 0 diverged"; +- a Ghidra-base number was used as a raw struct offset (`0x64` vs `0x60`), enumerating a vector's + spare capacity. + +Structural fixes now in place: compile-time-required `Coverage` on every descriptor (a hook without +one does not compile), guard regions that **localise** undeclared writes by offset rather than +reporting that a hash moved, replace-mode records, and `tracecmp --strict-coverage`. A hook claiming +"complete" while a guard caught an undeclared write counts as a **divergence**. + +**Read the trace, not the verdict.** Every hook bug above was visible to a reader and invisible to +the exit code. + +## 2. Write the prediction down before the run + +Commit the expected result *before* building, not before running. Lane U committed its prediction +before the build was even staged; lane P wrote its expected residual (`next_id` short by exactly 1 +on every completion call) and lane V confirmed it live. + +The strong form: **a prediction that survives a changed workload tests the model, not the +recording.** Lane U's call 9 was not the call its prediction was written against — the AI had picked +a different tech — and it unlocked three nodes at costs appearing in no earlier report. Numbers that +never occurred cannot have been memorised. + +Include a falsification section: how the model could be wrong, and the symptom of each way. + +## 3. Static reading finds what behavioural comparison cannot + +Both of these compared clean and were still wrong: + +- `ApplyTechEffect` had an early return that would have made **every real call a no-op**; +- the RNG divisor was wrong on 0.78% of draws and flipped **zero** decisions in 10⁶ trials. + +Conversely, behavioural compare found the `MoveFleet` rounding that static reading had missed. They +are different instruments. Use both; trust neither alone. + +## 4. Prefer the instruction stream to the decompiler for control flow + +**An inlined `std::vector` destructor looks exactly like a branch.** The `je` skips only the +`operator delete`; both arms converge a few hundred bytes later. This has produced two wrong +published findings — `turn-spine.md`'s "deferred end-of-turn tail" (there is no such branch) and a +near-miss on `Streamable::Write`. + +Any decompiler `if` that wraps a whole tail is suspect until you have found the converge point in +the disassembly. + +## 5. Size structs by enumeration, never by what the code touches + +`_Alval` is `std::allocator`, an **empty class**: it occupies a word and is never loaded or +stored, so it is invisible to touch-based analysis and undercounts by exactly 4. That produced a +`std::string`-is-0x18 scare against a correct campaign-wide 0x1c. + +An **enumeration** — serializer, constructor, copy constructor, or container stride — can show +**absence** where a touch-scan cannot. Same trap is live for `std::vector` here: +`{_Myfirst,_Mylast,_Myend,_Alval}` = `0x10`, **allocator-last**, the opposite of the shape the MSVC +textbooks describe. + +Related: the on-disk primitive is **not** the memory kind. A member held as `int16`/`int8` is +written by `WriteInt` and is **four bytes on the wire**. + +## 6. A path no save exercises is a hypothesis + +Label it as one. `spies2`, `SysMem` and `mts` were "closed" with count 0 in every save — no element +value has ever been observed. Lane A later found the element typing was wrong, and it was invisible +*precisely because* nothing exercised it. The flagging discipline is what made that checkable. + +**Saves can be manufactured.** No species-5 save existed, so lane V built one and closed a row that +had been disassembly-only for a day. Prefer building the workload over weakening the claim. + +## 7. Round-trip byte-identity is not coverage + +`ar.any` bodies round-trip trivially by copying bytes nobody understands. Measure the split between +items a field **names** and items a node merely **carries**. That reframing moved a "100% round-trip" +result to an honest 38%, and then to 99.9% by actual work. + +## 8. Two independent-looking checks sharing a hidden assumption are one check + +`save_reader.py` types items from its **own** kind catalog, not the schema passed to it — so on real +saves the catalog and the schema agreed with each other and were **both wrong**. Its failure mode is +silent agreement, not a desync. + +## 9. Rankers rank; they never filter + +A cohort/frequency ranker would have **discarded the correct answer**: the function that appends to +`ObservedTech` touches only two offsets on `ServerPlayer`, so every `--min>=1` cohort filter drops +it. What closed the case was a plain query plus one call-graph lookup. + +Displacement scan for recall, call graph for disambiguation. Neither alone sufficed. And report the +false-positive rate honestly — that scanner's class-level precision is ~13% by function; its value +is a 900× search-space cut, not accuracy. + +## 10. Derive experiment parameters from the ISA, not from assumed mnemonics + +An x87 sensitivity experiment was briefed as `0x027f / 0x127f / 0x137f`. In fact `0x027f` is 53-bit +(differing from `0x127f` only in infinity control, ignored since the 387) and `0x137f` is 64-bit +extended. Run literally it returns "all three identical" — true, and the conclusion drawn from it +would have been wrong on both axes, because it tests single precision not at all and rounding not at +all. The real probes are `0x007f` and `0x1a7f`. + +**Brief lanes to challenge the parameters, not just execute them.** Lane F did, and lane G rejected +its brief's premise outright and was right to. + +## 11. Correcting the record beats defending it + +Claims downgraded on evidence, by later lanes: "RNG matched 15/15" (workload luck, not a property); +B1's and B4's coverage after the harness audit; `StreamableEnum` typing; the deferred end-of-turn +tail; "the research roll draws exactly one NextFloat" (the branch it fires draws a second). + +If you find an earlier finding wrong, say so plainly and correct it in place. Nothing here is +anyone's reputation. + +## 12. Fix the oracle openly, never quietly + +Four defects were found in the reference save reader. The right move was **not** to patch them +mid-campaign in silence: they were recorded, then fixed openly with a test per defect and +byte-neutrality proven at item granularity (offset sequences identical across ~39,000 offsets per +save; typed-value deltas balancing to the byte). + +## 13. Run verification gates as separate commands + +Never `&&`-chain them. Once, a clean-room failure short-circuited the chain, skipping build and test +while a separate push still ran — an unverified tree reached `main`. + +On one merge **both** gates failed and both were real: a raw `FUN_xxxxxxxx` identifier had reached +`docs/` in the public-capable engine repo, and the **shim cross-build** hit `-Werror=unused-function` +on a WIN32-only TU. Host `ctest` passed cleanly both times and would have hidden both. + +**If a lane cannot cross-build its own shim TU, it must say so, and the integrator must run the +CT111 shim build before pushing.** + +## 14. Never hand-resolve a generated header + +Always regenerate from `ghidra/addresses.json` plus the `ghidra/addresses.d/.json` fragments. +Hand-resolving a merge conflict there silently dropped a new entry and broke the build — twice. + +Per-lane fragments exist because a shared single file caused three cross-lane sweeps in one day. +Duplicate names across fragments are a **hard error**, never last-wins: two lanes disagreeing about +an address is exactly what must not be papered over. + +## 15. Report thin coverage as loudly as divergences + +A green run that establishes twenty facts is not a strong result. `ComputeBudget` compared 4,284 +calls with 0 divergences — but only **20 distinct states**, with 4,278 of them the UI polling one +player, and **13 of 22 slots zero on every call**, including 5 of 6 declared input-boundary slots. + +The most useful sections of the best reports here have been the honest lists of what was *not* +covered.