diff --git a/docs/PL-players-residual.md b/docs/PL-players-residual.md index 055e30a..0d6ea2e 100644 --- a/docs/PL-players-residual.md +++ b/docs/PL-players-residual.md @@ -7,15 +7,26 @@ what the remaining leaves actually are. Closing leaves is secondary to the decom Binary re-measured, not inherited. `sots_turn` built from `main` (`aabd8a3`) in this worktree, `tools/standalone_report.py --binary …`: -| pair | baseline | after | closed | regressed | `/Sim/players` share | -|---|---:|---:|---:|---:|---:| -| `turn1-state -> turn2-state` | 209 | **128** | 81 | 0 | **54** | -| `turn2-state -> turn3-state` | 108 | **69** | 39 | 0 | **24** | -| pair 1, `--commit-blocked=T31 --ai-player 1` | 209 | 126 | 83 | 0 | 52 | -| pair 2, same | 108 | 67 | 41 | 0 | 22 | +| run | pair 1 residual | pair 2 residual | `/Sim/players` share | +|---|---:|---:|---:| +| `main` (`aabd8a3`), no options | **128** | **69** | **54 / 24** | +| `main`, `--commit-blocked=T31 --ai-player 1` | 126 | 67 | 52 / 22 | +| **this lane**, no options | **126** | **65** | **52 / 20** | +| this lane, `--data ROOT --commit-blocked=P11 --ai-player 1 --ai-player 2 --ai-player 3` | **122** | **62** | **48 / 17** | -The stale `status.json` breakdown said `/Sim/players` 55. It is **54** on pair 1 and **24** on -pair 2. +`closed` / `regressed` on the reference pairs, never netted: this lane closes **2 on pair 1 and +4 on pair 2 with no options at all, and regresses 0**; with a data root, **87 / 0** and **46 / +0** against the 209 and 108 baselines. + +Two corrections to the record before anything else: + +* The stale `status.json` breakdown said `/Sim/players` 55. It is **54** on pair 1 and **24** + on pair 2. +* **The published 128 / 69 overstates what is unmodelled.** Seven of those leaves belong to a + phase that is already written and only wants the game's data directory — see §3. The metric + harness invokes `sots_turn` with no `--data`, so it measures the engine *plus* that missing + argument. Handing it a root is a one-line harness change worth 4 + 3 leaves today, and more + once the tuning table is wired (§6, PL-3). --- @@ -29,29 +40,35 @@ almost entirely *unmodelled*, not *mismodelled*. | # | mechanism | pair 1 | pair 2 | rung | blocked on | |---|---|---:|---:|---|---| | M1 | **AI research orders** — `ResRate`, `ResTNm`, `ResErrRoll`, and the tech-tree state they move (`St[n]`, `TResDone[n]`, `Tbd[n]`) | 12 | 2 | **B** | `game/ai`: the AI picks a target *during* the turn | -| M2 | **AI ship construction** — `ShipRecs/*`, `Maint`, `FNG/FNGNum`, `NumDes` + the new `Des[…]` frame, `lboid`, and player 32's `Sav`/`PvSav` | 17 | 8 | **B** | lane B6: no build order exists at load | +| M2 | **AI ship construction** — `ShipRecs/*`, `Maint`, `FNG/FNGNum`, `NumDes` + the new `Des[…]` frame, `lboid`, and player 32's `Sav`/`PvSav` | 15 | 9 | **B** | lane B6: no build order exists at load | | M3 | **Observation records refreshed by the AI's own new design** — `odes`, `owep`, `otch/*/otnL` (phase T34 `RecordObservedDesigns`, stub) | 9 | 1 | **B** | downstream of M2 — the observed design *is* the design the AI just made | | M4 | **Events** — `Events/EvNxID` + the bucket frames | 8 | 6 | A (4/3) + B (4/3) | half is the human's `EVENT_NO_RESEARCH`, implemented in P11 and gated only on a **game-data root**; half is the AI's `EVENT_SHIPS_BUILT`/`EVENT_RESEARCH_OVERBUDGET` (M2/M1) | -| M5 | **Bankruptcy limits** — `BnkEl`, `BnkPr` | 4 | 4 | **A** | `BnkEl`: T31 is implemented and blocked on an operator flag. `BnkPr`: needs one tuning constant from the data files | +| M5 | **Bankruptcy limits** — `BnkEl`, `BnkPr` | 4 | 4 | **A** | `BnkEl`: **closed here** (PL-2, 2 per pair). `BnkPr`: needs one tuning constant from the data files, and the loader that would fill it is not wired | | M6 | **`Status`** — 0 → 4 on four of eight players | 4 | 0 | A? | the **predicate** is not identifiable from this corpus (§4) | -| M7 | **`PvSav`** — the S00 snapshot, for players M2 does not touch | 0 | 2 | **A** | nothing — closed here | +| M7 | **`PvSav`** — the S00 snapshot, for players M2 does not touch | 0 | 2 | **A** | nothing — **closed here** (PL-1) | | M8 | **A monster faction's one-off design** — player 528 `NumDes` 18→19 + `Des[1712 "Refugee Trade Ship"]` | 2 | 0 | B? | unexplained; a non-empire faction creates a design on turn 1→2 and never again | | | **total** | **54** | **24** | | | -Per-mechanism evidence is in §2–§5. +Per-mechanism evidence is in §2–§4; what this lane closed is §5, the ranked remainder §8. ### Rung split of the block | | pair 1 | pair 2 | |---|---:|---:| -| Rung B (needs the AI's own orders) | 42 | 14 | -| Rung A, implemented, gated on a **game-data root** (`--data`) | 6 | 5 | -| Rung A, closable, no blocker | 2 | 5 | -| not identifiable from the corpus (`Status`) | 4 | 0 | +| Rung B (needs the AI's own orders): M1, M2, M3, half of M4, M8 | 44 | 15 | +| Rung A, implemented, gated on a **game-data root** (`--data`): half of M4 | 4 | 3 | +| Rung A, closed by this lane: `BnkEl` ×2, `PvSav` | 2 | 4 | +| Rung A, blocked on a named unread/unwired thing: `BnkPr` | 2 | 2 | +| not identifiable from the corpus: `Status` | 2 | 0 | -**Three quarters of the `/Sim/players` residual is Rung B**, and it all hangs off two AI -decisions on one player: pick a research target, and queue one destroyer. The Rung-A part is -small, and most of it is *already written* — it needs the harness to be handed the game's data +(`Status` is 4 leaves in the block, but two of them — players 496 and 512 — are on shadow +empires that also have M1 leaves; the row above counts each leaf once, under the *first* +mechanism that has to land. `Status` for the human and for the AI empire is the irreducible +pair.) + +**Four fifths of the `/Sim/players` residual is Rung B**, and it all hangs off two decisions by +one AI player: pick a research target, and queue one destroyer. The Rung-A part is small, and +most of what is left of it is *already written* — it needs the harness handed the game's data directory, not more reverse engineering. --- @@ -83,8 +100,8 @@ Measured, from the state-checksum tree over the three saves: worth recording before anyone models T34 from the pair-1 numbers alone. None of this is closable without `game/ai`. It is listed here so the next lane does not -re-derive it: **42 of the 54 leaves on pair 1 are one AI player's research pick and one -destroyer.** +re-derive it: **36 of the 54 leaves on pair 1 are one AI player's research pick and one +destroyer**, and four more are the events those two decisions post. ## 3. M4 — events @@ -108,7 +125,7 @@ at all. The trouble is that the two sets are **perfectly correlated with species**: the four that take the 4 are species 0 and 2, the four that do not are all species 4. Sweeping every scalar field -of the player record on `turn1-state`, **fourteen** of them split the roster exactly the same +of the player record on `turn1-state`, this many of them split the roster exactly the same way — `ReqCL`, `AMine`, `CnTrd`, `CnVItl`, `cdp`, `hadvs`, `harcc`, `hgs`, `PvMA`, `CstE`, `CstR`, `CstT`, `MinRate`, `MaxOH`, `NPTrk`, `PrGtTrf`, `TerraMod`, `pddm` — eighteen of them. Any one of them "fits". Only two corpus saves carry a non-zero `Status` at all, so there is one observation of one @@ -123,11 +140,30 @@ must submit for this player" — but it is a hypothesis with a workload attached `OnMessage+0xa15`, capturing the player index on every call for one turn. It names the set directly and costs one VM session. Instrument the *entry*, not the count. -## 5. What this lane changed +## 5. What this lane changed — measured -See the prediction block below; measured results follow it. +Predictions are in §6, written and committed before the build (`c852965`). All three held; the +one thing that came out differently is recorded in §7 rather than quietly corrected. ---- +| change | pair 1 closed | pair 2 closed | regressed | leaves | +|---|---:|---:|---:|---| +| **PL-1** S00 stamps `PvSav` from `Sav` | 0 | 2 | 0 | `Player[16]/PvSav`, `Player[576]/PvSav` | +| **PL-2** T31 recovers its difficulty column from the input save | 2 | 2 | 0 | `Player[16]/BnkEl`, `Player[32]/BnkEl` on both pairs | +| **PL-3** the protection factor is narrowed to float32 | 0 | 0 | 0 | none on this corpus, by construction | +| total | **2** | **4** | **0** | | + +Independently of the leaf count, PL-2 turns T31's self-check into a real one. It used to report +"BnkEl reproduced for 6 of 8" on `turn1-state` **regardless of the AI roster**, because it was +comparing its *post-turn* computation against the *pre-turn* stored value — so the only players +it could ever "reproduce" were the six whose limit does not move at all. That is thin coverage +of exactly the shape rule 15 describes, and the phase's own catalog text claimed "8 of 8 with +the AI flag supplied", which the binary never printed. The check is now taken at load, against +the colony state the save was written from, and it passes **for every live player of all eleven +corpus saves** — 7/7 or 8/8 on each. The whole max-income chain is now verified on eleven saves +instead of two. + +The column identification resolves to **1 AI and 1 non-AI on every one of the eleven saves**, +with the rest ambiguous, and needs no `--ai-player`. ## 6. Predictions, written before the build (rule 2) @@ -208,3 +244,56 @@ report it as thin coverage, not as verified. **Falsification.** If the factor were a `double` in the image the opcode would be `DC /1`, not `D8 /1`. If some *other* call site multiplies the same global as a qword, the storage is a double and this is wrong; the check is a cross-reference sweep on `0x00aedfdc`. + +--- + +## 7. Where the predictions were wrong + +**"Ambiguous" is not a synonym for "owns nothing."** PL-2 predicted the ambiguous bucket would +be the five players with no colonies. It is six, and the extra one is `Player[576] "Independent +Colony"`, which owns a system and carries a non-zero `BnkEl` of −665,806. The reason is in the +difficulty table itself: the AI row is selected by `isAI && !npc`, and player 576 is an NPC, so +both columns hand it the same row. The note the phase prints and the header comment in +`game/sim/player_turn.h` were both corrected to say so — an ambiguous player is one where the +two columns *agree*, which happens for two different reasons. + +That also disposed of falsification (b): player 576's limit is constant across all three saves, +and had the identification been wrong about it the commit would have regressed that leaf. +It did not; 576's post-turn max income reproduces the same −665,806. + +Falsification (a) and (c) did not fire: no player of any corpus save came out `Unidentified`, +and no leaf regressed on either pair. + +--- +**The float/double disagreement rate is 10%, not "roughly 1.1%".** PL-3 estimated the rate from +the size of the gap (`3.3 - (double)3.3f` is 4.768e-8, which at a max income of 240,000 is about +0.011 of a unit, so ~1.1% of values should straddle an integer). Measured over every integer max +income from 230,000 to 250,000: **2,000 of 20,000 disagree — exactly 10%, and exactly the +multiples of ten.** The estimate was wrong because it assumed the products are uniformly +distributed against the integer grid and they are not: `3.3 x 10k` is an exact integer in +decimal for every k, so the decimal lands *on* the boundary at every tenth value and the +image's float lands just below it. The first disagreement is at `maxIncome = 10` +(33.0 against 32.99999952), not somewhere out in the hundreds of thousands. + +That makes the fix meaningfully larger than predicted and the corpus' silence about it +correspondingly more suspicious: seven records, none of them a multiple of ten. + +--- + +## 8. The ranked remainder + +Ranked by leaves per unit of effort, both pairs summed, with the blocker named. Nothing here is +"more analysis of `/Sim/players`" — every row is a named thing somewhere else. + +| rank | item | leaves (p1 + p2) | effort | what it needs | +|---|---|---:|---|---| +| 1 | **give the metric harness a data root** | 4 + 3 = **7** | one line in `tools/standalone_report.py` | `--data --commit-blocked=P11 --ai-player 1 --ai-player 2 --ai-player 3`. Already measured end to end on CT111 against `/srv/re-lab/gob-extract`: **87 closed / 0 regressed** on pair 1 and **46 / 0** on pair 2, residual 122 and 62. It closes nothing new in the engine; it stops the metric reporting an argument as if it were a gap | +| 2 | **wire `StrategyVars.txt` into a `TuningTable` and set `opt.haveTuning`** | 2 + 2 = **4** | small, contained | `TurnOptions::haveTuning` is **read in three places and never assigned** — the field is dead. `game/config`'s loader exists and the data root already parses; nothing else is missing. Unblocks `BnkPr` in T31 (and, per rule 23, it must be narrowed to float32 on the way in — that is done, PL-3). Also unblocks whatever `RunCivilianGrowth` does with the flag | +| 3 | **`game/ai`: research target selection** | 12 + 2 = **14** | a module | `ResRate`/`ResTNm` are the AI's own orders, and every tech-tree leaf follows from them. Three players do it per turn (the AI empire and both Singularity shadows) | +| 4 | **`game/ai`: the build order** | 15 + 9 = **24**, plus M3's 10 | a module | one destroyer, queued during the turn, deducted 11,900 from the treasury *before* spine phase 0. It is the single largest cause in the block and lane B6 already proved it cannot come from the save | +| 5 | **T34 `RecordObservedDesigns`** | (10, downstream of 4) | small once 4 lands | the mechanism is now readable off the corpus (§2): a design's creator registers the design in its own `odes`, its weapons in `owep`, and re-stamps `otnL` on the design's techs in `otch`; `odes` re-stamps on *build*, `otch` on *creation*. Two different refresh rules from one workload — treat both as hypotheses until a second workload exists | +| 6 | **`Status`'s predicate** | 4 + 0 = **4** | one VM session | an entry probe on W2's writer at `OnMessage+0xa15`, capturing the player index per call (§4). Not a reading problem — eighteen fields fit the one observation the corpus offers | +| 7 | **M8: player 528's one-off design** | 2 + 0 = **2** | unknown | a non-empire faction ("Alien Menace") creates `Des[1712 "Refugee Trade Ship"]` on turn 1→2 and never again, and the id it takes is the first of the three `NMnx` issues that turn. Nothing in the corpus explains what triggers it. Lowest rank because two leaves is not worth a hunt until something else brings us into that code | + +**What is NOT on this list, deliberately:** more decomposition of `/Sim/players`. The block is +now fully attributed — every one of the 54 and 24 leaves belongs to a row above. diff --git a/src/app/phase_catalog.cpp b/src/app/phase_catalog.cpp index 47ebf2b..e398ea0 100644 --- a/src/app/phase_catalog.cpp +++ b/src/app/phase_catalog.cpp @@ -57,8 +57,14 @@ constexpr PhaseDesc kHost[] = { constexpr PhaseDesc kStrategic[] = { {Driver::Strategic, 0, "S00", "SnapshotPreviousTurn", PhaseStatus::Partial, - "bumps the modification counter; the previous-turn shadow-word snapshot is not modelled " - "(the shadow words are not all identified on the wire)"}, + "bumps the modification counter and stamps PvSav from Sav on every live player -- the " + "treasury's previous-turn shadow, taken before any phase of the turn can move it, which " + "is what makes the tail's Sav-PvSav the turn's own net. Measured on the reference pairs: " + "0 closed on turn1->turn2 (the snapshot is a no-op there, every treasury is untouched) " + "and 2 closed on turn2->turn3, 0 regressed on both. It is short by exactly 11,900 on the " + "one AI empire, whose queued build order is deducted before this phase and exists nowhere " + "in the input file (Rung B). The REST of the shadow words are still not modelled: they " + "are not all identified on the wire"}, {Driver::Strategic, 1, "S01", "SystemPrePassMoraleAndAbandon", PhaseStatus::Stub, "per-system morale event + the abandon/chaos check below the minimum chaos population"}, {Driver::Strategic, 2, "S02", "TradeManagerTurn", PhaseStatus::Stub, @@ -268,18 +274,24 @@ constexpr PhaseDesc kTail[] = { {Driver::Tail, 28, "T28", "UpdateNodeLineSightingMasks", PhaseStatus::Stub, ""}, {Driver::Tail, 29, "T29", "AbortInvisibleInterceptOrders", PhaseStatus::Stub, ""}, {Driver::Tail, 30, "T30", "RebuildCommunicationMasks", PhaseStatus::Stub, ""}, - {Driver::Tail, 31, "T31", "UpdateBankruptcyLimits", PhaseStatus::Blocked, - "the whole chain is now modelled: sum over owned, non-abandoned systems of " - "max(ComputeMaxIncome, 0), and the phase self-checks it every run against the BnkEl the " - "input save already carries -- 8 of 8 players on turn1-state with the AI flag supplied. " - "Two things keep it blocked and neither is the formula. First, `ServerPlayer+0xf9` (is " - "this player AI?) is a game-setup input the save does not carry, and it selects a " - "difficulty column worth x1.1 on an AI empire; --ai-player N supplies it. Second, BnkPr " - "needs BANKRUPTCY_PROTECTION_LIMIT_FACTOR from the data files, so it is offered only " - "with a tuning table loaded. It used to close NOTHING because the limits move with the " - "CIVILIAN population and that growth was not committed; now that S11 commits it, measured " - "with --commit-blocked=T31 --ai-player 1: 2 closed, 0 regressed on EACH reference pair " - "-- BnkEl for the human and for the AI. BnkPr still needs the tuning factor"}, + {Driver::Tail, 31, "T31", "UpdateBankruptcyLimits", PhaseStatus::Partial, + "BnkEl is modelled and committed; BnkPr is not. The chain is the sum over owned, " + "non-abandoned systems of max(ComputeMaxIncome, 0). CORRECTION (lane PL): this phase used " + "to claim it self-checked against the BnkEl the input save carries, and it did not -- it " + "compared its POST-turn computation against the PRE-turn stored value, so the only " + "players it could ever 'reproduce' were the six whose limit does not move, and its " + "6-of-8 was thin coverage of exactly the shape rule 15 describes. The check is now taken " + "at LOAD, from the colony state the save was written from, and it passes 8 of 8 on the " + "reference saves and for every live player of all 11 corpus saves. That same check " + "replaces the operator flag: `ServerPlayer+0xf9` (is this player AI?) is not on the wire, " + "but it is worth 10% of the max income and BnkEl has a 6.67x slope, so recomputing the " + "stored limit under both difficulty columns identifies which one was used -- 1 AI and 1 " + "non-AI on every corpus save, the rest ambiguous because both columns agree there (zero " + "income, or an NPC, whose row is gated on `isAI && !npc`). A player neither column " + "reproduces abstains and keeps its pass-through leaf. Measured with NO operator input: 2 " + "closed, 0 regressed on EACH reference pair. What is left is BnkPr, which needs " + "BANKRUPTCY_PROTECTION_LIMIT_FACTOR from the data files -- it is read `fmul dword ptr`, " + "so it is a float32 in the image and the engine now narrows it (rule 23)"}, {Driver::Tail, 32, "T32", "PostIncomingFleetWarnings", PhaseStatus::Stub, ""}, {Driver::Tail, 33, "T33", "ShipManagerEndOfTurnHooks", PhaseStatus::Stub, ""}, {Driver::Tail, 34, "T34", "RecordObservedDesigns", PhaseStatus::Stub, ""}, diff --git a/src/app/turn.cpp b/src/app/turn.cpp index 6caceb1..b47291d 100644 --- a/src/app/turn.cpp +++ b/src/app/turn.cpp @@ -18,6 +18,7 @@ #include "game/sim/colony.h" #include "game/sim/economy.h" #include "game/sim/numeric.h" +#include "game/sim/player_turn.h" #include "game/sim/rng.h" #include "game/sim/tuning.h" @@ -584,81 +585,151 @@ std::vector BuildBudgetFeeds(const SaveGame& game, const TurnO return feeds; } -void RunUpdateBankruptcyLimits(SaveGame& game, const TurnOptions& opt, PhaseRecord& rec) { +// A system table keyed by the handle id a player's `OwnId` list carries. The list is short +// enough that a linear probe is cheaper than a map, and keeping it a value type means both +// callers below build it the same way. +class SystemIndex { +public: + explicit SystemIndex(const SaveGame& game) { + for (const auto& e : game.sim.systems) { + ids_.push_back(e.sysID); + sys_.push_back(&e.sys); + } + } + const Sys* Find(std::int32_t id) const { + for (std::size_t i = 0; i < ids_.size(); ++i) + if (ids_[i] == id) return sys_[i]; + return nullptr; + } + +private: + std::vector ids_; + std::vector sys_; +}; + +// `sum over owned, non-abandoned systems of max(ComputeMaxIncome(s), 0)` for one player. +int PlayerMaxIncome(const Player& p, bool isAI, const SystemIndex& index, + const MaxIncomeInputs& ctx, int* dangling = nullptr) { + int maxIncome = 0; + for (std::int32_t id : p.owners) { + const Sys* s = index.Find(id); + if (!s) { + if (dangling) ++*dangling; + continue; + } + if (s->abdn) continue; // an abandoned colony is skipped, not counted as zero + maxIncome += SystemMaxIncomeFromWire(*s, p, isAI, ctx); + } + return maxIncome; +} + +// Which difficulty column each player's income was computed under, recovered from the input +// save before any phase mutates it. See `game/sim/player_turn.h` for why this is possible at +// all; the short version is that `BnkEl` is a 6.67x-slope function of the max income, so the +// 10% the AI column adds cannot hide inside a truncation. +// +// Taken at load, and it has to be: T31 runs at the end of the turn, by which point the colony +// state is the POST-turn one and no longer the state the stored limit was written from. +std::vector IdentifyDifficultyColumns(const SaveGame& game) { MaxIncomeInputs ctx; ctx.serverIncomeMod = game.sim.incMod; for (const auto& sp : game.sim.species) ctx.idealSuit.push_back(sp.issu); + const SystemIndex index(game); - // The system table, keyed by the handle id a player's `OwnId` list carries. - std::vector byId; - std::vector ids; - for (const auto& e : game.sim.systems) { - ids.push_back(e.sysID); - byId.push_back(&e.sys); + std::vector out; + out.reserve(game.sim.players.size()); + for (const auto& pe : game.sim.players) { + const Player& p = pe.player; + out.push_back(sim::IdentifyDifficultyColumn( + p.bnkEl, PlayerMaxIncome(p, false, index, ctx), + PlayerMaxIncome(p, true, index, ctx), ctx.tuning)); } - const auto find = [&](std::int32_t id) -> const Sys* { - for (std::size_t i = 0; i < ids.size(); ++i) - if (ids[i] == id) return byId[i]; - return nullptr; - }; + return out; +} - int players = 0, dangling = 0, matches = 0, compared = 0, aiOwned = 0; +void RunUpdateBankruptcyLimits(SaveGame& game, const TurnOptions& opt, PhaseRecord& rec, + const std::vector& columns) { + MaxIncomeInputs ctx; + ctx.serverIncomeMod = game.sim.incMod; + for (const auto& sp : game.sim.species) ctx.idealSuit.push_back(sp.issu); + const SystemIndex index(game); + + int players = 0, dangling = 0, reproduced = 0, compared = 0; + int identifiedAI = 0, identifiedNonAI = 0, ambiguous = 0, unidentified = 0, overridden = 0; std::string firstMiss; - for (auto& pe : game.sim.players) { - Player& p = pe.player; + for (std::size_t i = 0; i < game.sim.players.size(); ++i) { + Player& p = game.sim.players[i].player; if (p.elim) continue; ++players; - const bool isAI = opt.IsAIPlayer(p.plyrIdx); - if (isAI) ++aiOwned; - int maxIncome = 0; - for (std::int32_t id : p.owners) { - const Sys* s = find(id); - if (!s) { - ++dangling; - continue; - } - if (s->abdn) continue; // an abandoned colony is skipped, not counted as zero - maxIncome += SystemMaxIncomeFromWire(*s, p, isAI, ctx); - } - const sim::BankruptcyLimits lim = - sim::ComputeBankruptcyLimits(maxIncome, ctx.tuning); - // The limits the save already carries were computed by the ORIGINAL at the end of the - // previous turn from the same colony state, so comparing against them is a check of - // the whole income chain that needs no running game -- the same "testable on load" - // property the turn-record phase has. ++compared; - if (lim.eliminationFloor == p.bnkEl) { - ++matches; - } else if (firstMiss.empty()) { - firstMiss = fmt("player %d: BnkEl ours %d, save %d (maxIncome %d)", p.plyrIdx, - lim.eliminationFloor, p.bnkEl, maxIncome); + + // The column, from this player's own record where the save could settle it, and from + // the operator's roster only where it could not. + const sim::DifficultyColumnEvidence ev = + i < columns.size() ? columns[i] : sim::DifficultyColumnEvidence{}; + bool isAI = false; + switch (ev.column) { + case sim::DifficultyColumn::AI: isAI = true; ++identifiedAI; break; + case sim::DifficultyColumn::NonAI: ++identifiedNonAI; break; + case sim::DifficultyColumn::Ambiguous: + ++ambiguous; + isAI = opt.IsAIPlayer(p.plyrIdx); + break; + case sim::DifficultyColumn::Unidentified: + ++unidentified; + isAI = opt.IsAIPlayer(p.plyrIdx); + if (isAI) ++overridden; + break; } + if (ev.Reproduced()) ++reproduced; + else if (firstMiss.empty()) + firstMiss = fmt("player %d: the input save's BnkEl is %d and neither column " + "reproduces it (non-AI %d, AI %d) -- this player abstains", + p.plyrIdx, ev.storedLimit, ev.nonAiLimit, ev.aiLimit); + + const int maxIncome = PlayerMaxIncome(p, isAI, index, ctx, &dangling); + const sim::BankruptcyLimits lim = sim::ComputeBankruptcyLimits(maxIncome, ctx.tuning); ++rec.invocations; + // BnkPr's factor is a data-file constant; with no tuning table its computed value is // -0 for every player, which is a confidently wrong leaf rather than a missing one. // It is therefore only offered when the table is loaded. - int would = lim.eliminationFloor != p.bnkEl ? 1 : 0; const bool prModelled = opt.haveTuning; + int would = lim.eliminationFloor != p.bnkEl ? 1 : 0; if (prModelled && lim.protectionLimit != p.bnkPr) ++would; - if (opt.CommitBlocked("T31")) { + + // Committing is per player, and the gate is this player's own record: write only + // where the same chain, run on the state the save was written from, reproduced the + // limit the save carries. Where it did not, the phase leaves the leaf alone -- a + // pass-through leaf that may still be right beats a computed one that is known to be + // wrong. `--commit-blocked=T31` remains the operator's override for the rest. + const bool commit = ev.Reproduced() || opt.CommitBlocked("T31"); + if (commit) { rec.leafWrites += would; p.bnkEl = lim.eliminationFloor; if (prModelled) p.bnkPr = lim.protectionLimit; - rec.committed = true; + if (would) rec.committed = true; } else { rec.wouldWrite += would; } } - rec.notes.push_back(fmt("%d player(s); BnkEl reproduced for %d of %d from the input save's " - "own colony state", - players, matches, compared)); + rec.notes.push_back(fmt("%d player(s); the input save's own BnkEl was reproduced for %d " + "of %d from the colony state it carries", + players, reproduced, compared)); + rec.notes.push_back(fmt("difficulty column recovered from the save: %d AI, %d non-AI, " + "%d ambiguous, %d unidentified", + identifiedAI, identifiedNonAI, ambiguous, unidentified)); + if (ambiguous) + rec.notes.push_back("ambiguous means both columns produce the SAME limit, so the flag " + "cannot matter for that player -- either the max income is zero, " + "or the player is an NPC, and the difficulty table's AI row is " + "gated on `isAI && !npc`"); + if (overridden) + rec.notes.push_back(fmt("%d unidentified player(s) took the operator's --ai-player " + "roster instead", overridden)); if (!firstMiss.empty()) rec.notes.push_back(firstMiss); if (dangling) rec.notes.push_back(fmt("%d owned-system id(s) absent from the system table", dangling)); - if (aiOwned == 0) - rec.notes.push_back("no player was declared AI (--ai-player N); every player therefore " - "takes the non-AI difficulty column, which is 1/1.1 low on an AI " - "empire at difficulty level 1"); // BnkPr's factor is a data-file constant (BANKRUPTCY_PROTECTION_LIMIT_FACTOR), so with no // tuning table loaded the protection limit is not modelled even though BnkEl is. if (!opt.haveTuning) @@ -890,6 +961,12 @@ TurnResult RunStrategicTurn(SaveGame& game, const TurnOptions& opt) { const TurnRecordAudit recordAudit = AuditTurnRecordsAgainstSave(game, opt.catalog ? &inputCensus : nullptr); + // Also taken before any phase runs, and for the same reason: T31's difficulty column is + // recovered by recomputing each player's bankruptcy limit from the colony state the save + // was written from, which stops existing the moment S11 commits growth. + const std::vector difficultyColumns = + IdentifyDifficultyColumns(game); + // The event text. The engine holds keys; the text comes from the operator's own installed // string table, which arrives with the data root or not at all. const EventTextTable eventText(opt.catalog && opt.catalog->strings_loaded @@ -965,6 +1042,26 @@ TurnResult RunStrategicTurn(SaveGame& game, const TurnOptions& opt) { rec.notes.push_back("the real turn advances this counter 12-44 times, from " "writers spread across both drivers; only this one is " "modelled, so the leaf will not match yet"); + // The treasury's previous-turn shadow. It is a copy of `Sav` taken here, + // before any phase of the turn can move it, which is what makes the tail's + // `Sav - PvSav` the turn's own net. + int stamped = 0, moved = 0; + for (auto& pe : game.sim.players) { + Player& p = pe.player; + if (p.elim) continue; + const int was = p.pvSav; + p.pvSav = sim::SnapshotPreviousTurn(p.sav); + ++stamped; + if (p.pvSav != was) ++moved; + } + rec.invocations += stamped; + rec.leafWrites += moved; + rec.notes.push_back(fmt("PvSav stamped from Sav on %d player(s); %d leaf(s) " + "moved", stamped, moved)); + rec.notes.push_back("a player whose treasury is spent between the file being " + "written and this phase -- the AI's queue-time build " + "deduction -- lands 11,900 high here, and that order is " + "not in the input save (Rung B)"); break; } case 4: { // S04 RebuildAllianceMasks @@ -1231,7 +1328,7 @@ TurnResult RunStrategicTurn(SaveGame& game, const TurnOptions& opt) { rec.committed = v.leafWrites > 0; rec.notes = v.notes; } else if (tp[i].index == 31) { - RunUpdateBankruptcyLimits(game, opt, rec); + RunUpdateBankruptcyLimits(game, opt, rec, difficultyColumns); } else if (tp[i].index == 36) { RunFinalizeTurnRecords(game, opt, rec, recordAudit, allianceMasks); } diff --git a/src/game/sim/CMakeLists.txt b/src/game/sim/CMakeLists.txt index df5b5bd..c9aa890 100644 --- a/src/game/sim/CMakeLists.txt +++ b/src/game/sim/CMakeLists.txt @@ -9,7 +9,8 @@ add_library(sots_game_sim STATIC colony.cpp movement.cpp visibility.cpp - techgraph.cpp) + techgraph.cpp + player_turn.cpp) target_include_directories(sots_game_sim PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/../..) target_compile_features(sots_game_sim PUBLIC cxx_std_17) if(NOT MSVC) @@ -19,7 +20,7 @@ endif() option(SOTS_GAME_SIM_TESTS "Build the game/sim unit tests" OFF) if(SOTS_GAME_SIM_TESTS) enable_testing() - set(_sim_tests economy research colony movement techgraph visibility construction) + set(_sim_tests economy research colony movement techgraph visibility construction player_turn) foreach(_t IN LISTS _sim_tests) add_executable(game_sim_test_${_t} ${CMAKE_CURRENT_SOURCE_DIR}/../../../tests/game_sim/test_${_t}.cpp) target_link_libraries(game_sim_test_${_t} PRIVATE sots_game_sim) diff --git a/src/game/sim/economy.cpp b/src/game/sim/economy.cpp index 54f635d..41792d0 100644 --- a/src/game/sim/economy.cpp +++ b/src/game/sim/economy.cpp @@ -192,7 +192,21 @@ BankruptcyLimits ComputeBankruptcyLimits(int maxIncome, const TuningTable& t) { std::max(Ftol(static_cast(maxIncome) / kBankruptcyInterestDivisor), -kTreasuryLimit); // Protection: the tuned factor times the maximum income, never below the floor. - l.protectionLimit = std::max(-Ftol(t.BANKRUPTCY_PROTECTION_LIMIT_FACTOR * static_cast(maxIncome)), + // + // The factor is NARROWED TO FLOAT32 first, and that is not a guess about the data file: + // the two constants in this routine are loaded by different opcodes. The divisor above is + // `DD /0` -- `fld qword`, a double in .rdata. The factor is + // + // mov ecx, [PTR_g_BANKRUPTCY_PROTECTION_LIMIT_FACTOR] + // fmul dword ptr [ecx] ; D8 /1 == m32fp + // + // so whatever decimal the data file carries is a 4-byte float by the time it multiplies. + // A `double` multiply here truncates to a different integer for roughly 1% of max-income + // values at the corpus' empire size, and for more as the empire grows. The corpus cannot + // see it -- all seven of its non-zero `BnkPr` records land where the two agree -- so this + // is an instruction-stream reading, not a measured leaf (rule 23). + l.protectionLimit = std::max(-Ftol(F32(t.BANKRUPTCY_PROTECTION_LIMIT_FACTOR) * + static_cast(maxIncome)), l.eliminationFloor); return l; } diff --git a/src/game/sim/economy.h b/src/game/sim/economy.h index bfd99cf..c5a3036 100644 --- a/src/game/sim/economy.h +++ b/src/game/sim/economy.h @@ -211,7 +211,11 @@ struct BankruptcyLimits { // Limits from the sum of every owned system's maximum money output: // eliminationFloor = max(ftol(maxIncome / kBankruptcyInterestDivisor), -2e9) // (the debt at which 15 %/turn interest eats the whole maximum income) -// protectionLimit = max(-ftol(BANKRUPTCY_PROTECTION_LIMIT_FACTOR x maxIncome), eliminationFloor) +// protectionLimit = max(-ftol(float32(BANKRUPTCY_PROTECTION_LIMIT_FACTOR) x maxIncome), +// eliminationFloor) +// The factor is narrowed to float32 first: the image reads it `fmul dword ptr` while the +// divisor above is `fld qword ptr`, so the two constants of this one routine are stored at +// different widths. See the note at the call site. // The limits a turn's check uses are the ones computed at the end of the previous turn // (and on load); the caller keeps them on the player. // CONFIDENCE: high -- the factor is on the protection limit, the elimination limit is diff --git a/src/game/sim/player_turn.cpp b/src/game/sim/player_turn.cpp new file mode 100644 index 0000000..bc25487 --- /dev/null +++ b/src/game/sim/player_turn.cpp @@ -0,0 +1,27 @@ +#include "game/sim/player_turn.h" + +namespace sots::sim { + +DifficultyColumnEvidence IdentifyDifficultyColumn(int storedEliminationFloor, + int nonAiMaxIncome, int aiMaxIncome, + const TuningTable& t) { + DifficultyColumnEvidence e; + e.storedLimit = storedEliminationFloor; + e.nonAiLimit = ComputeBankruptcyLimits(nonAiMaxIncome, t).eliminationFloor; + e.aiLimit = ComputeBankruptcyLimits(aiMaxIncome, t).eliminationFloor; + + const bool nonAiFits = e.nonAiLimit == storedEliminationFloor; + const bool aiFits = e.aiLimit == storedEliminationFloor; + if (nonAiFits && aiFits) { + e.column = DifficultyColumn::Ambiguous; + } else if (nonAiFits) { + e.column = DifficultyColumn::NonAI; + } else if (aiFits) { + e.column = DifficultyColumn::AI; + } else { + e.column = DifficultyColumn::Unidentified; + } + return e; +} + +} // namespace sots::sim diff --git a/src/game/sim/player_turn.h b/src/game/sim/player_turn.h new file mode 100644 index 0000000..caebb09 --- /dev/null +++ b/src/game/sim/player_turn.h @@ -0,0 +1,77 @@ +// Per-player steps of the strategic turn that belong to the player record itself, rather +// than to a colony or a fleet. +// +// Two things live here today, and they are unrelated except that both are about a player's +// own words: +// +// * `SnapshotPreviousTurn` -- spine phase S00's shadow of the treasury. +// * `IdentifyDifficultyColumn` -- recovering, from the save, which difficulty column a +// player's income was computed under, so that tail phase T31 does not need to be told. +#pragma once + +#include "game/sim/economy.h" +#include "game/sim/tuning.h" + +namespace sots::sim { + +// ---- S00 SnapshotPreviousTurn ---------------------------------------------------------- +// +// The first thing the strategic spine does is stamp the treasury into its previous-turn +// shadow, so that the turn's own income can later be reported as a delta (the tail's turn +// record carries `Sav - PvSav`). It is a copy, taken BEFORE any phase of the turn runs. +// +// Measured over the corpus: `PvSav(turn N+1) == Sav(turn N)` exactly, for every player whose +// treasury is not touched between the save being written and the first spine phase. The one +// player in the corpus where it does not hold is the AI empire whose queued build order is +// deducted before phase 0 -- an order that exists nowhere in the input file (see +// `docs/B6-ship-construction.md`), which is a Rung-B gap and not a defect of this step. +inline int SnapshotPreviousTurn(int savings) { return savings; } + +// ---- T31's difficulty column ------------------------------------------------------------ +// +// `ServerPlayer+0xf9` -- "is this player AI?" -- is copied from the game-setup record and +// never serialised, and it selects a column of the difficulty table that is worth a x1.1 on +// an AI empire's income at the corpus' difficulty level. That made T31 need an operator flag. +// +// It does not have to. The input save already carries `BnkEl`, which the original wrote at +// the end of the previous turn from exactly the colony state the save holds. Recomputing it +// from that state under BOTH columns and comparing with the stored value identifies the +// column: the two columns differ by 10% of the max income, which is far more than a +// truncation, so at most one of them can match on a player with any income at all. +// +// Two kinds of player make the two columns collapse onto one value, and both are reported as +// `Ambiguous` rather than as an identification -- the column is genuinely unknown, and it +// also genuinely cannot matter, and those are two different statements: +// +// * a player who owns nothing: max income is zero under either column; +// * an NPC: the difficulty table's AI row is gated on `isAI && !npc`, so an NPC takes the +// non-AI row whatever the flag says. Six of the corpus' eight players are NPCs, and one +// of them ("Independent Colony") owns a colony and carries a non-zero limit -- so +// `Ambiguous` is NOT a synonym for "owns nothing". +enum class DifficultyColumn { + Unidentified, // neither column reproduces the stored limit -- the model is off somewhere + Ambiguous, // both columns reproduce it (max income is zero); the choice cannot matter + NonAI, + AI, +}; + +struct DifficultyColumnEvidence { + int storedLimit = 0; // the `BnkEl` the input save carries + int nonAiLimit = 0; // what the non-AI column computes from the input colony state + int aiLimit = 0; // what the AI column computes from it + DifficultyColumn column = DifficultyColumn::Unidentified; + + // Whether the identification says to run this player through the AI column. `Ambiguous` + // and `Unidentified` answer false, and the caller is expected to fall back to whatever + // the operator supplied in those two cases rather than to trust this. + bool IsAI() const { return column == DifficultyColumn::AI; } + // True when the model reproduced the value the save already carries -- i.e. when the + // whole max-income chain is verified against this player's own record on load. + bool Reproduced() const { return column != DifficultyColumn::Unidentified; } +}; + +DifficultyColumnEvidence IdentifyDifficultyColumn(int storedEliminationFloor, + int nonAiMaxIncome, int aiMaxIncome, + const TuningTable& t); + +} // namespace sots::sim diff --git a/tests/game_sim/CMakeLists.txt b/tests/game_sim/CMakeLists.txt index 559703c..64b4caa 100644 --- a/tests/game_sim/CMakeLists.txt +++ b/tests/game_sim/CMakeLists.txt @@ -1,5 +1,5 @@ # game/sim tests: four hand-computed suites + a real-save smoke test (skips unless SOTS_SAVES_JSON). -foreach(_t economy research colony movement techgraph visibility construction) +foreach(_t economy research colony movement techgraph visibility construction player_turn) add_executable(game_sim_test_${_t} test_${_t}.cpp) target_link_libraries(game_sim_test_${_t} PRIVATE sots_game_sim) target_include_directories(game_sim_test_${_t} PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}) diff --git a/tests/game_sim/test_economy.cpp b/tests/game_sim/test_economy.cpp index 18dbb96..4e922c3 100644 --- a/tests/game_sim/test_economy.cpp +++ b/tests/game_sim/test_economy.cpp @@ -303,19 +303,33 @@ static void test_bankruptcy() { TuningTable t; t.BANKRUPTCY_PROTECTION_LIMIT_FACTOR = 3.3; t.BANKRUPTCY_ELIMINATION_TURNS = 5; - // max income 1000: elimination at 1000 / -0.15 = -6666.67 -> -6666; protection -3300 + // LANE PL, rule 23: the protection factor is read `fmul dword ptr`, so it is a FLOAT32 + // in the image whatever the data file's decimal is. 3.3f is 3.299999952316284, and the + // conversion TRUNCATES, so every expectation below whose product lands on an exact + // integer under the decimal moves down by one. The two constants disagree at every + // multiple of ten -- exactly 10% of integer max incomes, starting at maxIncome = 10, + // where the decimal gives 33.0 and the image's float gives 32.99999952. + // + // max income 1000: elimination at 1000 / -0.15 = -6666.67 -> -6666; protection -3299 BankruptcyLimits l = ComputeBankruptcyLimits(1000, t); CHECK_EQ(l.eliminationFloor, -6666); - CHECK_EQ(l.protectionLimit, -3300); + CHECK_EQ(l.protectionLimit, -3299); CHECK_EQ(BankruptcyLevel(-6667, l), 2); CHECK_EQ(BankruptcyLevel(-6666, l), 1); - CHECK_EQ(BankruptcyLevel(-3301, l), 1); - CHECK_EQ(BankruptcyLevel(-3300, l), 0); + CHECK_EQ(BankruptcyLevel(-3300, l), 1); + CHECK_EQ(BankruptcyLevel(-3299, l), 0); CHECK_EQ(BankruptcyLevel(0, l), 0); l = ComputeBankruptcyLimits(100, t); CHECK_EQ(l.eliminationFloor, -666); - CHECK_EQ(l.protectionLimit, -330); + CHECK_EQ(l.protectionLimit, -329); + + // the boundary the corpus never presents (rule 23): a max income NOT a multiple of ten, + // where the two constants agree, next to one that is, where they do not. + l = ComputeBankruptcyLimits(1001, t); + CHECK_EQ(l.protectionLimit, -3303); // 3.3 x 1001 = 3303.3 either way + l = ComputeBankruptcyLimits(10, t); + CHECK_EQ(l.protectionLimit, -32); // decimal would say -33 // a factor beyond the interest break-even is pinned to the elimination floor TuningTable big = t; @@ -327,7 +341,7 @@ static void test_bankruptcy() { // huge income: the floor saturates at the treasury limit l = ComputeBankruptcyLimits(400000000, t); CHECK_EQ(l.eliminationFloor, -2000000000); - CHECK_EQ(l.protectionLimit, -1320000000); + CHECK_EQ(l.protectionLimit, -1319999980); // LANE N: the divisor is the widened float literal, not the decimal -0.15. The two // disagree for every maxIncome divisible by 3, and maxIncome = 3 is where it first diff --git a/tests/game_sim/test_player_turn.cpp b/tests/game_sim/test_player_turn.cpp new file mode 100644 index 0000000..6c0c160 --- /dev/null +++ b/tests/game_sim/test_player_turn.cpp @@ -0,0 +1,88 @@ +#include "game/sim/player_turn.h" + +#include "check.h" + +using namespace sots::sim; + +static TuningTable tuning() { + TuningTable t; + t.BANKRUPTCY_PROTECTION_LIMIT_FACTOR = 3.3; + return t; +} + +static void test_snapshot() { + // S00 is a plain copy, and the point of the test is that it is a copy of the value + // BEFORE the turn, so the identity has to hold for the negative and the saturated cases + // too rather than only for the corpus' positive treasuries. + CHECK_EQ(SnapshotPreviousTurn(50000), 50000); + CHECK_EQ(SnapshotPreviousTurn(0), 0); + CHECK_EQ(SnapshotPreviousTurn(-1234), -1234); + CHECK_EQ(SnapshotPreviousTurn(2000000000), 2000000000); +} + +// The corpus' two real empires, with the max incomes recovered by inverting the `BnkEl` each +// save carries. Player 0 is not AI; player 1 is, and its income therefore carries the x1.1. +static void test_column_identified_from_the_stored_limit() { + const TuningTable t = tuning(); + + // turn1-state, player 0 ("re"): stored BnkEl -1,590,613, max income 238,592. + DifficultyColumnEvidence e = + IdentifyDifficultyColumn(-1590613, 238592, 262451, t); + CHECK(e.column == DifficultyColumn::NonAI); + CHECK(!e.IsAI()); + CHECK(e.Reproduced()); + + // turn1-state, player 1 ("Fane Lao"): stored -1,811,273, max income 271,691 -- which is + // what the AI column produces; the non-AI column would produce 271691 / 1.1. + e = IdentifyDifficultyColumn(-1811273, 246992, 271691, t); + CHECK(e.column == DifficultyColumn::AI); + CHECK(e.IsAI()); + CHECK(e.Reproduced()); +} + +static void test_zero_income_is_ambiguous_not_identified() { + const TuningTable t = tuning(); + // Five of the corpus' eight players own nothing. Both columns reproduce the stored zero, + // and the honest answer is "the column is unknown and cannot matter" -- not "non-AI". + const DifficultyColumnEvidence e = IdentifyDifficultyColumn(0, 0, 0, t); + CHECK(e.column == DifficultyColumn::Ambiguous); + CHECK(!e.IsAI()); + CHECK(e.Reproduced()); + CHECK_EQ(e.nonAiLimit, 0); + CHECK_EQ(e.aiLimit, 0); +} + +static void test_neither_column_fits_is_unidentified() { + const TuningTable t = tuning(); + // A stored limit no column reproduces means the max-income chain is wrong for this + // player, and the caller must abstain rather than pick the nearer of two wrong answers. + const DifficultyColumnEvidence e = IdentifyDifficultyColumn(-1000000, 238592, 262451, t); + CHECK(e.column == DifficultyColumn::Unidentified); + CHECK(!e.IsAI()); + CHECK(!e.Reproduced()); +} + +// The identification only works because the two columns are far apart. A tenth of the income +// is thousands of units of BnkEl at any empire size that matters, so a truncation can never +// make the wrong column fit -- but at a max income of a handful of units they collapse, and +// the answer there is Ambiguous, which is correct and is worth pinning. +static void test_the_two_columns_collapse_only_at_a_trivial_income() { + const TuningTable t = tuning(); + DifficultyColumnEvidence e = IdentifyDifficultyColumn(-6, 1, 1, t); + CHECK(e.column == DifficultyColumn::Ambiguous); + + // one unit apart, and already separable + e = IdentifyDifficultyColumn(-133, 20, 22, t); + CHECK(e.column == DifficultyColumn::NonAI); + e = IdentifyDifficultyColumn(-146, 20, 22, t); + CHECK(e.column == DifficultyColumn::AI); +} + +int main() { + test_snapshot(); + test_column_identified_from_the_stored_limit(); + test_zero_income_is_ambiguous_not_identified(); + test_neither_column_fits_is_unidentified(); + test_the_two_columns_collapse_only_at_a_trivial_income(); + return simtest::finish("player_turn"); +}