From bebdee1ceb5c6a074d44a20d54c4a91c5e0e22f3 Mon Sep 17 00:00:00 2001 From: alex Date: Tue, 8 Sep 2026 13:49:48 -0400 Subject: [PATCH] income-term: the output -> money chain read from the instruction stream; the BnkEl oracle goes 6/25 -> 25/25 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `output-term.md` §6 said the verified output total did not unblock the budget because a system's MONEY is a second chain. This is that chain, disassembled to the next function start throughout. The multiplier `formula-gaps.md` Q3 could not name is a three-row table the executable BUILDS IN CODE from .rdata float literals -- no data-file key, no GlobalConst slot, the same shape lane N found for the pop-type table. ServerPlayer+0x36c is an unnamed, unsaved pointer to {int id; float ai[3]; float other[3]}, filled from that table by ServerPlayer::Read and selected per player by `is-AI && !NPC`. Every corpus save carries aidf == 1, whose AI income column is 1.1f. The record's other two columns are a fleet-maintenance DIVISOR and a RESEARCH multiplier -- Q3 called the third a trade multiplier and it is not. 25/25 on the oracle, from 6/25. The prediction of WHERE the remaining misses were was wrong and is written down as wrong: the twelve Zuul records were not missing the suitability cost (every corpus colony sits at its species' ideal, so that whole term is multiplied by zero and stays unexercised). They were missing SpeciesDef +0x4c/+0x50, which are PER SPECIES and were carried as one global pair -- 400 output points, 2000 money, per Zuul colony, and the observed 4400 and 5566 shortfalls fall out to the unit. New wire fact: the Sim block's ISsp/ISsu pairs ARE server->IdealSuit[], the float[7] CalcSuitMod indexes. The array is randomised per game by the map generator and cross-checks against every ServerPlayer's own IdealSuit field in all 11 saves, so the suitability cost needs no data file. Also states plainly what this does NOT unblock: ComputeBudget's turn path takes its per-system money from ComputeOutput with the system's OWN rate sliders, not from ComputeMaxIncome, so P01/P02/P03/P05/P06 and the two research RNG words stay blocked on a strictly larger function. --- findings/subsystems/formula-gaps.md | 24 ++ findings/subsystems/income-term.md | 428 ++++++++++++++++++++++++++++ ghidra/addresses.d/lane-e1.json | 101 +++++++ tools/max_income_predict.py | 317 +++++++++++++++----- 4 files changed, 793 insertions(+), 77 deletions(-) create mode 100644 findings/subsystems/income-term.md create mode 100644 ghidra/addresses.d/lane-e1.json diff --git a/findings/subsystems/formula-gaps.md b/findings/subsystems/formula-gaps.md index 2e0c993..c1e5e80 100644 --- a/findings/subsystems/formula-gaps.md +++ b/findings/subsystems/formula-gaps.md @@ -72,6 +72,30 @@ money = ftol(t − cost) Note `SuitTol` therefore caps the hazard **cost** as well as extending the habitable range. `ADDICTION_INCOME_MOD` is applied inside PopIncome (0x0074d760 → 0x00746910 morale/addiction factor) — not verified line by line (MEDIUM). +### Q3 addendum — lane E1, 2026-09-08 (instruction-verified, and the block above is now superseded) + +The whole chain is read instruction by instruction in `findings/subsystems/income-term.md`, which +replaces the MEDIUM-confidence sketch above. Five corrections: + +1. **`TradePointsToMoney` returns a double, not an int.** The `ftol` is in the caller + (`ComputeOutputFromRates`, `out[3] = _ftol2(...)`), not here. +2. **The block term is `(trade − fmod(trade,5)) × 5`** — the literal at 0x009e2398 is used twice, + as the modulus *and* as the multiplier, so a whole five-point block is worth **25** money. +3. **`PopIncome` truncates twice per (group, species) row** — once inside `GroupIncome` + (`ftol(typeIncomeMod × count/14000)`) and again after the morale and addiction factors. Summing + the species and truncating once is wrong on any multi-species colony. +4. **`DifficultyMods(owner)->+4` resolves.** `ServerPlayer+0x36c` is an unnamed, unsaved pointer to + a 0x1c record `{int id; float ai[3]; float other[3]}` filled by `LoadDifficultyRow` 0x005a3990 + from a three-row table **built in code** by 0x005a3870 (`BuildDifficultyTable`) — no data-file + key. The selector 0x0059b490 takes the AI triple iff `p->[0xf9] && !p->NPC`. Every corpus save + carries `aidf = 1`, whose AI income column is **1.1f** — the multiplier the `BnkEl` oracle + measured. The record's other two columns are a fleet-maintenance **divisor** and a **research** + multiplier, not a trade multiplier as this note said. +5. **`CalcSuitMod`'s ideal is the SERVER's per-species array**, `server->IdealSuit[species]` + (0x0080f4b0, raw base +0xf8), which is on the wire as the Sim block's `ISsp`/`ISsu` pairs — not + the owner's `IdealSuit` field. The species is the system's population species (`indi->indsp` on + an independent colony), and a `vnh` system pays no cost at all. + ## Q4. `POPBONUS_INC` population increment `ServerSystem::AccrueSystemBonus` 0x0074d4f0 (disasm 0x0074d53b–0x0074d5be): diff --git a/findings/subsystems/income-term.md b/findings/subsystems/income-term.md new file mode 100644 index 0000000..a0d21df --- /dev/null +++ b/findings/subsystems/income-term.md @@ -0,0 +1,428 @@ +# The output → money term (lane E1, 2026-09-08) + +`ServerSystem::ComputeMaxIncome` and its `TradePointsToMoney` tail — the second chain +`ComputeBudget` needs, and the one `output-term.md` §6 named as the reason the verified output +total did **not** unblock `P01 P02 P03 P05 P06`. + +Evidence: own `objdump -d` pass over `Sword of the Stars.exe`, every range disassembled **to the +next function start** and the real boundary found (rule 17). Field names per +`findings/objects/struct-recovery.md`. Everything below marked *instruction-verified* is read off +the instruction stream; everything marked *inferred* is not. + +--- + +## 1. The call chain + +| addr | name | conv / boundary | note | +|---|---|---|---| +| 0x007521c0 | `ServerSystem::ComputeMaxIncome()` → int | thiscall, ends 0x0075223d | Ghidra size 126 = correct | +| 0x00751bb0 | `ServerSystem::ComputeOutputFromRates(int out[12], OutputRates*)` | thiscall `ret 8`, ends 0x00751fa0 | **repairs ships** — not compare-safe | +| 0x007505b0 | `ServerSystem::TradePointsToMoney(double trade)` → double | thiscall `ret 8`, ends 0x007506c8 | Ghidra size 281 = correct | +| 0x0074d760 | `ServerSystem::PopIncome(int groupType)` → double | thiscall `ret 4`, ends 0x0074d8e1 | | +| 0x0074b700 | `ServerSystem::SlaveIncome()` → double | thiscall, ends 0x0074b793 | | +| 0x00535e80 | `GroupIncome(int groupType, int64 count)` → int | **cdecl**, tail-jumps `_ftol2` | the per-capita income term | +| 0x00535e00 | `PopTypeRow(int t)` | cdecl | same 3-row table as the output term | +| 0x007484d0 | `ServerSystem::CalcSuitMod(int species)` → double | thiscall `ret 4`, ends 0x00748560 | the suitability money **cost** | +| 0x0080f470 | `StrategyServer::IncomeDifficultyMod(ServerPlayer*)` → float | thiscall `ret 4`, ends 0x0080f49f | **the missing ×1.1** | +| 0x0059b490 | `DifficultyMods::Select(ServerPlayer*)` → float* | thiscall `ret 4`, ends 0x0059b4b3 | picks the AI or the non-AI triple | +| 0x005a3990 | `LoadDifficultyRow(int level, DiffRec* out)` | cdecl | fills the 0x1c record | +| 0x005a3870 | `BuildDifficultyTable(vector* out)` | thiscall | **the table is built in code** — §3 | +| 0x0080dd10 | `ServerPlayer::GetIncMod()` → float | thiscall | `player+0x30c` | +| 0x0080dd20 | `ServerPlayer::GetSpeciesCostFactor()` → float | thiscall | `SpeciesDef(Species)->+0x24` | +| 0x0080f4b0 | `StrategyServer::IdealSuit(int species)` → float | thiscall `ret 4` | `server+0xf8 + species*4` | +| 0x00746890 | `ServerSystem::TerraformPointsNeeded()` → double | thiscall | always ≥ 0 — §2.4 | +| 0x009dd1e4 | `ceil` (MSVCR100) | import | resolved from the IAT | +| 0x00925086 | `_CIfmod` (MSVCR100) | import thunk | | + +--- + +## 2. The formula + +### 2.1 `ComputeMaxIncome()` — 0x007521c0 (instruction-verified) + +``` +rates = { SRt = 1.0, SRsc = SRtf = SRi = SRoh = SRs = 0.0, SRnr = 0 } +NormaliseOutputRates(&rates, this, 0) // 0x00747390, cdecl +int out[12] = {0} +ComputeOutputFromRates(out, &rates) // 0x00751bb0 +return (out[3] > 0) ? out[3] : 0 // a `jg`, so a negative colony contributes 0 +``` + +`NormaliseOutputRates` clamps only trade to `[0,1]` and rescales the other three to `1 − trade`; +with trade already 1.0 the other three stay 0 and `SRoh` is untouched. So the max-income rate +vector really is "all output to the trade channel, no over-harvest". + +### 2.2 `out[3]` inside `ComputeOutputFromRates` — the money channel (instruction-verified) + +The last four instructions of 0x00751bb0 are + +``` +out[3] = _ftol2( TradePointsToMoney( [ebp-0x28] + [ebp-0x40] + [ebp-0x18] ) ) +``` + +with + +* `[ebp-0x40]` = `round(round(ComputeTotalOutput(SRoh)) × SRt)` — the **trade points**; +* `[ebp-0x28]` = leftover **science** points, non-zero only when the repair block ran; +* `[ebp-0x18]` = leftover **terraforming** points, itself fed by leftover **industry** points. + +`round` is 0x008e5660 (`fistp`/`fild`, ties to even); the outer conversion is the truncating +`_ftol2`. Under the max-income rate vector all three of science, terraform and industry points are +`round(T × 0) = 0`, and §2.4 shows both cascade sources are then zero, so + +> **`out[3] = trunc( TradePointsToMoney( round_half_even( ComputeTotalOutput(0) ) ) )`.** + +### 2.3 `TradePointsToMoney(trade)` — 0x007505b0 (instruction-verified) + +``` +t = PopIncome(0) // 0x0074d760(this, 0) +t = ((trade − fmod(trade, 5.0)) × 5.0 + 0.0) + t // 5.0 @0x009e2398, 0.0 @0x009e1e68 +t = PopIncome(1) + t +t = SlaveIncome() + t // 0x0074b700 + +owner = sys->PID // +0x100 +t = float32( owner ? SpeciesDef(owner->Species)->f18 : 1.0 ) × t +t = IncomeDifficultyMod(sys->server, owner) // 0x0080f470, a float32 + × ( float32(owner ? owner->IncMod /*+0x30c*/ : 1.0) × t ) + +sp = owner ? (sys->indi ? sys->indi->[4] : owner->Species) : -1 // +0x1c8 +cost = float32( owner ? SpeciesDef(owner->Species)->f24 : 1.0 ) + × ( CalcSuitMod(sp) × 10000.0 × 1.5 ) // 10000.0 @0x009e9398, 1.5 @0x009e90b8 +return t − cost // a DOUBLE; the caller truncates +``` + +Three corrections to `formula-gaps.md` Q3, which had this block as MEDIUM confidence: + +1. **The function returns a double, not an int.** The `ftol` is in `ComputeOutputFromRates`, not here. +2. **`t` is `(trade − fmod(trade,5)) × 5`, not `× 5` of the block count** — i.e. whole 5-point blocks + are worth **25** money each, not 5. The literal at 0x009e2398 is used twice, once as the modulus + and once as the multiplier. +3. **Every per-player multiplier is narrowed to float32 before use** (`fstp DWORD [ebp+0xc]` / + `fld DWORD [ebp+0xc]`, three times), and the additive chain associates + `Slaves + (Pop1 + ((blocks + 0.0) + Pop0))`. x87 is not associative; this matters at the ulp. + +`sys->server` is `ServerSystem+0x10`, the **raw** StrategyServer base (B4 trap 1). + +### 2.4 The two cascade terms are zero under max-income (instruction-verified) + +* Industry: `infraNeed = ceil((1.0 − sys->Infra) / 3.3e-5)`. `Infra` is clamped to `≤ 1.0` by + `ApplyInfraBonus`, so `infraNeed ≥ 0`, so `leftoverIndustry = 0 − min(0, infraNeed) = 0`. +* Terraform: `TerraformPointsNeeded` (0x00746890) ends `|Δsuit| / |rate × sign / 20000|` — the + `fabs` at 0x00746906 is **after** the sign multiply, so the sign cancels and the result is + always `≥ 0`. `leftoverTerraform = 0 − min(0, ceil(that)) = 0`. + (This corrects the implication in `output-term.md`'s "unspent terraforming points cascade into + the money channel": they do cascade, but only when the terraform channel is actually funded.) +* Science: `out[7] = 0x00746830(sys, 0.0)`; with zero science points the repair block's guard + `esi > 0` fails and `[ebp-0x28]` keeps the `fldz` written at 0x00751d5f. *Inferred* — that + 0x00746830 returns 0 for an argument of 0 is read from its shape, not proven here. + +### 2.5 `PopIncome(t)` — 0x0074d760 (instruction-verified) + +``` +sum = 0.0 +for sp in 0..6: + n = GroupPopulation(t, sp) // 0x00747ba0, int64 + surplus = 0 + if (owner && sys->indi == 0 && t == 1 && sp == owner->Species): + MaxPop(sys, owner, sp, &A, 0, 0) // 0x0074a6d0, out slot 4 + MaxPop(sys, owner, sp, 0, 0, &B) // out slot 6 + surplus = (B > A) ? (B − A) : 0 + total = surplus + n + if (total > 0): + mo = (t == 1) ? MoraleOutputMod(sp) : 1.0 // 0x00746910, the SAME helper as output + ad = sys->addiction[sp] ? ADDICTION_INCOME_MOD : 1.0 // int[7] @+0x1e4; slot 0x00aeca48 + r = GroupIncome(t, total) // an int + sum += (double) trunc( (double)r × mo × ad ) +return sum + +SlaveIncome(): // t = 2, no morale, no surplus + same loop with n = SlaveCount(sp) (0x0074b610) and no morale factor + +GroupIncome(t, count): // 0x00535e80, cdecl + return _ftol2( float32(POPTYPE[t].income) × ((double)count / 14000.0) ) +``` + +**So the per-capita income rate is `typeIncomeMod / 14000` — no 1.8 and no 500000.** For imperial +population that is `1/14000` money per head; the output law over the same head is +`1.8/500000` points. Two different laws off two adjacent columns of the same three-row table. + +Note the **double truncation**: `GroupIncome` truncates, and then the morale/addiction product is +truncated again, per species, before it is summed. Summing first and truncating once is wrong on +any colony with more than one species or a non-unit morale/addiction factor. + +### 2.6 `CalcSuitMod(species)` — 0x007484d0 (instruction-verified) + +``` +if (sys->vnh /*+0xc6*/) return 0.0 +if (sys->PID == 0) return 20.0 // 0x009e2c08 − 0.0, and it logs a warning +if (owner->RebAI /*+0xfc*/) return 0.0 +d = | IdealSuit(species) − sys->Suit /*+0x64*/ | +return (d <= owner->SuitTol /*+0xb4*/) ? d : owner->SuitTol +``` + +`IdealSuit` is `server->float[0xf8 + species*4]`, the **species baseline**, not the player's own +`IdealSuit` field. In all 11 corpus saves the two agree for every player of that species, which is +what makes the oracle self-contained (§5). + +So the money cost of a badly-suited colony is `speciesCostFactor × min(|Δsuit|, SuitTol) × 15000`, +and `SuitTol` therefore caps the cost as well as extending the habitable range. + +--- + +## 3. The difficulty multiplier — where it comes from (instruction-verified) + +`formula-gaps.md` Q3 named `DifficultyMods(owner)->+4` and left it there. It resolves to a +**three-row table built in code from `.rdata` float literals**, exactly as lane N found for the +pop-type table. There is no data-file dependency and no `GlobalConst` key. + +### 3.1 Selection + +``` +StrategyServer::IncomeDifficultyMod(ServerPlayer* p): // 0x0080f470 + m = float32(server->IncMod) // raw base +0xbc == save tag `IncMod` + if (p) m = float32( float32(Select(p->diffMods /*+0x36c*/, p)[1]) × m ) + return m // float32 + +DifficultyMods::Select(rec, p): // 0x0059b490 + return (p && p->isAI /*+0xf9*/ && !p->NPC /*+0xfb*/) ? &rec->f[0] : &rec->f[3] +``` + +`ServerPlayer+0x36c` is an **unnamed, unsaved pointer** sitting between `aidf` (+0x368) and `civr` +(+0x370) — it is not in `struct-recovery.md` §2 because the serializer never touches it. +`ServerPlayer::Read` (0x008804d0) sets it at 0x00880fa3: + +``` +if (0 <= aidf && aidf < 3) { LoadDifficultyRow(aidf, p->diffMods); p->aidf = aidf; } +``` + +`p->isAI` at `+0xf9` is **not on the wire**. It is copied from the setup/network player record at +`+0xd` by 0x0077b620, alongside `PvMA`(+0x18c), `+0xfa` and `Elim`(+0xf8). 0x0080d7a0 reads it as +`NPC || RebAI ? 0 : (isAI ? 2 : 1)`, which is what fixes its polarity. This is a **game-setup +input the save does not carry**; §5 states how the engine takes it. + +### 3.2 The record and the table + +`LoadDifficultyRow(level, out)` (0x005a3990) writes the default `{1, 1,1,1, 1,1,1}` first, then +linear-searches the table for `id == level` and copies its six floats. So an out-of-range level +yields all-ones, not a crash. + +``` +struct DifficultyMods { // 0x1c bytes, stride confirmed by the 0x92492493 magic divide + int id; // +0x00 + float aiMaintDivisor; // +0x04 ComputeBudget: Maint /= ftol(this) + float aiIncomeMod; // +0x08 TradePointsToMoney, and trade-route income + float aiResearchMod; // +0x0c research points bought with money + float plMaintDivisor; // +0x10 + float plIncomeMod; // +0x14 + float plResearchMod; // +0x18 +}; +``` + +The three consumers, all reading through `Select`'s returned triple pointer: + +| triple offset | reader | what it scales | +|---|---|---| +| `+0` | `ComputeBudget` 0x0086338b | `Maint = Maint / ftol(m)` -- a fleet-upkeep **divisor** | +| `+4` | `IncomeDifficultyMod` 0x0080f470, and the trade manager at 0x00833938 | a system's money income, and a trade route's | +| `+8` | `ResearchPointsFromMoney` 0x0080e229, inlined again in `ComputeBudget` at 0x00863618 | research points bought with money | + +`BuildDifficultyTable` (0x005a3870) push_backs three rows: + +| id | AI: maint / | AI: **income x** | AI: research x | | else: maint / | income x | research x | +|---|---|---|---|---|---|---|---| +| 0 | 1.0 | 1.0 | 1.0 | | 1.5 @0x00a1b000 | 1.5 | 1.5 | +| 1 | 3.0 @0x00a0451c | **1.1 @0x009f957c** | 1.5 @0x00a1b000 | | 1.0 | 1.0 | 1.0 | +| 2 | 1e6 @0x009ebd7c | **1.7 @0x009f9580** | 2.0 @0x00a04518 | | 1.0 | 1.0 | 1.0 | + +Read as: on **easy** (id 0) the *player* gets the break -- 1.5x income, 1.5x research and +2/3 maintenance -- and the AI is unmodified; on **normal** (id 1) and **hard** (id 2) the *AI* +gets the break, and on hard its fleet maintenance is divided by a million, i.e. free. + +**Every player record in all 11 corpus saves carries `aidf = 1`**, so the live multiplier for an +AI-owned system in this corpus is exactly `1.1f = 1.100000023841858` — which is the ×1.1 the +`BnkEl` oracle measured and `formula-gaps.md` Q3 could not name. + +The other two columns are named here because they come off the same record and the same selector, +and both already have a home in `game::sim::BudgetInputs` (`maintenanceDivisor`, +`researchDifficultyMult`) that had no source until now. + +--- + +## 4. PREDICTION — written before the run + +The oracle is `tools/max_income_oracle.py`, which inverts the stored `BnkEl` to +`Σ max(ComputeMaxIncome(s), 0)` over each player's owned systems: 25 player-records over 11 saves. +Lane N's predictor scores **6/25** — every human-owned or independent record, no AI-owned one. + +### 4.1 What I expect + +1. Adding **only** the difficulty multiplier (`1.1` on AI-owned systems) does **not** reach 25/25. + The single-system AI empires close; the multi-system ones do not, because they miss the + suitability cost (§2.6), which is zero on a homeworld and non-zero on every other colony. +2. Adding the suitability cost, the species income/cost factors, the per-species truncation of + `PopIncome`, and the civilian species loop closes the rest: **25/25**. +3. `SpeciesDef +0x18` (income factor) and `+0x24` (cost factor) are data-file values. Q3 quotes + Zuul 1.1 / Morrigi 0.8 for the first and Zuul 0.7 for the second. If those are right the Zuul + saves close with them and miss without them. +4. `rbfl` (rebelling) zeroes `ComputeTotalOutput` but **not** the money chain — the population + income and the suitability cost still apply. Lane N's predictor returns 0 for such a system; + that is wrong, and no corpus system is rebelling, so it is untested either way. + +### 4.2 Falsification — how this could be wrong, and the symptom of each + +| way it could be wrong | symptom | +|---|---| +| `p->[0xf9]` is not "is AI" but something else | the ×1.1 lands on the wrong records; the human records that match today would break | +| the blocks term is `×5` not `(t − t mod 5) × 5` | every record off by a multiple of the block count | +| `POPTYPE[1].income` is not `0.33f` | civilian-bearing colonies off by a clean ratio; Zuul (no civilians) unaffected | +| the per-species truncation is really one truncation of the sum | off by at most 6 per colony, and only where two species or a non-unit factor coexist | +| `IdealSuit` is the player's field, not the server's species array | invisible in this corpus, where they are equal — so this stays **unverified** | +| `SpeciesDef +0x24` for species 4 (the NPC species) is not 1.0 | only the NPC-owned records miss | +| the science-cascade term is not zero | every record short by the same non-zero amount | + +--- + +## 5. Result + +**The oracle goes from 6/25 to 25/25.** No VM time was spent: `BnkEl` is an inversion of the +number under test, so the corpus states the answer for every player-record it holds. + +``` +tools/max_income_oracle.py --json oracle.json +tools/max_income_predict.py verify/results/saves/*.sav --oracle oracle.json + -> 25 match, 0 differ, 0 with no oracle record +``` + +### 5.1 Every prediction in §4.1, checked + +| predicted | outcome | verdict | +|---|---|---| +| the difficulty multiplier alone does not reach 25/25 | 13/25 with it; the twelve Zuul records still missed | HELD | +| the remaining terms close the rest | 25/25 | HELD, but **for a different reason than predicted** -- see §5.2 | +| the Zuul `SpeciesDef +0x18`/`+0x24` values matter | 1.1 / 0.7 give 25/25 | HELD (weakly: the cost factor is multiplied by a zero cost on every corpus colony, so **only the income factor is actually tested**) | +| `rbfl` does not zero the money chain | no corpus system is rebelling | UNEXERCISED, and now labelled as such in the code | +| `IdealSuit` is the server's species array, not the player's field | it is the server's, and it is **on the wire** -- see §5.3 | HELD, and no longer unfalsifiable | + +### 5.2 The prediction that was wrong + +§4.1 said the twelve Zuul misses were the suitability cost. They were not: every Zuul colony in the +corpus sits exactly at its species' ideal suitability, so `CalcSuitMod` is 0 and the cost term is +0 on all of them. The suitability cost is **completely unexercised by this corpus** and stays a +hypothesis, exactly like the morale and station branches of the output term. + +What the Zuul records were actually missing was in the *output* half, not the income half: +`SpeciesDef +0x4c` (base resource demand) and `+0x50` (resource output factor) are **per species** +and are read off the system **owner's** species. Lane N measured them live -- 0/10 for Human and +Tarkas, 10/40 for Zuul -- but `max_income_predict.py` carried them as one global pair defaulting to +the Human values. A Zuul colony's harvest term is `min(resAvail, 10) x 40 = 400` output points that +a 0/10 pair scores as **zero**; 400 points is 80 whole five-point blocks worth 5 money each after +the x5, i.e. exactly 2000 money per colony, and the observed shortfalls were 4400 (two colonies x +2000 x the 1.1 species income factor) and 5566 (the same, with a 1.15 `OutMod` and the AI's 1.1). +Both deltas fall out to the unit. + +The lesson is the campaign's own rule 8 in a new dress: a per-species table taken as a scalar +agrees with the oracle on the species it was measured from and disagrees on every other, and there +is no symptom until a second species appears in the corpus. + +### 5.3 A new wire fact: `ISsp`/`ISsu` is `server->IdealSuit[]` + +The Sim block's `ISsp`/`ISsu` pairs -- seven of them, in species-index order -- are exactly the +float[7] that `StrategyServer::IdealSuit` (0x0080f4b0) indexes at `raw base + 0xf8`. The array is +randomised per game by the map generator (`turn1-state` has Human 11.106, `human-turn2` has Human +10.220, `zuul-turn15` has Human 7.502), and every `ServerPlayer`'s own `IdealSuit` field carries the +same value for its own species in all 11 saves. So the suitability cost's ideal is fully held on +the wire and needs no data file. `struct-recovery.md` §5 lists the two tags without saying what +they are; this names them. + +### 5.4 Falsification actually run + +| run | score | what it shows | +|---|---|---| +| `--ai-rule non-npc-not-first` (the model) | **25/25** | | +| `--ai-rule none` (nobody is AI) | 14/25 | the eleven AI-owned records break -- the x1.1 is load-bearing | +| `--ai-rule all-non-npc` (both real players are AI) | 14/25 | the eleven human-owned records break, each by exactly x1.1 -- so the multiplier has to land on **precisely** the AI set, not merely somewhere | +| `--base-demand 0 --res-output 10` on a Zuul save | 0/2 | the per-species resource pair is load-bearing, by the 4400 computed in §5.2 | + +### 5.5 What this run did NOT cover -- read this before quoting the 25/25 + +* **The suitability cost is untested.** Every colony in the corpus is at its species' ideal, so + `CalcSuitMod` returns 0 everywhere and `SpeciesDef +0x24` is multiplied by zero. The `<=` + boundary, the `SuitTol` cap, the `vnh` early-out and the unowned-system 20.0 are all unexercised. +* **The slave income term is untested**: no colony carries slaves. +* **The addiction income modifier is untested**: `nadct` is 0 on every system. +* **The morale multiplier is untested**: every colony sits at 75, strictly between the thresholds. +* **The civilian capacity surplus is untested**: no colony is at its cap. +* **Only difficulty level 1 is exercised.** Levels 0 and 2, and the maintenance and research + columns of all three rows, are read from the initialiser and never run. +* **`aidf` is 1 on all 25 records**, so the level lookup itself is a constant here. +* **The AI flag is not on the wire**, so 25/25 pins *an* assignment of it (player 0 human, player 1 + AI, NPCs neither), not the flag's provenance. §5.4's second falsification run is what makes that + assignment non-trivial rather than a free parameter. +* Two of the 25 records (the human-turn2/turn3 pair) come from a game whose Sim-level `IncMod` is + 1.48 rather than 1.0, which is the only exercise the server income modifier gets. + +--- + +## 6. What this unblocks, and what it does not + +### 6.1 T31 `UpdateBankruptcyLimits` -- unblocked in substance + +`UpdateBankruptcyLimits` sums exactly `max(ComputeMaxIncome(s), 0)` over owned, non-abandoned +systems, which is the number the oracle validates. It is now wired into `sots_turn` and self-checks +every run against the `BnkEl` the input save already carries -- 8 of 8 players on `turn1-state`. + +It is still listed **blocked**, for two reasons that are not the formula: + +1. `ServerPlayer+0xf9` (is this player AI?) is a game-setup input the save does not carry; + `--ai-player N` supplies it, and without it an AI empire's limit comes out 1/1.1 low. +2. `BnkPr` needs `BANKRUPTCY_PROTECTION_LIMIT_FACTOR` from the data files, so it is only offered + with a tuning table loaded. Committing a value computed from an unloaded (zero) constant + regressed one leaf in the first measurement and was removed. + +Committing it closes **nothing** on the reference pair, and that is a fact about the rest of the +engine rather than about this chain: the limits move between `turn1` and `turn2` because the +**civilian population grows**, and that growth is not committed, so our limit equals the input +save's. Measured with `--commit-blocked=T31 --ai-player 1`: **0 closed, 0 regressed** (209 -> 204 +and 108 -> 103, unchanged from the baseline). The same numbers hold with every other blocked phase +committed alongside it. + +### 6.2 P01 `ComputeBudget` -- NOT unblocked, and the roadmap's item 1 was wrong about this + +`ComputeBudget` has two modes and they take their per-system money from **different functions** +(0x008631fd, the `[ebp+0xc]` test): + +``` +if (projected) money = ComputeMaxIncome(s) // 0x007521c0 -- what this lane closed +else { OutputRates r; ComputeOutput(s, &r); money = r.out[3] } // 0x00751fb0 +``` + +`ComputeOutput` (0x00751fb0) passes the system's **own** `Rts` sliders, not a max-mods vector. On +that path the science, construction and terraform channels are funded, which means: + +* the repair pass inside `ComputeOutputFromRates` runs (it is **not** side-effect free -- it repairs + ships in orbit), so its leftover science points can be non-zero; +* the unspent-industry and unspent-terraforming cascades into the money channel are live, and §2.4's + proof that both are zero **does not apply**; +* the build queue's consumption of construction points is in the same call. + +So the turn's real per-system money is a strictly larger problem than the one the `BnkEl` oracle can +falsify, and nothing in this corpus states its answer. P01/P02/P03/P05/P06 stay blocked, and the +next lane on them should target `ComputeOutputFromRates`'s full channel split rather than this +chain. The roadmap's item 1 claimed this chain unblocks them; it does not. + +### 6.3 The two research RNG words + +Both are downstream of the budget's research allocation: + +1. `ProcessResearch`'s `RNG::Chance(odds)` on the funded node, where `odds` is built from the + research **points** the allocation bought (0 or 1 word -- `Chance` draws nothing for `p <= 0` + or `p >= 1`); +2. the draw inside `ServerPlayer::OnTechResearched`'s effect callback, which fires only when a + tech actually completes and therefore only when the allocation was large enough (0 or 1 word, + measured live in the B3 compare run). + +Those points come from `ComputeBudget`'s research money, which is on the P01 path of §6.2 -- not +this one. **The generator model still cannot include them.** What has changed is the reason: it is +no longer "the money output of a system is unmodelled" but "the *projected-rate* money output is", +which is a narrower and differently-shaped gap, and it is now the only thing between the ledger and +those two words. diff --git a/ghidra/addresses.d/lane-e1.json b/ghidra/addresses.d/lane-e1.json new file mode 100644 index 0000000..368de11 --- /dev/null +++ b/ghidra/addresses.d/lane-e1.json @@ -0,0 +1,101 @@ +{ + "_note": "Lane E1, the output -> money chain. Six addresses this lane read are already held elsewhere and AGREE, so they are dropped from this fragment rather than duplicated (rule 14): ServerSystem_ComputeMaxIncome 0x007521c0 (lane-k.json), ServerSystem_ComputeOutputFromRates 0x00751bb0 and ServerSystem_TradePointsToMoney 0x007505b0 and ServerSystem_CalcSuitMod 0x007484d0 (addresses.json), GroupIncome 0x00535e80 and PopTypeRow 0x00535e00 (lane-n.json). Every one of those six was re-read from the instruction stream this lane and the address, convention and boundary matched what was already recorded.", + "entries": [ + { + "name": "ServerSystem_PopIncome", + "addr": "0x0074d760", + "convention": "thiscall", + "prototype": "double (ServerSystem* sys, int groupType) // `ret 4`, real end 0x0074d8e1. The income analogue of PopOutput 0x0074d8f0, and NOT the same law: it sums, over species 0..6, `(double)ftol( (double)GroupIncome(groupType, count) x moraleMod x addictionMod )` -- so the value truncates TWICE per (group, species) row, once inside GroupIncome and once after both factors. moraleMod is 0x00746910 (the same helper the output term uses) and applies to groupType 1 only; addictionMod is the float behind slot 0x00aeca48 when the system's int[7] addiction table at +0x1e4 has a non-zero entry for that species. For groupType 1 and the owner's own species on a non-independent system the count first gains the capacity surplus from two calls to 0x0074a6d0 (out slot 4, then out slot 6), max(0, B - A)", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerSystem_SlaveIncome", + "addr": "0x0074b700", + "convention": "thiscall", + "prototype": "double (ServerSystem* sys) // plain `ret`, real end 0x0074b793. groupType 2 of the same loop as PopIncome, over SlaveCount(species) (0x0074b610): no morale factor and no capacity surplus, but the addiction factor still applies. Unexercised: slave counts are 0 on every call in the corpus", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerSystem_SlaveCount", + "addr": "0x0074b610", + "convention": "thiscall", + "prototype": "int64 (ServerSystem* sys, int species) // the slave-group population of one species; the group-2 counterpart of GroupPopulation 0x00747ba0", + "status": "unverified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08) -- reached from SlaveIncome, body not read" + }, + { + "name": "ServerSystem_ComputeOutput", + "addr": "0x00751fb0", + "convention": "thiscall", + "prototype": "void (ServerSystem* sys, int out[12]) // zeroes `out`, returns immediately when the caller's pointer is null or the system has no owner (+0x100), else calls ComputeOutputFromRates(out, &sys->Rts /*+0x88*/) -- the system's OWN rate sliders, not a max-mods vector. THE DISTINCTION THAT MATTERS: ComputeBudget's real (non-projected) per-system money is this function's out[3], while its projected mode and UpdateBankruptcyLimits use ComputeMaxIncome 0x007521c0. They are different numbers: this path funds the science, construction and terraform channels, so the repair pass runs and the unspent-industry and unspent-terraforming cascades into the money channel are live", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "StrategyServer_IncomeDifficultyMod", + "addr": "0x0080f470", + "convention": "thiscall", + "prototype": "float (StrategyServer* srv, ServerPlayer* p) // `ret 4`, real end 0x0080f49f. Returns float32( float32(DifficultyMods_Select(p->diffMods /*+0x36c*/, p)[1]) x float32(srv->IncMod /*raw base +0xbc, the Sim block's `IncMod` tag*/) ), or just the server modifier when p is null. Every step is stored back through a 4-byte float. `ecx` here is the RAW StrategyServer base (ServerSystem+0x10), four bytes above the base the class's own methods get. THIS IS THE MISSING x1.1: at the difficulty level every corpus save carries (aidf == 1) the AI column of the table is 1.1f", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "DifficultyMods_Select", + "addr": "0x0059b490", + "convention": "thiscall", + "prototype": "float* (DifficultyMods* rec, ServerPlayer* p) // `ret 4`, real end 0x0059b4b3. Returns &rec->f[0] (the AI triple, at +0x04) when p is non-null AND p->[0xf9] (is-AI) is set AND p->NPC (+0xfb) is clear; otherwise &rec->f[3] (the non-AI triple, at +0x10). The three consumers read offset +0 (fleet maintenance divisor, ComputeBudget 0x0086338b), +4 (system and trade-route money, 0x0080f470 and 0x00833938) and +8 (research points bought with money, 0x0080e229 and 0x00863618)", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "LoadDifficultyRow", + "addr": "0x005a3990", + "convention": "cdecl", + "prototype": "void (int level, DifficultyMods* out) // real end 0x005a3a53 (Ghidra size 193 stops 2 bytes short). memcpy's the default {id 1, 1.0f x6} into `out` FIRST, then builds the table with BuildDifficultyTable 0x005a3870 and linear-searches it for id == level (stride 0x1c, from the 0x92492493 magic divide), copying the six floats on a hit. An out-of-range level therefore yields all ones rather than failing. Called from ServerPlayer::Read 0x008804d0 at 0x00880fa3, gated on 0 <= aidf < 3, which is also where ServerPlayer+0x368 (`aidf`) is stored", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "BuildDifficultyTable", + "addr": "0x005a3870", + "convention": "thiscall", + "prototype": "vector* (vector* out) // real end 0x005a3989. THE TABLE IS BUILT IN CODE, from .rdata float literals -- no data-file key, no GlobalConst slot, same shape as lane N's pop-type table. Three rows of {int id; float ai[3]; float other[3]} (0x1c): id 0 = ai {1,1,1} / other {1.5,1.5,1.5} (0x00a1b000); id 1 = ai {3.0 (0x00a0451c), 1.1 (0x009f957c), 1.5} / other {1,1,1}; id 2 = ai {1e6 (0x009ebd7c), 1.7 (0x009f9580), 2.0 (0x00a04518)} / other {1,1,1}. Read as: level 0 gives the break to the human player, levels 1 and 2 give it to the AI, and on level 2 the AI's fleet maintenance is divided by a million", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerPlayer_GetIncMod", + "addr": "0x0080dd10", + "convention": "thiscall", + "prototype": "float (ServerPlayer* p) // seven bytes: `fld DWORD [ecx+0x30c]; ret`. The save's per-player `IncMod`", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerPlayer_GetSpeciesCostFactor", + "addr": "0x0080dd20", + "convention": "thiscall", + "prototype": "float (ServerPlayer* p) // `SpeciesDef(p->Species /*+0x5c*/)->+0x24`, the multiplier on the suitability MONEY cost (Zuul 0.7). A data-file value", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "StrategyServer_IdealSuit", + "addr": "0x0080f4b0", + "convention": "thiscall", + "prototype": "float (StrategyServer* srv, int species) // `ret 4`; one instruction of work: `fld DWORD [ecx + species*4 + 0xf8]`. The per-species ideal-suitability array on the RAW server base. IT IS ON THE WIRE: the Sim block's `ISsp`/`ISsu` pairs are this float[7] in species-index order, and the array is randomised per game by the map generator -- verified against every ServerPlayer's own `IdealSuit` field in all 11 corpus saves. CalcSuitMod reads THIS, not the player's field", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerSystem_TerraformPointsNeeded", + "addr": "0x00746890", + "convention": "thiscall", + "prototype": "double (ServerSystem* sys) // real end 0x0074690d. Returns 0 with no owner, else float32(|IdealSuit(species) - sys->Suit|) / |owner->TerraMod (+0x134) x [0x00a1f928] / 20000|. The sign term (-1.0 when the planet's suitability is STRICTLY above the ideal) is multiplied in BEFORE the fabs at 0x00746906 and therefore cancels: the result is always >= 0. That is why the unspent-terraform cascade into the money channel is provably zero under the max-income rate vector", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + } + ] +} diff --git a/tools/max_income_predict.py b/tools/max_income_predict.py index 840b9c2..a32a0f3 100644 --- a/tools/max_income_predict.py +++ b/tools/max_income_predict.py @@ -9,22 +9,32 @@ which is the per-system money output that blocks `ComputeBudget` and four other This script goes the other way: it computes the same number from the colony state and compares, so the formula is falsified per player-record rather than per lane. -The chain it implements (lane N, findings/subsystems/output-term.md, live-verified on VM140): +The chain it implements (lane N `findings/subsystems/output-term.md` for the output half, +lane E1 `findings/subsystems/income-term.md` for the money half, both read off the +instruction stream): total = ComputeTotalOutput(SRoh = 0) # max mods -> trade rate 1, rest 0 - = ( overHarvestDemand x speciesResourceOutput - + (TRes + resAvail) x stripMineFraction x 0.9 - + populationOutput ) - x OutMod x sysOutMod x setupOutMod x RebOutMod x ScOutMod trade = roundHalfEven(total) - money = ftol( TradePointsToMoney(trade) ) + money = trunc( TradePointsToMoney(trade) ) + maxIncome_s = max(money, 0) -Population output per head is `typeOutputMod x 1.8 / 500000`; population INCOME per head is -`typeIncomeModifier / 14000`, with no 1.8 -- two different laws off the same table. +with -Everything the executable carries is hard-coded here as such. The two per-species fields the -data files supply (SpeciesDef +0x4c base resource demand, +0x50 resource output factor) are -options, defaulting to the values measured live on VM140 for the species this corpus contains. + TradePointsToMoney(trade) = + diffMod * ( f32(IncMod) * ( f32(speciesIncomeFactor) + * ( Slaves + (PopIncome(1) + (((trade - trade mod 5) * 5 + 0) + PopIncome(0))) ) ) ) + - f32(speciesCostFactor) * ( CalcSuitMod * 10000 * 1.5 ) + + diffMod = f32( f32(DifficultyMods(owner)[1]) * f32(server.IncMod) ) + +Population output per head is `typeOutputMod * 1.8 / 500000`; population INCOME per head is +`typeIncomeModifier / 14000`, with no 1.8 -- two different laws off two adjacent columns of the +same three-row table, and the income one truncates twice (once inside GroupIncome, once after +the morale/addiction product) PER SPECIES. + +The difficulty table is built in code from .rdata float literals (0x005a3870); every corpus save +carries `aidf == 1`, whose AI income modifier is 1.1f. Which players count as AI is NOT on the +wire -- see --ai-rule. """ import argparse import json @@ -39,14 +49,37 @@ import save_reader as sr # noqa: E402 # ---- constants the executable carries itself ------------------------------------------------ OUT_FACTOR = struct.unpack(" (aiMaintDiv, aiIncome, aiTrade, plMaintDiv, plIncome, plTrade) +DIFFICULTY_TABLE = { + 0: (1.0, 1.0, 1.0, 1.5, 1.5, 1.5), + 1: (3.0, 1.100000023841858, 1.5, 1.0, 1.0, 1.0), + 2: (1000000.0, 1.7000000476837158, 2.0, 1.0, 1.0, 1.0), +} +DIFFICULTY_DEFAULT = (1.0, 1.0, 1.0, 1.0, 1.0, 1.0) # LoadDifficultyRow's memcpy'd default + +# ---- values the data files supply, measured live on VM140 (2026-09-08, lane N) --------------- +# SpeciesDef +0x4c (base resource demand) and +0x50 (resource output factor) are PER SPECIES and +# are read off the SYSTEM OWNER's species, not the system's population species. Taking them as +# one global pair is what cost lane N's predictor every Zuul record: a Zuul colony's harvest term +# is min(resAvail, 10) * 40 = 400 output points that a 0/10 pair scores as zero, and 400 points +# is 400/5 whole blocks worth 5 money each, i.e. exactly the 2000-per-colony shortfall observed. +SPECIES_BASE_DEMAND = {5: 10} # SpeciesDef +0x4c; 0 for Human and Tarkas +SPECIES_RES_OUTPUT = {5: 40.0} # SpeciesDef +0x50; 10 for Human and Tarkas +SPECIES_BASE_DEMAND_DEFAULT = 0 +SPECIES_RES_OUTPUT_DEFAULT = 10.0 +# SpeciesDef +0x18 income factor / +0x24 cost factor, per formula-gaps.md Q3 (NOT measured live). +SPECIES_INCOME_FACTOR = {5: 1.1, 6: 0.8} +SPECIES_COST_FACTOR = {5: 0.7} def f32(v): @@ -54,6 +87,7 @@ def f32(v): def ftol(v): + """_ftol2 -- truncation toward zero.""" return math.trunc(v) @@ -103,14 +137,15 @@ def group_output(group, count, morale, stations, tuning, owned, independent): b = tuning["STATION_BONUS_IMPERIAL_OUTPUT"] sf = 1.0 + stations * (b if b > 0 else 0.0) mo = 1.0 - if group == 1 and owned and not independent: - mo = morale_output_mod(morale, tuning) + if group == 1: + mo = morale_output_mod(morale, tuning, owned, independent) v = POPTYPE_OUT[group] * (sf * OUT_FACTOR) * mo * q return v if v > 0 else 0.0 -def morale_output_mod(m, tuning): - if m == 0: +def morale_output_mod(m, tuning, owned, independent): + """0x00746910 -- 1.0 with no owner, on an independent system, or with cm == 0.""" + if not owned or independent or m == 0: return 1.0 if m >= tuning["MORALE_INCREASE_OUTPUT"]: x = tuning["MORALE_INCREASE_OUTPUT_MOD"] @@ -129,6 +164,8 @@ LIVE_TUNING = { "MORALE_DECREASE_OUTPUT": 20, "MORALE_DECREASE_OUTPUT_MOD": f32(0.5), "SLAVES_OUTPUT_MOD": f32(3.0), + "SLAVES_INCOME_MOD": f32(3.0), + "ADDICTION_INCOME_MOD": f32(0.9), } @@ -169,32 +206,43 @@ def population(node, name, group, species): return total -def morale_table(node): - """`cm` on the wire is a count then (species, value) pairs.""" - cm = sub(node, "cm") +def sparse_table(node, name, key_tag, val_tag): + """`cm`/`nadct` on the wire: a count then (index, value) pairs.""" out = {} - if cm is None: + holder = sub(node, name) if name else node + if holder is None: return out pending = None - for c in kids(cm): - if c.name == "msp": + for c in kids(holder): + if c.name == key_tag: pending = c.value - elif c.name == "mv" and pending is not None: + elif c.name == val_tag and pending is not None: out[pending] = c.value pending = None return out +def morale_table(node): + return sparse_table(node, "cm", "msp", "mv") + + +def addiction_table(node): + """The int[7] at ServerSystem+0x1e4: `nadct` then sparse (`ads`,`adt`) pairs.""" + return sparse_table(node, None, "ads", "adt") + + def read_state(path): r = sr.read_save(path) - systems, players = [], [] + systems, players, sim = [], [], None for _, n in walk(r.tree): + if n.name == "Sim" and sim is None: + sim = n names = {c.name for c in kids(n)} if {"Pop", "Rts", "Infra", "pbon"} <= names: systems.append(n) elif {"OutMod", "IncMod", "ScOutMod", "BnkEl", "PlyrIdx"} <= names: players.append(n) - return systems, players + return systems, players, sim def system_species(node, owner_species): @@ -218,7 +266,123 @@ def is_independent(node): return bool(field(node, "hindi")) -def predict(save, base_demand, res_output, verbose=False): +# ---- the income chain ------------------------------------------------------------------------ +def group_income(t, count, tuning): + """0x00535e80: _ftol2( f32(POPTYPE[t].income) * (count / 14000) ).""" + inc = POPTYPE_INC[t] + if inc is None: + inc = tuning["SLAVES_INCOME_MOD"] + return ftol(f32(inc) * (float(count) / INCOME_DIVISOR)) + + +def pop_income(t, counts, morale, addicted, tuning, owned, independent): + """0x0074d760 / 0x0074b700. `counts` is species -> population for this group type.""" + total = 0.0 + for sp in range(7): + n = counts.get(sp, 0) + if not n > 0: + continue + mo = morale_output_mod(morale.get(sp, 0), tuning, owned, independent) if t == 1 else 1.0 + ad = tuning["ADDICTION_INCOME_MOD"] if addicted.get(sp) else 1.0 + r = group_income(t, n, tuning) + total += float(ftol(float(r) * mo * ad)) + return total + + +def calc_suit_mod(node, p, species, ideal_suit): + """0x007484d0.""" + if field(node, "vnh"): + return 0.0 + if not field(node, "PID"): + return UNOWNED_SUIT_MOD + if field(p, "RebAI"): + return 0.0 + ideal = ideal_suit.get(species) + if ideal is None: + raise KeyError("no IdealSuit known for species %r" % (species,)) + tol = f32(field(p, "SuitTol") or 0.0) + d = abs(f32(ideal) - f32(field(node, "Suit") or 0.0)) + return d if d <= tol else tol + + +def difficulty_income_mod(p, is_ai, server_inc_mod): + """0x0080f470 + 0x0059b490 + 0x005a3990: float32 throughout.""" + m = f32(server_inc_mod) + if p is None: + return m + level = field(p, "aidf") + row = DIFFICULTY_TABLE.get(level, DIFFICULTY_DEFAULT) + # Select(): the AI triple is f[0..2], the non-AI triple f[3..5]; ->+4 is index 1 of the triple. + mods = row[0:3] if (is_ai and not field(p, "NPC")) else row[3:6] + return f32(f32(mods[1]) * m) + + +def system_income(s, p, base_demand, res_output, ideal_suit, server_inc_mod, is_ai, tuning): + owner_species = field(p, "Species") + if base_demand is None: + base_demand = SPECIES_BASE_DEMAND.get(owner_species, SPECIES_BASE_DEMAND_DEFAULT) + if res_output is None: + res_output = SPECIES_RES_OUTPUT.get(owner_species, SPECIES_RES_OUTPUT_DEFAULT) + sp = system_species(s, owner_species) + strip = bool(field(p, "AMine")) + independent = is_independent(s) + + res = field(s, "Res") or 0 + avail = res + ((field(s, "MRes") or 0) + (field(s, "ARes2") or 0) if strip else 0) + pop = (field(s, "Pop") or 0) + (field(s, "pbon") or 0) + morale = morale_table(s) + addicted = addiction_table(s) + + # ---- the output half (lane N, live-verified) -------------------------------------------- + if field(s, "rbfl"): + total = 0.0 + else: + civ_own = population(s, "Pop2", 1, sp) + population(s, "pbon2", 1, sp) + harvest = over_harvest_demand(0.0, avail, pop, base_demand) * f32(res_output) + resource = (float((field(s, "TRes") or 0) + avail) + * float(strip_mine_fraction(pop, field(s, "Infra") or 0.0, + field(s, "ibon") or 0.0)) + * RES_FACTOR) + imperial = group_output(0, pop, 0, 0, tuning, True, independent) + civilian = group_output(1, civ_own, morale.get(sp, 0), 0, tuning, True, independent) + total = (civilian + (imperial + 0.0)) + (harvest + resource) + total *= f32(field(p, "OutMod") or 1.0) + total *= f32(field(s, "OutMod") or 1.0) + total *= f32(field(p, "RebOutMod") or 1.0) + total *= f32(field(p, "ScOutMod") or 1.0) + + # ---- the money half (lane E1) ------------------------------------------------------------- + trade = round_half_even(total) + + imperial_counts = {sp: pop} if pop > 0 else {} + civ_counts = {} + for q in range(7): + c = population(s, "Pop2", 1, q) + population(s, "pbon2", 1, q) + if c: + civ_counts[q] = c + + t = pop_income(0, imperial_counts, morale, addicted, tuning, True, independent) + t = ((trade - math.fmod(trade, BLOCK)) * BLOCK + 0.0) + t + t = pop_income(1, civ_counts, morale, addicted, tuning, True, independent) + t + t = 0.0 + t # SlaveIncome(): no slaves in this corpus + + t = f32(SPECIES_INCOME_FACTOR.get(owner_species, 1.0)) * t + t = difficulty_income_mod(p, is_ai, server_inc_mod) * (f32(field(p, "IncMod") or 1.0) * t) + + cost = f32(SPECIES_COST_FACTOR.get(owner_species, 1.0)) * ( + calc_suit_mod(s, p, sp, ideal_suit) * SUIT_COST_A * SUIT_COST_B) + return ftol(t - cost) + + +AI_RULES = { + # `ServerPlayer+0xf9` is a setup input, not a save field (income-term.md §3.1). + "non-npc-not-first": lambda p, i: not field(p, "NPC") and (field(p, "PlyrIdx") or 0) != 0, + "none": lambda p, i: False, + "all-non-npc": lambda p, i: not field(p, "NPC"), +} + + +def predict(save, base_demand, res_output, ai_rule, verbose=False): """Return {BnkEl: (predicted maxIncome, per-system detail)}. The join key is `BnkEl` rather than any id: a system stores its owner as a HANDLE id @@ -226,7 +390,30 @@ def predict(save, base_demand, res_output, verbose=False): schemes. The oracle inverts the same `BnkEl` the player node carries, so keying on it needs no id mapping at all and cannot silently pair the wrong two records. """ - systems, players = read_state(save) + systems, players, sim = read_state(save) + server_inc_mod = field(sim, "IncMod", 1.0) if sim is not None else 1.0 + + # `server->IdealSuit[species]` IS on the wire: the Sim block's `ISsp`/`ISsu` pairs are that + # float[7], in species-index order, and they are randomised per game by the map generator. + # Each ServerPlayer's own `IdealSuit` field carries the same value for its species, so the + # two are cross-checked here rather than one being trusted blindly (rule 8: two checks that + # share a hidden assumption are one check -- these two do not share a source). + ideal_suit = {} + if sim is not None: + idx = 0 + for c in kids(sim): + if c.name == "ISsu": + ideal_suit[idx] = c.value + idx += 1 + for p in players: + s, v = field(p, "Species"), field(p, "IdealSuit") + if s is None or v is None: + continue + if s in ideal_suit and ideal_suit[s] != v: + print(" WARNING: species %d ISsu %r disagrees with a player's IdealSuit %r" + % (s, ideal_suit[s], v), file=sys.stderr) + ideal_suit.setdefault(s, v) + # A player's handle id is not a named field, but every system names its owner's, so the # set of distinct non-zero `PID` values is the set of owning players -- in the same order # the player records appear. Pair them by position among the players that own anything. @@ -244,69 +431,39 @@ def predict(save, base_demand, res_output, verbose=False): handle_of[id(p)] = h out = {} - for p in players: + for i, p in enumerate(players): h = handle_of.get(id(p)) + is_ai = AI_RULES[ai_rule](p, i) total = 0 detail = [] if h is not None: for s in systems: if field(s, "PID") != h: continue - m = system_income(s, p, base_demand, res_output) + m = system_income(s, p, base_demand, res_output, ideal_suit, + server_inc_mod, is_ai, LIVE_TUNING) detail.append((field(s, "Idx"), field(s, "Name"), m)) total += max(m, 0) out[field(p, "BnkEl")] = (total, detail) if verbose and detail: - print(" player handle %s (%d system(s))" % (h, len(detail))) + print(" player handle %s sp=%s ai=%s npc=%s (%d system(s))" + % (h, field(p, "Species"), is_ai, field(p, "NPC"), len(detail))) for sid, nm, m in detail: print(" sys %-4s %-16s money=%d" % (sid, nm, m)) return out -def system_income(s, p, base_demand, res_output): - rts = sub(s, "Rts") - owner_species = field(p, "Species") - sp = system_species(s, owner_species) - strip = bool(field(p, "AMine")) - - res = field(s, "Res") or 0 - avail = res + ((field(s, "MRes") or 0) + (field(s, "ARes2") or 0) if strip else 0) - pop = (field(s, "Pop") or 0) + (field(s, "pbon") or 0) - civ = population(s, "Pop2", 1, sp) + population(s, "pbon2", 1, sp) - morale = morale_table(s).get(sp, 0) - independent = is_independent(s) - - harvest = over_harvest_demand(0.0, avail, pop, base_demand) * f32(res_output) - resource = (float((field(s, "TRes") or 0) + avail) - * float(strip_mine_fraction(pop, field(s, "Infra") or 0.0, field(s, "ibon") or 0.0)) - * RES_FACTOR) - imperial = group_output(0, pop, 0, 0, LIVE_TUNING, True, independent) - civilian = group_output(1, civ, morale, 0, LIVE_TUNING, True, independent) - base = (civilian + (imperial + 0.0)) + (harvest + resource) - - if field(s, "rbfl"): - return 0 - total = base - total *= f32(field(p, "OutMod") or 1.0) - total *= f32(field(s, "OutMod") or 1.0) - total *= f32(field(p, "RebOutMod") or 1.0) - total *= f32(field(p, "ScOutMod") or 1.0) - - trade = round_half_even(total) - t = (trade - math.fmod(trade, 5.0)) * 5.0 - t += ftol(POPTYPE_INC[0] * (float(pop) / INCOME_DIVISOR)) - t += ftol(POPTYPE_INC[1] * (float(civ) / INCOME_DIVISOR)) - t *= f32(field(p, "IncMod") or 1.0) - return ftol(t) - - def main(): ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) ap.add_argument("saves", nargs="*") ap.add_argument("--oracle", help="JSON from max_income_oracle.py --json") - ap.add_argument("--base-demand", type=int, default=SPECIES_BASE_DEMAND) - ap.add_argument("--res-output", type=float, default=SPECIES_RES_OUTPUT) + ap.add_argument("--base-demand", type=int, default=None, + help="override SpeciesDef +0x4c for every species") + ap.add_argument("--res-output", type=float, default=None, + help="override SpeciesDef +0x50 for every species") + ap.add_argument("--ai-rule", choices=sorted(AI_RULES), default="non-npc-not-first", + help="which players count as AI (ServerPlayer+0xf9 is not on the wire)") ap.add_argument("-v", "--verbose", action="store_true") a = ap.parse_args() @@ -314,15 +471,21 @@ def main(): if a.oracle: with open(a.oracle) as fh: raw = json.load(fh) - for row in (raw if isinstance(raw, list) else raw.get("records", [])): - oracle[(os.path.basename(row.get("save", "")), row.get("BnkEl"))] = row.get("maxIncome") + if isinstance(raw, dict) and raw and isinstance(next(iter(raw.values())), list): + for name, rows in raw.items(): + for row in rows: + oracle[(os.path.basename(name), row.get("BnkEl"))] = row.get("maxIncome") + else: + for row in (raw if isinstance(raw, list) else raw.get("records", [])): + oracle[(os.path.basename(row.get("save", "")), row.get("BnkEl"))] = \ + row.get("maxIncome") hits = misses = unknown = 0 for save in a.saves: name = os.path.basename(save) print("==", name) for bnkel, (total, detail) in sorted(predict(save, a.base_demand, a.res_output, - a.verbose).items()): + a.ai_rule, a.verbose).items()): if not detail: continue want = oracle.get((name, bnkel)) @@ -334,7 +497,7 @@ def main(): print(" BnkEl=%-12s predicted=%-12d MATCH" % (bnkel, total)) else: misses += 1 - print(" BnkEl=%-12s predicted=%-12d oracle=%-12d delta=%+d (%.4f x)" + print(" BnkEl=%-12s predicted=%-12d oracle=%-12d delta=%+d (%.6f x)" % (bnkel, total, want, total - want, (total / want) if want else float("nan"))) print("\n%d match, %d differ, %d with no oracle record" % (hits, misses, unknown))