diff --git a/findings/subsystems/formula-gaps.md b/findings/subsystems/formula-gaps.md index 2e0c993..c1e5e80 100644 --- a/findings/subsystems/formula-gaps.md +++ b/findings/subsystems/formula-gaps.md @@ -72,6 +72,30 @@ money = ftol(t − cost) Note `SuitTol` therefore caps the hazard **cost** as well as extending the habitable range. `ADDICTION_INCOME_MOD` is applied inside PopIncome (0x0074d760 → 0x00746910 morale/addiction factor) — not verified line by line (MEDIUM). +### Q3 addendum — lane E1, 2026-09-08 (instruction-verified, and the block above is now superseded) + +The whole chain is read instruction by instruction in `findings/subsystems/income-term.md`, which +replaces the MEDIUM-confidence sketch above. Five corrections: + +1. **`TradePointsToMoney` returns a double, not an int.** The `ftol` is in the caller + (`ComputeOutputFromRates`, `out[3] = _ftol2(...)`), not here. +2. **The block term is `(trade − fmod(trade,5)) × 5`** — the literal at 0x009e2398 is used twice, + as the modulus *and* as the multiplier, so a whole five-point block is worth **25** money. +3. **`PopIncome` truncates twice per (group, species) row** — once inside `GroupIncome` + (`ftol(typeIncomeMod × count/14000)`) and again after the morale and addiction factors. Summing + the species and truncating once is wrong on any multi-species colony. +4. **`DifficultyMods(owner)->+4` resolves.** `ServerPlayer+0x36c` is an unnamed, unsaved pointer to + a 0x1c record `{int id; float ai[3]; float other[3]}` filled by `LoadDifficultyRow` 0x005a3990 + from a three-row table **built in code** by 0x005a3870 (`BuildDifficultyTable`) — no data-file + key. The selector 0x0059b490 takes the AI triple iff `p->[0xf9] && !p->NPC`. Every corpus save + carries `aidf = 1`, whose AI income column is **1.1f** — the multiplier the `BnkEl` oracle + measured. The record's other two columns are a fleet-maintenance **divisor** and a **research** + multiplier, not a trade multiplier as this note said. +5. **`CalcSuitMod`'s ideal is the SERVER's per-species array**, `server->IdealSuit[species]` + (0x0080f4b0, raw base +0xf8), which is on the wire as the Sim block's `ISsp`/`ISsu` pairs — not + the owner's `IdealSuit` field. The species is the system's population species (`indi->indsp` on + an independent colony), and a `vnh` system pays no cost at all. + ## Q4. `POPBONUS_INC` population increment `ServerSystem::AccrueSystemBonus` 0x0074d4f0 (disasm 0x0074d53b–0x0074d5be): diff --git a/findings/subsystems/income-term.md b/findings/subsystems/income-term.md new file mode 100644 index 0000000..a0d21df --- /dev/null +++ b/findings/subsystems/income-term.md @@ -0,0 +1,428 @@ +# The output → money term (lane E1, 2026-09-08) + +`ServerSystem::ComputeMaxIncome` and its `TradePointsToMoney` tail — the second chain +`ComputeBudget` needs, and the one `output-term.md` §6 named as the reason the verified output +total did **not** unblock `P01 P02 P03 P05 P06`. + +Evidence: own `objdump -d` pass over `Sword of the Stars.exe`, every range disassembled **to the +next function start** and the real boundary found (rule 17). Field names per +`findings/objects/struct-recovery.md`. Everything below marked *instruction-verified* is read off +the instruction stream; everything marked *inferred* is not. + +--- + +## 1. The call chain + +| addr | name | conv / boundary | note | +|---|---|---|---| +| 0x007521c0 | `ServerSystem::ComputeMaxIncome()` → int | thiscall, ends 0x0075223d | Ghidra size 126 = correct | +| 0x00751bb0 | `ServerSystem::ComputeOutputFromRates(int out[12], OutputRates*)` | thiscall `ret 8`, ends 0x00751fa0 | **repairs ships** — not compare-safe | +| 0x007505b0 | `ServerSystem::TradePointsToMoney(double trade)` → double | thiscall `ret 8`, ends 0x007506c8 | Ghidra size 281 = correct | +| 0x0074d760 | `ServerSystem::PopIncome(int groupType)` → double | thiscall `ret 4`, ends 0x0074d8e1 | | +| 0x0074b700 | `ServerSystem::SlaveIncome()` → double | thiscall, ends 0x0074b793 | | +| 0x00535e80 | `GroupIncome(int groupType, int64 count)` → int | **cdecl**, tail-jumps `_ftol2` | the per-capita income term | +| 0x00535e00 | `PopTypeRow(int t)` | cdecl | same 3-row table as the output term | +| 0x007484d0 | `ServerSystem::CalcSuitMod(int species)` → double | thiscall `ret 4`, ends 0x00748560 | the suitability money **cost** | +| 0x0080f470 | `StrategyServer::IncomeDifficultyMod(ServerPlayer*)` → float | thiscall `ret 4`, ends 0x0080f49f | **the missing ×1.1** | +| 0x0059b490 | `DifficultyMods::Select(ServerPlayer*)` → float* | thiscall `ret 4`, ends 0x0059b4b3 | picks the AI or the non-AI triple | +| 0x005a3990 | `LoadDifficultyRow(int level, DiffRec* out)` | cdecl | fills the 0x1c record | +| 0x005a3870 | `BuildDifficultyTable(vector* out)` | thiscall | **the table is built in code** — §3 | +| 0x0080dd10 | `ServerPlayer::GetIncMod()` → float | thiscall | `player+0x30c` | +| 0x0080dd20 | `ServerPlayer::GetSpeciesCostFactor()` → float | thiscall | `SpeciesDef(Species)->+0x24` | +| 0x0080f4b0 | `StrategyServer::IdealSuit(int species)` → float | thiscall `ret 4` | `server+0xf8 + species*4` | +| 0x00746890 | `ServerSystem::TerraformPointsNeeded()` → double | thiscall | always ≥ 0 — §2.4 | +| 0x009dd1e4 | `ceil` (MSVCR100) | import | resolved from the IAT | +| 0x00925086 | `_CIfmod` (MSVCR100) | import thunk | | + +--- + +## 2. The formula + +### 2.1 `ComputeMaxIncome()` — 0x007521c0 (instruction-verified) + +``` +rates = { SRt = 1.0, SRsc = SRtf = SRi = SRoh = SRs = 0.0, SRnr = 0 } +NormaliseOutputRates(&rates, this, 0) // 0x00747390, cdecl +int out[12] = {0} +ComputeOutputFromRates(out, &rates) // 0x00751bb0 +return (out[3] > 0) ? out[3] : 0 // a `jg`, so a negative colony contributes 0 +``` + +`NormaliseOutputRates` clamps only trade to `[0,1]` and rescales the other three to `1 − trade`; +with trade already 1.0 the other three stay 0 and `SRoh` is untouched. So the max-income rate +vector really is "all output to the trade channel, no over-harvest". + +### 2.2 `out[3]` inside `ComputeOutputFromRates` — the money channel (instruction-verified) + +The last four instructions of 0x00751bb0 are + +``` +out[3] = _ftol2( TradePointsToMoney( [ebp-0x28] + [ebp-0x40] + [ebp-0x18] ) ) +``` + +with + +* `[ebp-0x40]` = `round(round(ComputeTotalOutput(SRoh)) × SRt)` — the **trade points**; +* `[ebp-0x28]` = leftover **science** points, non-zero only when the repair block ran; +* `[ebp-0x18]` = leftover **terraforming** points, itself fed by leftover **industry** points. + +`round` is 0x008e5660 (`fistp`/`fild`, ties to even); the outer conversion is the truncating +`_ftol2`. Under the max-income rate vector all three of science, terraform and industry points are +`round(T × 0) = 0`, and §2.4 shows both cascade sources are then zero, so + +> **`out[3] = trunc( TradePointsToMoney( round_half_even( ComputeTotalOutput(0) ) ) )`.** + +### 2.3 `TradePointsToMoney(trade)` — 0x007505b0 (instruction-verified) + +``` +t = PopIncome(0) // 0x0074d760(this, 0) +t = ((trade − fmod(trade, 5.0)) × 5.0 + 0.0) + t // 5.0 @0x009e2398, 0.0 @0x009e1e68 +t = PopIncome(1) + t +t = SlaveIncome() + t // 0x0074b700 + +owner = sys->PID // +0x100 +t = float32( owner ? SpeciesDef(owner->Species)->f18 : 1.0 ) × t +t = IncomeDifficultyMod(sys->server, owner) // 0x0080f470, a float32 + × ( float32(owner ? owner->IncMod /*+0x30c*/ : 1.0) × t ) + +sp = owner ? (sys->indi ? sys->indi->[4] : owner->Species) : -1 // +0x1c8 +cost = float32( owner ? SpeciesDef(owner->Species)->f24 : 1.0 ) + × ( CalcSuitMod(sp) × 10000.0 × 1.5 ) // 10000.0 @0x009e9398, 1.5 @0x009e90b8 +return t − cost // a DOUBLE; the caller truncates +``` + +Three corrections to `formula-gaps.md` Q3, which had this block as MEDIUM confidence: + +1. **The function returns a double, not an int.** The `ftol` is in `ComputeOutputFromRates`, not here. +2. **`t` is `(trade − fmod(trade,5)) × 5`, not `× 5` of the block count** — i.e. whole 5-point blocks + are worth **25** money each, not 5. The literal at 0x009e2398 is used twice, once as the modulus + and once as the multiplier. +3. **Every per-player multiplier is narrowed to float32 before use** (`fstp DWORD [ebp+0xc]` / + `fld DWORD [ebp+0xc]`, three times), and the additive chain associates + `Slaves + (Pop1 + ((blocks + 0.0) + Pop0))`. x87 is not associative; this matters at the ulp. + +`sys->server` is `ServerSystem+0x10`, the **raw** StrategyServer base (B4 trap 1). + +### 2.4 The two cascade terms are zero under max-income (instruction-verified) + +* Industry: `infraNeed = ceil((1.0 − sys->Infra) / 3.3e-5)`. `Infra` is clamped to `≤ 1.0` by + `ApplyInfraBonus`, so `infraNeed ≥ 0`, so `leftoverIndustry = 0 − min(0, infraNeed) = 0`. +* Terraform: `TerraformPointsNeeded` (0x00746890) ends `|Δsuit| / |rate × sign / 20000|` — the + `fabs` at 0x00746906 is **after** the sign multiply, so the sign cancels and the result is + always `≥ 0`. `leftoverTerraform = 0 − min(0, ceil(that)) = 0`. + (This corrects the implication in `output-term.md`'s "unspent terraforming points cascade into + the money channel": they do cascade, but only when the terraform channel is actually funded.) +* Science: `out[7] = 0x00746830(sys, 0.0)`; with zero science points the repair block's guard + `esi > 0` fails and `[ebp-0x28]` keeps the `fldz` written at 0x00751d5f. *Inferred* — that + 0x00746830 returns 0 for an argument of 0 is read from its shape, not proven here. + +### 2.5 `PopIncome(t)` — 0x0074d760 (instruction-verified) + +``` +sum = 0.0 +for sp in 0..6: + n = GroupPopulation(t, sp) // 0x00747ba0, int64 + surplus = 0 + if (owner && sys->indi == 0 && t == 1 && sp == owner->Species): + MaxPop(sys, owner, sp, &A, 0, 0) // 0x0074a6d0, out slot 4 + MaxPop(sys, owner, sp, 0, 0, &B) // out slot 6 + surplus = (B > A) ? (B − A) : 0 + total = surplus + n + if (total > 0): + mo = (t == 1) ? MoraleOutputMod(sp) : 1.0 // 0x00746910, the SAME helper as output + ad = sys->addiction[sp] ? ADDICTION_INCOME_MOD : 1.0 // int[7] @+0x1e4; slot 0x00aeca48 + r = GroupIncome(t, total) // an int + sum += (double) trunc( (double)r × mo × ad ) +return sum + +SlaveIncome(): // t = 2, no morale, no surplus + same loop with n = SlaveCount(sp) (0x0074b610) and no morale factor + +GroupIncome(t, count): // 0x00535e80, cdecl + return _ftol2( float32(POPTYPE[t].income) × ((double)count / 14000.0) ) +``` + +**So the per-capita income rate is `typeIncomeMod / 14000` — no 1.8 and no 500000.** For imperial +population that is `1/14000` money per head; the output law over the same head is +`1.8/500000` points. Two different laws off two adjacent columns of the same three-row table. + +Note the **double truncation**: `GroupIncome` truncates, and then the morale/addiction product is +truncated again, per species, before it is summed. Summing first and truncating once is wrong on +any colony with more than one species or a non-unit morale/addiction factor. + +### 2.6 `CalcSuitMod(species)` — 0x007484d0 (instruction-verified) + +``` +if (sys->vnh /*+0xc6*/) return 0.0 +if (sys->PID == 0) return 20.0 // 0x009e2c08 − 0.0, and it logs a warning +if (owner->RebAI /*+0xfc*/) return 0.0 +d = | IdealSuit(species) − sys->Suit /*+0x64*/ | +return (d <= owner->SuitTol /*+0xb4*/) ? d : owner->SuitTol +``` + +`IdealSuit` is `server->float[0xf8 + species*4]`, the **species baseline**, not the player's own +`IdealSuit` field. In all 11 corpus saves the two agree for every player of that species, which is +what makes the oracle self-contained (§5). + +So the money cost of a badly-suited colony is `speciesCostFactor × min(|Δsuit|, SuitTol) × 15000`, +and `SuitTol` therefore caps the cost as well as extending the habitable range. + +--- + +## 3. The difficulty multiplier — where it comes from (instruction-verified) + +`formula-gaps.md` Q3 named `DifficultyMods(owner)->+4` and left it there. It resolves to a +**three-row table built in code from `.rdata` float literals**, exactly as lane N found for the +pop-type table. There is no data-file dependency and no `GlobalConst` key. + +### 3.1 Selection + +``` +StrategyServer::IncomeDifficultyMod(ServerPlayer* p): // 0x0080f470 + m = float32(server->IncMod) // raw base +0xbc == save tag `IncMod` + if (p) m = float32( float32(Select(p->diffMods /*+0x36c*/, p)[1]) × m ) + return m // float32 + +DifficultyMods::Select(rec, p): // 0x0059b490 + return (p && p->isAI /*+0xf9*/ && !p->NPC /*+0xfb*/) ? &rec->f[0] : &rec->f[3] +``` + +`ServerPlayer+0x36c` is an **unnamed, unsaved pointer** sitting between `aidf` (+0x368) and `civr` +(+0x370) — it is not in `struct-recovery.md` §2 because the serializer never touches it. +`ServerPlayer::Read` (0x008804d0) sets it at 0x00880fa3: + +``` +if (0 <= aidf && aidf < 3) { LoadDifficultyRow(aidf, p->diffMods); p->aidf = aidf; } +``` + +`p->isAI` at `+0xf9` is **not on the wire**. It is copied from the setup/network player record at +`+0xd` by 0x0077b620, alongside `PvMA`(+0x18c), `+0xfa` and `Elim`(+0xf8). 0x0080d7a0 reads it as +`NPC || RebAI ? 0 : (isAI ? 2 : 1)`, which is what fixes its polarity. This is a **game-setup +input the save does not carry**; §5 states how the engine takes it. + +### 3.2 The record and the table + +`LoadDifficultyRow(level, out)` (0x005a3990) writes the default `{1, 1,1,1, 1,1,1}` first, then +linear-searches the table for `id == level` and copies its six floats. So an out-of-range level +yields all-ones, not a crash. + +``` +struct DifficultyMods { // 0x1c bytes, stride confirmed by the 0x92492493 magic divide + int id; // +0x00 + float aiMaintDivisor; // +0x04 ComputeBudget: Maint /= ftol(this) + float aiIncomeMod; // +0x08 TradePointsToMoney, and trade-route income + float aiResearchMod; // +0x0c research points bought with money + float plMaintDivisor; // +0x10 + float plIncomeMod; // +0x14 + float plResearchMod; // +0x18 +}; +``` + +The three consumers, all reading through `Select`'s returned triple pointer: + +| triple offset | reader | what it scales | +|---|---|---| +| `+0` | `ComputeBudget` 0x0086338b | `Maint = Maint / ftol(m)` -- a fleet-upkeep **divisor** | +| `+4` | `IncomeDifficultyMod` 0x0080f470, and the trade manager at 0x00833938 | a system's money income, and a trade route's | +| `+8` | `ResearchPointsFromMoney` 0x0080e229, inlined again in `ComputeBudget` at 0x00863618 | research points bought with money | + +`BuildDifficultyTable` (0x005a3870) push_backs three rows: + +| id | AI: maint / | AI: **income x** | AI: research x | | else: maint / | income x | research x | +|---|---|---|---|---|---|---|---| +| 0 | 1.0 | 1.0 | 1.0 | | 1.5 @0x00a1b000 | 1.5 | 1.5 | +| 1 | 3.0 @0x00a0451c | **1.1 @0x009f957c** | 1.5 @0x00a1b000 | | 1.0 | 1.0 | 1.0 | +| 2 | 1e6 @0x009ebd7c | **1.7 @0x009f9580** | 2.0 @0x00a04518 | | 1.0 | 1.0 | 1.0 | + +Read as: on **easy** (id 0) the *player* gets the break -- 1.5x income, 1.5x research and +2/3 maintenance -- and the AI is unmodified; on **normal** (id 1) and **hard** (id 2) the *AI* +gets the break, and on hard its fleet maintenance is divided by a million, i.e. free. + +**Every player record in all 11 corpus saves carries `aidf = 1`**, so the live multiplier for an +AI-owned system in this corpus is exactly `1.1f = 1.100000023841858` — which is the ×1.1 the +`BnkEl` oracle measured and `formula-gaps.md` Q3 could not name. + +The other two columns are named here because they come off the same record and the same selector, +and both already have a home in `game::sim::BudgetInputs` (`maintenanceDivisor`, +`researchDifficultyMult`) that had no source until now. + +--- + +## 4. PREDICTION — written before the run + +The oracle is `tools/max_income_oracle.py`, which inverts the stored `BnkEl` to +`Σ max(ComputeMaxIncome(s), 0)` over each player's owned systems: 25 player-records over 11 saves. +Lane N's predictor scores **6/25** — every human-owned or independent record, no AI-owned one. + +### 4.1 What I expect + +1. Adding **only** the difficulty multiplier (`1.1` on AI-owned systems) does **not** reach 25/25. + The single-system AI empires close; the multi-system ones do not, because they miss the + suitability cost (§2.6), which is zero on a homeworld and non-zero on every other colony. +2. Adding the suitability cost, the species income/cost factors, the per-species truncation of + `PopIncome`, and the civilian species loop closes the rest: **25/25**. +3. `SpeciesDef +0x18` (income factor) and `+0x24` (cost factor) are data-file values. Q3 quotes + Zuul 1.1 / Morrigi 0.8 for the first and Zuul 0.7 for the second. If those are right the Zuul + saves close with them and miss without them. +4. `rbfl` (rebelling) zeroes `ComputeTotalOutput` but **not** the money chain — the population + income and the suitability cost still apply. Lane N's predictor returns 0 for such a system; + that is wrong, and no corpus system is rebelling, so it is untested either way. + +### 4.2 Falsification — how this could be wrong, and the symptom of each + +| way it could be wrong | symptom | +|---|---| +| `p->[0xf9]` is not "is AI" but something else | the ×1.1 lands on the wrong records; the human records that match today would break | +| the blocks term is `×5` not `(t − t mod 5) × 5` | every record off by a multiple of the block count | +| `POPTYPE[1].income` is not `0.33f` | civilian-bearing colonies off by a clean ratio; Zuul (no civilians) unaffected | +| the per-species truncation is really one truncation of the sum | off by at most 6 per colony, and only where two species or a non-unit factor coexist | +| `IdealSuit` is the player's field, not the server's species array | invisible in this corpus, where they are equal — so this stays **unverified** | +| `SpeciesDef +0x24` for species 4 (the NPC species) is not 1.0 | only the NPC-owned records miss | +| the science-cascade term is not zero | every record short by the same non-zero amount | + +--- + +## 5. Result + +**The oracle goes from 6/25 to 25/25.** No VM time was spent: `BnkEl` is an inversion of the +number under test, so the corpus states the answer for every player-record it holds. + +``` +tools/max_income_oracle.py --json oracle.json +tools/max_income_predict.py verify/results/saves/*.sav --oracle oracle.json + -> 25 match, 0 differ, 0 with no oracle record +``` + +### 5.1 Every prediction in §4.1, checked + +| predicted | outcome | verdict | +|---|---|---| +| the difficulty multiplier alone does not reach 25/25 | 13/25 with it; the twelve Zuul records still missed | HELD | +| the remaining terms close the rest | 25/25 | HELD, but **for a different reason than predicted** -- see §5.2 | +| the Zuul `SpeciesDef +0x18`/`+0x24` values matter | 1.1 / 0.7 give 25/25 | HELD (weakly: the cost factor is multiplied by a zero cost on every corpus colony, so **only the income factor is actually tested**) | +| `rbfl` does not zero the money chain | no corpus system is rebelling | UNEXERCISED, and now labelled as such in the code | +| `IdealSuit` is the server's species array, not the player's field | it is the server's, and it is **on the wire** -- see §5.3 | HELD, and no longer unfalsifiable | + +### 5.2 The prediction that was wrong + +§4.1 said the twelve Zuul misses were the suitability cost. They were not: every Zuul colony in the +corpus sits exactly at its species' ideal suitability, so `CalcSuitMod` is 0 and the cost term is +0 on all of them. The suitability cost is **completely unexercised by this corpus** and stays a +hypothesis, exactly like the morale and station branches of the output term. + +What the Zuul records were actually missing was in the *output* half, not the income half: +`SpeciesDef +0x4c` (base resource demand) and `+0x50` (resource output factor) are **per species** +and are read off the system **owner's** species. Lane N measured them live -- 0/10 for Human and +Tarkas, 10/40 for Zuul -- but `max_income_predict.py` carried them as one global pair defaulting to +the Human values. A Zuul colony's harvest term is `min(resAvail, 10) x 40 = 400` output points that +a 0/10 pair scores as **zero**; 400 points is 80 whole five-point blocks worth 5 money each after +the x5, i.e. exactly 2000 money per colony, and the observed shortfalls were 4400 (two colonies x +2000 x the 1.1 species income factor) and 5566 (the same, with a 1.15 `OutMod` and the AI's 1.1). +Both deltas fall out to the unit. + +The lesson is the campaign's own rule 8 in a new dress: a per-species table taken as a scalar +agrees with the oracle on the species it was measured from and disagrees on every other, and there +is no symptom until a second species appears in the corpus. + +### 5.3 A new wire fact: `ISsp`/`ISsu` is `server->IdealSuit[]` + +The Sim block's `ISsp`/`ISsu` pairs -- seven of them, in species-index order -- are exactly the +float[7] that `StrategyServer::IdealSuit` (0x0080f4b0) indexes at `raw base + 0xf8`. The array is +randomised per game by the map generator (`turn1-state` has Human 11.106, `human-turn2` has Human +10.220, `zuul-turn15` has Human 7.502), and every `ServerPlayer`'s own `IdealSuit` field carries the +same value for its own species in all 11 saves. So the suitability cost's ideal is fully held on +the wire and needs no data file. `struct-recovery.md` §5 lists the two tags without saying what +they are; this names them. + +### 5.4 Falsification actually run + +| run | score | what it shows | +|---|---|---| +| `--ai-rule non-npc-not-first` (the model) | **25/25** | | +| `--ai-rule none` (nobody is AI) | 14/25 | the eleven AI-owned records break -- the x1.1 is load-bearing | +| `--ai-rule all-non-npc` (both real players are AI) | 14/25 | the eleven human-owned records break, each by exactly x1.1 -- so the multiplier has to land on **precisely** the AI set, not merely somewhere | +| `--base-demand 0 --res-output 10` on a Zuul save | 0/2 | the per-species resource pair is load-bearing, by the 4400 computed in §5.2 | + +### 5.5 What this run did NOT cover -- read this before quoting the 25/25 + +* **The suitability cost is untested.** Every colony in the corpus is at its species' ideal, so + `CalcSuitMod` returns 0 everywhere and `SpeciesDef +0x24` is multiplied by zero. The `<=` + boundary, the `SuitTol` cap, the `vnh` early-out and the unowned-system 20.0 are all unexercised. +* **The slave income term is untested**: no colony carries slaves. +* **The addiction income modifier is untested**: `nadct` is 0 on every system. +* **The morale multiplier is untested**: every colony sits at 75, strictly between the thresholds. +* **The civilian capacity surplus is untested**: no colony is at its cap. +* **Only difficulty level 1 is exercised.** Levels 0 and 2, and the maintenance and research + columns of all three rows, are read from the initialiser and never run. +* **`aidf` is 1 on all 25 records**, so the level lookup itself is a constant here. +* **The AI flag is not on the wire**, so 25/25 pins *an* assignment of it (player 0 human, player 1 + AI, NPCs neither), not the flag's provenance. §5.4's second falsification run is what makes that + assignment non-trivial rather than a free parameter. +* Two of the 25 records (the human-turn2/turn3 pair) come from a game whose Sim-level `IncMod` is + 1.48 rather than 1.0, which is the only exercise the server income modifier gets. + +--- + +## 6. What this unblocks, and what it does not + +### 6.1 T31 `UpdateBankruptcyLimits` -- unblocked in substance + +`UpdateBankruptcyLimits` sums exactly `max(ComputeMaxIncome(s), 0)` over owned, non-abandoned +systems, which is the number the oracle validates. It is now wired into `sots_turn` and self-checks +every run against the `BnkEl` the input save already carries -- 8 of 8 players on `turn1-state`. + +It is still listed **blocked**, for two reasons that are not the formula: + +1. `ServerPlayer+0xf9` (is this player AI?) is a game-setup input the save does not carry; + `--ai-player N` supplies it, and without it an AI empire's limit comes out 1/1.1 low. +2. `BnkPr` needs `BANKRUPTCY_PROTECTION_LIMIT_FACTOR` from the data files, so it is only offered + with a tuning table loaded. Committing a value computed from an unloaded (zero) constant + regressed one leaf in the first measurement and was removed. + +Committing it closes **nothing** on the reference pair, and that is a fact about the rest of the +engine rather than about this chain: the limits move between `turn1` and `turn2` because the +**civilian population grows**, and that growth is not committed, so our limit equals the input +save's. Measured with `--commit-blocked=T31 --ai-player 1`: **0 closed, 0 regressed** (209 -> 204 +and 108 -> 103, unchanged from the baseline). The same numbers hold with every other blocked phase +committed alongside it. + +### 6.2 P01 `ComputeBudget` -- NOT unblocked, and the roadmap's item 1 was wrong about this + +`ComputeBudget` has two modes and they take their per-system money from **different functions** +(0x008631fd, the `[ebp+0xc]` test): + +``` +if (projected) money = ComputeMaxIncome(s) // 0x007521c0 -- what this lane closed +else { OutputRates r; ComputeOutput(s, &r); money = r.out[3] } // 0x00751fb0 +``` + +`ComputeOutput` (0x00751fb0) passes the system's **own** `Rts` sliders, not a max-mods vector. On +that path the science, construction and terraform channels are funded, which means: + +* the repair pass inside `ComputeOutputFromRates` runs (it is **not** side-effect free -- it repairs + ships in orbit), so its leftover science points can be non-zero; +* the unspent-industry and unspent-terraforming cascades into the money channel are live, and §2.4's + proof that both are zero **does not apply**; +* the build queue's consumption of construction points is in the same call. + +So the turn's real per-system money is a strictly larger problem than the one the `BnkEl` oracle can +falsify, and nothing in this corpus states its answer. P01/P02/P03/P05/P06 stay blocked, and the +next lane on them should target `ComputeOutputFromRates`'s full channel split rather than this +chain. The roadmap's item 1 claimed this chain unblocks them; it does not. + +### 6.3 The two research RNG words + +Both are downstream of the budget's research allocation: + +1. `ProcessResearch`'s `RNG::Chance(odds)` on the funded node, where `odds` is built from the + research **points** the allocation bought (0 or 1 word -- `Chance` draws nothing for `p <= 0` + or `p >= 1`); +2. the draw inside `ServerPlayer::OnTechResearched`'s effect callback, which fires only when a + tech actually completes and therefore only when the allocation was large enough (0 or 1 word, + measured live in the B3 compare run). + +Those points come from `ComputeBudget`'s research money, which is on the P01 path of §6.2 -- not +this one. **The generator model still cannot include them.** What has changed is the reason: it is +no longer "the money output of a system is unmodelled" but "the *projected-rate* money output is", +which is a narrower and differently-shaped gap, and it is now the only thing between the ledger and +those two words. diff --git a/ghidra/addresses.d/lane-e1.json b/ghidra/addresses.d/lane-e1.json new file mode 100644 index 0000000..368de11 --- /dev/null +++ b/ghidra/addresses.d/lane-e1.json @@ -0,0 +1,101 @@ +{ + "_note": "Lane E1, the output -> money chain. Six addresses this lane read are already held elsewhere and AGREE, so they are dropped from this fragment rather than duplicated (rule 14): ServerSystem_ComputeMaxIncome 0x007521c0 (lane-k.json), ServerSystem_ComputeOutputFromRates 0x00751bb0 and ServerSystem_TradePointsToMoney 0x007505b0 and ServerSystem_CalcSuitMod 0x007484d0 (addresses.json), GroupIncome 0x00535e80 and PopTypeRow 0x00535e00 (lane-n.json). Every one of those six was re-read from the instruction stream this lane and the address, convention and boundary matched what was already recorded.", + "entries": [ + { + "name": "ServerSystem_PopIncome", + "addr": "0x0074d760", + "convention": "thiscall", + "prototype": "double (ServerSystem* sys, int groupType) // `ret 4`, real end 0x0074d8e1. The income analogue of PopOutput 0x0074d8f0, and NOT the same law: it sums, over species 0..6, `(double)ftol( (double)GroupIncome(groupType, count) x moraleMod x addictionMod )` -- so the value truncates TWICE per (group, species) row, once inside GroupIncome and once after both factors. moraleMod is 0x00746910 (the same helper the output term uses) and applies to groupType 1 only; addictionMod is the float behind slot 0x00aeca48 when the system's int[7] addiction table at +0x1e4 has a non-zero entry for that species. For groupType 1 and the owner's own species on a non-independent system the count first gains the capacity surplus from two calls to 0x0074a6d0 (out slot 4, then out slot 6), max(0, B - A)", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerSystem_SlaveIncome", + "addr": "0x0074b700", + "convention": "thiscall", + "prototype": "double (ServerSystem* sys) // plain `ret`, real end 0x0074b793. groupType 2 of the same loop as PopIncome, over SlaveCount(species) (0x0074b610): no morale factor and no capacity surplus, but the addiction factor still applies. Unexercised: slave counts are 0 on every call in the corpus", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerSystem_SlaveCount", + "addr": "0x0074b610", + "convention": "thiscall", + "prototype": "int64 (ServerSystem* sys, int species) // the slave-group population of one species; the group-2 counterpart of GroupPopulation 0x00747ba0", + "status": "unverified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08) -- reached from SlaveIncome, body not read" + }, + { + "name": "ServerSystem_ComputeOutput", + "addr": "0x00751fb0", + "convention": "thiscall", + "prototype": "void (ServerSystem* sys, int out[12]) // zeroes `out`, returns immediately when the caller's pointer is null or the system has no owner (+0x100), else calls ComputeOutputFromRates(out, &sys->Rts /*+0x88*/) -- the system's OWN rate sliders, not a max-mods vector. THE DISTINCTION THAT MATTERS: ComputeBudget's real (non-projected) per-system money is this function's out[3], while its projected mode and UpdateBankruptcyLimits use ComputeMaxIncome 0x007521c0. They are different numbers: this path funds the science, construction and terraform channels, so the repair pass runs and the unspent-industry and unspent-terraforming cascades into the money channel are live", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "StrategyServer_IncomeDifficultyMod", + "addr": "0x0080f470", + "convention": "thiscall", + "prototype": "float (StrategyServer* srv, ServerPlayer* p) // `ret 4`, real end 0x0080f49f. Returns float32( float32(DifficultyMods_Select(p->diffMods /*+0x36c*/, p)[1]) x float32(srv->IncMod /*raw base +0xbc, the Sim block's `IncMod` tag*/) ), or just the server modifier when p is null. Every step is stored back through a 4-byte float. `ecx` here is the RAW StrategyServer base (ServerSystem+0x10), four bytes above the base the class's own methods get. THIS IS THE MISSING x1.1: at the difficulty level every corpus save carries (aidf == 1) the AI column of the table is 1.1f", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "DifficultyMods_Select", + "addr": "0x0059b490", + "convention": "thiscall", + "prototype": "float* (DifficultyMods* rec, ServerPlayer* p) // `ret 4`, real end 0x0059b4b3. Returns &rec->f[0] (the AI triple, at +0x04) when p is non-null AND p->[0xf9] (is-AI) is set AND p->NPC (+0xfb) is clear; otherwise &rec->f[3] (the non-AI triple, at +0x10). The three consumers read offset +0 (fleet maintenance divisor, ComputeBudget 0x0086338b), +4 (system and trade-route money, 0x0080f470 and 0x00833938) and +8 (research points bought with money, 0x0080e229 and 0x00863618)", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "LoadDifficultyRow", + "addr": "0x005a3990", + "convention": "cdecl", + "prototype": "void (int level, DifficultyMods* out) // real end 0x005a3a53 (Ghidra size 193 stops 2 bytes short). memcpy's the default {id 1, 1.0f x6} into `out` FIRST, then builds the table with BuildDifficultyTable 0x005a3870 and linear-searches it for id == level (stride 0x1c, from the 0x92492493 magic divide), copying the six floats on a hit. An out-of-range level therefore yields all ones rather than failing. Called from ServerPlayer::Read 0x008804d0 at 0x00880fa3, gated on 0 <= aidf < 3, which is also where ServerPlayer+0x368 (`aidf`) is stored", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "BuildDifficultyTable", + "addr": "0x005a3870", + "convention": "thiscall", + "prototype": "vector* (vector* out) // real end 0x005a3989. THE TABLE IS BUILT IN CODE, from .rdata float literals -- no data-file key, no GlobalConst slot, same shape as lane N's pop-type table. Three rows of {int id; float ai[3]; float other[3]} (0x1c): id 0 = ai {1,1,1} / other {1.5,1.5,1.5} (0x00a1b000); id 1 = ai {3.0 (0x00a0451c), 1.1 (0x009f957c), 1.5} / other {1,1,1}; id 2 = ai {1e6 (0x009ebd7c), 1.7 (0x009f9580), 2.0 (0x00a04518)} / other {1,1,1}. Read as: level 0 gives the break to the human player, levels 1 and 2 give it to the AI, and on level 2 the AI's fleet maintenance is divided by a million", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerPlayer_GetIncMod", + "addr": "0x0080dd10", + "convention": "thiscall", + "prototype": "float (ServerPlayer* p) // seven bytes: `fld DWORD [ecx+0x30c]; ret`. The save's per-player `IncMod`", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerPlayer_GetSpeciesCostFactor", + "addr": "0x0080dd20", + "convention": "thiscall", + "prototype": "float (ServerPlayer* p) // `SpeciesDef(p->Species /*+0x5c*/)->+0x24`, the multiplier on the suitability MONEY cost (Zuul 0.7). A data-file value", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "StrategyServer_IdealSuit", + "addr": "0x0080f4b0", + "convention": "thiscall", + "prototype": "float (StrategyServer* srv, int species) // `ret 4`; one instruction of work: `fld DWORD [ecx + species*4 + 0xf8]`. The per-species ideal-suitability array on the RAW server base. IT IS ON THE WIRE: the Sim block's `ISsp`/`ISsu` pairs are this float[7] in species-index order, and the array is randomised per game by the map generator -- verified against every ServerPlayer's own `IdealSuit` field in all 11 corpus saves. CalcSuitMod reads THIS, not the player's field", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + }, + { + "name": "ServerSystem_TerraformPointsNeeded", + "addr": "0x00746890", + "convention": "thiscall", + "prototype": "double (ServerSystem* sys) // real end 0x0074690d. Returns 0 with no owner, else float32(|IdealSuit(species) - sys->Suit|) / |owner->TerraMod (+0x134) x [0x00a1f928] / 20000|. The sign term (-1.0 when the planet's suitability is STRICTLY above the ideal) is multiplied in BEFORE the fabs at 0x00746906 and therefore cancels: the result is always >= 0. That is why the unspent-terraform cascade into the money channel is provably zero under the max-income rate vector", + "status": "verified", + "source": "findings/subsystems/income-term.md (lane E1 2026-09-08)" + } + ] +} diff --git a/tools/max_income_predict.py b/tools/max_income_predict.py index 840b9c2..a32a0f3 100644 --- a/tools/max_income_predict.py +++ b/tools/max_income_predict.py @@ -9,22 +9,32 @@ which is the per-system money output that blocks `ComputeBudget` and four other This script goes the other way: it computes the same number from the colony state and compares, so the formula is falsified per player-record rather than per lane. -The chain it implements (lane N, findings/subsystems/output-term.md, live-verified on VM140): +The chain it implements (lane N `findings/subsystems/output-term.md` for the output half, +lane E1 `findings/subsystems/income-term.md` for the money half, both read off the +instruction stream): total = ComputeTotalOutput(SRoh = 0) # max mods -> trade rate 1, rest 0 - = ( overHarvestDemand x speciesResourceOutput - + (TRes + resAvail) x stripMineFraction x 0.9 - + populationOutput ) - x OutMod x sysOutMod x setupOutMod x RebOutMod x ScOutMod trade = roundHalfEven(total) - money = ftol( TradePointsToMoney(trade) ) + money = trunc( TradePointsToMoney(trade) ) + maxIncome_s = max(money, 0) -Population output per head is `typeOutputMod x 1.8 / 500000`; population INCOME per head is -`typeIncomeModifier / 14000`, with no 1.8 -- two different laws off the same table. +with -Everything the executable carries is hard-coded here as such. The two per-species fields the -data files supply (SpeciesDef +0x4c base resource demand, +0x50 resource output factor) are -options, defaulting to the values measured live on VM140 for the species this corpus contains. + TradePointsToMoney(trade) = + diffMod * ( f32(IncMod) * ( f32(speciesIncomeFactor) + * ( Slaves + (PopIncome(1) + (((trade - trade mod 5) * 5 + 0) + PopIncome(0))) ) ) ) + - f32(speciesCostFactor) * ( CalcSuitMod * 10000 * 1.5 ) + + diffMod = f32( f32(DifficultyMods(owner)[1]) * f32(server.IncMod) ) + +Population output per head is `typeOutputMod * 1.8 / 500000`; population INCOME per head is +`typeIncomeModifier / 14000`, with no 1.8 -- two different laws off two adjacent columns of the +same three-row table, and the income one truncates twice (once inside GroupIncome, once after +the morale/addiction product) PER SPECIES. + +The difficulty table is built in code from .rdata float literals (0x005a3870); every corpus save +carries `aidf == 1`, whose AI income modifier is 1.1f. Which players count as AI is NOT on the +wire -- see --ai-rule. """ import argparse import json @@ -39,14 +49,37 @@ import save_reader as sr # noqa: E402 # ---- constants the executable carries itself ------------------------------------------------ OUT_FACTOR = struct.unpack(" (aiMaintDiv, aiIncome, aiTrade, plMaintDiv, plIncome, plTrade) +DIFFICULTY_TABLE = { + 0: (1.0, 1.0, 1.0, 1.5, 1.5, 1.5), + 1: (3.0, 1.100000023841858, 1.5, 1.0, 1.0, 1.0), + 2: (1000000.0, 1.7000000476837158, 2.0, 1.0, 1.0, 1.0), +} +DIFFICULTY_DEFAULT = (1.0, 1.0, 1.0, 1.0, 1.0, 1.0) # LoadDifficultyRow's memcpy'd default + +# ---- values the data files supply, measured live on VM140 (2026-09-08, lane N) --------------- +# SpeciesDef +0x4c (base resource demand) and +0x50 (resource output factor) are PER SPECIES and +# are read off the SYSTEM OWNER's species, not the system's population species. Taking them as +# one global pair is what cost lane N's predictor every Zuul record: a Zuul colony's harvest term +# is min(resAvail, 10) * 40 = 400 output points that a 0/10 pair scores as zero, and 400 points +# is 400/5 whole blocks worth 5 money each, i.e. exactly the 2000-per-colony shortfall observed. +SPECIES_BASE_DEMAND = {5: 10} # SpeciesDef +0x4c; 0 for Human and Tarkas +SPECIES_RES_OUTPUT = {5: 40.0} # SpeciesDef +0x50; 10 for Human and Tarkas +SPECIES_BASE_DEMAND_DEFAULT = 0 +SPECIES_RES_OUTPUT_DEFAULT = 10.0 +# SpeciesDef +0x18 income factor / +0x24 cost factor, per formula-gaps.md Q3 (NOT measured live). +SPECIES_INCOME_FACTOR = {5: 1.1, 6: 0.8} +SPECIES_COST_FACTOR = {5: 0.7} def f32(v): @@ -54,6 +87,7 @@ def f32(v): def ftol(v): + """_ftol2 -- truncation toward zero.""" return math.trunc(v) @@ -103,14 +137,15 @@ def group_output(group, count, morale, stations, tuning, owned, independent): b = tuning["STATION_BONUS_IMPERIAL_OUTPUT"] sf = 1.0 + stations * (b if b > 0 else 0.0) mo = 1.0 - if group == 1 and owned and not independent: - mo = morale_output_mod(morale, tuning) + if group == 1: + mo = morale_output_mod(morale, tuning, owned, independent) v = POPTYPE_OUT[group] * (sf * OUT_FACTOR) * mo * q return v if v > 0 else 0.0 -def morale_output_mod(m, tuning): - if m == 0: +def morale_output_mod(m, tuning, owned, independent): + """0x00746910 -- 1.0 with no owner, on an independent system, or with cm == 0.""" + if not owned or independent or m == 0: return 1.0 if m >= tuning["MORALE_INCREASE_OUTPUT"]: x = tuning["MORALE_INCREASE_OUTPUT_MOD"] @@ -129,6 +164,8 @@ LIVE_TUNING = { "MORALE_DECREASE_OUTPUT": 20, "MORALE_DECREASE_OUTPUT_MOD": f32(0.5), "SLAVES_OUTPUT_MOD": f32(3.0), + "SLAVES_INCOME_MOD": f32(3.0), + "ADDICTION_INCOME_MOD": f32(0.9), } @@ -169,32 +206,43 @@ def population(node, name, group, species): return total -def morale_table(node): - """`cm` on the wire is a count then (species, value) pairs.""" - cm = sub(node, "cm") +def sparse_table(node, name, key_tag, val_tag): + """`cm`/`nadct` on the wire: a count then (index, value) pairs.""" out = {} - if cm is None: + holder = sub(node, name) if name else node + if holder is None: return out pending = None - for c in kids(cm): - if c.name == "msp": + for c in kids(holder): + if c.name == key_tag: pending = c.value - elif c.name == "mv" and pending is not None: + elif c.name == val_tag and pending is not None: out[pending] = c.value pending = None return out +def morale_table(node): + return sparse_table(node, "cm", "msp", "mv") + + +def addiction_table(node): + """The int[7] at ServerSystem+0x1e4: `nadct` then sparse (`ads`,`adt`) pairs.""" + return sparse_table(node, None, "ads", "adt") + + def read_state(path): r = sr.read_save(path) - systems, players = [], [] + systems, players, sim = [], [], None for _, n in walk(r.tree): + if n.name == "Sim" and sim is None: + sim = n names = {c.name for c in kids(n)} if {"Pop", "Rts", "Infra", "pbon"} <= names: systems.append(n) elif {"OutMod", "IncMod", "ScOutMod", "BnkEl", "PlyrIdx"} <= names: players.append(n) - return systems, players + return systems, players, sim def system_species(node, owner_species): @@ -218,7 +266,123 @@ def is_independent(node): return bool(field(node, "hindi")) -def predict(save, base_demand, res_output, verbose=False): +# ---- the income chain ------------------------------------------------------------------------ +def group_income(t, count, tuning): + """0x00535e80: _ftol2( f32(POPTYPE[t].income) * (count / 14000) ).""" + inc = POPTYPE_INC[t] + if inc is None: + inc = tuning["SLAVES_INCOME_MOD"] + return ftol(f32(inc) * (float(count) / INCOME_DIVISOR)) + + +def pop_income(t, counts, morale, addicted, tuning, owned, independent): + """0x0074d760 / 0x0074b700. `counts` is species -> population for this group type.""" + total = 0.0 + for sp in range(7): + n = counts.get(sp, 0) + if not n > 0: + continue + mo = morale_output_mod(morale.get(sp, 0), tuning, owned, independent) if t == 1 else 1.0 + ad = tuning["ADDICTION_INCOME_MOD"] if addicted.get(sp) else 1.0 + r = group_income(t, n, tuning) + total += float(ftol(float(r) * mo * ad)) + return total + + +def calc_suit_mod(node, p, species, ideal_suit): + """0x007484d0.""" + if field(node, "vnh"): + return 0.0 + if not field(node, "PID"): + return UNOWNED_SUIT_MOD + if field(p, "RebAI"): + return 0.0 + ideal = ideal_suit.get(species) + if ideal is None: + raise KeyError("no IdealSuit known for species %r" % (species,)) + tol = f32(field(p, "SuitTol") or 0.0) + d = abs(f32(ideal) - f32(field(node, "Suit") or 0.0)) + return d if d <= tol else tol + + +def difficulty_income_mod(p, is_ai, server_inc_mod): + """0x0080f470 + 0x0059b490 + 0x005a3990: float32 throughout.""" + m = f32(server_inc_mod) + if p is None: + return m + level = field(p, "aidf") + row = DIFFICULTY_TABLE.get(level, DIFFICULTY_DEFAULT) + # Select(): the AI triple is f[0..2], the non-AI triple f[3..5]; ->+4 is index 1 of the triple. + mods = row[0:3] if (is_ai and not field(p, "NPC")) else row[3:6] + return f32(f32(mods[1]) * m) + + +def system_income(s, p, base_demand, res_output, ideal_suit, server_inc_mod, is_ai, tuning): + owner_species = field(p, "Species") + if base_demand is None: + base_demand = SPECIES_BASE_DEMAND.get(owner_species, SPECIES_BASE_DEMAND_DEFAULT) + if res_output is None: + res_output = SPECIES_RES_OUTPUT.get(owner_species, SPECIES_RES_OUTPUT_DEFAULT) + sp = system_species(s, owner_species) + strip = bool(field(p, "AMine")) + independent = is_independent(s) + + res = field(s, "Res") or 0 + avail = res + ((field(s, "MRes") or 0) + (field(s, "ARes2") or 0) if strip else 0) + pop = (field(s, "Pop") or 0) + (field(s, "pbon") or 0) + morale = morale_table(s) + addicted = addiction_table(s) + + # ---- the output half (lane N, live-verified) -------------------------------------------- + if field(s, "rbfl"): + total = 0.0 + else: + civ_own = population(s, "Pop2", 1, sp) + population(s, "pbon2", 1, sp) + harvest = over_harvest_demand(0.0, avail, pop, base_demand) * f32(res_output) + resource = (float((field(s, "TRes") or 0) + avail) + * float(strip_mine_fraction(pop, field(s, "Infra") or 0.0, + field(s, "ibon") or 0.0)) + * RES_FACTOR) + imperial = group_output(0, pop, 0, 0, tuning, True, independent) + civilian = group_output(1, civ_own, morale.get(sp, 0), 0, tuning, True, independent) + total = (civilian + (imperial + 0.0)) + (harvest + resource) + total *= f32(field(p, "OutMod") or 1.0) + total *= f32(field(s, "OutMod") or 1.0) + total *= f32(field(p, "RebOutMod") or 1.0) + total *= f32(field(p, "ScOutMod") or 1.0) + + # ---- the money half (lane E1) ------------------------------------------------------------- + trade = round_half_even(total) + + imperial_counts = {sp: pop} if pop > 0 else {} + civ_counts = {} + for q in range(7): + c = population(s, "Pop2", 1, q) + population(s, "pbon2", 1, q) + if c: + civ_counts[q] = c + + t = pop_income(0, imperial_counts, morale, addicted, tuning, True, independent) + t = ((trade - math.fmod(trade, BLOCK)) * BLOCK + 0.0) + t + t = pop_income(1, civ_counts, morale, addicted, tuning, True, independent) + t + t = 0.0 + t # SlaveIncome(): no slaves in this corpus + + t = f32(SPECIES_INCOME_FACTOR.get(owner_species, 1.0)) * t + t = difficulty_income_mod(p, is_ai, server_inc_mod) * (f32(field(p, "IncMod") or 1.0) * t) + + cost = f32(SPECIES_COST_FACTOR.get(owner_species, 1.0)) * ( + calc_suit_mod(s, p, sp, ideal_suit) * SUIT_COST_A * SUIT_COST_B) + return ftol(t - cost) + + +AI_RULES = { + # `ServerPlayer+0xf9` is a setup input, not a save field (income-term.md §3.1). + "non-npc-not-first": lambda p, i: not field(p, "NPC") and (field(p, "PlyrIdx") or 0) != 0, + "none": lambda p, i: False, + "all-non-npc": lambda p, i: not field(p, "NPC"), +} + + +def predict(save, base_demand, res_output, ai_rule, verbose=False): """Return {BnkEl: (predicted maxIncome, per-system detail)}. The join key is `BnkEl` rather than any id: a system stores its owner as a HANDLE id @@ -226,7 +390,30 @@ def predict(save, base_demand, res_output, verbose=False): schemes. The oracle inverts the same `BnkEl` the player node carries, so keying on it needs no id mapping at all and cannot silently pair the wrong two records. """ - systems, players = read_state(save) + systems, players, sim = read_state(save) + server_inc_mod = field(sim, "IncMod", 1.0) if sim is not None else 1.0 + + # `server->IdealSuit[species]` IS on the wire: the Sim block's `ISsp`/`ISsu` pairs are that + # float[7], in species-index order, and they are randomised per game by the map generator. + # Each ServerPlayer's own `IdealSuit` field carries the same value for its species, so the + # two are cross-checked here rather than one being trusted blindly (rule 8: two checks that + # share a hidden assumption are one check -- these two do not share a source). + ideal_suit = {} + if sim is not None: + idx = 0 + for c in kids(sim): + if c.name == "ISsu": + ideal_suit[idx] = c.value + idx += 1 + for p in players: + s, v = field(p, "Species"), field(p, "IdealSuit") + if s is None or v is None: + continue + if s in ideal_suit and ideal_suit[s] != v: + print(" WARNING: species %d ISsu %r disagrees with a player's IdealSuit %r" + % (s, ideal_suit[s], v), file=sys.stderr) + ideal_suit.setdefault(s, v) + # A player's handle id is not a named field, but every system names its owner's, so the # set of distinct non-zero `PID` values is the set of owning players -- in the same order # the player records appear. Pair them by position among the players that own anything. @@ -244,69 +431,39 @@ def predict(save, base_demand, res_output, verbose=False): handle_of[id(p)] = h out = {} - for p in players: + for i, p in enumerate(players): h = handle_of.get(id(p)) + is_ai = AI_RULES[ai_rule](p, i) total = 0 detail = [] if h is not None: for s in systems: if field(s, "PID") != h: continue - m = system_income(s, p, base_demand, res_output) + m = system_income(s, p, base_demand, res_output, ideal_suit, + server_inc_mod, is_ai, LIVE_TUNING) detail.append((field(s, "Idx"), field(s, "Name"), m)) total += max(m, 0) out[field(p, "BnkEl")] = (total, detail) if verbose and detail: - print(" player handle %s (%d system(s))" % (h, len(detail))) + print(" player handle %s sp=%s ai=%s npc=%s (%d system(s))" + % (h, field(p, "Species"), is_ai, field(p, "NPC"), len(detail))) for sid, nm, m in detail: print(" sys %-4s %-16s money=%d" % (sid, nm, m)) return out -def system_income(s, p, base_demand, res_output): - rts = sub(s, "Rts") - owner_species = field(p, "Species") - sp = system_species(s, owner_species) - strip = bool(field(p, "AMine")) - - res = field(s, "Res") or 0 - avail = res + ((field(s, "MRes") or 0) + (field(s, "ARes2") or 0) if strip else 0) - pop = (field(s, "Pop") or 0) + (field(s, "pbon") or 0) - civ = population(s, "Pop2", 1, sp) + population(s, "pbon2", 1, sp) - morale = morale_table(s).get(sp, 0) - independent = is_independent(s) - - harvest = over_harvest_demand(0.0, avail, pop, base_demand) * f32(res_output) - resource = (float((field(s, "TRes") or 0) + avail) - * float(strip_mine_fraction(pop, field(s, "Infra") or 0.0, field(s, "ibon") or 0.0)) - * RES_FACTOR) - imperial = group_output(0, pop, 0, 0, LIVE_TUNING, True, independent) - civilian = group_output(1, civ, morale, 0, LIVE_TUNING, True, independent) - base = (civilian + (imperial + 0.0)) + (harvest + resource) - - if field(s, "rbfl"): - return 0 - total = base - total *= f32(field(p, "OutMod") or 1.0) - total *= f32(field(s, "OutMod") or 1.0) - total *= f32(field(p, "RebOutMod") or 1.0) - total *= f32(field(p, "ScOutMod") or 1.0) - - trade = round_half_even(total) - t = (trade - math.fmod(trade, 5.0)) * 5.0 - t += ftol(POPTYPE_INC[0] * (float(pop) / INCOME_DIVISOR)) - t += ftol(POPTYPE_INC[1] * (float(civ) / INCOME_DIVISOR)) - t *= f32(field(p, "IncMod") or 1.0) - return ftol(t) - - def main(): ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) ap.add_argument("saves", nargs="*") ap.add_argument("--oracle", help="JSON from max_income_oracle.py --json") - ap.add_argument("--base-demand", type=int, default=SPECIES_BASE_DEMAND) - ap.add_argument("--res-output", type=float, default=SPECIES_RES_OUTPUT) + ap.add_argument("--base-demand", type=int, default=None, + help="override SpeciesDef +0x4c for every species") + ap.add_argument("--res-output", type=float, default=None, + help="override SpeciesDef +0x50 for every species") + ap.add_argument("--ai-rule", choices=sorted(AI_RULES), default="non-npc-not-first", + help="which players count as AI (ServerPlayer+0xf9 is not on the wire)") ap.add_argument("-v", "--verbose", action="store_true") a = ap.parse_args() @@ -314,15 +471,21 @@ def main(): if a.oracle: with open(a.oracle) as fh: raw = json.load(fh) - for row in (raw if isinstance(raw, list) else raw.get("records", [])): - oracle[(os.path.basename(row.get("save", "")), row.get("BnkEl"))] = row.get("maxIncome") + if isinstance(raw, dict) and raw and isinstance(next(iter(raw.values())), list): + for name, rows in raw.items(): + for row in rows: + oracle[(os.path.basename(name), row.get("BnkEl"))] = row.get("maxIncome") + else: + for row in (raw if isinstance(raw, list) else raw.get("records", [])): + oracle[(os.path.basename(row.get("save", "")), row.get("BnkEl"))] = \ + row.get("maxIncome") hits = misses = unknown = 0 for save in a.saves: name = os.path.basename(save) print("==", name) for bnkel, (total, detail) in sorted(predict(save, a.base_demand, a.res_output, - a.verbose).items()): + a.ai_rule, a.verbose).items()): if not detail: continue want = oracle.get((name, bnkel)) @@ -334,7 +497,7 @@ def main(): print(" BnkEl=%-12s predicted=%-12d MATCH" % (bnkel, total)) else: misses += 1 - print(" BnkEl=%-12s predicted=%-12d oracle=%-12d delta=%+d (%.4f x)" + print(" BnkEl=%-12s predicted=%-12d oracle=%-12d delta=%+d (%.6f x)" % (bnkel, total, want, total - want, (total / want) if want else float("nan"))) print("\n%d match, %d differ, %d with no oracle record" % (hits, misses, unknown))