alex/sots-re
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Compare commits

base: alex:f69e07824799264dde97d2815c1a1acea912ebb7
Branches Tags
alex:main
...
compare: alex:1bae6f1bfea6e374096ed0aa7cf920cc22d5e205
Branches Tags
alex:main

3 commits
f69e078247 ... 1bae6f1bfe

Author SHA1 Message Date
alex
1bae6f1bfe board: lane ID - one allocator partitioned by node nibble; techId is the sorted index; 282 named 2026-09-08 20:28:15 -04:00
alex
c60ee36a0e lane ID: zero-residual id account for the canonical turn, and the collision story 
Diffing the four master id lists in the Sim block accounts for every object created on
turn 2->3: ship 1760 (server counter 110), fleet 1776 (111), fleet 34 (client node 2,
counter 2), and DesignIDs unchanged at 43 -- so no design was created that turn and
1776 is a fleet, not a design. Sharpens the correction to turn-command-replay.md row 2.
Also works out what explanation (B) implies for CB's reloaded run: the client re-issues
18, collides in its own map, and the autosave still matches because only the server's
state is serialized.
 2026-09-08 20:26:44 -04:00
alex
975c19a57d lane ID: one id allocator on sixteen node counters, and techId is a sorted index 
The client-side allocator is not a second allocator. StrategyServer and StrategyClient
are both StrategySim, which owns an IDMap at +0x80; each sim allocates from its own map
on its own local node index. StrategyServer::InitGameForPlayer sets that index to
PlyrIdx + 1 (node 0 is the server's) and seeds the client from the server's counter for
that node. IDMap::Initialize names the save's NM* tags: NMSz nodes, NMLc local node,
NMnx that node's counter -- confirming B5's labelled hypothesis and adding the other two.
Cross-checked on 20 saves: nodes 1, 2 and 3 all occur, counters run from 1 per node, and
2,600 ids collide zero times. Corrects turn-command-replay.md row 2: design 18 IS in
turn2-state.sav, so the canonical pair needs one minted id, not two. One open item, with
the one-hook probe named: a reloaded save produced fleet 34 rather than 18, and nothing I
read restores a client counter.

techId is the 0-based index into the master tech list sorted by _stricmp -- read out of
MasterTechTree's ctor, which sorts a copy of the parse-order list and then writes
def->techId = i. 282 is XNC_TrnsHum2, which lane L4 had already observed live and nobody
connected. tools/techid_table.py derives all 293 offline and refuses to print unless the
four observed points agree.
 2026-09-08 20:22:24 -04:00
5 changed files with 670 additions and 0 deletions
Show stats Expand all files Collapse all files

6
campaign/board.md
View file

@ -380,3 +380,9 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`).
| two type corrections, and the coverage figures were OPTIMISTIC too | verify | verified | high | 100% | 2026-09-08 | (1) `Game::CombatReport`: `auto`/`cdst` are **bools** and `dur`/`cdt`/`cdi` are **floats** - all five had been ints. Four are 0/1 everywhere so the bytes never moved, but **`cdt` was being read as 1070805848 for what is 1.598**. (2) `Game::CombatWeaponReport`'s damage quartet is flat with float dami/damt; the old shape reached them through an `obj_flex` nested `dams` frame **the binary never writes** - which made CoverageArchive charge **one phantom typed item per weapon report**. The four affected saves have 8/13/21/32 weapon reports and their totals moved by exactly 8/13/21/32, **so the pre-fix coverage figures were slightly OPTIMISTIC as well as too low** |
| FTPnts stays carried, and names its own workload | verify | backlog | — | 0% | 2026-09-08 | The one item the recovery itself marks `unresolved`, count 0 in every `Lay`. **It names the element class from the DECORATED HELPER NAME - exactly the SysMem/mts/nalat trap: the framing belongs to the helper, not the class.** Carried, with `Game::FieldTemplate::Point` deliberately left unbound. **The workload that settles it: a save with a stored fleet tactical formation** - set a fleet's combat layout in the tactical setup screen, then save. Everything else still carried is a never-filled container costing zero items on every save |
| LAB: CT111 direct SSH is publickey-denied | meta | verified | high | 100% | 2026-09-08 | Multiple lanes have now hit this and worked around it differently. **Direct `ssh 192.168.10.138` is denied; the working route is `ssh spicy` then `pct exec 111`.** Lane WS could not reach it at all and **installed g++-mingw-w64-i686 locally, cross-building against GCC 10-posix rather than CT111's toolchain** - so its "shim clean" was against a different compiler and the integrator re-ran the real one. Two facts it surfaced that are worth keeping: `TRmin` wanted a member named `min`, **which windows.h defines as a macro** (it is `mine` for that reason), and **the win32-threads variant of that compiler cannot build zip_archive.cpp at all** (`std::mutex` absent) - the posix variant is required. ReVa also stayed down (`CONNECTION_CLOSED`) the whole session; the server is `active` and CT111 is healthy at load ~15 from our own builds |
| THERE IS NO CLIENT-SIDE ALLOCATOR - one allocator, partitioned by node | objects | verified | high | 100% | 2026-09-08 | Lane ID, and it dissolves the question rather than answering it. **StrategyServer and StrategyClient are BOTH `Game::StrategySim`** (RTTI; StrategyServer carries vftables at offset 0 AND offset 4 - earned rule 1's "two bases 4 bytes apart", and the +4 one IS the StrategySim subobject). StrategySim owns an `IDMap` at **StrategySim+0x80**, so each sim allocates from **its own map on its own node index** using lane B5's one function. The entire protocol is a single branch in StrategySim::CreateDesign 0x008827e0: **`explicitId != 0 ? use it : AllocateOnLocalNode(this+0x80)`** - the client mints and puts the id in the command; the server runs THE SAME CODE and mints nothing. **`localNode = (PlyrIdx == -1) ? 0 : PlyrIdx + 1`** (four instructions at 0x007c8ecb): node 0 is the server's, player k gets node k+1. IDMap::Initialize caps nodes at **16 - that IS the nibble** - zeroes every counter and seeds only the local one. **NO COLLISION IS POSSIBLE**: the low nibble partitions the space; ~2,600 ids across 20 saves and 4 games, zero collisions, with nodes 1, 2 and 3 all occurring and each starting at counter 1 |
| save tags named: NMSz / NMLc / NMnx | objects | verified | high | 100% | 2026-09-08 | Falls straight out of IDMap::Initialize(numNodes, localNode, startId) 0x008b9ad0: **NMSz = node count, NMLc = local node, NMnx = that node's counter** - confirming lane B5's labelled hypothesis and adding the other two. A client is seeded from the SERVER's counter for its node via IDMap::GetNodeCounter. **Zero-residual id account of the reference turn** from the Sim block's master lists: ship 1760 (counter 110), fleet 1776 (111), fleet 34 (client counter 2), DesignIDs unchanged 43->43. TWO CORRECTIONS to turn-command-replay.md 4: 1712 was issued in turn 1->2, and **1776 is a fleet, not a design**; and **design 18 is already in turn2-state.sav**, so the canonical replay must mint exactly ONE id (fleet 34), not two |
| techId = the stricmp-SORTED index, closed offline | objects | verified | high | 95% | 2026-09-08 | No capture needed. Read out of MasterTechTree's constructor: it copies the parse-order list to a second vector, **std::sorts the copy with an inlined `_stricmp(a->name, b->name) < 0`**, then walks it writing `def->techId = i` at 0x0058ba29 - and the sorted vector doubles as the lower_bound name index, **which is WHY it is sorted**. That also explains lane RB's dead end: the parse-order table we held is THE OTHER VECTOR. **282 = XNC_TrnsHum2** - and **it was already in our notes, observed live by lane L4 in run R2 and never connected** (rule 18's corollary, paid for again). All four observed points hit exactly and it reproduces L4's six-member family as {282, 284, 286, 288, 290, 292}, so **Rung C-set's "we can name all k" CAN CLOSE**. `tools/techid_table.py` derives all 293 offline and **refuses to print unless the four observed points agree**. FLAGGED: the _stricmp half is disassembly-only - 48 of 293 ids differ from byte-wise order and no observed point discriminates |
| ID declined the third target, correctly | meta | verified | high | 100% | 2026-09-08 | It did NOT run the cbtrap set: that needs an End Turn on VM140, **which rewrites the three oracle autosaves**, and lane CB had already judged it the weaker check next to RB's zero-residual ModCount = 24. **Confirmatory value did not justify putting the reference bytes at risk.** It stays cheap for anyone who backs up SavedGames first, and the exact hashes are recorded in its finding. VM140 released with the oracle verified intact and the game never launched |
| ID's falsified prediction, with the probe named | verify | backlog | — | 0% | 2026-09-08 | Its prediction that client counters restart at 0 on load is **falsified by a corpus save**: lane CB's reloaded run produced fleet **34**, not 18. Either something restores the counter (six IDMap/InitGame functions read, no such write found) or **the client makes one earlier unsaved allocation per turn** - the second fits everything and implies the id collision has already happened harmlessly in CB's run. One hook on 0x008b8ae0 logging (map, node, id, return address) separates them |
| LAB TRAP: cmd.exe eats parentheses in save names | meta | verified | high | 100% | 2026-09-08 | `ssh ... certutil -hashfile "...(Autosave).sav"` fails with **"Check the spelling"**, which reads exactly like a missing file. Lane ID's first attempt **reported the three oracle saves as absent, and they were never absent** - a false alarm about the campaign's most important bytes |

334
findings/subsystems/id-allocation.md Normal file
View file

@ -0,0 +1,334 @@
# Object id allocation: one allocator, sixteen node counters, and why the client's ids are free
Lane ID, 2026-09-08. **Host work only — no VM run was needed and none was taken.** VM140 was held
and released untouched; its `SavedGames` set is byte-identical to the oracle (§8).
Closes row 2 of `turn-command-replay.md` §4 — "the client's id allocator" — which lane RB flagged as
"a watchpoint, not a week of reading". It turned out to be neither: it is arithmetic on ids the
corpus already held, confirmed against the instruction stream and cross-checked on twenty saves.
Predictions were written before any of the work below and are quoted verbatim where they were
falsified (§3.2). Consumes: `combat-retreat-pipeline.md` (B5, `IDMap::AllocateID`),
`turn-command-replay.md` (RB), `ai-order-emission.md` (AI4), `struct-recovery.md`.
---
## 0. Lead: there is no second allocator
RB's gap list says "**two id spaces**, and the small one is part of the wire protocol". There are
two id *spaces*, and RB is right that a reimplementation which allocates on apply gets every
AI-created id wrong. But there is **one allocator, one implementation, and one formula** — B5's
`IDMap::AllocateID` 0x008b8ae0, `id = (counter << 4) | (node & 0xF)`.
What differs is **which node's counter** is used, and that is a property of the *sim object*, not of
the code:
> `Game::StrategyServer` and `Game::StrategyClient` are **both** `Game::StrategySim`
> (RTTI: `StrategyClient` bases `[StrategyClient, StrategySim]` at offset 0; `StrategyServer` bases
> `[StrategyServer, IStreamable, StrategySim]` with vftables at offset 0 **and** offset 4 — that is
> the "two bases 4 bytes apart" of earned rule 1, and the +4 one is the `StrategySim` subobject).
>
> `StrategySim` owns an `IDMap` at **`StrategySim + 0x80`** — i.e. `StrategyServer + 0x84`,
> `StrategyClient + 0x80`. Every sim allocates from **its own** map, on **its own** local node
> index at `IDMap + 0x18`.
So the object-creation code is base-class code that runs unchanged on both sides. The whole
client-allocates/server-honours protocol is one branch in `StrategySim::CreateDesign` 0x008827e0:
```
008828b3 mov eax,[ebp+0x10] ; explicitId, the third argument
008828b6 test eax,eax
008828b8 jne 0x8828c5 ; non-zero -> use the id the command carried
008828ba lea ecx,[edi+0x80] ; else: this sim's own IDMap
008828c0 call 0x8b8b70 ; IDMap::AllocateOnLocalNode
008828c5 ... call 0x8b9350 ; IDMap::Insert(map, design+0xa0, id)
```
The client calls it with `explicitId = 0` and mints; the server calls it with the id off the wire
and mints nothing.
## 1. The arithmetic, which is the whole result
B5's formula, run backwards on RB's numbers:
| id | | counter | node |
|---:|---|---:|---:|
| **18** design | *client-allocated* | **1** | **2** |
| **34** fleet | *client-allocated* | **2** | **2** |
| 1712 | from the master counter | 107 | 0 |
| 1728 | ship in the input save | 108 | 0 |
| 1776 | | 111 | 0 |
| 272, 288 systems; 32, 496, 512 players | | 17, 18, 2, 31, 32 | 0 |
Every id that comes out of a save carries node nibble **0**. Both client-allocated ids carry node
nibble **2**, with counters **1** and **2** — the first two ids that node ever issued.
### 1.1 On twenty saves, four games, three distinct client nodes
Nibble histogram over `FltID`/`DesID`/`ShipID`/`SysID`/`PlayerID` in the whole corpus
(`verify/results/saves/*.sav`, read with `verify/save-reader/save_reader.py --dump`):
| save | ids | nibbles | the non-zero ones |
|---|---:|---|---|
| `turn1-state` | 98 | `{0: 98}` | — |
| `turn2-state` | 102 | `{0: 101, 2: 1}` | 18 |
| `turn3-state` | 104 | `{0: 102, 2: 2}` | 18, 34 |
| `cb-turn2to3-autosave` | 104 | `{0: 102, 2: 2}` | 18, 34 |
| `human-turn3-noderoute` | 90 | `{0: 86, 2: 4}` | 18, 34, 50, 66 |
| `human-turn8-traderoutes` | 151 | `{0: 141, 2: 10}` | 18 … 242 |
| `human-turn15-spyprogram` | 295 | `{0: 252, **1: 1**, 2: 42}` | **17**, 18 … 818 |
| `zuul-turn15-orders` | 150 | `{0: 139, 2: 10, **3: 1**}` | 18 … 162, **19** |
| `zuul-turn17-orders2` | 167 | `{0: 151, 2: 14, **3: 2**}` | 18 … 226, **19, 35** |
| `zuul-turn23-fleet23` | 222 | `{0: 198, 2: 22, 3: 2}` | … |
Three facts fall straight out, and none of them needed a hook.
1. **Nodes 1, 2 and 3 all occur.** "The client" is not singular. Each node's counters run
**1, 2, 3, …** contiguously from the start of the game — node 2's first id is `18` in *every*
save in the corpus, node 3's first is `19`, node 1's first is `17`.
2. **The partition is absolute.** ~2,600 ids across four games, zero collisions between spaces.
That is the entire purpose of the nibble and of `IDMap::Initialize`'s `numNodes <= 0x10` cap.
3. **Node = owning player's index + 1.** Every node-nibble object belongs to exactly one player:
| save | id | owner `PID` | that player's `PlyrIdx` | node |
|---|---:|---:|---:|---:|
| `turn3-state` | design 18, fleet 34 | 32 | 1 | 2 |
| `human-turn15` | design 17 | 16 | 0 | 1 |
| `human-turn15` | design 18 | 32 | 1 | 2 |
| `zuul-turn17` | designs 18, 34, … | 32 | 1 | 2 |
| `zuul-turn17` | designs 19, 35 | 496 | 2 | 3 |
## 2. The node index, read from the instruction stream
`StrategyServer::InitGameForPlayer` 0x007c8d90 builds the CreateGame message for one player's
client. Four instructions do the assignment:
```
007c8ddf mov esi,[eax+0x28] ; player->PlyrIdx
...
007c8ecb cmp esi,0xffffffff
007c8ece je 0x7c8ed3
007c8ed0 inc esi ; localNode = PlyrIdx + 1
007c8ed1 jmp 0x7c8ed5
007c8ed3 xor esi,esi ; PlyrIdx == -1 -> localNode = 0
007c8ed5 ... [ebx+0x90] - [ebx+0x8c], /20 ; numNodes = the server map's node count
007c8ef2 mov [ebp-0x134],ecx ; msg->numNodes
007c8ef8 push esi
007c8ef9 lea ecx,[ebx+0x84] ; the SERVER's IDMap
007c8eff mov [ebp-0x130],esi ; msg->localNode
007c8f05 call 0x8b8b80 ; IDMap::GetNodeCounter(server map, localNode)
007c8f1a mov [ebp-0x12c],eax ; msg->startId
```
The message is `{numNodes, localNode = PlyrIdx + 1, startId = the server's counter for that node}`.
The client receives it as **case 0** of `StrategyClient::RaiseEvent` 0x00783ee0 (jump table
0x00784200, handler 0x00783f05) and `StrategySim::OnCreateGame` 0x00776f20 calls
`IDMap::Initialize` on its own map at `+0x80` with exactly those three words.
So `PlyrIdx + 1` is not an inference from the saves any more. The saves and the disassembly agree,
which is the only reason either is worth reporting.
## 3. Seeding: what a save carries, and what it cannot
`IDMap::Initialize(numNodes, localNode, startId)` 0x008b9ad0, `ret 0xc`:
* refuses `numNodes > 0x10` — **sixteen nodes, and that is the four-bit nibble**;
* refuses `localNode` outside `[0, numNodes)`;
* resizes the 0x14-stride node vector, then **zeroes every node's counter** (0x008b9b20);
* writes `startId` into **`nodes[localNode].counter` alone** (0x008b9b38);
* stores `localNode` at `this->+0x18` (0x008b9b3d).
### 3.1 The save's `NM*` tags are now named
`StrategyServer::Read` 0x007d27a0 reads three ints and passes them straight in:
```
NMSz -> numNodes NMLc -> localNode NMnx -> startId
```
That **confirms and extends B5's labelled hypothesis** ("the counter is almost certainly NMnx").
It is NMnx, it is the counter *for node NMLc only*, and the two tags beside it are the node count
and the local node index. Corpus:
| save | NMSz | NMLc | NMnx | ModCount | Frame |
|---|---:|---:|---:|---:|---:|
| `turn1-state` | 16 | 0 | 106 | 0 | 1 |
| `turn2-state` | 16 | 0 | 109 | 12 | 2 |
| `turn3-state` | 16 | 0 | **111** | 24 | 3 |
`StrategyServer::LoadGame` 0x007dd530 sets a fresh game up with `Initialize(16, 0, 0)`, which is
where `NMSz 16` / `NMLc 0` come from and why the host's ids all carry nibble 0.
**A correction to `turn-command-replay.md` §4 row 2** (earned rule 11). It says the server "issued
1712 and 1776 the same turn", and that design 18 and fleet 34 are "objects that do not exist in the
input save". Neither is right on the canonical pair, and the master id lists say so exactly.
`turn2-state → turn3-state` moves NMnx **109 → 111**, so the server issued counters **110 and 111**
and nothing else. Diffing the four master lists in the `Sim` block gives a **zero-residual account
of every object created on the reference turn** — the id-level counterpart of RB's `ModCount = 24`:
| id | counter | node | list | what |
|---:|---:|---:|---|---|
| **1760** | 110 | 0 | `ShipID` 16 → 17 | the ship the build order built |
| **1776** | 111 | 0 | `FleetIDs` 7 → 8 | a server-created fleet |
| **34** | 2 | **2** | `FleetIDs` 7 → 8 | **the client-created fleet** |
| — | — | — | `DesignIDs` **43 → 43** | **no design was created that turn** |
| — | — | — | `PlayerIDs` 8, `SystemIDs` 28 | unchanged |
Three objects, three ids, and the counters on both nodes are contiguous with no gaps. So:
* 1712 is counter **107** and was issued during turn **1→2**, not 2→3 (NMnx 106 → 109 covers
counters 107, 108, 109 = ids 1712, 1728, 1744). 1776 is a **fleet**, not a design.
* **Design 18 is already in `turn2-state.sav`** — in the master `DesignIDs` list (43 entries, and
still 43 at turn 3) and in player 32's block. It was allocated during turn 1→2. The turn-2→3
build order therefore names a design the input save already holds, and **only fleet 34 is new**.
A faithful replay of the canonical pair mints exactly **one** client id, not two. That is a
materially smaller gap than the row claims.
Everything else in the row stands, including the load-bearing part: allocate on apply and you get
`1792` where the original has `34`.
### 3.2 A falsified prediction, and the open item it leaves
Prediction ID-P3 said, in full:
> If NMLc = 0 holds, then the node-2 counter that issued 18 and 34 is NOT in the save, is NOT
> restored by Initialize (which zeroes it), and therefore **restarts at zero on every load**.
NMLc is 0, and the first two clauses are read directly out of the code: no save carries a per-node
array, `Initialize` zeroes the rest, `IDMap::Insert` 0x008b9350 **never touches a counter** (read to
the end: it logs a duplicate, refuses id 0, bounds-checks `id & 0xF`, writes the id into `obj->+4`
and inserts into `nodes[n]`'s map — no counter write), and `InitGameForPlayer` seeds a client from
`GetNodeCounter`, which therefore returns 0 for every client node in a freshly loaded game.
**The third clause is falsified by a save already in the corpus.** `cb-turn2to3-autosave.sav` is a
fresh process that *loaded* `turn2-state.sav` (which contains design 18 = node 2, counter 1) and
pressed End Turn. The fleet it created is **34 — counter 2**, not 18. If the client's counter had
started at 0 the fleet would have been 18, colliding with a design that already exists.
Two explanations survive, and this lane cannot separate them without a hook:
* **(A)** something restores the server's node-2 counter to 1 during the load, and the client is
seeded with it. I read `Initialize`, `Insert`, `AllocateID`, `FindNode`, `GetNodeCounter` and
`InitGameForPlayer` and found no such write. Absence in six functions is not absence.
* **(B)** the counter really does start at 0 and the client made **one earlier allocation that turn
which never reached the save** — a local design object, a task, an order — taking counter 1, so
the fleet got counter 2. This fits the read code exactly and requires nothing unread.
**(B) has a specific form that fits everything, including the master-list diff above.** In the
*continuous* reference session (turn 1 → 3 in one process, which is where `turn2-state` and
`turn3-state` come from) the client allocated 18 for its design in turn 1→2 and 34 for its fleet in
turn 2→3 — counters 1 then 2, no gap, nothing extra needed. In CB's *reloaded* run the client's
counter started at 0 again, so its first allocation of the turn would be counter 1 = **18**, an id
the loaded board already holds; `IDMap::Insert` logs and inserts anyway; the fleet then takes
counter 2 = 34. On that reading **the collision has already happened, in a run whose autosave
matched the oracle** — which is exactly what you would expect, because the colliding object lives in
the *client's* map and only the server's state is serialized. It also predicts what the earlier
allocation was: whatever the AI client makes each turn before it makes a fleet.
That is a story, not a measurement. **The probe is one hook and it is cheap** (earned rule 18, and rule 20 — instrument the *entry*):
detour `IDMap::AllocateID` 0x008b8ae0 and log `(this, nodeIndex, resulting id, return address)` for
one `turn2-state → turn3-state` End Turn. (A) predicts the client's very first allocation that turn
returns 34; (B) predicts it returns 18 from a site that is not the fleet creator. **A count alone
cannot separate them** — that is exactly the shape rule 20 exists for.
Whichever it is, the consequence for a reimplementation is the same and is stated in §5.
### 3.3 The collision hazard, stated as a hypothesis
If (B) is right, then a client's counter is a pure function of **allocation order within the
process** and a save/load loses it. Load a save in which a client has already allocated *n* objects
and let it allocate again, and it re-issues `(1<<4)|k` — an id the map already holds. `IDMap::Insert`
logs `IDMap: Object already exists with id %d.` and **inserts anyway** (0x008b9367: the log is
followed by fall-through, not a return). That would be an original defect, not ours.
Labelled hypothesis. The same one-hook probe settles it, and `human-turn15-spyprogram.sav` — node 2
at counter 51 — is the workload that would make it loud.
## 4. Layout, for the record
```
IDMap (StrategySim + 0x80)
+0x08 / +0x0c / +0x10 vector<NodeEntry>, STRIDE 0x14 (allocator +0x10, rule 5)
+0x18 int localNodeIndex
NodeEntry 0x14 bytes
+0x00 .. +0x0f std::map<int, void*> id -> object, for THIS node
+0x10 int counter pre-incremented; 0 is never issued
```
`AllocateID(node)`: 0 if `node == -1` or the node does not exist; else `++nodes[node].counter`
(pre-increment, and on wrap to 0 it logs and increments again, so `INVALID_NETWORK_ID` is never
handed out); `id = (counter << 4) | (node & 0xF)`, with an overflow log if the counter no longer
round-trips.
## 5. What a faithful replay has to do
1. **A per-node counter array, not one counter.** Sixteen of them.
2. **Node 0 for the sim that owns the board; node `PlyrIdx + 1` for each player's client.**
3. **Restore only node `NMLc`'s counter from `NMnx`.** There is nothing else in the save to
restore, and inventing a per-node array in the save format would diverge from the original.
4. **Mint the id at command *emission*, not at apply**, and carry it in the command — the applier
must honour a non-zero id and mint only on zero (`StrategySim::CreateDesign`'s branch).
5. **Do not model the client counter as a function of the save.** It is a function of allocation
order in the process, and until §3.2 is settled a replay of the canonical pair should take the
client's ids from the capture rather than derive them.
For the canonical pair specifically, (4) plus "design 18 is already in the input save" means the
replay needs exactly one minted id, `34`, and it is the *second* thing node 2 allocates that turn.
## 6. What this lane did NOT do
1. **No hook, no VM run, no new instrument.** Every number is from the checked-in corpus and the
image. That is also the limit: §3.2 is open precisely because nothing ran.
2. **The second `LoadTechFile` loop and the message *send* were not read.** I read
`InitGameForPlayer` up to the point where the message is filled, and `OnCreateGame` from the
point where it is received; the transport between them is assumed, not read. It is a single
process, so nothing observable depends on it — but it is not read.
3. **`FUN_008a9030`'s counter at `+0x23c` is a different mechanism** (a Mars scene object with its
own `IDMap` at `+0x68`, `Initialize(1, 0, 0)`, and a separate per-object counter gated on
`obj->+0x20 & 0xF == 2`). It is not game state and it is not this. Recorded so the next lane
that greps for `& 0xF` does not chase it.
4. **Multiplayer is untested.** Every save in the corpus is `NMLc 0`. A real client machine would
save with `NMLc = k` and its own counter, and *the server's* copy of that counter is the thing
§3.2 is about. Nothing here has been exercised across two machines.
5. **`ShipID` allocation was not traced to its site.** Ships in the corpus are all node 0, so they
are minted server-side; I did not read which function does it.
## 7. Reproducing every number here
```sh
cd ~/sots-re
uv run python3 verify/save-reader/save_reader.py --dump verify/results/saves/turn3-state.sav \
| grep -E 'NMSz|NMLc|NMnx'
# nibble histogram: see the one-screen script in section 1.1's method note below
uv run python3 dumps/b6dis.py 0x008b9ad0 # IDMap::Initialize
uv run python3 dumps/b6dis.py 0x007c8d90 # InitGameForPlayer; the +1 is at 0x007c8ed0
```
The histogram is `re.match(r'\s*@[0-9a-f]+\s+(\S+)\s+int\??\s+(-?\d+)', line)` over the dump,
keeping the five id tags and counting `value & 15`. No tool was added for it — it is four lines and
a tool would only hide what it does.
## 8. VM140 released
The game was never launched and nothing was written. `C:\SOTS\SavedGames` verified over SSH before
release — **8 files**, and the three oracle hashes match:
| file | bytes | sha256[0:16] | |
|---|---:|---|---|
| `(Autosave).sav` | 67,219 | `978041acd168b56e` | ✅ oracle |
| `(Autosave EndTurn).sav` | 66,732 | `bb4fd9ac89f41e3b` | ✅ oracle |
| `(Autosave Backup).sav` | 67,219 | `978041acd168b56e` | ✅ oracle |
| `MyGameverify1rtD.sav` | 66,732 | `bb4fd9ac89f41e3b` | |
| `MyGameverify1rtE.sav` | 66,732 | `bb4fd9ac89f41e3b` | |
| `MyGameverify1verify1.sav` | 66,732 | `bb4fd9ac89f41e3b` | |
| `ref-turn2.sav` | 66,739 | `ab4ac2d7e2977260` | |
| `zuul-turn5.sav` | 59,131 | `48559ab5b719b332` | |
Note for the next VM lane: a filename with parentheses cannot be hashed through
`ssh … 'certutil -hashfile "…(Autosave).sav"'` — `cmd.exe` eats the parens and returns
"Check the spelling", which reads exactly like a missing file. Use
`powershell -NoProfile -c "(Get-FileHash -LiteralPath '…' -Algorithm SHA256).Hash"`. My first
attempt reported the three oracle files as absent and they were never absent.
**Please mark VM140 free.**

146
findings/subsystems/techid-name-map.md Normal file
View file

@ -0,0 +1,146 @@
# `techId` -> tech name: an offline lookup, not a capture to transcribe
Lane ID, 2026-09-08. Host work only.
Closes row 8 of `turn-command-replay.md` §4 and the open value that kept Rung C-set's "we can name
all *k*" from closing.
> **282 is `XNC_TrnsHum2`.**
And it was **already in our own notes**: lane L4 observed it live in run R2
(`ai-order-capture.md`, the two-capture table — "player 512 target: `XNC_TrnsHum2`, techId 282").
Nobody connected it, and lane CB then produced 282 as an unmapped value. Earned rule 18's corollary,
paid for again: **search the notes before the binary.**
---
## 1. The rule
> **`techId` is the 0-based index of the tech's name in the master tech list sorted by
> `_stricmp` — case-insensitively.**
293 nodes in stock `TechTree/MasterTechList.tech`, so `techId` runs 0..292. That is a complete,
offline, data-derived mapping: **no capture, no transcription, no VM.** The replayer can stop
carrying the observed name in its own column.
Every observed point, including the two the rule was not fitted to:
| techId | name | how it was observed |
|---:|---|---|
| 90 | `DRV_PlsFiss` | RB / L4 capture, player 496 |
| 144 | `IND_Waldo` | RB / L4 capture, player 32 |
| **282** | **`XNC_TrnsHum2`** | L4 `ai-order-capture.md` run R2, player 512 — **independent of the fit** |
| 288 | `XNC_TrnsMorr2` | RB / L4 capture, player 512 |
The rule was fitted on the three RB reported and then hit 282 on the nose against a live
observation taken days earlier by a different lane on a different run. It also reproduces L4's
structural claim about the family: the six-member `XNC_Trns<Species>2` set is exactly
**{282 Hum2, 284 Hvr2, 286 Lir2, 288 Morr2, 290 Trk2, 292 Zuul2}**, interleaved with the `…3`
variants — which is why L4 saw two picks "six apart".
## 2. Read from the instruction stream, not fitted
`MasterTechTree::MasterTechTree` 0x0058b870, immediately after the `LoadTechFile` loop:
```
0058b9df lea eax,[ebx+0x14] ; the PARSE-ORDER list
0058b9e3 lea ecx,[ebx+0x24]
0058b9e6 call 0x8b3210 ; vector::operator= -- +0x24 is a COPY of +0x14
0058b9eb..0058b9ff
push pred / push count=(last-first)/4 / push last / push first
call 0x583b10 ; MSVC std::sort::_Sort
0058ba19 xor eax,eax
0058ba23 mov edx,[ebx+0x24]
0058ba26 mov ecx,[edx+eax*4] ; sortedList[i]
0058ba29 mov [ecx],eax ; **def->techId = i**
0058ba31 inc eax
0058ba37 jl 0x58ba23
```
Three things this settles that the fit could not.
**It is a sorted COPY.** `+0x14` keeps parse order; `+0x24` is the sorted one and the id is its
index. That is why the parse-order pointer-table dump in `strategic-turn-internals.md`
(`turn3/techtable.txt`, which starts `IND_Waldo, IND_RefCoat, …`) does not match the ids, and why
RB correctly concluded the ids "are not indices into anything we hold" — we held the wrong list.
**The comparator is `_stricmp`, so the order is case-insensitive.** It is inlined into
`_Unguarded_partition` FUN_00581130 at 0x00581190 and 0x005811b8 as
`_stricmp(a->name, b->name) < 0`, reading a `std::string` at `TechDef+0x04` (SSO-tested at `+0x14`
against `0x10`); the import at `0x009dd328` resolves to **`MSVCR100.dll _stricmp`**.
This matters. **48 of the 293 ids differ between byte-wise ASCII order and case-insensitive order**
— the `CCC_AI*`/`CCC_Adv*` run at 21–28, `DRN_COL`/`DRN_Cmbt`, `DRV_NodFoc`/`DRV_Node`,
`DRV_ROOT`/`DRV_Rad…`, `IND_DSCon`/`IND_Decon`, `IND_ROOT`/`IND_RefCoat`, `WEP_APRtech`/`WEP_Ac…`,
`WEP_KKMsl`/`WEP_KelTrp`, `WEP_MWMsl`/`WEP_Mas…`, `WEP_Nukes`/`WEP_NukMine`, `WEP_Pls*`. **None of
the four observed points discriminates**, so this half is disassembly-only and the corpus cannot
confirm it. Flagged as such (earned rule 6): if a future capture ever reports a techId in one of
those 48 slots, it is a real test and it should be reported as one. No two names collide under
case-folding (checked over all 293), so `std::sort`'s instability cannot bite.
**Why sorted at all.** The same `+0x24` vector is the binary-search index: `FUN_0057f120`
(`std::lower_bound`) resolves every `requires` / `allows` name reference against it at 0x0058baed.
The id is the index into a lookup table that had to be sorted anyway.
## 3. The two tech id spaces, which is what made this look hard
| | space | base | size | where |
|---|---|---:|---:|---|
| **wire / command** | index into the `_stricmp`-sorted master list | 0 | 293 | `TechDef+0x00`; the research-target gate payload |
| **`TechID` enum** | index into `MasterTechTree->resolved[]` | **10000** | **196** | a `{name, id}` table in `.rdata`, `CCC_AdvSens = 10001` |
`MasterTechTree::GetTechDef` 0x0057d610 and `IsTech` 0x0057d5d0 take the **enum** id:
`if (id == 0xc5) return 0; id -= 10000; if ((unsigned)id > 0xc3) return 0; return resolved[id];`
(`0xc5` = 197 is the "no tech" sentinel.)
`TechTree::HasResearched` 0x0057d810 is the bridge and shows both in four instructions:
```
eax = enumId - 10000; eax = master->resolved[eax]; ; ENUM space -> TechDef*
eax = *eax; ; TechDef+0x00 = the WIRE techId
eax = this->+0x10[eax]; ; per-player node vector, indexed by WIRE id
return eax && eax->+0x14 == 4;
```
The enum space exists so that C++ code can name a specific tech; the wire space is a dense index
over *all* techs. `TechTree::IsResearchable` 0x0057e820 — called by the research-target gate applier
at 0x0088ff49 with the payload word straight off the command — bounds-checks against the length of
the 293-entry vector, which is what proves the payload is in the wire space.
## 4. The table, and how to regenerate it
`tools/techid_table.py` derives the whole mapping from the data file (or from the checked-in parse
oracle) and self-checks the four observed points:
```sh
uv run python3 tools/techid_table.py # prints all 293
uv run python3 tools/techid_table.py 282 288 144 90 # or just some
```
It is deliberately short: the rule is one `sorted(key=str.lower)` and a tool that obscured that
would be worse than no tool. It reads `$SOTS_GOB_DIR/TechTree/MasterTechList.tech` if that is set
(the same variable the engine's `realdata_test` uses) and otherwise the checked-in parse oracle, so
it needs no data tree. **It refuses to print a table whose four observed points do not agree** —
a short parse shifts every id after the gap, and the observed points are the only check it has.
Honest bound: the raw-`.tech` regex path is exercised here only against a synthetic three-block
fixture (both `tech {` and `tech` + newline + `{`); the numbers in this note all come through the
parse-oracle path.
## 5. What this lane did NOT do
1. **No live confirmation of its own.** The 282 point is lane L4's observation, re-read. Nothing new
ran. The strongest available further test is free: any future capture reporting a techId in the
48 sort-ambiguous slots.
2. **Only stock data.** A mod adding a tech file renumbers **every id after the insertion point**,
because the id is a sorted index over the union. That is a property of the original and a real
hazard for anyone replaying a modded game; untested here.
3. **The second `LoadTechFile` loop in the ctor (0x0058ba7e) was not read.** If it can add nodes
*after* 0x0058ba37, ids would be assigned before those nodes exist. Stock data has exactly one
tech file (`data-model.md`) and the 293-entry fit holds, so it does not bite here — but it is
unread, and it is the one way the rule could be incomplete.
4. **The 196-entry `.rdata` enum table was not dumped in full.** It is at the address behind
`MasterTechTree::ResolveTechIds` 0x00581c10; `scratchpad/techfx/techtable196.txt` has an old
dump of it (`CCC_AdvSens = 10001`, `IND_Waldo = 10002`, …). Note that
`nvo-tshn-visible-owner.md` calls `CCC_AdvSens` "tech id 10000"; the table says 10001. One of
the two is off by one and this lane did not resolve which.

92
ghidra/addresses.d/lane-id.json Normal file
View file

@ -0,0 +1,92 @@
{
"entries": [
{
"name": "IDMap_Initialize",
"addr": "0x008b9ad0",
"convention": "thiscall",
"prototype": "bool (IDMap* this, int numNodes, int localNode, int startId) // ret 0xc. THE SEEDING ENTRY POINT, and it names the save's three NM* tags. Calls FUN_008b8f80 (clear), then REFUSES numNodes > 0x10 ('IDMap: Cannot support %d nodes.') -- SIXTEEN is the hard cap, and it is the low nibble of IDMap_AllocateID's id. Refuses localNode outside [0, numNodes) ('IDMap: Node %d does not exist, valid nodes are 0-%d.'). On success: resize the 0x14-stride node vector at this->+0x08 via FUN_008b99d0, ZERO EVERY NODE'S COUNTER (the loop at 0x008b9b20 stores 0 to +eax+0x10, eax += 0x14), then write startId into nodes[localNode].counter ALONE (0x008b9b38), then this->+0x18 = localNode (0x008b9b3d). SO: exactly one node's counter is ever seeded; every other node starts at 0, and nothing else in the class ever restores one -- IDMap_Insert 0x008b9350 does NOT touch a counter. Four call sites: StrategyServer_Read +0x105 = Initialize(NMSz, NMLc, NMnx) on S+0x84 -- WHICH NAMES THE TAGS: NMSz = node count, NMLc = LOCAL NODE INDEX, NMnx = that node's counter; StrategyServer_LoadGame +0x107 = Initialize(16, 0, 0); StrategySim_OnCreateGame +0x45 = Initialize(msg->+4, msg->+8, msg->+0xc) on the CLIENT's map at +0x80; and FUN_008a9f80 +0x1ce = Initialize(1, 0, 0) on an unrelated Mars scene object's map at +0x68",
"status": "verified",
"source": "findings/subsystems/id-allocation.md"
},
{
"name": "IDMap_AllocateOnLocalNode",
"addr": "0x008b8b70",
"convention": "thiscall",
"prototype": "int (IDMap* this) // 10 B: push [ecx+0x18] (the local node index); call IDMap_AllocateID; ret. The IDMap-relative form of the same thunk IDMap_AllocateLocalID 0x0080f710 wraps from the StrategySim side (that one is `ecx += 0x80` then jmp here). Five direct call sites, all StrategySim methods reached on BOTH the server and a client: StrategySim_CreateDesign +0xe0, FUN_008713a0 +0x3df, SystemBuildQueue_AttachBuiltShip 0x0088e7f0 +0x42, TradeManager_SpawnEncounterSquadron 0x0088f070 +0x216 and +0x28f",
"status": "verified",
"source": "findings/subsystems/id-allocation.md"
},
{
"name": "IDMap_GetNodeCounter",
"addr": "0x008b8b80",
"convention": "thiscall",
"prototype": "int (IDMap* this, int nodeIndex) // 29 B, ret 4: node = IDMap_FindNode(nodeIndex); return node ? node->+0x10 : 0. A READ of a per-node counter with no side effect. Its only interesting caller is StrategyServer_InitGameForPlayer 0x007c8f05, which uses it to SEED a client: the CreateGame message carries the server's current counter for the node that client is about to own. Since a save restores only ONE node's counter (Initialize zeroes the rest), this returns 0 for every client node in a freshly loaded game",
"status": "verified",
"source": "findings/subsystems/id-allocation.md"
},
{
"name": "IDMap_FindNode",
"addr": "0x008b8a70",
"convention": "thiscall",
"prototype": "void* (IDMap* this, int nodeIndex) // 101 B, ret 4. Bounds-checks nodeIndex against (this->+0x0c - this->+0x08)/0x14 -- the 0x66666667 / sar 3 divide-by-20 that pins the NodeEntry stride at 0x14 -- logs 'IDMap: Node %d does not exist, valid nodes are 0-%d.' and returns 0 when out of range; else returns _Myfirst + nodeIndex*0x14. Shared by IDMap_AllocateID and IDMap_GetNodeCounter",
"status": "verified",
"source": "findings/subsystems/id-allocation.md"
},
{
"name": "StrategySim_OnCreateGame",
"addr": "0x00776f20",
"convention": "thiscall",
"prototype": "void (StrategySim* this, CreateGameMsg* msg) // ret 4. CASE 0 of StrategyClient::RaiseEvent 0x00783ee0's 0x2c-entry jump table at 0x00784200 (the handler body is at 0x00783f05). FIRST ACT: IDMap_Initialize(this+0x80, msg->+0x04 numNodes, msg->+0x08 localNode, msg->+0x0c startId) -- THIS IS WHERE A CLIENT'S ID SPACE IS SET UP, and the only path by which an IDMap ever gets a local node index other than 0. Then copies the rest of the message into the sim: +0x10 -> this->+0x08, +0x48 -> this->+0x10, +0x14 -> this->+0xb8, the two floats at +0x18/+0x1c -> this->+0xbc/+0xc0, the bools at +0x20/+0x21 -> this->+0xc4/+0xc5, +0x6c -> this->+0x154, +0x4c -> this->+0xf8, then the vectors from this+0x40 on",
"status": "verified",
"source": "findings/subsystems/id-allocation.md"
},
{
"name": "StrategyServer_InitGameForPlayer",
"addr": "0x007c8d90",
"convention": "thiscall",
"prototype": "void (StrategyServer* this /*the S frame*/, int playerObjectId, int arg2) // THE NODE-INDEX ASSIGNMENT, in four instructions. Resolves the player through this->+0x84 (the id-to-object lookup FUN_008b9240) and takes esi = player->PlyrIdx(+0x28). Then at 0x007c8ecb: `cmp esi,-1 / je L / inc esi / jmp / L: xor esi,esi` -- localNode = (PlyrIdx == -1) ? 0 : PlyrIdx + 1. NODE 0 IS THE SERVER'S; PLAYER k GETS NODE k+1. It then fills the CreateGame message on the stack at [ebp-0x138]: +0x00 vtable 0x00a252c0, +0x04 numNodes = (server IDMap node vector length, the /20 divide at 0x007c8ee1 over ebx+0x8c/+0x90), +0x08 localNode = that esi, +0x0c startId = IDMap_GetNodeCounter(S+0x84, localNode). Sends it, and StrategySim_OnCreateGame is what receives it. See findings/subsystems/id-allocation.md section 3 for why startId is always 0 for a client after a save load",
"status": "verified",
"source": "findings/subsystems/id-allocation.md"
},
{
"name": "StrategyServer_LoadGame",
"addr": "0x007dd530",
"convention": "thiscall",
"prototype": "void (StrategyServer* this, ...) // At +0x107 it calls IDMap_Initialize(this+0x84, 16, 0, 0) -- SIXTEEN NODES, LOCAL NODE 0, COUNTER 0. That is where the corpus's NMSz 16 / NMLc 0 come from, and it is why every id created by the host carries low nibble 0",
"status": "verified",
"source": "findings/subsystems/id-allocation.md"
},
{
"name": "StrategySim_CreateDesign",
"addr": "0x008827e0",
"convention": "thiscall",
"prototype": "void (StrategySim* this, ServerPlayer* owner, void* params, int explicitId, char, char, char, char) // ret 0x18. THE CLIENT-ALLOCATES / SERVER-HONOURS SPLIT, in one branch. operator new(0x1a8) -> ctor FUN_00874c70 -> FUN_0057c6d0(params); design->+0x130 = owner, design->+0x134 = this->+0x08. Then at 0x008828b3: `if (explicitId != 0) use it; else id = IDMap_AllocateOnLocalNode(this+0x80)`, followed by IDMap_Insert(this+0x80, design+0xa0, id). BECAUSE StrategyClient AND StrategyServer BOTH DERIVE FROM StrategySim, this is the SAME code on both sides: the client runs it with explicitId 0 and mints an id on ITS node, puts that id in the turn command, and the server runs it again with explicitId set and mints nothing. A reimplementation that allocates on apply produces every AI-created id wrong",
"status": "verified",
"source": "findings/subsystems/id-allocation.md"
},
{
"name": "MasterTechTree_SortAndNumber",
"addr": "0x0058b9df",
"convention": "label",
"status": "verified",
"prototype": "THE techId -> NAME MAP, in eight instructions inside MasterTechTree_ctor 0x0058b870, immediately after the LoadTechFile loop. 0x0058b9df: vector::operator=(this+0x24, this+0x14) -- the sorted list at +0x24 is a COPY of the parse-order list at +0x14, which is why a parse-order dump does not match the ids. 0x0058b9ff: std::sort(first=[this+0x24], last=[this+0x28], ideal=(last-first)/4, pred) = FUN_00583b10, MSVC _Sort (the _ISORT_MAX 0x20 test and the _Ideal 3/4 halving are both there); the predicate is INLINED in FUN_00581130 at 0x00581190 and 0x005811b8 as `_stricmp(a->name, b->name) < 0` on TechDef+0x04 (a std::string, SSO-tested at +0x14 against 0x10) -- MSVCR100 _stricmp, so the order is CASE-INSENSITIVE, not byte-wise. 0x0058ba19..0x0058ba37: `for (i = 0; i < n; ++i) sortedList[i]->+0x00 = i;` -- THE WIRE techId IS LITERALLY THE 0-BASED INDEX INTO THE _stricmp-SORTED MASTER LIST. The same sorted vector is then the binary-search index: FUN_0057f120 (std::lower_bound) resolves every `requires`/`allows` name against it at 0x0058baed. NOT the 10000-based TechID enum, which is a separate 196-entry .rdata table",
"source": "findings/subsystems/techid-name-map.md"
},
{
"name": "TechTree_IsResearchable",
"addr": "0x0057e820",
"convention": "thiscall",
"prototype": "bool (TechTree* this /*per-player*/, int techId) // THE WIRE-SPACE BOUNDS CHECK, and it is what proves the space. Rejects techId < 0 and techId >= (this->+0x14 - this->+0x10)/4 -- the per-player node vector, 293 entries in stock data. Then esi = masterList[techId] from this->+0x04 -> +0x24/+0x28 (the SORTED vector, MasterTechTree_SortAndNumber) and edx = this->+0x10[techId] (the player's node). Reads the name as a std::string at masterEntry+0x04 with the SSO test at +0x18 -- so TechDef is {+0x00 int techId, +0x04 std::string name, ...}. Called by the research-target gate applier at 0x0088ff49 with the techId straight off the command payload ([block-0x28])",
"status": "verified",
"source": "findings/subsystems/techid-name-map.md"
},
{
"name": "MSVC_Sort",
"addr": "0x00583b10",
"convention": "cdecl",
"prototype": "void (T** first, T** last, int ideal, Pred pred) // MSVC std::sort's _Sort: the `(last-first)/4 <= 0x20` insertion-sort cutoff at 0x00583b26, the `ideal -= ideal/2 + ideal/4` heap-sort fallback counter, and _Unguarded_partition FUN_00581130. Identified here because MasterTechTree_ctor uses it to build the tech id space; the same body will be reached from anywhere else that sorts a pointer vector",
"status": "verified",
"source": "findings/subsystems/techid-name-map.md"
}
]
}

92
tools/techid_table.py Normal file
View file

@ -0,0 +1,92 @@
#!/usr/bin/env python3
"""techId -> tech name, derived offline from the tech tree data.
The wire techId is the 0-based index of the tech's name in the master tech list sorted with
_stricmp -- case-insensitively. Read from MasterTechTree's constructor: it copies the parse-order
list to a second vector, std::sort()s that copy with an inlined `_stricmp(a->name, b->name) < 0`,
then walks it writing `def->techId = i`. See findings/subsystems/techid-name-map.md.
This is NOT the 10000-based TechID enum, which is a separate 196-entry .rdata table.
Usage:
tools/techid_table.py # all of them
tools/techid_table.py 282 288 144 90 # just these
"""
import glob
import json
import os
import re
import sys
ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..")
# Four (techId, name) pairs observed live, by two lanes on different runs. 282 is the one the
# rule was NOT fitted to -- it comes from lane L4's run R2 (findings/subsystems/ai-order-capture.md)
# and is the check that this file is right rather than merely self-consistent.
OBSERVED = {90: "DRV_PlsFiss", 144: "IND_Waldo", 282: "XNC_TrnsHum2", 288: "XNC_TrnsMorr2"}
# Where the 293 names can come from, in order of preference: the raw game file under
# $SOTS_GOB_DIR (the same variable the engine's realdata_test uses), then the checked-in parse
# oracle in sots-engine's tests, which needs no data tree at all.
SOURCES = [p for p in [
os.path.join(os.environ["SOTS_GOB_DIR"], "TechTree", "MasterTechList.tech")
if os.environ.get("SOTS_GOB_DIR") else None,
os.path.join(ROOT, "..", "sots-engine", "tests", "mars_parse", "build", "oracle",
"TechTree", "MasterTechList.tech.json"),
] if p]
def names_from_tech_file(path):
"""Pull every `tech { name "X" ... }` block name out of the raw .tech file."""
text = open(path, encoding="latin-1").read()
return re.findall(r'^\s*tech\b[^\n]*\n(?:[^\n]*\n)*?\s*name\s+"([^"]+)"', text, re.M)
def names_from_oracle(path):
return [t["name"] for t in json.load(open(path))["tech"]]
def load_names():
for pat in SOURCES:
for path in sorted(glob.glob(pat)):
if os.path.exists(path):
return (names_from_oracle(path) if path.endswith(".json")
else names_from_tech_file(path)), path
for path in sorted(glob.glob(os.path.join(ROOT, "**", "MasterTechList.tech"), recursive=True)):
return names_from_tech_file(path), path
sys.exit("no MasterTechList.tech (or its parse oracle) found; see SOURCES in this file")
def table():
names, src = load_names()
if len(set(n.lower() for n in names)) != len(names):
sys.exit("case-folded duplicate tech names: the sorted order is not total, stop")
# _stricmp order. str.lower reproduces it exactly for these pure-ASCII names.
return sorted(names, key=str.lower), src
def main(argv):
ids, src = table()
bad = [(i, want, ids[i] if i < len(ids) else "<out of range>")
for i, want in OBSERVED.items() if i >= len(ids) or ids[i] != want]
if bad:
print(f"# {len(ids)} names parsed from {src}", file=sys.stderr)
for i, want, got in bad:
print(f"MISMATCH techId {i}: observed {want!r}, derived {got!r}", file=sys.stderr)
print("The observed points are the only check this file has. Do not trust the table "
"until they agree -- a short parse (a missed block) shifts EVERY id after it.",
file=sys.stderr)
return 1
print(f"# {len(ids)} techs from {os.path.relpath(src, ROOT)}; "
f"{len(OBSERVED)}/{len(OBSERVED)} observed points agree", file=sys.stderr)
wanted = [int(a) for a in argv] if argv else range(len(ids))
for i in wanted:
if 0 <= i < len(ids):
print(f"{i:4d} {ids[i]}{' <- observed' if i in OBSERVED else ''}")
else:
print(f"{i:4d} <out of range 0..{len(ids) - 1}>")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))