alex/sots-re
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

strings/config: battle-load smoking gun (affinity), lockstep sim, net map, Mods override

Browse source
This commit is contained in:
alex 2026-09-07 14:58:36 -04:00
parent ef8d557499
commit 994b182f7a
3 changed files with 160 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
campaign/board.md
View file

@ -19,7 +19,9 @@ Status flow: `backlog → in-progress → mapped → verified` (or `blocked`).
| D3D9 device init | subsystem | backlog | — | 0% | 2026-09-07 | `Mars::DrawDevice`/`Camera` → Direct3DCreate9/CreateDevice | | D3D9 device init | subsystem | backlog | — | 0% | 2026-09-07 | `Mars::DrawDevice`/`Camera` → Direct3DCreate9/CreateDevice |
| `Mars::Buffer::gobio` | subsystem | in-progress | — | 0% | 2026-09-07 | .gob I/O; anchors the verify/ parser | | `Mars::Buffer::gobio` | subsystem | in-progress | — | 0% | 2026-09-07 | .gob I/O; anchors the verify/ parser |
| serializable-types vs save structs | verify | in-progress | — | 0% | 2026-09-07 | cross-ref `serializable-types.txt` with community save editors | | serializable-types vs save structs | verify | in-progress | — | 0% | 2026-09-07 | cross-ref `serializable-types.txt` with community save editors |
| battle-load path | subsystem | in-progress | — | 0% | 2026-09-07 | primary bug (many-core slowdown); LAA already on | | battle-load path | subsystem | in-progress | — | 0% | 2026-09-07 | SMOKING GUN: ForceSingleCore -> SetProcessAffinityMask; Ghidra agent pinning the fn |
| class hierarchy + key vftables | meta | in-progress | — | — | 2026-09-07 | Ghidra recon: inheritance + vftable sizes for core classes | | class hierarchy + key vftables | meta | in-progress | — | — | 2026-09-07 | Ghidra recon: inheritance + vftable sizes for core classes |
| data-model (.gob data files) | subsystem | in-progress | — | — | 2026-09-07 | harvest tech/ship/weapon/race tables from archives | | data-model (.gob data files) | subsystem | mapped | high | 100% | 2026-09-07 | findings/subsystems/data-model.md - tech/weapons/sections/races/AI all data-driven |
| string / config intel | meta | in-progress | — | — | 2026-09-07 | ini keys, paths, scripting, net msgs, battle-load strings | | string / config intel | meta | mapped | high | 100% | 2026-09-07 | findings/subsystems/strings-and-config.md |
| Mars brace-block parser | subsystem | backlog | — | 0% | 2026-09-07 | one recursive KV parser; reimpl lever + feeds verify/parsers |
| networking (SNM/FNM + GameSpy) | subsystem | backlog | — | 0% | 2026-09-07 | ~60 SNM strategy msgs, FNM file-xfer/host-migration; lockstep sim |

2
campaign/open-questions.md
View file

@ -10,3 +10,5 @@ Each links to the finding that raised it. Promoted to backlog or closed by **re-
(from [[00-strategy]]) (from [[00-strategy]])
- **Save-editor struct coverage** — community editors cover players/tech/systems; fleet/combat - **Save-editor struct coverage** — community editors cover players/tech/systems; fleet/combat
state coverage may be partial. Treat as strong hints, verify each field against real saves. state coverage may be partial. Treat as strong hints, verify each field against real saves.
- **Lockstep determinism** — `@SYNC ERROR DETECTED: Frame %d` indicates a deterministic lockstep sim. Big for reimplementation + old-vs-new differential (determinism makes equivalence checkable frame-by-frame). Confirm scope. (from [[strings-and-config]])
- **Mods override mechanism** — `.gob` = `gobio::ZipFileSystem` with a `NativeFileSystem` loose-file override; this is how `./Mods` wins over archived files. Verify load order. (from [[strings-and-config]])

153
findings/subsystems/strings-and-config.md Normal file
View file

@ -0,0 +1,153 @@
# SOTS1 — String Intelligence (Sword of the Stars.exe)
**Binary:** `/srv/re-lab/sots-game/Sword of the Stars.exe` (7,898,624 bytes)
**Type:** PE32 i386, GUI, 5 sections, MSVC-2010 (MSVCP100/MSVCR100), DirectX-9.
**Extraction:** `strings -n 5` (ASCII) + `strings -e l -n 5` (UTF-16).
Counts: **28,719 unique ASCII strings**, only **8 unique UTF-16** (this build stores nearly everything as ASCII/UTF-8; localized text lives in the `.gob` archives, not the exe).
---
## 0. TL;DR — highest-value findings
- **Battle-load / multicore bug is right here in the open:** the exe ships a **`ForceSingleCore`** config key (also literal `FORCESINGLE`) that drives **`SetProcessAffinityMask`**, logging **`Limiting process affinity to CPU-%i...`**. These three strings sit adjacent in `.rdata` (offsets 6,510,208 / 6,511,852 / 6,523,472). This is the classic SOTS1 fix for the combat-load hang on multi-core CPUs.
- **Build stamp:** version **`1.8.1`**, compiled **`Wed Dec 13 03:38:31 2017`** (this is the GOG 2018 rerelease of the original 2006 game).
- **PDB leak:** `E:\sots.clean\Distribution\Demo\Sword of the Stars - Retail.pdb` — confirms build tree layout and that this is the "Retail" config off the `sots.clean` branch.
- **No embedded scripting VM** (no Lua/Python). Game logic is **data-driven** via `Data/**/*.txt` + `*.csv` + `*.tech`; `*.script` files are GUI/HUD layout definitions, not a bytecode language.
- **Namespaces:** engine code lives under **`Mars`** (the engine) and game code under **`Game`** (RTTI decorations like `.?AV...@Game@@` / `@Mars@@` are everywhere and are a goldmine for the class map).
- **Networking = GameSpy SDK** (IRC-style Chat/Peer + NatNeg + master server) over **WSOCK32**, plus two in-house message families: **`SNM*`** (Strategy Net Messages, ~60 types) and **`FNM*`** (File Network Messages — chunked reliable transfer used for host migration / savegame sync).
---
## 1. Config keys / sots.ini / registry
SOTS does **not** embed literal `[Section]` header strings; it reads config through the Win32 profile API (`GetPrivateProfileStringA`, `GetPrivateProfileSectionA`, `WritePrivateProfileStringA/SectionA`) and the registry (`RegOpenKeyExA` on `Software\Sword of the Stars`).
**Config files / stores (samples):**
- `sots.ini`, `sots.cfg`
- Registry key `Software\Sword of the Stars` (also `Software\GameSpy\GameSpy 3D\Registration`)
- `Profiles/Profile_*.ini`, `Profiles/*.profile`, `Profiles/lastprofile.dat`
- `CurrentProfile`, `ProfileID`, `ProfileName`, `lastprofile`
**Full user-facing knob list (mined config keys):**
*CPU / performance (BUG-relevant):*
- `ForceSingleCore` / `FORCESINGLE`
*Video / render:*
- `adapter`, `width`, `height`, `fullscreen`, `windowed`, `fullscreenWidth`, `fullscreenHeight`, `antialiasing`, `brightness`, `vertexColors`
*Audio:*
- `MusicVolume` (plus per-sound `*_sound_minrange`/`*_sound_maxrange`, `SOUND_START_DELAY`)
*Gameplay defaults (game-setup options):*
- `MaxPlayers` ("must be an integer between 2 and 8"), max colonies (1–10)
- `EnableAlliances`, `EnablePlayerPasswords`, `EnableTeams`, `EnableTeamsGrouped`
- `DefaultAutoRefuel`, `DefaultAutoResolveCombat`, `DefaultHideFleetLines`, `DefaultShowAxes`, `DefaultShowTacticalInfo`, `DefaultShowTacticalOverlay`, `DefaultSubdueStars`
- `AutoPause`
*Networking:*
- `HostName`, `HostPort`, `hostname`, `hostport`, `host_species`, `MaxTxMessageSize`
---
## 2. File paths, extensions & asset pipeline
**Extension histogram (top): ** `.X` (DirectX mesh, dominant model format), `.fx` (D3D9 effect/shader), `.tga`/`.dds`/`.bmp`/`.jpg` (textures), `.txt` + `.csv` (data tables), `.wav`/`.bik` (audio/Bink video), `.tech` (tech tree), `.script`/`.def`/`.effect`/`.shipsection` (custom defs), `.gob` (archive), `.sav`/`.dat`/`.ini`/`.cfg`.
**Directory layout (from ~261 path strings):**
- `Data/` — `globals.txt`, `Species.txt`, `encounters.txt`, `music.csv`, `Combat/*.txt` (`ship.txt`, `planet.txt`, `terrain.txt`, `sensors.txt`, `camera.txt`, `drone.txt`, `spyship.txt`, `gravboat.txt`, `shipai.txt`, `damfx.txt`, `ctechvars.txt`…), `Strategy/playercolors.txt`, `Strategy/ai/affinity_weapon.csv`, `Strategy/ai/affinity_section.csv`
- `Species/<Race>/...` — Human, Hiver, Tarkas, Liir, Zuul, Morrigi, `_NPC` (art/debris, sections, `_Spy.shipsection`)
- `Models/` — `Planets/`, `Shields/` (`sld_*` per hull class/tech), `Asteroids/Asteroids.csv`, `Trade/`, `Skysphere/`
- `Weapons/` — `_weapons.txt`, `_turrets.txt`, `_defaultweapons.txt`, `art/*.X`
- `TechTree/` — `MasterTechList.tech`, `tech.fx`, `techBases.fx`, `TechTreeDisplay.script`
- `Scenarios/` — `Gathering`, `ProgressionWars`, `UpstartApes*`, `BrokenEmpire`, `CivilWar`, `EndFlesh`, `HiverInvasion*`, `CrownJewels`, `Tournament`, `Antiquarians` (each a `.txt` + supporting `.csv`s)
- `GUI/` — `Combat/*.script` (HUD), `StarMap/Lines.script`, `Mouse/mouse_*.tga`, `SplashArt/`, `LoadScreen_Anim0*.tga`
- `Render/` — many `.fx` (`DefaultScene`, `StandardMaterial`, `GasCloud`, `CloakBadges`, `NodeDrive`, `StratStars`, `CombatStars`, `Miniship`, `StarMapBlobs_*`…)
- `Sounds/`, `Movies/` (`sotsintro.bik`), `Locale/<lang>/Desc*.txt`, `Avatars/AvatarTable.txt`, `Badges/BadgeTable.txt`
**`.gob` archive format:** the `gobio` namespace (`gobio::IFileSystem`, `NativeFileSystem`, `ZipFileSystem`, `Buffer`/`IBuffer`, `gobio_D3DXInclude`) shows **.gob is a ZIP-based virtual filesystem** with a native-directory fallback (so loose files can override archived ones — useful for asset extraction/modding). Runtime archives: `sots.gob` (1.5 GB) and `sots_local_en.gob` (628 MB, localization).
---
## 3. Scripting
**No embedded Lua/Python/bytecode VM.** Grep for `lua_`/`luaL_`/`Py_`/`.pyc`/`tolua` returned only incidental substrings. The engine is fully data-driven:
- `*.script` files are **GUI/HUD layout descriptions** parsed by the engine (`GUI/Combat/CombatHUD.script`, `SensorHUD.script`, `NoHUD.script`, `Models/Planets/PlanetResources.script`, `TechTree/TechTreeDisplay.script`, `GUI/StarMap/Lines.script`). Errors: `Error creating extra script %s.`, `Failed to open script %s!`, `StringTable: %s not found in script.`
- Scenario/AI behavior is described in `.txt`/`.csv` tables (`ScenarioKeyFile: Unrecognized script object id: %s`, `Script described %d slots, but max players is %d.`). "Script" here = declarative data, not code.
---
## 4. Format strings & error/log messages (subsystem map)
~991 printf-style and ~2,498 error/assert strings. They name subsystems cleanly:
- **Combat sim:** `CombatPlayerAI::Update`, `(Combat Player %i) ...`, `Reserves::PickReserve`, `ShipBase::Damage/Load`, `CombatFieldTemplate`, `SpawnShips`, `ICarrier`/`Carrier runway`, `MissileBase`, `Bolt`/`Mine`/`ChainLightning`, `@SYNC ERROR DETECTED: Frame %d` (determinism check).
- **Strategy sim:** `StrategyClient`/`StrategyServer`/`StrategyApp`, `BuildQueue::*`, `HandleEvent: Cannot init/term/sync fleet|ship|design ...`, `Alliances:`, `SpecialProject:`, `PLAGUE:`, `Sensors: %f seconds to update sensors for all players.`
- **Tech:** `MasterTechTree`, `TechTree::GetSync()/GetInit()`, `Unable to find mesh for tech %s.`
- **Renderer / D3D9:** `Could not create D3D Device!`, `Created D3D device with full features` / `with fallback (%i)`, `*** Losing/Restoring D3D Device Objects ***`, `Adapter: %d: %s`, `Resolution: %dx%d @ %d-Hz`, `Antialiasing: %s`, `D3D available (SDK version %08x).`
- **Serialization:** the `Mars::StreamableHelper<...>` / `VectorHelper<...>` templates (see §8) wrap every `*Params` struct — this is the save/network serialization layer.
- **Crash handling:** `MiniDumpWriteDump` (via `DBGHELP.DLL`) — the game writes minidumps.
---
## 5. Network
**Transport:** `WSOCK32.dll`; `IPHLPAPI.DLL` (NAT/adapter discovery).
**GameSpy SDK** (matchmaking + chat):
- Master/query servers: `%s.master.gamespy.com`, `%s.available.gamespy.com`, `%s.ms%d.gamespy.com`; `GameSpyHTTP/1.0`; registry `Software\GameSpy\GameSpy 3D\Registration`; `gamespy_splash_A.tga`.
- **NAT negotiation:** `NatNeg error: %d (progress %d).`, `NatNeg init error: %d.`, extensive `Network: ... NAT negotiation ...` logging.
- **IRC-style Chat/Peer protocol** (format strings): `PRIVMSG %s :%s`, `JOIN %s %s`, `PART %s :%s`, `MODE %s ...`, `NICK %s`, `TOPIC`, `NOTICE`, `USER`, `LOGIN`, plus GameSpy key exchange (`GETKEY`, `SETCKEY`, `CRYPT des %d %s`) and browser tokens (`\hostname\gamemode`, `#GSP!%s`).
**In-house P2P group layer** ("Network:" logger): host/client group model with **host migration** — `Network: New group created. %s is host.`, `DoHost()/DoConnect()/DoDisconnect()`, `HostMigration: Begin/Setup migration...`, `HostMigration failed: ...`, `Rejoin: ...`.
**`SNM*` — Strategy Net Message types** (~60; the strategy-layer protocol enum). Samples: `SNMJoin`/`SNMJoinReply`, `SNMEndTurn`, `SNMHostCombat`/`SNMLaunchCombat`/`SNMJoinCombat`, `SNMEncounterQuery*`/`SNMEncounterResults`, `SNMChat`/`SNMLobbyChat`, `SNMAlliance*`, `SNMSlot*` (lobby slot mgmt + passwords: `SNMSlotReqPW`/`SNMSlotPWAttempt`/`SNMSlotPWAccepted`), `SNMSetPlayerStatus`, `SNMRunAI`/`SNMKillAI`/`SNMMakePlayerAI`, `SNMSetTimers`, `SNMRefreshSlots`.
**`FNM*` — File Network Messages** (`Mars` namespace): `FNMBase`, `FNMChunk`/`FNMChunkReply`, `FNMDone`/`FNMDoneReply`, `FNMSendNotify`/`FNMSendReply`, `FNMError` — a chunked reliable **file-transfer** protocol (drives `FileTransfer: Cannot open %s to send to %s.` and the host-migration savegame push).
---
## 6. Version / build
- **`1.8.1`** (game version; `Invalid version number string (must conform to '0.0.0'...)`).
- Build timestamp: **`Wed Dec 13 03:38:31 2017`**.
- Window class / mutex: `Kerberos_SwordOfTheStars_WndCls`, `Kerberos_SwordOfTheStars_Mutex`.
- Title format: `Sword of the Stars%s (%s %s)`; MOTD URL `www.kerberos-productions.com/motd`.
- **PDB:** `E:\sots.clean\Distribution\Demo\Sword of the Stars - Retail.pdb`.
- Toolchain: MSVC 2010 (`MSVCP100.dll`, `MSVCR100.dll`). Middleware DLLs: `d3d9.dll`, `d3dx9_42.dll`, `DSOUND.dll`, `binkw32.dll` (Bink video), `WINMM.dll`, `DBGHELP.DLL`.
- Editions gated in-exe: `SOTS_COLLECTORSEDITION_ENABLED_TITLE`, `SOTS_COMPLETECOLLECTION_ENABLED_TITLE`, `SOTS_ARGOSNAVALYARD_TITLE`.
---
## 7. BUG watch — CPU / thread / core / affinity (battle-load hang)
Smoking-gun strings, with byte offsets in the file:
| Offset | String | Meaning |
|---|---|---|
| 6,510,208 | `Limiting process affinity to CPU-%i...` | log emitted when single-core mode engages |
| 6,511,852 | `ForceSingleCore` | ini/cfg key that triggers it |
| 6,523,472 | `FORCESINGLE` | uppercased variant (likely token/registry) |
| 7,172,098 | `SetProcessAffinityMask` | KERNEL32 import performing the pin |
| 5,266,079 / 5,266,317 | `F ;Cpu` | fragment near the CPU-detect block |
Supporting threading imports/strings: `CreateThread`, `CreateMutexA`, `GetCurrentThreadId`, `InterlockedExchange`, `InterlockedCompareExchange`, `IsProcessorFeaturePresent`, `SetThreadPriority`, `_configthreadlocale`, `Initializing Streaming sound update thread...`, `std::_Lockit`. The **audio streaming thread** and the combat sim are the likely contenders in the multi-core race; `ForceSingleCore`→`SetProcessAffinityMask(CPU-i)` is the shipped mitigation. Determinism guard `@SYNC ERROR DETECTED: Frame %d` shows the sim expects lockstep — worth correlating with the load path.
*(Note: most `.?AV...Affinity@Game@@` and `AITAttack*` hits are gameplay "diplomatic affinity" / AI, NOT CPU affinity — don't confuse them.)*
---
## 8. Data model (tables / params structs)
No SQL/SQLite. Two shapes:
**A. CSV/TXT tables** with a schema-ish loader: `[%s,%i] CSV row parsing failed.`, `Field '%s' precedes 'name' field and will be ignored in %s.`, `AsteroidEdictDB: Table is empty!`, `Sprite Table: Sprite %s multiply defined/not found.`, `StringTable`/`GlobalConsts: %s being registered after loading const table.`, `Weapon not found: "%s" - Was it added to the index file?`. There is also XML/DTD validation (MSXML) for some data.
**B. Binary-serialized `*Params` structs** (save/network via `Mars::StreamableHelper<T>` / `VectorHelper<T>`). The full RTTI list gives the effective "record types":
`StrategyGameParams`, `StrategyGameCreateParams`, `StrategyGameLoadParams`, `StrategyPlayerParams`, `StrategySessionParams`, `StrategySimParams`, `StrategyTimerParams`, `StrategyScriptParams`, `StrategyHostParams`, `StrategyLaunchParams`, `SystemParams`, `StarMapParams`, `NodePointParams`, `ShipParams` (with nested `Section`, `GunBank`), `CombatEncounterParams`, `CombatLaunchParams`, `EncounterParams`, `LaunchParams`, `SyncParams`, `LocustEncounterParams`, `RefugeesEncounterParams`, `SwarmEncounterParams`, `VNHomeEncounterParams`, `VonNeumannEncounterParams`. These are the serialized game-state schema — the right targets for savegame/netpacket structure work.
**Engine/game split:** RTTI shows engine classes under **`@Mars@@`** (Frame, ModelFrame, PhysicalEntity, ListFramework, gobio, FNM*, Streamable) and game logic under **`@Game@@`** (all the strategy/combat classes). Use this to partition the code map for the Ghidra agent.
---
*Extraction scripts left in CT111: `/tmp/extract.py`, `/tmp/g2.sh`, `/tmp/g3.sh`, `/tmp/g4.sh`.*