153 lines
13 KiB
Markdown
153 lines
13 KiB
Markdown
# SOTS1 — String Intelligence (Sword of the Stars.exe)
|
||
|
||
**Binary:** `/srv/re-lab/sots-game/Sword of the Stars.exe` (7,898,624 bytes)
|
||
**Type:** PE32 i386, GUI, 5 sections, MSVC-2010 (MSVCP100/MSVCR100), DirectX-9.
|
||
**Extraction:** `strings -n 5` (ASCII) + `strings -e l -n 5` (UTF-16).
|
||
Counts: **28,719 unique ASCII strings**, only **8 unique UTF-16** (this build stores nearly everything as ASCII/UTF-8; localized text lives in the `.gob` archives, not the exe).
|
||
|
||
---
|
||
|
||
## 0. TL;DR — highest-value findings
|
||
|
||
- **Battle-load / multicore bug is right here in the open:** the exe ships a **`ForceSingleCore`** config key (also literal `FORCESINGLE`) that drives **`SetProcessAffinityMask`**, logging **`Limiting process affinity to CPU-%i...`**. These three strings sit adjacent in `.rdata` (offsets 6,510,208 / 6,511,852 / 6,523,472). This is the classic SOTS1 fix for the combat-load hang on multi-core CPUs.
|
||
- **Build stamp:** version **`1.8.1`**, compiled **`Wed Dec 13 03:38:31 2017`** (this is the GOG 2018 rerelease of the original 2006 game).
|
||
- **PDB leak:** `E:\sots.clean\Distribution\Demo\Sword of the Stars - Retail.pdb` — confirms build tree layout and that this is the "Retail" config off the `sots.clean` branch.
|
||
- **No embedded scripting VM** (no Lua/Python). Game logic is **data-driven** via `Data/**/*.txt` + `*.csv` + `*.tech`; `*.script` files are GUI/HUD layout definitions, not a bytecode language.
|
||
- **Namespaces:** engine code lives under **`Mars`** (the engine) and game code under **`Game`** (RTTI decorations like `.?AV...@Game@@` / `@Mars@@` are everywhere and are a goldmine for the class map).
|
||
- **Networking = GameSpy SDK** (IRC-style Chat/Peer + NatNeg + master server) over **WSOCK32**, plus two in-house message families: **`SNM*`** (Strategy Net Messages, ~60 types) and **`FNM*`** (File Network Messages — chunked reliable transfer used for host migration / savegame sync).
|
||
|
||
---
|
||
|
||
## 1. Config keys / sots.ini / registry
|
||
|
||
SOTS does **not** embed literal `[Section]` header strings; it reads config through the Win32 profile API (`GetPrivateProfileStringA`, `GetPrivateProfileSectionA`, `WritePrivateProfileStringA/SectionA`) and the registry (`RegOpenKeyExA` on `Software\Sword of the Stars`).
|
||
|
||
**Config files / stores (samples):**
|
||
- `sots.ini`, `sots.cfg`
|
||
- Registry key `Software\Sword of the Stars` (also `Software\GameSpy\GameSpy 3D\Registration`)
|
||
- `Profiles/Profile_*.ini`, `Profiles/*.profile`, `Profiles/lastprofile.dat`
|
||
- `CurrentProfile`, `ProfileID`, `ProfileName`, `lastprofile`
|
||
|
||
**Full user-facing knob list (mined config keys):**
|
||
|
||
*CPU / performance (BUG-relevant):*
|
||
- `ForceSingleCore` / `FORCESINGLE`
|
||
|
||
*Video / render:*
|
||
- `adapter`, `width`, `height`, `fullscreen`, `windowed`, `fullscreenWidth`, `fullscreenHeight`, `antialiasing`, `brightness`, `vertexColors`
|
||
|
||
*Audio:*
|
||
- `MusicVolume` (plus per-sound `*_sound_minrange`/`*_sound_maxrange`, `SOUND_START_DELAY`)
|
||
|
||
*Gameplay defaults (game-setup options):*
|
||
- `MaxPlayers` ("must be an integer between 2 and 8"), max colonies (1–10)
|
||
- `EnableAlliances`, `EnablePlayerPasswords`, `EnableTeams`, `EnableTeamsGrouped`
|
||
- `DefaultAutoRefuel`, `DefaultAutoResolveCombat`, `DefaultHideFleetLines`, `DefaultShowAxes`, `DefaultShowTacticalInfo`, `DefaultShowTacticalOverlay`, `DefaultSubdueStars`
|
||
- `AutoPause`
|
||
|
||
*Networking:*
|
||
- `HostName`, `HostPort`, `hostname`, `hostport`, `host_species`, `MaxTxMessageSize`
|
||
|
||
---
|
||
|
||
## 2. File paths, extensions & asset pipeline
|
||
|
||
**Extension histogram (top): ** `.X` (DirectX mesh, dominant model format), `.fx` (D3D9 effect/shader), `.tga`/`.dds`/`.bmp`/`.jpg` (textures), `.txt` + `.csv` (data tables), `.wav`/`.bik` (audio/Bink video), `.tech` (tech tree), `.script`/`.def`/`.effect`/`.shipsection` (custom defs), `.gob` (archive), `.sav`/`.dat`/`.ini`/`.cfg`.
|
||
|
||
**Directory layout (from ~261 path strings):**
|
||
- `Data/` — `globals.txt`, `Species.txt`, `encounters.txt`, `music.csv`, `Combat/*.txt` (`ship.txt`, `planet.txt`, `terrain.txt`, `sensors.txt`, `camera.txt`, `drone.txt`, `spyship.txt`, `gravboat.txt`, `shipai.txt`, `damfx.txt`, `ctechvars.txt`…), `Strategy/playercolors.txt`, `Strategy/ai/affinity_weapon.csv`, `Strategy/ai/affinity_section.csv`
|
||
- `Species/<Race>/...` — Human, Hiver, Tarkas, Liir, Zuul, Morrigi, `_NPC` (art/debris, sections, `_Spy.shipsection`)
|
||
- `Models/` — `Planets/`, `Shields/` (`sld_*` per hull class/tech), `Asteroids/Asteroids.csv`, `Trade/`, `Skysphere/`
|
||
- `Weapons/` — `_weapons.txt`, `_turrets.txt`, `_defaultweapons.txt`, `art/*.X`
|
||
- `TechTree/` — `MasterTechList.tech`, `tech.fx`, `techBases.fx`, `TechTreeDisplay.script`
|
||
- `Scenarios/` — `Gathering`, `ProgressionWars`, `UpstartApes*`, `BrokenEmpire`, `CivilWar`, `EndFlesh`, `HiverInvasion*`, `CrownJewels`, `Tournament`, `Antiquarians` (each a `.txt` + supporting `.csv`s)
|
||
- `GUI/` — `Combat/*.script` (HUD), `StarMap/Lines.script`, `Mouse/mouse_*.tga`, `SplashArt/`, `LoadScreen_Anim0*.tga`
|
||
- `Render/` — many `.fx` (`DefaultScene`, `StandardMaterial`, `GasCloud`, `CloakBadges`, `NodeDrive`, `StratStars`, `CombatStars`, `Miniship`, `StarMapBlobs_*`…)
|
||
- `Sounds/`, `Movies/` (`sotsintro.bik`), `Locale/<lang>/Desc*.txt`, `Avatars/AvatarTable.txt`, `Badges/BadgeTable.txt`
|
||
|
||
**`.gob` archive format:** the `gobio` namespace (`gobio::IFileSystem`, `NativeFileSystem`, `ZipFileSystem`, `Buffer`/`IBuffer`, `gobio_D3DXInclude`) shows **.gob is a ZIP-based virtual filesystem** with a native-directory fallback (so loose files can override archived ones — useful for asset extraction/modding). Runtime archives: `sots.gob` (1.5 GB) and `sots_local_en.gob` (628 MB, localization).
|
||
|
||
---
|
||
|
||
## 3. Scripting
|
||
|
||
**No embedded Lua/Python/bytecode VM.** Grep for `lua_`/`luaL_`/`Py_`/`.pyc`/`tolua` returned only incidental substrings. The engine is fully data-driven:
|
||
- `*.script` files are **GUI/HUD layout descriptions** parsed by the engine (`GUI/Combat/CombatHUD.script`, `SensorHUD.script`, `NoHUD.script`, `Models/Planets/PlanetResources.script`, `TechTree/TechTreeDisplay.script`, `GUI/StarMap/Lines.script`). Errors: `Error creating extra script %s.`, `Failed to open script %s!`, `StringTable: %s not found in script.`
|
||
- Scenario/AI behavior is described in `.txt`/`.csv` tables (`ScenarioKeyFile: Unrecognized script object id: %s`, `Script described %d slots, but max players is %d.`). "Script" here = declarative data, not code.
|
||
|
||
---
|
||
|
||
## 4. Format strings & error/log messages (subsystem map)
|
||
|
||
~991 printf-style and ~2,498 error/assert strings. They name subsystems cleanly:
|
||
|
||
- **Combat sim:** `CombatPlayerAI::Update`, `(Combat Player %i) ...`, `Reserves::PickReserve`, `ShipBase::Damage/Load`, `CombatFieldTemplate`, `SpawnShips`, `ICarrier`/`Carrier runway`, `MissileBase`, `Bolt`/`Mine`/`ChainLightning`, `@SYNC ERROR DETECTED: Frame %d` (determinism check).
|
||
- **Strategy sim:** `StrategyClient`/`StrategyServer`/`StrategyApp`, `BuildQueue::*`, `HandleEvent: Cannot init/term/sync fleet|ship|design ...`, `Alliances:`, `SpecialProject:`, `PLAGUE:`, `Sensors: %f seconds to update sensors for all players.`
|
||
- **Tech:** `MasterTechTree`, `TechTree::GetSync()/GetInit()`, `Unable to find mesh for tech %s.`
|
||
- **Renderer / D3D9:** `Could not create D3D Device!`, `Created D3D device with full features` / `with fallback (%i)`, `*** Losing/Restoring D3D Device Objects ***`, `Adapter: %d: %s`, `Resolution: %dx%d @ %d-Hz`, `Antialiasing: %s`, `D3D available (SDK version %08x).`
|
||
- **Serialization:** the `Mars::StreamableHelper<...>` / `VectorHelper<...>` templates (see §8) wrap every `*Params` struct — this is the save/network serialization layer.
|
||
- **Crash handling:** `MiniDumpWriteDump` (via `DBGHELP.DLL`) — the game writes minidumps.
|
||
|
||
---
|
||
|
||
## 5. Network
|
||
|
||
**Transport:** `WSOCK32.dll`; `IPHLPAPI.DLL` (NAT/adapter discovery).
|
||
|
||
**GameSpy SDK** (matchmaking + chat):
|
||
- Master/query servers: `%s.master.gamespy.com`, `%s.available.gamespy.com`, `%s.ms%d.gamespy.com`; `GameSpyHTTP/1.0`; registry `Software\GameSpy\GameSpy 3D\Registration`; `gamespy_splash_A.tga`.
|
||
- **NAT negotiation:** `NatNeg error: %d (progress %d).`, `NatNeg init error: %d.`, extensive `Network: ... NAT negotiation ...` logging.
|
||
- **IRC-style Chat/Peer protocol** (format strings): `PRIVMSG %s :%s`, `JOIN %s %s`, `PART %s :%s`, `MODE %s ...`, `NICK %s`, `TOPIC`, `NOTICE`, `USER`, `LOGIN`, plus GameSpy key exchange (`GETKEY`, `SETCKEY`, `CRYPT des %d %s`) and browser tokens (`\hostname\gamemode`, `#GSP!%s`).
|
||
|
||
**In-house P2P group layer** ("Network:" logger): host/client group model with **host migration** — `Network: New group created. %s is host.`, `DoHost()/DoConnect()/DoDisconnect()`, `HostMigration: Begin/Setup migration...`, `HostMigration failed: ...`, `Rejoin: ...`.
|
||
|
||
**`SNM*` — Strategy Net Message types** (~60; the strategy-layer protocol enum). Samples: `SNMJoin`/`SNMJoinReply`, `SNMEndTurn`, `SNMHostCombat`/`SNMLaunchCombat`/`SNMJoinCombat`, `SNMEncounterQuery*`/`SNMEncounterResults`, `SNMChat`/`SNMLobbyChat`, `SNMAlliance*`, `SNMSlot*` (lobby slot mgmt + passwords: `SNMSlotReqPW`/`SNMSlotPWAttempt`/`SNMSlotPWAccepted`), `SNMSetPlayerStatus`, `SNMRunAI`/`SNMKillAI`/`SNMMakePlayerAI`, `SNMSetTimers`, `SNMRefreshSlots`.
|
||
|
||
**`FNM*` — File Network Messages** (`Mars` namespace): `FNMBase`, `FNMChunk`/`FNMChunkReply`, `FNMDone`/`FNMDoneReply`, `FNMSendNotify`/`FNMSendReply`, `FNMError` — a chunked reliable **file-transfer** protocol (drives `FileTransfer: Cannot open %s to send to %s.` and the host-migration savegame push).
|
||
|
||
---
|
||
|
||
## 6. Version / build
|
||
|
||
- **`1.8.1`** (game version; `Invalid version number string (must conform to '0.0.0'...)`).
|
||
- Build timestamp: **`Wed Dec 13 03:38:31 2017`**.
|
||
- Window class / mutex: `Kerberos_SwordOfTheStars_WndCls`, `Kerberos_SwordOfTheStars_Mutex`.
|
||
- Title format: `Sword of the Stars%s (%s %s)`; MOTD URL `www.kerberos-productions.com/motd`.
|
||
- **PDB:** `E:\sots.clean\Distribution\Demo\Sword of the Stars - Retail.pdb`.
|
||
- Toolchain: MSVC 2010 (`MSVCP100.dll`, `MSVCR100.dll`). Middleware DLLs: `d3d9.dll`, `d3dx9_42.dll`, `DSOUND.dll`, `binkw32.dll` (Bink video), `WINMM.dll`, `DBGHELP.DLL`.
|
||
- Editions gated in-exe: `SOTS_COLLECTORSEDITION_ENABLED_TITLE`, `SOTS_COMPLETECOLLECTION_ENABLED_TITLE`, `SOTS_ARGOSNAVALYARD_TITLE`.
|
||
|
||
---
|
||
|
||
## 7. BUG watch — CPU / thread / core / affinity (battle-load hang)
|
||
|
||
Smoking-gun strings, with byte offsets in the file:
|
||
|
||
| Offset | String | Meaning |
|
||
|---|---|---|
|
||
| 6,510,208 | `Limiting process affinity to CPU-%i...` | log emitted when single-core mode engages |
|
||
| 6,511,852 | `ForceSingleCore` | ini/cfg key that triggers it |
|
||
| 6,523,472 | `FORCESINGLE` | uppercased variant (likely token/registry) |
|
||
| 7,172,098 | `SetProcessAffinityMask` | KERNEL32 import performing the pin |
|
||
| 5,266,079 / 5,266,317 | `F ;Cpu` | fragment near the CPU-detect block |
|
||
|
||
Supporting threading imports/strings: `CreateThread`, `CreateMutexA`, `GetCurrentThreadId`, `InterlockedExchange`, `InterlockedCompareExchange`, `IsProcessorFeaturePresent`, `SetThreadPriority`, `_configthreadlocale`, `Initializing Streaming sound update thread...`, `std::_Lockit`. The **audio streaming thread** and the combat sim are the likely contenders in the multi-core race; `ForceSingleCore`→`SetProcessAffinityMask(CPU-i)` is the shipped mitigation. Determinism guard `@SYNC ERROR DETECTED: Frame %d` shows the sim expects lockstep — worth correlating with the load path.
|
||
|
||
*(Note: most `.?AV...Affinity@Game@@` and `AITAttack*` hits are gameplay "diplomatic affinity" / AI, NOT CPU affinity — don't confuse them.)*
|
||
|
||
---
|
||
|
||
## 8. Data model (tables / params structs)
|
||
|
||
No SQL/SQLite. Two shapes:
|
||
|
||
**A. CSV/TXT tables** with a schema-ish loader: `[%s,%i] CSV row parsing failed.`, `Field '%s' precedes 'name' field and will be ignored in %s.`, `AsteroidEdictDB: Table is empty!`, `Sprite Table: Sprite %s multiply defined/not found.`, `StringTable`/`GlobalConsts: %s being registered after loading const table.`, `Weapon not found: "%s" - Was it added to the index file?`. There is also XML/DTD validation (MSXML) for some data.
|
||
|
||
**B. Binary-serialized `*Params` structs** (save/network via `Mars::StreamableHelper<T>` / `VectorHelper<T>`). The full RTTI list gives the effective "record types":
|
||
`StrategyGameParams`, `StrategyGameCreateParams`, `StrategyGameLoadParams`, `StrategyPlayerParams`, `StrategySessionParams`, `StrategySimParams`, `StrategyTimerParams`, `StrategyScriptParams`, `StrategyHostParams`, `StrategyLaunchParams`, `SystemParams`, `StarMapParams`, `NodePointParams`, `ShipParams` (with nested `Section`, `GunBank`), `CombatEncounterParams`, `CombatLaunchParams`, `EncounterParams`, `LaunchParams`, `SyncParams`, `LocustEncounterParams`, `RefugeesEncounterParams`, `SwarmEncounterParams`, `VNHomeEncounterParams`, `VonNeumannEncounterParams`. These are the serialized game-state schema — the right targets for savegame/netpacket structure work.
|
||
|
||
**Engine/game split:** RTTI shows engine classes under **`@Mars@@`** (Frame, ModelFrame, PhysicalEntity, ListFramework, gobio, FNM*, Streamable) and game logic under **`@Game@@`** (all the strategy/combat classes). Use this to partition the code map for the Ghidra agent.
|
||
|
||
---
|
||
|
||
*Extraction scripts left in CT111: `/tmp/extract.py`, `/tmp/g2.sh`, `/tmp/g3.sh`, `/tmp/g4.sh`.*
|