alex/sots-re
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
sots-re/findings/subsystems/strings-and-config.md

13 KiB
Raw Blame History

SOTS1 — String Intelligence (Sword of the Stars.exe)

Binary: /srv/re-lab/sots-game/Sword of the Stars.exe (7,898,624 bytes) Type: PE32 i386, GUI, 5 sections, MSVC-2010 (MSVCP100/MSVCR100), DirectX-9. Extraction: strings -n 5 (ASCII) + strings -e l -n 5 (UTF-16). Counts: 28,719 unique ASCII strings, only 8 unique UTF-16 (this build stores nearly everything as ASCII/UTF-8; localized text lives in the .gob archives, not the exe).

0. TL;DR — highest-value findings

  • Battle-load / multicore bug is right here in the open: the exe ships a ForceSingleCore config key (also literal FORCESINGLE) that drives SetProcessAffinityMask, logging Limiting process affinity to CPU-%i.... These three strings sit adjacent in .rdata (offsets 6,510,208 / 6,511,852 / 6,523,472). This is the classic SOTS1 fix for the combat-load hang on multi-core CPUs.
  • Build stamp: version 1.8.1, compiled Wed Dec 13 03:38:31 2017 (this is the GOG 2018 rerelease of the original 2006 game).
  • PDB leak: E:\sots.clean\Distribution\Demo\Sword of the Stars - Retail.pdb — confirms build tree layout and that this is the "Retail" config off the sots.clean branch.
  • No embedded scripting VM (no Lua/Python). Game logic is data-driven via Data/**/*.txt + *.csv + *.tech; *.script files are GUI/HUD layout definitions, not a bytecode language.
  • Namespaces: engine code lives under Mars (the engine) and game code under Game (RTTI decorations like .?AV...@Game@@ / @Mars@@ are everywhere and are a goldmine for the class map).
  • Networking = GameSpy SDK (IRC-style Chat/Peer + NatNeg + master server) over WSOCK32, plus two in-house message families: SNM* (Strategy Net Messages, ~60 types) and FNM* (File Network Messages — chunked reliable transfer used for host migration / savegame sync).

1. Config keys / sots.ini / registry

SOTS does not embed literal [Section] header strings; it reads config through the Win32 profile API (GetPrivateProfileStringA, GetPrivateProfileSectionA, WritePrivateProfileStringA/SectionA) and the registry (RegOpenKeyExA on Software\Sword of the Stars).

Config files / stores (samples):

  • sots.ini, sots.cfg
  • Registry key Software\Sword of the Stars (also Software\GameSpy\GameSpy 3D\Registration)
  • Profiles/Profile_*.ini, Profiles/*.profile, Profiles/lastprofile.dat
  • CurrentProfile, ProfileID, ProfileName, lastprofile

Full user-facing knob list (mined config keys):

CPU / performance (BUG-relevant):

  • ForceSingleCore / FORCESINGLE

Video / render:

  • adapter, width, height, fullscreen, windowed, fullscreenWidth, fullscreenHeight, antialiasing, brightness, vertexColors

Audio:

  • MusicVolume (plus per-sound *_sound_minrange/*_sound_maxrange, SOUND_START_DELAY)

Gameplay defaults (game-setup options):

  • MaxPlayers ("must be an integer between 2 and 8"), max colonies (1–10)
  • EnableAlliances, EnablePlayerPasswords, EnableTeams, EnableTeamsGrouped
  • DefaultAutoRefuel, DefaultAutoResolveCombat, DefaultHideFleetLines, DefaultShowAxes, DefaultShowTacticalInfo, DefaultShowTacticalOverlay, DefaultSubdueStars
  • AutoPause

Networking:

  • HostName, HostPort, hostname, hostport, host_species, MaxTxMessageSize

2. File paths, extensions & asset pipeline

**Extension histogram (top): ** .X (DirectX mesh, dominant model format), .fx (D3D9 effect/shader), .tga/.dds/.bmp/.jpg (textures), .txt + .csv (data tables), .wav/.bik (audio/Bink video), .tech (tech tree), .script/.def/.effect/.shipsection (custom defs), .gob (archive), .sav/.dat/.ini/.cfg.

Directory layout (from ~261 path strings):

  • Data/ — globals.txt, Species.txt, encounters.txt, music.csv, Combat/*.txt (ship.txt, planet.txt, terrain.txt, sensors.txt, camera.txt, drone.txt, spyship.txt, gravboat.txt, shipai.txt, damfx.txt, ctechvars.txt…), Strategy/playercolors.txt, Strategy/ai/affinity_weapon.csv, Strategy/ai/affinity_section.csv
  • Species/<Race>/... — Human, Hiver, Tarkas, Liir, Zuul, Morrigi, _NPC (art/debris, sections, _Spy.shipsection)
  • Models/ — Planets/, Shields/ (sld_* per hull class/tech), Asteroids/Asteroids.csv, Trade/, Skysphere/
  • Weapons/ — _weapons.txt, _turrets.txt, _defaultweapons.txt, art/*.X
  • TechTree/ — MasterTechList.tech, tech.fx, techBases.fx, TechTreeDisplay.script
  • Scenarios/ — Gathering, ProgressionWars, UpstartApes*, BrokenEmpire, CivilWar, EndFlesh, HiverInvasion*, CrownJewels, Tournament, Antiquarians (each a .txt + supporting .csvs)
  • GUI/ — Combat/*.script (HUD), StarMap/Lines.script, Mouse/mouse_*.tga, SplashArt/, LoadScreen_Anim0*.tga
  • Render/ — many .fx (DefaultScene, StandardMaterial, GasCloud, CloakBadges, NodeDrive, StratStars, CombatStars, Miniship, StarMapBlobs_*…)
  • Sounds/, Movies/ (sotsintro.bik), Locale/<lang>/Desc*.txt, Avatars/AvatarTable.txt, Badges/BadgeTable.txt

.gob archive format: the gobio namespace (gobio::IFileSystem, NativeFileSystem, ZipFileSystem, Buffer/IBuffer, gobio_D3DXInclude) shows .gob is a ZIP-based virtual filesystem with a native-directory fallback (so loose files can override archived ones — useful for asset extraction/modding). Runtime archives: sots.gob (1.5 GB) and sots_local_en.gob (628 MB, localization).

3. Scripting

No embedded Lua/Python/bytecode VM. Grep for lua_/luaL_/Py_/.pyc/tolua returned only incidental substrings. The engine is fully data-driven:

  • *.script files are GUI/HUD layout descriptions parsed by the engine (GUI/Combat/CombatHUD.script, SensorHUD.script, NoHUD.script, Models/Planets/PlanetResources.script, TechTree/TechTreeDisplay.script, GUI/StarMap/Lines.script). Errors: Error creating extra script %s., Failed to open script %s!, StringTable: %s not found in script.
  • Scenario/AI behavior is described in .txt/.csv tables (ScenarioKeyFile: Unrecognized script object id: %s, Script described %d slots, but max players is %d.). "Script" here = declarative data, not code.

4. Format strings & error/log messages (subsystem map)

~991 printf-style and ~2,498 error/assert strings. They name subsystems cleanly:

  • Combat sim: CombatPlayerAI::Update, (Combat Player %i) ..., Reserves::PickReserve, ShipBase::Damage/Load, CombatFieldTemplate, SpawnShips, ICarrier/Carrier runway, MissileBase, Bolt/Mine/ChainLightning, @SYNC ERROR DETECTED: Frame %d (determinism check).
  • Strategy sim: StrategyClient/StrategyServer/StrategyApp, BuildQueue::*, HandleEvent: Cannot init/term/sync fleet|ship|design ..., Alliances:, SpecialProject:, PLAGUE:, Sensors: %f seconds to update sensors for all players.
  • Tech: MasterTechTree, TechTree::GetSync()/GetInit(), Unable to find mesh for tech %s.
  • Renderer / D3D9: Could not create D3D Device!, Created D3D device with full features / with fallback (%i), *** Losing/Restoring D3D Device Objects ***, Adapter: %d: %s, Resolution: %dx%d @ %d-Hz, Antialiasing: %s, D3D available (SDK version %08x).
  • Serialization: the Mars::StreamableHelper<...> / VectorHelper<...> templates (see §8) wrap every *Params struct — this is the save/network serialization layer.
  • Crash handling: MiniDumpWriteDump (via DBGHELP.DLL) — the game writes minidumps.

5. Network

Transport: WSOCK32.dll; IPHLPAPI.DLL (NAT/adapter discovery).

GameSpy SDK (matchmaking + chat):

  • Master/query servers: %s.master.gamespy.com, %s.available.gamespy.com, %s.ms%d.gamespy.com; GameSpyHTTP/1.0; registry Software\GameSpy\GameSpy 3D\Registration; gamespy_splash_A.tga.
  • NAT negotiation: NatNeg error: %d (progress %d)., NatNeg init error: %d., extensive Network: ... NAT negotiation ... logging.
  • IRC-style Chat/Peer protocol (format strings): PRIVMSG %s :%s, JOIN %s %s, PART %s :%s, MODE %s ..., NICK %s, TOPIC, NOTICE, USER, LOGIN, plus GameSpy key exchange (GETKEY, SETCKEY, CRYPT des %d %s) and browser tokens (\hostname\gamemode, #GSP!%s).

In-house P2P group layer ("Network:" logger): host/client group model with host migration — Network: New group created. %s is host., DoHost()/DoConnect()/DoDisconnect(), HostMigration: Begin/Setup migration..., HostMigration failed: ..., Rejoin: ....

SNM* — Strategy Net Message types (~60; the strategy-layer protocol enum). Samples: SNMJoin/SNMJoinReply, SNMEndTurn, SNMHostCombat/SNMLaunchCombat/SNMJoinCombat, SNMEncounterQuery*/SNMEncounterResults, SNMChat/SNMLobbyChat, SNMAlliance*, SNMSlot* (lobby slot mgmt + passwords: SNMSlotReqPW/SNMSlotPWAttempt/SNMSlotPWAccepted), SNMSetPlayerStatus, SNMRunAI/SNMKillAI/SNMMakePlayerAI, SNMSetTimers, SNMRefreshSlots.

FNM* — File Network Messages (Mars namespace): FNMBase, FNMChunk/FNMChunkReply, FNMDone/FNMDoneReply, FNMSendNotify/FNMSendReply, FNMError — a chunked reliable file-transfer protocol (drives FileTransfer: Cannot open %s to send to %s. and the host-migration savegame push).

6. Version / build

  • 1.8.1 (game version; Invalid version number string (must conform to '0.0.0'...)).
  • Build timestamp: Wed Dec 13 03:38:31 2017.
  • Window class / mutex: Kerberos_SwordOfTheStars_WndCls, Kerberos_SwordOfTheStars_Mutex.
  • Title format: Sword of the Stars%s (%s %s); MOTD URL www.kerberos-productions.com/motd.
  • PDB: E:\sots.clean\Distribution\Demo\Sword of the Stars - Retail.pdb.
  • Toolchain: MSVC 2010 (MSVCP100.dll, MSVCR100.dll). Middleware DLLs: d3d9.dll, d3dx9_42.dll, DSOUND.dll, binkw32.dll (Bink video), WINMM.dll, DBGHELP.DLL.
  • Editions gated in-exe: SOTS_COLLECTORSEDITION_ENABLED_TITLE, SOTS_COMPLETECOLLECTION_ENABLED_TITLE, SOTS_ARGOSNAVALYARD_TITLE.

7. BUG watch — CPU / thread / core / affinity (battle-load hang)

Smoking-gun strings, with byte offsets in the file:

Offset String Meaning
6,510,208 Limiting process affinity to CPU-%i... log emitted when single-core mode engages
6,511,852 ForceSingleCore ini/cfg key that triggers it
6,523,472 FORCESINGLE uppercased variant (likely token/registry)
7,172,098 SetProcessAffinityMask KERNEL32 import performing the pin
5,266,079 / 5,266,317 F ;Cpu fragment near the CPU-detect block

Supporting threading imports/strings: CreateThread, CreateMutexA, GetCurrentThreadId, InterlockedExchange, InterlockedCompareExchange, IsProcessorFeaturePresent, SetThreadPriority, _configthreadlocale, Initializing Streaming sound update thread..., std::_Lockit. The audio streaming thread and the combat sim are the likely contenders in the multi-core race; ForceSingleCore→SetProcessAffinityMask(CPU-i) is the shipped mitigation. Determinism guard @SYNC ERROR DETECTED: Frame %d shows the sim expects lockstep — worth correlating with the load path.

(Note: most .?AV...Affinity@Game@@ and AITAttack* hits are gameplay "diplomatic affinity" / AI, NOT CPU affinity — don't confuse them.)

8. Data model (tables / params structs)

No SQL/SQLite. Two shapes:

A. CSV/TXT tables with a schema-ish loader: [%s,%i] CSV row parsing failed., Field '%s' precedes 'name' field and will be ignored in %s., AsteroidEdictDB: Table is empty!, Sprite Table: Sprite %s multiply defined/not found., StringTable/GlobalConsts: %s being registered after loading const table., Weapon not found: "%s" - Was it added to the index file?. There is also XML/DTD validation (MSXML) for some data.

B. Binary-serialized *Params structs (save/network via Mars::StreamableHelper<T> / VectorHelper<T>). The full RTTI list gives the effective "record types": StrategyGameParams, StrategyGameCreateParams, StrategyGameLoadParams, StrategyPlayerParams, StrategySessionParams, StrategySimParams, StrategyTimerParams, StrategyScriptParams, StrategyHostParams, StrategyLaunchParams, SystemParams, StarMapParams, NodePointParams, ShipParams (with nested Section, GunBank), CombatEncounterParams, CombatLaunchParams, EncounterParams, LaunchParams, SyncParams, LocustEncounterParams, RefugeesEncounterParams, SwarmEncounterParams, VNHomeEncounterParams, VonNeumannEncounterParams. These are the serialized game-state schema — the right targets for savegame/netpacket structure work.

Engine/game split: RTTI shows engine classes under @Mars@@ (Frame, ModelFrame, PhysicalEntity, ListFramework, gobio, FNM*, Streamable) and game logic under @Game@@ (all the strategy/combat classes). Use this to partition the code map for the Ghidra agent.

Extraction scripts left in CT111: /tmp/extract.py, /tmp/g2.sh, /tmp/g3.sh, /tmp/g4.sh.