2170782a81
SvSctOb is a StreamableHelper<SVScriptObject> -- a polymorphic pointer holding a Game::SVSOSots, which writes two variant lists each dispatched by the key item before it (xscn -> xsc, EncID -> EncObj). Neither map is on the wire; both were read out of the game's factories (see the notes repo). The shapes apply the key in both directions, so a body goes back out as whatever it came in as, and an unmodelled key still round-trips as a Node. 18 new shapes: SVSOSots, the four scenario bodies (traps / crowdefs / indsys / gmtrigger -- indsys really does serialize nothing, its Read and Write are both the shared `ret 4` stub) and the eight encounter bodies the saves exercise. The four factory ids no save carries (7 SystemKiller, 8 PuppetMaster, 14 Locust, 21 Ortgay) are deliberately NOT typed: their serializers are recovered but nothing could check a shape for them. DOpts and SVSOVonNeumann::trev are VectorHelper<Mars::String>, so read_elem / write_elem / SchemaBuilder::carr grew the std::string branch lane G listed as missing. spies2 is VectorHelper<int>: the TYPE is certain from the helper's own decorated name, but the count is 0 in all 28 systems of all four saves, so no element value has ever been observed -- the shape is a hypothesis about behaviour even though it is a fact about type. Same for SysMem and mts. Conformance 56 shapes / 657 items -> 74 / 769, still 0 MISMATCH, and every new binding is 0 wire-only and 0 shape-only. Coverage 97.1/97.2/97.2/97.6 -> 98.0/98.0/98.0/98.4 with the byte-identical round trip preserved; ratchet 95.0 -> 97.5. CD is now the only remaining region of size, and it stays opaque: the recovered 44-item Game::TurnCommands sequence cannot be aligned to the save's 35 items even as a subsequence (item 4 is 8 bytes, so a bool where the recovery says i32; and the 27 trailing ints have only 22 i32 slots to come from), which proves the no-orders diagnosis rather than assuming it. Two unit tests added that need no saves: the string-array element branch (including the empty string, which is four zero bytes and so looks like int 0) and the SvSctOb variant dispatch round trip. ctest 34/34, clean_room_check OK, test_save skips cleanly with SOTS_SAVES_DIR unset. sots_stream_schema.h unchanged: streams.py and gen_stream_schema.py were re-run and the output is byte-identical apart from the provenance line.
268 lines
13 KiB
C++
268 lines
13 KiB
C++
// Conformance: every typed shape in shapes.h vs the wire schema recovered from
|
|
// the game's own Mars::IStreamable serializers (include/generated/sots_stream_schema.h).
|
|
//
|
|
// Why a check and not a generator. The recovered table is a *specification*, not
|
|
// a program: the recovery is a linear pass over the game's Write and cannot see
|
|
// Write's branches, so a conditional field (StarShip's BQ2, gated by hbq) is
|
|
// listed unconditionally and the sequence is a superset of any single record.
|
|
// Container loops are flattened the same way. A codec driven straight off the
|
|
// table would desynchronise on the first branch. The hand-written io() shapes
|
|
// stay the codec — they can express the conditionals and the nesting that the
|
|
// binary facts cannot supply — and this test is what proves they agree with the
|
|
// binary, item for item, in order.
|
|
//
|
|
// SchemaProbe runs each io() with every branch taken, which is the same "all
|
|
// branches" view the recovery has, so the two sequences are comparable. They
|
|
// are aligned with an LCS and three numbers come out:
|
|
//
|
|
// matched the shape and the binary agree on tag and on-disk primitive
|
|
// MISMATCH same tag, different primitive — a real bug, and a hard failure
|
|
// wire-only the binary writes an item the shape does not name: either a
|
|
// conditional the shape models with opt_*/when, or genuine
|
|
// coverage debt
|
|
// shape-only the shape names an item the recovery did not resolve
|
|
//
|
|
// Opaque items (ar.any / ar.raw_frame — a body carried as a Node) are counted
|
|
// separately: they round-trip byte-for-byte but are not understood, and the
|
|
// count is this codebase's honest coverage number.
|
|
//
|
|
// No game data, no saves: the whole test is the generated table plus shapes.h.
|
|
#include <cstdio>
|
|
#include <cstring>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "generated/sots_stream_schema.h"
|
|
#include "mars/stream/probe.h"
|
|
#include "mars/stream/shapes.h"
|
|
|
|
using namespace mars::stream;
|
|
namespace sh = mars::stream::shapes;
|
|
|
|
static int fails = 0;
|
|
#define CHECK(cond) \
|
|
do { \
|
|
if (!(cond)) { \
|
|
std::printf("FAIL %s:%d: %s\n", __FILE__, __LINE__, #cond); \
|
|
++fails; \
|
|
} \
|
|
} while (0)
|
|
|
|
// --- primitive compatibility -------------------------------------------------
|
|
// The generated Prim is the DISK type: a member the original holds as int16 or
|
|
// int8 is written by Stream::WriteInt and is I32 on the wire, so there is no
|
|
// narrow integer to reconcile here.
|
|
static bool prim_ok(SchemaProbe::P got, sots::wire::Prim want) {
|
|
using P = SchemaProbe::P;
|
|
using W = sots::wire::Prim;
|
|
if (want == W::Unknown) return true; // recovery could not type it
|
|
switch (got) {
|
|
case P::Unknown: return true; // opaque on our side: any disk type fits
|
|
case P::I32: return want == W::I32;
|
|
case P::I64: return want == W::I64;
|
|
case P::F32: return want == W::F32;
|
|
case P::Bool: return want == W::Bool;
|
|
case P::Str: return want == W::Str;
|
|
case P::Frame: return want == W::Frame;
|
|
case P::Raw: return want == W::Raw || want == W::I64;
|
|
}
|
|
return false;
|
|
}
|
|
|
|
struct Result {
|
|
int matched = 0, mismatch = 0, wire_only = 0, shape_only = 0;
|
|
int opaque = 0;
|
|
};
|
|
|
|
// LCS alignment on (tag, compatible primitive). Sequences are short (<= 120).
|
|
static Result align(const std::vector<SchemaProbe::Item>& a, const sots::wire::Class& c, bool verbose) {
|
|
const size_t n = a.size(), m = c.count;
|
|
std::vector<std::vector<int>> dp(n + 1, std::vector<int>(m + 1, 0));
|
|
auto eq = [&](size_t i, size_t j) {
|
|
return a[i].tag == c.fields[j].tag && prim_ok(a[i].prim, c.fields[j].prim);
|
|
};
|
|
for (size_t i = n; i-- > 0;)
|
|
for (size_t j = m; j-- > 0;)
|
|
dp[i][j] = eq(i, j) ? dp[i + 1][j + 1] + 1 : std::max(dp[i + 1][j], dp[i][j + 1]);
|
|
|
|
Result r;
|
|
for (const SchemaProbe::Item& it : a) r.opaque += it.opaque;
|
|
size_t i = 0, j = 0;
|
|
while (i < n && j < m) {
|
|
if (eq(i, j)) {
|
|
++r.matched;
|
|
++i;
|
|
++j;
|
|
continue;
|
|
}
|
|
// Same tag, incompatible primitive: not a gap, a disagreement.
|
|
if (a[i].tag == c.fields[j].tag) {
|
|
++r.mismatch;
|
|
std::printf(" MISMATCH %-20s shape %-6s vs wire %-6s\n", a[i].tag.c_str(),
|
|
probe_prim_name(a[i].prim), sots::wire::prim_name(c.fields[j].prim));
|
|
++i;
|
|
++j;
|
|
continue;
|
|
}
|
|
if (dp[i + 1][j] >= dp[i][j + 1]) {
|
|
++r.shape_only;
|
|
if (verbose)
|
|
std::printf(" shape-only %-20s %s\n", a[i].tag.c_str(), probe_prim_name(a[i].prim));
|
|
++i;
|
|
} else {
|
|
++r.wire_only;
|
|
if (verbose)
|
|
std::printf(" wire-only %-20s %s %s\n", c.fields[j].tag,
|
|
sots::wire::prim_name(c.fields[j].prim),
|
|
sots::wire::shape_name(c.fields[j].shape));
|
|
++j;
|
|
}
|
|
}
|
|
for (; i < n; ++i) {
|
|
++r.shape_only;
|
|
if (verbose) std::printf(" shape-only %-20s %s\n", a[i].tag.c_str(), probe_prim_name(a[i].prim));
|
|
}
|
|
for (; j < m; ++j) {
|
|
++r.wire_only;
|
|
if (verbose)
|
|
std::printf(" wire-only %-20s %s %s\n", c.fields[j].tag, sots::wire::prim_name(c.fields[j].prim),
|
|
sots::wire::shape_name(c.fields[j].shape));
|
|
}
|
|
return r;
|
|
}
|
|
|
|
static int total_matched = 0, total_mismatch = 0, total_wire_only = 0, total_opaque = 0;
|
|
static int bound = 0, unresolved_classes = 0;
|
|
|
|
template <class T>
|
|
static void check(const char* shape_name, const char* cls) {
|
|
const sots::wire::Class* c = sots::wire::find(cls);
|
|
if (!c) {
|
|
std::printf(" %-18s -> %-34s NOT IN TABLE\n", shape_name, cls);
|
|
++fails;
|
|
return;
|
|
}
|
|
SchemaProbe p = SchemaProbe::of<T>();
|
|
bool verbose = std::getenv("SOTS_WIRE_VERBOSE") != nullptr;
|
|
Result r = align(p.items, *c, verbose);
|
|
++bound;
|
|
total_matched += r.matched;
|
|
total_mismatch += r.mismatch;
|
|
total_wire_only += r.wire_only;
|
|
total_opaque += r.opaque;
|
|
std::printf(" %-18s -> %-34s [%-8s %2u/%-2u] shape %2zu wire %2u match %2d"
|
|
" wire-only %2d shape-only %2d opaque %d\n",
|
|
shape_name, cls, c->grade, c->read_agree, c->read_comparable, p.items.size(), c->count,
|
|
r.matched, r.wire_only, r.shape_only, r.opaque);
|
|
// A tag both sides name, typed differently, is a real disagreement between
|
|
// our codec and the binary. Nothing else here is allowed to fail the build:
|
|
// wire-only is conditional-field or coverage debt and is reported, not fatal.
|
|
CHECK(r.mismatch == 0);
|
|
}
|
|
|
|
int main() {
|
|
std::printf("wire schema: %zu classes, generated from the game's serializers\n\n",
|
|
sots::wire::kClassCount);
|
|
|
|
// --- the table's own integrity -------------------------------------------------
|
|
for (const sots::wire::Class& c : sots::wire::kClasses) {
|
|
if (c.count == 0) continue;
|
|
for (uint16_t k = 0; k < c.count; ++k) unresolved_classes += c.fields[k].unresolved;
|
|
}
|
|
|
|
std::printf("shape -> class (recovery tier, Read/Write cross-check)\n");
|
|
// Bindings taken from the campaign's own shape->Write table
|
|
// (sots-re tools/serializers_golden.py DISK_ORDER, resolved through RTTI),
|
|
// extended with unambiguous name matches.
|
|
check<sh::Summary>("Summary", "Game::StrategyGameInfo");
|
|
check<sh::Slot>("Slot", "Game::SlotDef");
|
|
check<sh::PlayerSettings>("PlayerSettings", "Game::StrategyPlayerGameSettings");
|
|
check<sh::PlayerColor>("PlayerColor", "Game::PlayerColorID");
|
|
check<sh::Session>("Session", "Game::StrategySessionParams");
|
|
check<sh::CreateParams>("CreateParams", "Game::StrategyGameCreateParams");
|
|
check<sh::Scrp>("Scrp", "Game::StrategyScriptParams");
|
|
check<sh::MapP>("MapP", "Game::StarMapParams");
|
|
check<sh::Planet>("Planet", "Game::SystemParams");
|
|
check<sh::Sim>("Sim", "Game::StrategyServer");
|
|
check<sh::Sys>("Sys", "Game::ServerSystem");
|
|
check<sh::Player>("Player", "Game::ServerPlayer");
|
|
check<sh::Fleet>("Fleet", "Game::StarFleet");
|
|
check<sh::Ship>("Ship", "Game::StarShip");
|
|
check<sh::FlightPlan>("FlightPlan", "Game::FlightPlan");
|
|
check<sh::Waypoint>("Waypoint", "Game::FlightPlan::Waypoint");
|
|
check<sh::PrisonerHold>("PrisonerHold", "Game::PrisonerHold");
|
|
check<sh::Otch>("Otch", "Game::ObservedTech");
|
|
check<sh::Owep>("Owep", "Game::ObservedWeapon");
|
|
check<sh::Odes>("Odes", "Game::ObservedDesign");
|
|
check<sh::Rts>("Rts", "Game::StarSystem::OutputRates");
|
|
check<sh::PlayerView>("PlayerView", "Game::StarSystem::PlayerView");
|
|
check<sh::IndependenceInfo>("IndependenceInfo", "Game::IndependenceInfo");
|
|
check<sh::BuildOrder>("BuildOrder", "Game::ShipBuildOrder");
|
|
check<sh::BuildQueue>("BuildQueue", "Game::BuildQueue");
|
|
check<sh::Prep>("Prep", "Game::PlayerReport");
|
|
check<sh::DipStat>("DipStat", "Game::DiplomacyStats");
|
|
check<sh::Alliances>("Alliances", "Game::PlayerAlliances");
|
|
check<sh::NodeRoute>("NodeRoute", "Game::NodeRoute");
|
|
check<sh::SystemEvent>("SystemEvent", "Game::SystemEvent");
|
|
check<sh::PlayerTurnStats>("PlayerTurnStats", "Game::PlayerTurnStats");
|
|
check<sh::PlayerTurnHistory>("PlayerTurnHistory", "Game::PlayerTurnHistory");
|
|
check<sh::TurnStats>("TurnStats", "Game::GameTurnHistory");
|
|
check<sh::PopG>("PopG", "Game::PopulationGroup");
|
|
check<sh::Population>("Population", "Game::Population");
|
|
check<sh::Morale>("Morale", "Game::Morale");
|
|
check<sh::MoraleEvent>("MoraleEvent", "Game::MoraleEvent");
|
|
// Game::SimpleNodePath is a different, 2-item type — it is what MapP.nodePaths
|
|
// holds (VectorHelper<SimpleNodePath>), and it is empty in every real save.
|
|
check<sh::NodePath>("NodePath", "Game::NodePath");
|
|
|
|
// Bodies typed this round from the recovered schema (previously opaque Nodes).
|
|
check<sh::EventStorage>("EventStorage", "Game::EventStorage");
|
|
check<sh::TurnEvents>("TurnEvents", "Game::EventStorage::TurnEvents");
|
|
check<sh::EventRec>("EventRec", "Game::EventStorage::Event");
|
|
check<sh::FleetNameGen>("FleetNameGen", "Game::FleetNameGenerator");
|
|
check<sh::SpeciesRatios>("SpeciesRatios", "Game::SpeciesRatios");
|
|
check<sh::CivilianRatios>("CivilianRatios", "Game::CivilianRatios");
|
|
check<sh::TechTree>("TechTree", "Game::TechTree");
|
|
check<sh::ShipRecords>("ShipRecords", "Game::ShipRecords");
|
|
check<sh::AIEncounterFlags>("AIEncounterFlags", "Game::AIEncounterFlags");
|
|
check<sh::PlayerAid>("PlayerAid", "Game::PlayerAid");
|
|
check<sh::CommMessages>("CommMessages", "Game::CommMessageContainer");
|
|
check<sh::SpyReport>("SpyReport", "Game::SpyReport");
|
|
check<sh::SpyManager>("SpyManager", "Game::ServerSpyManager");
|
|
check<sh::ProjectNames>("ProjectNames", "Game::SpecialProjectNameGen");
|
|
check<sh::TradeManager>("TradeManager", "Game::ServerTradeManagerImpl");
|
|
check<sh::TradeSector>("TradeSector", "Game::ServerTradeSector");
|
|
check<sh::ShipSectionID>("ShipSectionID", "Game::ShipSectionID");
|
|
check<sh::DesignSection>("DesignSection", "Game::ShipDesignDef::Section");
|
|
|
|
// --- SvSctOb: Game::SVSOSots and the variant bodies it dispatches to -------
|
|
// The variant SELECTION (xscn name / EncID -> class) is not on the wire and so
|
|
// is not checkable here; it came from the game's own factories. What this
|
|
// does check is that each body we bind agrees with that class's serializer.
|
|
check<sh::ScriptObjects>("ScriptObjects", "Game::SVSOSots");
|
|
check<sh::SVSOVonNeumann>("SVSOVonNeumann", "Game::SVSOVonNeumann");
|
|
check<sh::VonNeumannDefeat>("VonNeumannDefeat", "Game::SVSOVonNeumann::DefeatRecord");
|
|
check<sh::SVSOSwarm>("SVSOSwarm", "Game::SVSOSwarm");
|
|
check<sh::SwarmInfestation>("SwarmInfestation", "Game::SVSOSwarm::Infestation");
|
|
check<sh::SVSODerelict>("SVSODerelict", "Game::SVSODerelict");
|
|
check<sh::SVSOMonitor>("SVSOMonitor", "Game::SVSOMonitor");
|
|
check<sh::SVSOSlaversRefuel>("SVSOSlaversRefuel", "Game::SVSOSlaversRefuel");
|
|
check<sh::SVSOSwarmQueen>("SVSOSwarmQueen", "Game::SVSOSwarmQueen");
|
|
check<sh::SVSOSwarmQueenHive>("SVSOSwarmQueenHive", "Game::SVSOSwarmQueen::HiveInfo");
|
|
check<sh::SVSOSwarmQueenQueen>("SVSOSwarmQueenQueen", "Game::SVSOSwarmQueen::QueenInfo");
|
|
check<sh::SVSOCrowRuins>("SVSOCrowRuins", "Game::SVSOCrowRuins");
|
|
check<sh::SVSORefugees>("SVSORefugees", "Game::SVSORefugees");
|
|
check<sh::RefugeeStatus>("RefugeeStatus", "Game::SVSORefugees::PlayerStatus");
|
|
check<sh::SVSOTraps>("SVSOTraps", "Game::SVSOTraps");
|
|
check<sh::SVSOTrap>("SVSOTrap", "Game::SVSOTraps::Trap");
|
|
check<sh::SVSOCrowDefenders>("SVSOCrowDefenders", "Game::SVSOCrowDefenders");
|
|
check<sh::SVSOGrandMenaceTrigger>("SVSOGrandMenaceTrigger", "Game::SVSOGrandMenaceTrigger");
|
|
|
|
std::printf(
|
|
"\ntotals: %d shapes bound, %d items matched, %d MISMATCH, %d wire-only, "
|
|
"%d opaque item(s) in bound shapes\n",
|
|
bound, total_matched, total_mismatch, total_wire_only, total_opaque);
|
|
std::printf("table: %d item(s) the recovery itself could not type\n", unresolved_classes);
|
|
std::printf("test_wire_schema: %s (%d failures)\n", fails ? "FAILED" : "ok", fails);
|
|
return fails ? 1 : 0;
|
|
}
|