sots-re/tools/DASHBOARD_README.md

tools/dashboard.py — what every number means

Generates campaign/DASHBOARD.md from the campaign tracker, the findings, the verify artefacts and the sots-engine checkout. Pure Python 3 stdlib (/usr/bin/python3), no Ghidra, no network. Idempotent: run it after every integration; it overwrites the dashboard and computes the delta against the previous one before doing so.

tools/dashboard.py                 # write campaign/DASHBOARD.md
tools/dashboard.py --print         # also echo to stdout
tools/dashboard.py --no-write      # render to stdout only, touch nothing
tools/dashboard.py --engine PATH   # sots-engine checkout (default ~/sots-engine, or $SOTS_ENGINE)

Nothing is fatal: an unreadable or malformed input produces a ? or a missing row and an entry in the footer warnings: line. Check that line first when a number looks wrong.

Header

Item	Source
Generated timestamp	wall clock, UTC
sots-re commit	git log -1 --format=%h,%cs in this repo
sots-engine commit + count	same in the engine checkout; count = git rev-list --count HEAD
North star	bold sentence under ## North star in findings/00-strategy.md

1. Map coverage

Source: campaign/board.md, every line starting with | except the header and separator. Rows are split on | for the first six cells only (Target, Type, Status, Conf, Coverage, Updated); everything after is the Notes cell, so backticks or pipes in notes cannot break a row. Rows with fewer than seven cells are skipped with a warning; unknown statuses are counted but warned.

• mapped-or-better = mapped + verified (verified implies mapped).
• Type breakdown maps board Type values to rows: object → objects, control-flow, subsystem → subsystems, engine, verify, phase2, meta. Any other type appears as an other row plus a warning.

2. Binary understanding

Number	Source / heuristic
RTTI type descriptors	non-empty line count of findings/objects/rtti-raw.txt
Game:: / Mars:: classes	line counts of classes-game.txt / classes-mars.txt
serializable types	line count of serializable-types.txt — used as the "classes of interest" denominator because save-layout recovery targets exactly these
classes with recovered layouts	heuristic: distinct backticked Game::X / Mars::X names that appear in ##, ### or #### headings of findings/objects/struct-recovery.md and schema-gaps-resolved.md. Each such heading introduces a member table or a settled layout. Nested/POD helpers with their own heading count; classes mentioned only in prose or table cells do not. Over-counts slightly when a heading names a class whose layout is only partial (e.g. TechTree); under-counts classes recovered elsewhere (e.g. save-editor-structs.md).
functions	regex N functions in findings/01-fingerprint.md; falls back to the hardcoded 41,411 with a warning
named/annotated	number of entries[] in ghidra/addresses.json. This is the address contract handed to sots-engine, not Ghidra's total rename count (the Ghidra project has ~60 spine functions + 52 serializers + ~600 g_KEY labels renamed; those are not exported anywhere parseable).
addresses verified	entries whose status starts with verified (verified, verified-by-save, …)

3. Data layer

Number	Source
files parsed / total	Total: N parsed, M failed line in findings/subsystems/data-parsers.md (written by verify/parsers/verify.py); total = N + M (the 70 skipped HLSL/prose files are not data and are not counted)
block kinds	number of top-level keys in verify/results/data-catalogs/schema_stats.json
dangling cross-refs	sum of the lengths of every *_dangling list in crosslink.json (case-mismatch and "no requires" lists are informational and not counted)
oracle mars-parse / mars-text	the **total** row of the first table in each sots-engine/docs/mars-*.md whose header has a files column and an agree/oracle match column; reported as agree/files. Docs without such a row are warned about.
saves strict	count of *.sav in verify/results/saves/ and the strict exit E -- X errors, Y warnings phrase in the first three lines of turn2-strict-issues.txt; clean iff exit 0 and 0 errors. The N/N form assumes the issues file speaks for all real saves, as the board save_reader.py row states.
design rules	first N/N designs phrase in verify/design-rules/SHIP_DESIGN_RULES.md

4. Engine accrual

Modules = every directory under sots-engine/src/ that has its own CMakeLists.txt, plus any src/... directory whose sources are compiled directly from the root CMakeLists.txt (currently src/shim, built only under if(WIN32)). third_party/ is never a module.

Column	Source
LOC	wc -l equivalent over *.cpp *.h *.c in the module directory, excluding third_party
Test files	number of files under sots-engine/tests/<name>/ where <name> is the module path with / replaced by _ (mars/parse → tests/mars_parse), skipping build/
Checks	lines in those tests' *.cpp matching CHECK*(, REQUIRE*(, ASSERT*(, EXPECT*(, check(, assert(. Counts call sites, not executed assertions (a loop of checks counts once).
Wired	yes if add_subdirectory(src/<mod>) appears in the root CMakeLists.txt; direct (WIN32) if its sources are listed directly inside a WIN32 block; no otherwise
Doc	docs/<name-with-dashes>.md if it exists, else the first docs/*.md whose first five lines mention the module's leaf name (shim → M0.md)

"Verified & merged vs in flight" uses the board: engine:* rows by status; the bar is verified / all engine rows. In flight = in-progress.

5. Phase 2 milestones

Board rows whose Target starts with P2-M. Glyphs: ✅ verified or mapped, 🔄 in-progress, ⬜ backlog, ⛔ blocked. Notes are truncated to 90 characters.

6. Verification ledger

One line per evidence source: saves strict and design rules (as in section 3), each parsed oracle, verify/harness/compare/ directory present, verify/results/shim/m0.log present, and the board status of the determinism oracle row.

7. Open questions

campaign/open-questions.md: every bullet beginning - **. A bullet is resolved/parked if its bold text starts with RESOLVED, Resolved or (parked); everything else is open. "Most recent" = the last five open bullets in file order (the file is append-ordered), newest first, first 100 characters. Backlog counts are numbered or bulleted items under each ## heading of campaign/backlog.md.

8. Delta

The previous DASHBOARD.md carries a machine-readable footer comment <!-- dashboard-metrics {...} -->. It is read before the file is overwritten; the section shows old → new (±) for verified targets, mapped-or-better, engine LOC, test files, checks, addresses verified, recovered layouts and open questions. A dashboard without that comment (or the first run) reports "first run".

Adding a number

Parse it in its own small function that returns None on failure and calls warn(...), render it with a ? fallback, and document it here. Never let a missing file raise.