sots-re/verify/tooling/README.md

2.2 KiB

Reproducible local gate and turn measurements

Run on the developer host or a leased lab host; no shared remote overlay or reused build tree. Every output path must be new and outside source/input roots. Keep it in durable storage.

python3 tools/gate.py --engine /absolute/sots-engine \
  --corpus /absolute/saves --out /durable/new-run --profile host --shim --jobs 4

Host mode explicitly allows seven asset/trace-dependent skips listed in host-tests.json. All other tests must execute, and four corpus tests must report a positive count matching the input manifest. Full mode requires --profile full --data PATH --shim and no skipped tests; supply additional dependencies explicitly with repeated --sots-input SOTS_NAME=PATH. Inherited SOTS_* variables are discarded. Build uses the hashed/copied trace comparator, not a HOME-directory fallback. Source snapshot, tool snapshot, compiler settings, inputs, binary, shim, full commands, JUnit and positive execution summaries are retained. Truncated or missing output fails the gate.

python3 tools/standalone_report.py --binary /durable/new-run/build-host/src/app/sots_turn \
  --provenance /durable/new-run/manifest.json --out /durable/new-replay \
  --pair /absolute/input.sav /absolute/oracle.sav

Repeat --pair for more workloads. Use repeated --engine-arg=ARG for explicit runtime inputs such as --engine-arg=--data --engine-arg=/owner/assets or captured commands. File arguments are hashed; output/metric/roundtrip overrides and arbitrary positional arguments are rejected. Output saves and full residuals are retained. Binary, provenance and input mutation invalidates the run.

Valid divergence is measured; invalid execution is failed. --require-match additionally requires equal file/inflated/state results and exits nonzero on inequality. It NEVER produces campaign accepted status: only independent verification of the integrated contract can do that.

Publish with tools/select_evidence.py, then tools/dashboard.py. A selected snapshot is explicitly identified, not claimed to match every later working-tree edit. Source-bound contract promotion checks actual candidate/integrated content independently of HEAD.

Tests: python3 -m unittest discover -s verify/tooling -v.