4.5 KiB
Astra integration followup — repair gate/report acceptance gaps
First session delivered useful initial tooling but incomplete enforcement. Same file ownership, fresh bounded session openai/gpt-5.6-terra. Read independent-review.md if present and your checkpoint. No delegation, commits, live lab operations or architecture changes. This is a precise Astra approved repair contract, not permission to waive any of the initial requirements.
Fix these inspected defects and write adversarial tests proving them:
corpusNonzerocannot be absence of "0 save" in ctest --output-on-failure. Parse per-test JUnit system-out positive summary counts for all four corpus tests; require count equals manifest top-level saves count (tests scan top-level, do not hash nested unexecuted .sav as exercised). Missing summaries, zero/mismatched count and duplicates fail. JUnit executed identity multiset must exactly match discovered/expected inventory (including tests absent from XML -> fail).- Source copy integrity: compare every copied byte hash/mode to pre-copy snapshot BEFORE build.
startswith('build')excludes actual source filenames like build_turn.cpp. Exclude only top-level known build output directories or explicitly named runtime artifact directories. Track deleted files and symlink semantics or reject source symlinks explicitly. Snapshot includes untracked files. - Require tools versions and complete hashes for actual supplied SOTS_* file/dir dependencies, assets and trace inputs. Explicitly sanitize inherited SOTS_* environment or validate/hash each consumed input. Record roots, actual env settings, compiler/build settings and skip limitations.
- --shim should be a boolean flag; use source cmake/toolchain-mingw-i686.cmake. Full requires --data and shim build; --shim on host also builds. Check resulting DLL hash. Initial --shim PATH requires directory but passes directory to CMAKE_TOOLCHAIN_FILE, unusable. Correct docs/help/tests.
- RE identity mutation: don't hash transient campaign runtime/publication state as executable source. Bind actual RE tooling inputs via immutable copies or hash list before+after; list excludes only runtime/evidence output areas explicitly. Engine source must be stable before+after. Gate should run while other independent campaign checkpoints change without falsely claiming tools are mutable. Detect actual gate/checksum/inventory source mutation.
- run/errors: subprocess exceptions need failed manifest, expected actual statuses must hold,
input roots/output roots must not overlap, jobs positive, binary must exist for passing gate.
All required checks explicitly present, not all(empty) success.
allover strings is unsafe. - Standalone reporter: verify passed gate schema and binary hash/source identity thoroughly. Preserve full stderr/stdout and complete paths/diffs. --roundtrip only if CLI semantics means verify then SIMULATE (inspect src/app/main.cpp; don't accidentally test identity instead of turn). Distinguish accepted ON SELECTED WORKLOADS from campaign replacement acceptance; reporter is measurement by default and --accept file equality criterion explicitly documented. Hash all engine-arg file inputs (commands/data) actually used, prevent extra args overriding --out/metric or injecting --roundtrip-only/etc. Detect binary/input mutation after run too. Keep required coverage and equality separation. Add negative-path tests missing pairs, binary mismatch, malformed/failed manifest, missing coverage, pair failure, wrong reference and unexpected args.
Do not rerun integrated gate until lead says source stable. Unit tests and fixture negative paths now; checkpoint exact manifest/CLI shape and results. Main will run fresh final gate later.
Astra resolution of independent review R2
The equality reporter must NOT claim campaign acceptance. Replace --accept with --require-match
(no legacy alias). This flag only requires all selected pairs to match and affects exit status;
report status remains measured on a valid measurement, including when equality holds. Add
matchRequired and allPairsMatch booleans and an explicit scope statement. A failed required
match has status failed and retained residual evidence. ONLY campaign controls + independent
verdict/integrated criteria can produce accepted status. This removes no-op-as-accepted semantics
without pretending a general equality tool can infer whether a workload exercised research.
Test identical input/oracle can be measured but never gets accepted; full gate source validation
still required. Pilot provides required positive phase/branch controls separately.