56 lines
4.5 KiB
Markdown
56 lines
4.5 KiB
Markdown
# Astra integration followup — repair gate/report acceptance gaps
|
|
|
|
First session delivered useful initial tooling but incomplete enforcement. Same file ownership,
|
|
fresh bounded session openai/gpt-5.6-terra. Read independent-review.md if present and your checkpoint.
|
|
No delegation, commits, live lab operations or architecture changes. This is a precise Astra
|
|
approved repair contract, not permission to waive any of the initial requirements.
|
|
|
|
Fix these inspected defects and write adversarial tests proving them:
|
|
|
|
1. `corpusNonzero` cannot be absence of "0 save" in ctest --output-on-failure. Parse per-test
|
|
JUnit system-out positive summary counts for all four corpus tests; require count equals manifest
|
|
top-level saves count (tests scan top-level, do not hash nested unexecuted .sav as exercised).
|
|
Missing summaries, zero/mismatched count and duplicates fail. JUnit executed identity multiset
|
|
must exactly match discovered/expected inventory (including tests absent from XML -> fail).
|
|
2. Source copy integrity: compare every copied byte hash/mode to pre-copy snapshot BEFORE build.
|
|
`startswith('build')` excludes actual source filenames like build_turn.cpp. Exclude only top-level
|
|
known build output directories or explicitly named runtime artifact directories. Track deleted
|
|
files and symlink semantics or reject source symlinks explicitly. Snapshot includes untracked files.
|
|
3. Require tools versions and complete hashes for actual supplied SOTS_* file/dir dependencies,
|
|
assets and trace inputs. Explicitly sanitize inherited SOTS_* environment or validate/hash each
|
|
consumed input. Record roots, actual env settings, compiler/build settings and skip limitations.
|
|
4. --shim should be a boolean flag; use source cmake/toolchain-mingw-i686.cmake. Full requires
|
|
--data and shim build; --shim on host also builds. Check resulting DLL hash. Initial --shim PATH
|
|
requires directory but passes directory to CMAKE_TOOLCHAIN_FILE, unusable. Correct docs/help/tests.
|
|
5. RE identity mutation: don't hash transient campaign runtime/publication state as executable
|
|
source. Bind actual RE tooling inputs via immutable copies or hash list before+after; list excludes
|
|
only runtime/evidence output areas explicitly. Engine source must be stable before+after. Gate
|
|
should run while other independent campaign checkpoints change without falsely claiming tools
|
|
are mutable. Detect actual gate/checksum/inventory source mutation.
|
|
6. run/errors: subprocess exceptions need failed manifest, expected actual statuses must hold,
|
|
input roots/output roots must not overlap, jobs positive, binary must exist for passing gate.
|
|
All required checks explicitly present, not all(empty) success. `all` over strings is unsafe.
|
|
7. Standalone reporter: verify passed gate schema and binary hash/source identity thoroughly.
|
|
Preserve full stderr/stdout and complete paths/diffs. --roundtrip only if CLI semantics means
|
|
verify then SIMULATE (inspect src/app/main.cpp; don't accidentally test identity instead of turn).
|
|
Distinguish accepted ON SELECTED WORKLOADS from campaign replacement acceptance; reporter is
|
|
measurement by default and --accept file equality criterion explicitly documented. Hash all
|
|
engine-arg file inputs (commands/data) actually used, prevent extra args overriding --out/metric
|
|
or injecting --roundtrip-only/etc. Detect binary/input mutation after run too. Keep required
|
|
coverage and equality separation. Add negative-path tests missing pairs, binary mismatch,
|
|
malformed/failed manifest, missing coverage, pair failure, wrong reference and unexpected args.
|
|
|
|
Do not rerun integrated gate until lead says source stable. Unit tests and fixture negative paths
|
|
now; checkpoint exact manifest/CLI shape and results. Main will run fresh final gate later.
|
|
|
|
## Astra resolution of independent review R2
|
|
|
|
The equality reporter must NOT claim campaign acceptance. Replace --accept with --require-match
|
|
(no legacy alias). This flag only requires all selected pairs to match and affects exit status;
|
|
report status remains `measured` on a valid measurement, including when equality holds. Add
|
|
`matchRequired` and `allPairsMatch` booleans and an explicit scope statement. A failed required
|
|
match has status failed and retained residual evidence. ONLY campaign controls + independent
|
|
verdict/integrated criteria can produce accepted status. This removes no-op-as-accepted semantics
|
|
without pretending a general equality tool can infer whether a workload exercised research.
|
|
Test identical input/oracle can be measured but never gets `accepted`; full gate source validation
|
|
still required. Pilot provides required positive phase/branch controls separately.
|