223 lines
16 KiB
Markdown
223 lines
16 KiB
Markdown
# Canonical campaign controls
|
|
|
|
`sots-re/campaign/` is the state authority. Start with your contract, its checkpoint, open
|
|
surprises, and [current policy](../guides/multi-agent-workflow.md). Board/dashboard are generated
|
|
projections. Historical evidence is not current acceptance. No automatic commits or lab operations.
|
|
|
|
Framework development is complete. The `controls-bootstrap` contract reached scoped acceptance;
|
|
see [rollout result](rollout/RESULT.md) and its current contract/verdict for source-bound evidence.
|
|
Active work is reverse engineering. Change tooling only to unblock a named RE experiment.
|
|
|
|
## Contract format
|
|
|
|
`contract.schema.json` is strict JSON Schema (unknown fields and duplicate JSON keys fail).
|
|
The standard-library validator implements the schema's used subset. A populated example is
|
|
[contracts/controls-bootstrap.json](contracts/controls-bootstrap.json); its lifecycle is in the record.
|
|
|
|
Required fields:
|
|
|
|
| Field | Structure |
|
|
|---|---|
|
|
| `id`, `title`, `status` | Slug, short title, lifecycle state |
|
|
| `owner` | `{ "name": "worker-identity", "role": "implementer" }` |
|
|
| `baseline` | `{ "engine": {"path":"/absolute/canonical/engine","commit":"full-commit-id"}, "re": {"path":"/absolute/canonical/re","commit":"full-commit-id"} }` |
|
|
| `scope`, `inputs`, `effects` | Arrays of explicit nonempty strings; include full write set and runtime inputs |
|
|
| `original_dependencies` | String array, including original-assisted portions and unavailable inputs |
|
|
| `dependencies` | Array of other contract IDs; all must be accepted before ready/implementing |
|
|
| `acceptance` | Array of `{ "id": "unique-criterion", "axis": "validation-scope", "criterion": "executable requirement" }` |
|
|
| `predictions`, `stop_conditions` | String arrays of predictions and conditions that halt work |
|
|
| `checkpoint` | `null` or `campaign/runtime/checkpoints/<id>.json` |
|
|
|
|
Optional `evidence` is an array of
|
|
`{id,axis,path,sha256,source,integrated,source_binding,binaries,inputs,outcomes}`.
|
|
`path` is an existing canonical RE-relative artifact; `sha256` hashes its actual bytes; `source`
|
|
equals the contract's complete baseline object. Store understanding,
|
|
implementation, original dependencies, and validation scope as separate acceptance/evidence axes.
|
|
There is no generic `verified` scalar. Baseline commit IDs describe starting repositories;
|
|
dirty source identity is machine-bound by `source_binding`, never inferred from those commits.
|
|
Criteria need distinct states, branch exposure, positive execution, complete writes/elements,
|
|
allocations/IDs/events/RNG/runtime inputs, synthetic and original-game differentials as applicable.
|
|
The CLI checks package identity and declared axes; the independent reviewer evaluates the actual
|
|
criteria, gate outcomes, full manifests and integrated reproduction. A passing measurement alone
|
|
does not establish acceptance.
|
|
|
|
### Source-bound evidence interface (R4)
|
|
|
|
`source_binding` is `{engine:{path,commit,sha256},re:{path,commit,sha256}}`. Generate it with:
|
|
|
|
```sh
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re source-binding research-replacement --engine-worktree /absolute/candidate-engine --re-worktree /absolute/candidate-re
|
|
```
|
|
|
|
Omit both worktree arguments to bind the canonical integrated trees. Paths must be Git worktree
|
|
roots in the respective baseline repositories. `commit` is the actual current HEAD; `sha256`
|
|
is the deterministic digest of the actual file manifest, including dirty/untracked nonignored
|
|
files, deleted tracked paths (`null`), file bytes and Unix modes. Symlinks/submodules fail closed.
|
|
The fixed manifest policy uses `git ls-files --cached --others --exclude-standard`; ignored
|
|
untracked build/output files are not source. Python cache directories are excluded. In RE only,
|
|
`verify/results/` and `campaign/` are excluded **except** `campaign/models.json`,
|
|
`campaign/contract.schema.json`, and `campaign/agents/**`. These exclusions prevent mutable
|
|
contracts/checkpoints/evidence/projections from hashing themselves. Relevant RE tools, tests,
|
|
generated facts and guides remain bound. Any consumed item outside that source inventory must
|
|
appear among immutable input/binary artifacts. The independent reviewer checks inventory adequacy.
|
|
|
|
`binaries` and `inputs` are nonempty arrays of `{path,sha256}` artifact references; for tooling
|
|
contracts, bind the executable scripts/interpreter identity package and fixture input package.
|
|
`outcomes` exactly covers the acceptance criterion IDs for that evidence axis, with entries
|
|
`{criterion,status,artifact:{path,sha256}}`; promotion requires `status: "pass"`. Outcome artifacts
|
|
contain positive execution, branch/state exposures, reproduction recipe and required effect/input
|
|
accounting. The CLI checks identities, hashes and declared outcomes, **not arbitrary criterion
|
|
semantics**. The independent verifier must reproduce and challenge those claims.
|
|
|
|
For example, an outcome for the bootstrap contract is:
|
|
|
|
```json
|
|
{"criterion":"controls-negative-paths","status":"pass","artifact":{"path":"verify/results/controls/result.json","sha256":"<actual 64-hex artifact hash>"}}
|
|
```
|
|
|
|
Capture bindings when producing evidence; do not attach a fresh source hash to old measurements.
|
|
Every evidence/verdict/promotion check rehashes referenced sources and artifacts. Same-HEAD byte
|
|
changes reject old evidence and verdicts. Integrated records require canonical paths, lead in
|
|
integration state, and one identical binding across **all** final integrated evidence. A lead's
|
|
`integrated` boolean cannot substitute for this check. Verdicts bind the full evidence array and
|
|
the source-binding array; old verdicts lacking these identities must be reproduced.
|
|
|
|
This contract wrapper is separate from gate measurement schema **`sots-gate/1`**, whose `source`
|
|
still has `engine`/`re`. Reference its immutable manifest/binary/input package; do not rename its
|
|
fields to match contract `source`. Reporter output is measured evidence, with `--require-match`
|
|
for required equality, and gains acceptance only through independent contract/integration gates.
|
|
|
|
## State and transactions
|
|
|
|
Every command requires `--state-root /absolute/canonical/sots-re` (the repository, not `campaign/`).
|
|
No sibling inference. Control records stay below canonical `campaign/runtime/`; contracts remain
|
|
in `campaign/contracts/`. Immutable hashed artifacts may be referenced anywhere inside canonical
|
|
RE, including existing `verify/` corpora, without copying them. Absolute/traversing artifact paths,
|
|
outside symlinks, Git internals and named secret/private-key locations are rejected; aliases are
|
|
checked after resolution too. Never reference secrets or commit owner-supplied binaries/assets.
|
|
JSON writes are atomic and fsynced; a canonical `flock` serializes
|
|
CLI mutations, WIP decisions, and resource acquisition. Do not hand-edit active state concurrently
|
|
with commands. Interrupted multi-file operations retain blocking records and require inspection.
|
|
|
|
Runtime APIs (JSON files; no server):
|
|
|
|
- `runtime/checkpoints/*.json`: `sots-checkpoint/1`, contract, actor/role/model/session, timestamp,
|
|
contract `basis` digest, bounded summary (6000 characters), up to 32 `{path,sha256}` artifacts,
|
|
and one `next_action` (2000 characters). Include observations versus decisions, source identities,
|
|
tests, blockers, resources/access/cleanup, exact next action in the summary/artifacts.
|
|
Do not attach the checkpoint's own contract as an artifact: saving the pointer changes that
|
|
file. Its task metadata is already covered by `basis`; the CLI rejects this self-reference.
|
|
- `runtime/surprises/*.json`: `sots-surprise/1`, id, contract, `status: open|resolved`, summary,
|
|
discriminating probe, actor/model/session provenance where applicable, optional decision ID.
|
|
- `runtime/decisions/*.json`: `sots-decision/1`, Astra resolution, explanation/probe, invalidated
|
|
evidence and checkpoint; prior verdict is marked invalidated. Resolution returns needs-revision
|
|
only when all surprises are closed. Re-probe and rebuild evidence; resolution is not acceptance.
|
|
- `runtime/verdicts/<contract>.json`: independent verifier actor/session/model, pass/fail,
|
|
explanation, contract basis, complete evidence digest and source-bindings digest.
|
|
- `runtime/transitions/*.json`: actor/model, previous/next lifecycle state, timestamp.
|
|
- `runtime/leases/<resource>.json`: owner, random token, held/released, acquisition/release provenance.
|
|
- `runtime/runs/run-*.json`, `.jsonl`, `.stderr.log`: requested model/config, command, worktree
|
|
manifests before/after, expanded prompt hash, effective configuration hashes, canonical config
|
|
file hashes, actual events/session/model when emitted, completion/checkpoint status. Effective
|
|
provider config is hashed rather than persisted because it can contain credentials.
|
|
`active-<contract>.json` reserves the contract. Interrupted running reservations never auto-expire.
|
|
|
|
Lifecycle: `proposed -> ready -> implementing -> verification -> integration -> accepted`.
|
|
Blocked and needs-revision edges support repairs; no skipping stages. Ready requires scope,
|
|
inputs, acceptance, stop conditions, valid pinned baseline and accepted dependencies. Implementing
|
|
is atomically capped at two concurrent contracts; lead schedules only one pilot before enabling
|
|
two independent slices. Verification requires fresh checkpoint/artifacts after implementation start.
|
|
Integration requires lead plus independent passing verifier bound to current source/evidence.
|
|
Accepted requires every declared axis in integrated evidence, passing independent verdict over
|
|
that final package, and no open surprises. Adding integrated evidence changes the evidence digest:
|
|
the verifier must attest the integrated package again. Handoff/promotion/end checkpoints must be
|
|
within 15 minutes; recovery start has no age limit.
|
|
|
|
Role/model registry: lead/architecture-review/analyst/implementer/verifier/lab =
|
|
`openai/gpt-5.6-sol`; resolver = `openai/gpt-6-astra`.
|
|
CLI identity fields are **claims, not authenticated model authority**. The runner requests the
|
|
registry model explicitly and records emitted provenance. Editable JSON, agent permissions and
|
|
shell-accessible tooling are not a security boundary. No silent routing fallback.
|
|
|
|
## Commands
|
|
|
|
Run from either repository using the canonical tool path when necessary. Examples:
|
|
|
|
```sh
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re validate
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re list
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re status research-replacement
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re checkpoint controls-bootstrap --actor controls-architect --role architecture-review --model openai/gpt-6-astra --session rollout-controls --summary 'Source identities, observations, decisions, tests and blockers are in the attached checkpoint.' --artifact campaign/rollout/controls-worker-state.md --next-action 'Run the independent controls review.'
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re transition controls-bootstrap ready --actor controls-architect --role architecture-review --model openai/gpt-6-astra
|
|
```
|
|
|
|
`surprise CONTRACT --summary TEXT --probe TEXT` blocks immediately. `resolve SURPRISE_ID
|
|
--explanation TEXT --probe TEXT` requires claimed Astra lead/resolver. Both also require
|
|
`--actor NAME --role ROLE --model MODEL`. `evidence CONTRACT --record campaign/path.json`
|
|
uses the same identity flags; record format is the evidence object above. Integrated records
|
|
require lead and integration state. `verdict CONTRACT --session SESSION --verdict pass|fail
|
|
--explanation TEXT` requires verifier identity flags and independent actor/session.
|
|
|
|
```sh
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re lease acquire windows-vm --actor lab-one --role lab --model openai/gpt-5.5
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re lease show windows-vm
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re lease release windows-vm --actor lab-one --role lab --model openai/gpt-5.5 --token TOKEN_FROM_ACQUIRE
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re lease release windows-vm --actor lead --role lead --model openai/gpt-6-astra --lead-release --reason 'Confirmed prior operator stopped; access and cleanup checked.'
|
|
```
|
|
|
|
No stale lease stealing. Explicit lead release requires an explanation and lab preconditions,
|
|
side effects, cleanup, and access verification in the operator checkpoint. Treat lease tokens
|
|
as local owner capabilities, not secrets to put in a board/dashboard.
|
|
|
|
## Fresh bounded launches
|
|
|
|
Prepare **two actual linked worktrees**, each distinct from its canonical source repository,
|
|
at the contract's full baseline commit. No auto commits/worktree creation. Launch uses explicit
|
|
canonical `OPENCODE_CONFIG`, checks matching repo-local agent/model/40 steps, and sets the final
|
|
environment overlay to bind requested role/model/steps. Other inherited config overrides are
|
|
cleared. `opencode models` must list the exact requested model even for dry runs.
|
|
|
|
```sh
|
|
python3 tools/run_agent.py --state-root /home/alex/sots-re --role implementer --actor worker-one --contract slice-one --engine-worktree /home/alex/worktrees/slice-one-engine --re-worktree /home/alex/worktrees/slice-one-re --cwd engine --dry-run
|
|
```
|
|
|
|
Remove `--dry-run` to execute. Normal worker launch requires a valid durable checkpoint, matching
|
|
owner/role/status, no open surprises, baseline HEADs and canonical Git common-directory identity.
|
|
Recovery checks checkpoint identity/basis and artifact hashes regardless of age, rechecks any
|
|
source-bound evidence, and validates paired Git worktree/baseline identity. Missing ordinary-worker
|
|
state still blocks. Bootstrap lead/architecture-review can start without a checkpoint; they still
|
|
need paired worktrees. Astra lead/resolver may launch a blocked contract with open surprises and
|
|
without a worker checkpoint in **resolution-only** scope: read evidence and write decisions/state,
|
|
no implementation. Its prompt and permission overlay carry that limit, and worktree source changes
|
|
fail completion. Ordinary affected workers stay blocked. Other Astra architecture actors receive
|
|
explicit architecture authority within their owned scope. Each run is a fresh
|
|
`opencode run --format json --model ... --agent ...`; no resume/continue option is used. The prompt
|
|
supplies the run ID to use as checkpoint `--session`; actual OpenCode session IDs are captured
|
|
separately when emitted. On exit, a checkpoint after start matching actor/role/model/run ID is
|
|
mandatory or the run is marked incomplete. Completion additionally requires a zero exit, no
|
|
`type:error`, a successful `step_finish` with `part.reason: "stop"`, one nonempty actual session ID,
|
|
and consistent explicitly emitted model IDs. Text/tool-call/length events alone cannot complete a
|
|
run. Missing model emission is recorded `observed_model_status: "unavailable"`, never invented.
|
|
|
|
The runner checks the exact Git baseline again under reservation lock, rejects intervening contract
|
|
changes, and checks canonical model/prompt/config files and effective configuration for drift at
|
|
completion. Expanded role prompt/model/steps are frozen in the last-layer environment overlay.
|
|
`opencode debug config` runs in the actual launch cwd with the actual environment before execution
|
|
and again at completion. Permissions explicitly allow read/search, ordinary worker shell commands,
|
|
and external-directory access to the assigned paired trees plus canonical RE; role-specific edit
|
|
denies survive. Task delegation and interactive questions are denied. No `--auto` is required.
|
|
Noninteractive command/environment tests and a live loader check cover this overlay; an actual
|
|
normal worker launch smoke is a separate lead integration check. Permissions and before/after
|
|
checks are operational guards, not a sandbox or continuous filesystem audit.
|
|
Compaction is a backstop; no token threshold or machine-enforced 20-tool-call claim.
|
|
|
|
## Local verification
|
|
|
|
```sh
|
|
python3 -m unittest discover -s verify/campaign -p 'test_*.py' -v
|
|
python3 tools/campaign.py --state-root /home/alex/sots-re validate
|
|
```
|
|
|
|
Tests create commits only in disposable fixture repositories, linked worktrees and fake OpenCode processes; they do
|
|
not launch agents or touch the lab. Historical rollout artifacts retain the scope and source identity
|
|
of their runs; consult the accepted contract and current evidence, not an old handoff's status prose.
|