Merge feat/sf-session-01: the session-types contract crate and package, and the synthetic lockstep session over the bus
This commit is contained in:
commit
f48cab5258
91 changed files with 32943 additions and 1 deletions
|
|
@ -37,6 +37,15 @@ retroactively to existing [public feed](../../feed-protocol.md),
|
|||
sentence against bus-v1, with the test that proves each row, the measurements and the
|
||||
draft's own contradictions. A review artifact, not a contract.
|
||||
|
||||
Two derived specifications, written by CONTRACT-01 because the slices that need them cannot
|
||||
be built without them:
|
||||
|
||||
- [Seed derivation v1](seed-derivation-v1.md) — independent per-agent seeds from one recorded
|
||||
master seed and stable agent ids, with test vectors in both languages.
|
||||
- [Checkpoint envelope v1](checkpoint-envelope-v1.md) — the exact bytes of the new `FLYSESS1`
|
||||
envelope and the durable commit sequence. `FLYSIM01` is unchanged and stays separately
|
||||
readable.
|
||||
|
||||
For context: [modular-session analysis](../malecns-modular-sessions.md) and
|
||||
[Melee audit](../melee-framework-audit.md). Each contract owns its named subject; step ordering
|
||||
wins over an informal diagram, and Flybus owns transport/resource rules. Resolve contradictions
|
||||
|
|
|
|||
181
docs/design/session-framework/checkpoint-envelope-v1.md
Normal file
181
docs/design/session-framework/checkpoint-envelope-v1.md
Normal file
|
|
@ -0,0 +1,181 @@
|
|||
# Checkpoint envelope v1: `FLYSESS1`
|
||||
|
||||
Status: **draft 1**, 2026-09-22. Specified by CONTRACT-01 of the
|
||||
[implementation guide](implementation.md); required by
|
||||
[session artifacts, native media and recovery](state-media-v1.md) section 4, which says to
|
||||
"use a new envelope version; specify exact byte layout before production files". Reference
|
||||
implementations of the layout: `services/flysim/crates/fly-session-types/src/checkpoint.rs`
|
||||
and `packages/session-types/src/checkpoint.ts`; fixture:
|
||||
`.../fly-session-types/fixtures/checkpoint-envelope.json`.
|
||||
|
||||
This is the byte layout and the durable commit sequence. The store itself, generations,
|
||||
rotation, the writer thread and the capture RPC flow are the STATE-01 slice.
|
||||
|
||||
## 1. Why a new format
|
||||
|
||||
The historical envelope (`FLYSIM01`, `crates/flybrain-core/src/envelope.rs`) is a magic, a
|
||||
`u32` manifest length, a JSON manifest, `u32`-prefixed chunks in manifest order and a CRC32
|
||||
footer, with chunk names restricted to ASCII letters so the TypeScript reader can never name a
|
||||
prototype key. It stays exactly as it is, and its reader stays separately readable: nothing in
|
||||
this document changes a byte of it, and a `FLYSIM01` file is refused by a `FLYSESS1` reader at
|
||||
the magic.
|
||||
|
||||
A coherent all-participant session checkpoint needs what that format does not have:
|
||||
|
||||
- payload names that are `Id`s (`agent-fly-a`, `executor-fly-a`), so the letters-only
|
||||
constraint is widened **deliberately, in a new version**, rather than quietly;
|
||||
- a per-payload content digest, because state-media-v1 section 1 makes digests mandatory on
|
||||
checkpoint payloads and a group install must be able to fail one participant's bytes;
|
||||
- a payload table with explicit offsets and lengths, so a reader can map one participant's
|
||||
payload without walking every preceding chunk;
|
||||
- SHA-256 over the whole prefix instead of CRC32, matching the `Digest` type these contracts
|
||||
already use everywhere else.
|
||||
|
||||
## 2. Byte layout
|
||||
|
||||
All integers are unsigned little-endian. All digests are raw 32-byte SHA-256 (the manifest
|
||||
records the same digests as lowercase hex `Digest` strings).
|
||||
|
||||
### Header, 32 bytes
|
||||
|
||||
| Offset | Size | Field |
|
||||
| ---: | ---: | --- |
|
||||
| 0 | 8 | Magic, ASCII `FLYSESS1` |
|
||||
| 8 | 4 | `envelopeVersion`, `1` |
|
||||
| 12 | 4 | `headerBytes`, `32` |
|
||||
| 16 | 4 | `manifestBytes` |
|
||||
| 20 | 4 | `payloadCount`, at most 64 |
|
||||
| 24 | 4 | `tableOffset` |
|
||||
| 28 | 4 | Reserved, must be zero |
|
||||
|
||||
### Manifest
|
||||
|
||||
`manifestBytes` bytes of canonical JSON (RFC 8785) at offset 32, no trailing newline. It is
|
||||
canonical so the envelope's own digest is stable under reserialization, and a reader rejects a
|
||||
manifest that is not already canonical rather than silently accepting a second spelling.
|
||||
|
||||
### Payload table
|
||||
|
||||
At `tableOffset`, which is `32 + manifestBytes` rounded up to a multiple of 8.
|
||||
`payloadCount` entries of 112 bytes each, in write order:
|
||||
|
||||
| Offset in entry | Size | Field |
|
||||
| ---: | ---: | --- |
|
||||
| 0 | 64 | Name: an `Id` in ASCII, NUL-padded, no bytes after the terminator |
|
||||
| 64 | 8 | `offset` |
|
||||
| 72 | 8 | `byteLength` |
|
||||
| 80 | 32 | SHA-256 of exactly `byteLength` bytes at `offset` |
|
||||
|
||||
### Payloads
|
||||
|
||||
Each payload starts at its declared offset. The first starts at the end of the table rounded
|
||||
up to a multiple of 8; each subsequent one starts at the previous payload's end rounded up the
|
||||
same way. Padding bytes are zero. Offsets are ascending and non-overlapping, which a reader
|
||||
checks rather than assumes.
|
||||
|
||||
### Footer, 48 bytes
|
||||
|
||||
| Offset from end | Size | Field |
|
||||
| ---: | ---: | --- |
|
||||
| 48 | 8 | `fileBytes`, the total length including the footer |
|
||||
| 40 | 32 | SHA-256 of every byte before the footer |
|
||||
| 8 | 8 | Magic, ASCII `FLYSESSF` |
|
||||
|
||||
A truncated file therefore fails at the footer magic or the recorded length, not at an
|
||||
arbitrary payload.
|
||||
|
||||
## 3. Manifest fields
|
||||
|
||||
State-media-v1 section 4 lists what the manifest records. The names below are the JSON field
|
||||
names; a manifest missing any of them is not a complete checkpoint.
|
||||
|
||||
| Field | Contents |
|
||||
| --- | --- |
|
||||
| `envelopeVersion` | `1` |
|
||||
| `checkpointId` | `Id`, the identity every participant's capture shares |
|
||||
| `sourceScope` | `Scope`: session, epoch and the committed step |
|
||||
| `episodeId` | `Id` |
|
||||
| `worldTime` | `RationalNs`, the environment's logical time at that boundary |
|
||||
| `schedulerId` | The coordinator's scheduler identity, `lockstep-v1` in v1 |
|
||||
| `compositionDigest` | Coordinator scheduler and configuration identity |
|
||||
| `portMap` | The exact port-to-agent map, `[{portId, agentId}]` |
|
||||
| `compatibility` | Backend, content, patch, controller, parser and state-format identities |
|
||||
| `agents` | Per agent: profile, dataset and model identities, resolved seed, tick count, remainder and the payload name holding its state |
|
||||
| `coordinator` | Task ledger, prior world inspection, per-agent executor state, admission state and event watermarks, each as a payload name or an inline value |
|
||||
| `helperState` | External-helper state required for exact resume, as payload names |
|
||||
| `payloads` | `[{name, byteLength, digest}]`, mirroring the payload table |
|
||||
|
||||
`payloads` is redundant with the table on purpose: the table is what a reader needs to map
|
||||
bytes, and the manifest is what a store lists, compares and reports without opening the
|
||||
payload area. A reader checks that the two agree.
|
||||
|
||||
What the manifest must **not** contain (state-media-v1 section 4): a transient bus `storeId`,
|
||||
artifact ID, owner token, mapping or pointer. Payload bytes and durable content identity are
|
||||
the only things that survive; on restore the durable store imports fresh bus artifacts, and
|
||||
`sourceScope` is provenance, not a claim on the current router.
|
||||
|
||||
## 4. What a reader enforces
|
||||
|
||||
In this order, so a corrupt file fails on its own terms rather than on a derived value:
|
||||
|
||||
1. Length at least header plus footer; magic; version; `headerBytes`; reserved word zero.
|
||||
2. Footer magic, `fileBytes` equal to the actual length, and the prefix digest.
|
||||
3. Manifest inside the payload area, valid strict JSON (duplicate keys, invalid UTF-8 and
|
||||
non-finite numbers refused) and already canonical.
|
||||
4. `tableOffset` exactly at the laid-out position; the table inside the payload area.
|
||||
5. Per entry: an `Id` name with no bytes after its terminator, names unique, the declared
|
||||
offset exactly at the aligned end of the previous payload, the payload inside the payload
|
||||
area, and its digest matching its bytes.
|
||||
6. No padding between the last payload and the footer.
|
||||
7. The required manifest field set, `envelopeVersion` of 1, and a `payloads` list that matches
|
||||
the table name for name, length for length and digest for digest.
|
||||
|
||||
Failing any of these is a corrupt or foreign file. The group install rule of state-media-v1
|
||||
section 5 then applies: corrupt any participant and installation fails as a group.
|
||||
|
||||
## 5. Durable commit
|
||||
|
||||
State-media-v1 section 6, in the order the writer performs it:
|
||||
|
||||
1. Write the envelope to a temporary generation file in the store directory.
|
||||
2. `fsync` the file.
|
||||
3. `rename` it to its final generation name.
|
||||
4. `fsync` the store directory.
|
||||
5. Write the store manifest to its own temporary file, `fsync`, `rename`, `fsync` the
|
||||
directory.
|
||||
|
||||
**The store manifest rename is the durable commit point.** Before it, the generation file is
|
||||
an unreferenced temporary that is never a restore candidate. After it, and only after it, the
|
||||
writer reports a saved acknowledgment and moves the high-water mark.
|
||||
|
||||
Consequences the writer must respect rather than reinterpret:
|
||||
|
||||
- Bus publications for `captured`, `queued`, `committed`, `failed` and `superseded` are
|
||||
distinct events; only durable completion produces the saved acknowledgment.
|
||||
- A lost save reply never advances durable metadata: the coordinator resolves the same
|
||||
operation or fails the epoch, and an unreferenced generation stays unreferenced.
|
||||
- A failed write releases its owned ephemeral captures under the configured retry policy and
|
||||
reports the failure. It never reports false durability.
|
||||
- The writer owns the bus artifact handles until the bytes are committed or the job fails, and
|
||||
drops them afterwards; durable files are outside the bus's ephemeral collection.
|
||||
- No per-payload `fsync` inside one envelope: the single file `fsync` in step 2 covers it.
|
||||
|
||||
## 6. Fixture
|
||||
|
||||
`fixtures/checkpoint-envelope.json` holds one complete envelope: the manifest, five payloads
|
||||
(one agent, one executor, the task ledger, the prior inspection and a world payload), the
|
||||
envelope's base64 bytes, its exact layout (header size, manifest offset and length, table
|
||||
offset, every entry's offset, length and digest, footer offset, total length) and six
|
||||
corruptions a reader must refuse, each naming the byte to flip.
|
||||
|
||||
The two implementations are held to it from both directions: each parses the fixture and
|
||||
checks every recorded offset, and the TypeScript side re-encodes the same manifest and
|
||||
payloads and requires the bytes to be identical to the fixture. A layout change that only one
|
||||
language makes therefore fails on the next test run.
|
||||
|
||||
## 7. Out of scope
|
||||
|
||||
Generations, rotation, hot versus durable copies, the capture queue and its bounds, the
|
||||
`State.Capture` / `State.StageRestore` / `State.ActivateRestore` flow, compatibility
|
||||
comparison rules and group fencing. Those are STATE-01, over this layout. `FLYSIM01` and the
|
||||
legacy composition keep their own format and their own reader, unchanged.
|
||||
|
|
@ -154,6 +154,11 @@ Lifecycle/capture replies are retained until Worker.Acknowledge:
|
|||
not another consumer's bus delivery. Already released/unknown IDs are ignored. Serial
|
||||
watermarks reject reuse after acknowledgment without an unbounded tombstone list.
|
||||
|
||||
**Amendment, 2026-09-22 (CONTRACT-01):** those ids are domain request ids in the `req-<U64>`
|
||||
serial form, not arbitrary `Id`s. The serial watermark rule in the sentence above cannot reject
|
||||
reuse after acknowledgment unless the acknowledged id carries its serial, so a bus callId or a
|
||||
bare `Id` is refused there.
|
||||
|
||||
Bound unacknowledged lifecycle replies at 16, then BUSY before application. Status and
|
||||
Acknowledge use a cache of their last 16 replies; current/previous step records have their
|
||||
separate finite retention. Caches containing big artifacts consume bus owner/byte budgets;
|
||||
|
|
|
|||
101
docs/design/session-framework/seed-derivation-v1.md
Normal file
101
docs/design/session-framework/seed-derivation-v1.md
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
# Seed derivation v1
|
||||
|
||||
Status: **draft 1**, 2026-09-22. Specified by CONTRACT-01 of the
|
||||
[implementation guide](implementation.md), required by
|
||||
[worker interfaces](workers-v1.md) section 2 before the real-agent slice. Reference
|
||||
implementations: `services/flysim/crates/fly-session-types/src/seed.rs` and
|
||||
`packages/session-types/src/seed.ts`; test vectors:
|
||||
`services/flysim/crates/fly-session-types/fixtures/seed-vectors.json`.
|
||||
|
||||
## 1. What this is for
|
||||
|
||||
`Agent.Initialize` takes `seed`, a signed 32-bit integer, matching the current RNG input.
|
||||
Workers-v1 section 2 requires that the coordinator derive independent per-agent seeds from
|
||||
**its recorded master seed and stable agent IDs** under a versioned algorithm, and that the
|
||||
algorithm be specified and tested before the real agent slice. This is that algorithm.
|
||||
|
||||
It is a reproducibility rule, not a secret: a run manifest records the master seed in the
|
||||
clear, and anyone with the manifest can recompute every agent's seed. It is not a key
|
||||
derivation function and must not be used as one.
|
||||
|
||||
`seed-derivation-v1` is part of composition identity. Changing any byte of it requires a new
|
||||
identifier (`seed-derivation-v2`), because two runs that agree on every other identity but
|
||||
disagree here are not the same experiment.
|
||||
|
||||
## 2. Inputs
|
||||
|
||||
| Input | Type | Source |
|
||||
| --- | --- | --- |
|
||||
| `masterSeed` | `U64` decimal string | Recorded once per run by the application/supervisor |
|
||||
| `agentId` | `Id` | The configured agent identity, stable across restarts and epochs |
|
||||
|
||||
Both are the ipc-v1 section 2 scalars. An `agentId` that is not an `Id` is an error, not
|
||||
something to normalize. The master seed is the whole 64-bit range: a 32-bit master seed would
|
||||
be no wider than the seed it derives.
|
||||
|
||||
## 3. Derivation
|
||||
|
||||
```text
|
||||
material = "flybrain/seed-derivation-v1" LF masterSeed LF agentId LF
|
||||
digest = SHA-256(material)
|
||||
lanes = digest read as eight big-endian uint32 values, in order
|
||||
seed = the first nonzero lane, reinterpreted as a two's-complement int32
|
||||
```
|
||||
|
||||
`LF` is one `0x0a` byte. `masterSeed` is its canonical decimal form: `"0"`, or no leading
|
||||
zero. The prefix is a domain separator, so a digest from this algorithm can never collide with
|
||||
one taken over some other pair of strings.
|
||||
|
||||
Zero lanes are skipped because the pinned kernel's RNG is an xorshift generator, whose state
|
||||
must not be zero: a derivation that could hand out `0` would silently produce a stalled
|
||||
generator. If every one of the eight lanes were zero, the material is rehashed with a counter
|
||||
suffix (`material || "1" LF`, then `"2" LF`, then `"3" LF`) and the search repeats; no input
|
||||
has ever needed it, and four rounds exhausted is an error rather than a fallback seed.
|
||||
|
||||
The seed is the *negative* number when the lane's high bit is set. That is deliberate: the
|
||||
existing RNG input is a signed 32-bit integer, and half the range is negative.
|
||||
|
||||
## 4. Properties
|
||||
|
||||
- **Deterministic.** The seed is a function of the two recorded inputs and nothing else: not
|
||||
of wall time, agent order, port assignment, worker process or thread count.
|
||||
- **Independent per agent.** Distinct agent IDs give unrelated seeds; there is no arithmetic
|
||||
relationship between `fly-a` and `fly-b` for a caller to exploit or accidentally rely on.
|
||||
- **Stable across recovery.** Restore, episode reset and a new epoch do not re-derive a
|
||||
different seed for the same agent ID under the same master seed. The seed is persisted as run
|
||||
configuration and state, and the capture compatibility digest covers the resolved seed
|
||||
(workers-v1 section 2), so a checkpoint cannot be installed into a differently seeded
|
||||
instance.
|
||||
- **Equal IDs give equal seeds.** That is the only way to get identical seeds, and workers-v1
|
||||
allows identical seeds only when an experiment declares them. A composition therefore
|
||||
refuses a repeated agent ID rather than quietly sharing a seed between two agents.
|
||||
|
||||
Non-properties, stated so nobody assumes them: this is not uniform over the int32 range beyond
|
||||
what SHA-256 gives, it is not a stream (one seed per agent per run, not per step), and it says
|
||||
nothing about how a model consumes its seed.
|
||||
|
||||
## 5. Test vectors
|
||||
|
||||
`fixtures/seed-vectors.json` carries the full table: five master seeds (`0`, `1`, `42`, `2^63`
|
||||
and the `U64` maximum) across four agent IDs, each with the exact material string, its SHA-256
|
||||
and the derived seed, plus one four-agent composition and the inputs that must be refused.
|
||||
Both implementations reproduce every row, and each records the material as well as the seed so
|
||||
a third implementation can find where it diverges.
|
||||
|
||||
The first two rows:
|
||||
|
||||
| masterSeed | agentId | material | seed |
|
||||
| --- | --- | --- | ---: |
|
||||
| `0` | `fly-a` | `flybrain/seed-derivation-v1\n0\nfly-a\n` | 1828176714 |
|
||||
| `0` | `fly-b` | `flybrain/seed-derivation-v1\n0\nfly-b\n` | 1218785088 |
|
||||
|
||||
Refused: an agent ID that is not an `Id` (uppercase, empty, over 64 characters), a master seed
|
||||
that is not a canonical `U64`, and a composition with a repeated agent ID.
|
||||
|
||||
## 6. Out of scope
|
||||
|
||||
Choosing the master seed, recording it in the run manifest, and the hand-selected explicit
|
||||
seeds that workers-v1 allows for the first synthetic composition. This document defines only
|
||||
the derivation. A profile that needs several independent streams inside one agent derives them
|
||||
from the agent's own seed under its own documented rule; that is a profile concern, not a
|
||||
session one.
|
||||
|
|
@ -30,6 +30,7 @@ Starting → Ready(k) → Preparing(k) → Applying(k) → Observing(k+1)
|
|||
|
||||
Ready(k) → Paused(k) → Ready(k)
|
||||
Ready(k) / Paused(k) → Capturing(k) → same boundary
|
||||
pause requested mid-step → Committing(k) → Ready(k+1) → Paused(k+1)
|
||||
any unresolved partial failure → Failed → Restoring(new epoch) → Paused(k)
|
||||
terminal episode → Paused(k) → Resetting(new epoch) → Ready(0)
|
||||
```
|
||||
|
|
@ -38,6 +39,12 @@ terminal episode → Paused(k) → Resetting(new epoch) → Ready(0)
|
|||
transition carry `scope.step=k`; result fields identify `nextStep=k+1` where applicable.
|
||||
Do not send Agent.Commit with step `k+1` merely because the observation is newer.
|
||||
|
||||
**Amendment, 2026-09-22.** The mid-step pause line above adds no new edge: a pause requested
|
||||
during a transition is served by the ordinary `Committing(k) → Ready(k+1)` edge followed by
|
||||
`Ready(k+1) → Paused(k+1)`. It is written into the machine because section 6 requires the
|
||||
transition to finish first, so the only boundary such a pause can land on is the one the
|
||||
transition just committed.
|
||||
|
||||
## 3. Transaction sequence
|
||||
|
||||
### Phase A: prepare all agents concurrently
|
||||
|
|
|
|||
18
package-lock.json
generated
18
package-lock.json
generated
|
|
@ -14,6 +14,7 @@
|
|||
"apps/stage": {
|
||||
"name": "@flybrain/stage",
|
||||
"version": "0.1.1",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@flybrain/brain": "*",
|
||||
"@flybrain/feed": "*",
|
||||
|
|
@ -892,6 +893,10 @@
|
|||
"resolved": "packages/feed",
|
||||
"link": true
|
||||
},
|
||||
"node_modules/@flybrain/session-types": {
|
||||
"resolved": "packages/session-types",
|
||||
"link": true
|
||||
},
|
||||
"node_modules/@flybrain/stage": {
|
||||
"resolved": "apps/stage",
|
||||
"link": true
|
||||
|
|
@ -3666,6 +3671,7 @@
|
|||
"packages/brain": {
|
||||
"name": "@flybrain/brain",
|
||||
"version": "0.1.1",
|
||||
"license": "Apache-2.0",
|
||||
"devDependencies": {
|
||||
"@types/node": "22.17.0",
|
||||
"@types/three": "0.178.1",
|
||||
|
|
@ -3685,6 +3691,7 @@
|
|||
"packages/feed": {
|
||||
"name": "@flybrain/feed",
|
||||
"version": "0.1.0",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"ws": "8.21.3"
|
||||
},
|
||||
|
|
@ -3696,9 +3703,20 @@
|
|||
"typescript": "5.9.2"
|
||||
}
|
||||
},
|
||||
"packages/session-types": {
|
||||
"name": "@flybrain/session-types",
|
||||
"version": "0.1.0",
|
||||
"license": "Apache-2.0",
|
||||
"devDependencies": {
|
||||
"@types/node": "22.17.0",
|
||||
"tsx": "4.20.3",
|
||||
"typescript": "5.9.2"
|
||||
}
|
||||
},
|
||||
"services/bridge": {
|
||||
"name": "@flybrain/bridge",
|
||||
"version": "0.1.0",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@flybrain/feed": "0.1.0",
|
||||
"@twurple/api": "8.1.4",
|
||||
|
|
|
|||
76
packages/session-types/README.md
Normal file
76
packages/session-types/README.md
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
# @flybrain/session-types
|
||||
|
||||
The session framework contracts in TypeScript: types, validation, canonical JSON (RFC 8785)
|
||||
and canonical digests.
|
||||
|
||||
The other half of [`services/flysim/crates/fly-session-types`](../../services/flysim/crates/fly-session-types).
|
||||
Same rules, same canonical bytes, same digests, and the same fixture corpus: this package
|
||||
loads the crate's `fixtures/` directory rather than keeping a copy, so a case written once
|
||||
holds both languages to it. Nothing here opens a socket; it reads, validates and hashes.
|
||||
|
||||
This is the internal session path (`docs/design/session-framework/`). The public feed and
|
||||
control contracts are unchanged and still live in [`@flybrain/feed`](../feed).
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | Contents |
|
||||
| --- | --- |
|
||||
| `canonical` | `canonicalize`, `digestOf`, `parseStrict`, `requireEnvelopeFit`, `rejectBusIdentities` |
|
||||
| `scalar` | `Id`, `U64`, `Digest`, `Scope`, `RationalNs` with checked arithmetic, and the four identities as branded types |
|
||||
| `reader` | `Reader`, which reads one object field by field and then refuses any field it did not read |
|
||||
| `common` | `readScope`, `readSchemaRef`, `readTypedValue`, `operationKeyDigest`, `bodyDigest` |
|
||||
| `media` | View and audio descriptors and refs, and the `State.*` payloads |
|
||||
| `workers` | The closed enums and every Agent/Environment/Worker method payload |
|
||||
| `rpc` | `SessionRpcRequest`, the success and failure replies, `ErrorCode`, `MutationCertainty` |
|
||||
| `publishing` | `SessionDescriptor`, `CommittedSnapshot` |
|
||||
| `trace` | The step-v1 section 8 record and the behaviour-only comparator |
|
||||
| `seed` | `seed-derivation-v1` |
|
||||
| `checkpoint` | The `FLYSESS1` envelope layout |
|
||||
| `fixtures` | Loading the shared corpus |
|
||||
|
||||
## Reading a payload
|
||||
|
||||
Every reader takes `unknown`, validates, and hands back a value whose fields are exactly the
|
||||
ones it read. A payload with an unknown or misspelled field fails instead of silently
|
||||
defaulting, and a round trip through a reader is the test that no field is dropped.
|
||||
|
||||
```ts
|
||||
import { canonicalize, digestOf, readScope, readPrepareParams, bodyDigest } from '@flybrain/session-types';
|
||||
|
||||
const scope = readScope(payload.scope);
|
||||
const params = readPrepareParams(payload.params);
|
||||
const digest = bodyDigest('Agent.Prepare', scope, params); // the ipc-v1 section 5 comparison
|
||||
```
|
||||
|
||||
Rules that need another value in hand are separate functions, because a payload cannot check
|
||||
them alone: `validatePortControlAgainst`, `validateBatch`, `validateSensoryInputAgainst`,
|
||||
`validateObservationAgainst`, `validateStepResultAgainst`, `validateSnapshotAgainst`,
|
||||
`validateTelemetryRoles`, `validateRemainder`, `validateCommitAgainstScope`.
|
||||
|
||||
## Canonical JSON
|
||||
|
||||
Three rules make the two implementations agree byte for byte:
|
||||
|
||||
- object keys sort by UTF-16 code unit, which is what comparing JavaScript strings does;
|
||||
- numbers print with `String(number)`, the ECMAScript algorithm RFC 8785 requires;
|
||||
- a number is canonicalizable when it is finite and, if integral, no larger in magnitude than
|
||||
`Number.MAX_SAFE_INTEGER`. Larger integers are refused rather than rounded: every counter
|
||||
and clock in these contracts is a `U64` decimal string. The rule is on the value, not on how
|
||||
it was written, because `JSON.parse` cannot tell `1e21` from the same digits written out.
|
||||
|
||||
`parseStrict` is a small recursive-descent parser rather than a wrapper around `JSON.parse`,
|
||||
which keeps the last of two duplicate keys instead of failing.
|
||||
|
||||
Digests use `node:crypto`. This package is contract tooling for services and tests, not
|
||||
browser code; the presentation layer consumes the public feed package instead.
|
||||
|
||||
## Tests
|
||||
|
||||
```sh
|
||||
npm test --workspace @flybrain/session-types
|
||||
npm run typecheck --workspace @flybrain/session-types
|
||||
```
|
||||
|
||||
Nine files, all fixture-driven. The one that says the most about the two implementations is in
|
||||
`tests/checkpoint.test.ts`: a `FLYSESS1` envelope written here is byte-identical to the one the
|
||||
Rust crate wrote into the fixture.
|
||||
22
packages/session-types/package.json
Normal file
22
packages/session-types/package.json
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
{
|
||||
"name": "@flybrain/session-types",
|
||||
"version": "0.1.0",
|
||||
"description": "TypeScript types, validation, canonical JSON (RFC 8785) and canonical digests for the session framework contracts, sharing the fixture corpus of services/flysim/crates/fly-session-types.",
|
||||
"license": "Apache-2.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "./src/index.ts",
|
||||
"types": "./src/index.ts",
|
||||
"exports": {
|
||||
".": "./src/index.ts"
|
||||
},
|
||||
"scripts": {
|
||||
"test": "node --import tsx --test tests/**/*.test.ts",
|
||||
"typecheck": "tsc -p tsconfig.json --pretty false"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "22.17.0",
|
||||
"tsx": "4.20.3",
|
||||
"typescript": "5.9.2"
|
||||
}
|
||||
}
|
||||
381
packages/session-types/src/canonical.ts
Normal file
381
packages/session-types/src/canonical.ts
Normal file
|
|
@ -0,0 +1,381 @@
|
|||
/**
|
||||
* Canonical JSON (RFC 8785), strict parsing and canonical digests.
|
||||
*
|
||||
* The Rust crate `services/flysim/crates/fly-session-types` is the other half of this
|
||||
* contract; `fixtures/valid.json` records the canonical bytes and digest of every accepted
|
||||
* payload, and both languages assert against it.
|
||||
*
|
||||
* Three rules make the two agree:
|
||||
*
|
||||
* - object keys sort by UTF-16 code unit, which is what comparing JavaScript strings does;
|
||||
* - numbers print with `String(number)`, the ECMAScript algorithm RFC 8785 requires;
|
||||
* - a number is canonicalizable when it is finite and, if integral, no larger in magnitude
|
||||
* than `Number.MAX_SAFE_INTEGER`. Larger integers are refused rather than rounded: every
|
||||
* counter and clock in these contracts is a `U64` decimal string. The rule is on the value,
|
||||
* not on how it was written, because `JSON.parse` cannot tell `1e21` from the same digits
|
||||
* written out.
|
||||
*/
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
/** The largest JSON envelope, in bytes (bus-v1 section 4). */
|
||||
export const MAX_ENVELOPE_BYTES = 65_536;
|
||||
|
||||
/** Thrown by everything in this package. One error type, like the bus's `WireError`. */
|
||||
export class ContractError extends Error {
|
||||
constructor(message: string) {
|
||||
super(message);
|
||||
this.name = 'ContractError';
|
||||
}
|
||||
}
|
||||
|
||||
export function fail(message: string): never {
|
||||
throw new ContractError(message);
|
||||
}
|
||||
|
||||
/** A JSON value, as strictly parsed. */
|
||||
export type Json = null | boolean | number | string | Json[] | { [key: string]: Json };
|
||||
|
||||
/** `String(number)` for a canonicalizable number. */
|
||||
function numberToString(value: number): string {
|
||||
if (!Number.isFinite(value)) {
|
||||
fail(`canonical JSON: ${String(value)} is not a finite number`);
|
||||
}
|
||||
if (Number.isInteger(value) && Math.abs(value) > Number.MAX_SAFE_INTEGER) {
|
||||
fail(`canonical JSON: ${String(value)} is an integral value outside the exact double range`);
|
||||
}
|
||||
// String(-0) is already "0".
|
||||
return String(value);
|
||||
}
|
||||
|
||||
function writeString(out: string[], value: string): void {
|
||||
out.push('"');
|
||||
for (const character of value) {
|
||||
switch (character) {
|
||||
case '"':
|
||||
out.push('\\"');
|
||||
break;
|
||||
case '\\':
|
||||
out.push('\\\\');
|
||||
break;
|
||||
case '\b':
|
||||
out.push('\\b');
|
||||
break;
|
||||
case '\t':
|
||||
out.push('\\t');
|
||||
break;
|
||||
case '\n':
|
||||
out.push('\\n');
|
||||
break;
|
||||
case '\f':
|
||||
out.push('\\f');
|
||||
break;
|
||||
case '\r':
|
||||
out.push('\\r');
|
||||
break;
|
||||
default: {
|
||||
const point = character.codePointAt(0) ?? 0;
|
||||
if (point < 0x20) {
|
||||
out.push(`\\u${point.toString(16).padStart(4, '0')}`);
|
||||
} else {
|
||||
out.push(character);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out.push('"');
|
||||
}
|
||||
|
||||
function write(out: string[], value: unknown): void {
|
||||
if (value === null) {
|
||||
out.push('null');
|
||||
return;
|
||||
}
|
||||
switch (typeof value) {
|
||||
case 'boolean':
|
||||
out.push(value ? 'true' : 'false');
|
||||
return;
|
||||
case 'number':
|
||||
out.push(numberToString(value));
|
||||
return;
|
||||
case 'string':
|
||||
writeString(out, value);
|
||||
return;
|
||||
case 'object':
|
||||
break;
|
||||
default:
|
||||
fail(`canonical JSON: ${typeof value} is not a JSON value`);
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
out.push('[');
|
||||
value.forEach((item, index) => {
|
||||
if (index > 0) out.push(',');
|
||||
write(out, item);
|
||||
});
|
||||
out.push(']');
|
||||
return;
|
||||
}
|
||||
const entries = Object.entries(value as Record<string, unknown>);
|
||||
for (const [key, item] of entries) {
|
||||
if (item === undefined) fail(`canonical JSON: ${key} is undefined, which is not a JSON value`);
|
||||
}
|
||||
// Comparing JavaScript strings compares UTF-16 code units, which is the order RFC 8785
|
||||
// section 3.2.3 specifies.
|
||||
entries.sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0));
|
||||
out.push('{');
|
||||
entries.forEach(([key, item], index) => {
|
||||
if (index > 0) out.push(',');
|
||||
writeString(out, key);
|
||||
out.push(':');
|
||||
write(out, item);
|
||||
});
|
||||
out.push('}');
|
||||
}
|
||||
|
||||
/** The canonical JSON text of `value`. */
|
||||
export function canonicalize(value: unknown): string {
|
||||
const out: string[] = [];
|
||||
write(out, value);
|
||||
return out.join('');
|
||||
}
|
||||
|
||||
/** Lowercase hex SHA-256 of `bytes`. */
|
||||
export function sha256Hex(bytes: Uint8Array | string): string {
|
||||
return createHash('sha256')
|
||||
.update(typeof bytes === 'string' ? Buffer.from(bytes, 'utf8') : bytes)
|
||||
.digest('hex');
|
||||
}
|
||||
|
||||
/** The canonical digest of a JSON value: SHA-256 over its canonical JSON bytes. */
|
||||
export function digestOf(value: unknown): string {
|
||||
return sha256Hex(canonicalize(value));
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses JSON strictly: duplicate keys at any depth, invalid UTF-8, `NaN`/`Infinity`,
|
||||
* trailing bytes and control characters inside strings are all refused.
|
||||
*
|
||||
* `JSON.parse` keeps the last of two duplicate keys instead of failing, so this is a small
|
||||
* recursive-descent parser rather than a wrapper around it.
|
||||
*/
|
||||
export function parseStrict(input: Uint8Array | string): Json {
|
||||
let text: string;
|
||||
if (typeof input === 'string') {
|
||||
text = input;
|
||||
} else {
|
||||
try {
|
||||
text = new TextDecoder('utf-8', { fatal: true }).decode(input);
|
||||
} catch {
|
||||
return fail('invalid UTF-8');
|
||||
}
|
||||
}
|
||||
const parser = new Parser(text);
|
||||
const value = parser.value();
|
||||
parser.skipWhitespace();
|
||||
if (!parser.atEnd()) fail('invalid JSON: trailing data');
|
||||
return value;
|
||||
}
|
||||
|
||||
class Parser {
|
||||
private index = 0;
|
||||
|
||||
constructor(private readonly text: string) {}
|
||||
|
||||
atEnd(): boolean {
|
||||
return this.index >= this.text.length;
|
||||
}
|
||||
|
||||
skipWhitespace(): void {
|
||||
while (this.index < this.text.length && ' \t\n\r'.includes(this.text[this.index] as string)) {
|
||||
this.index += 1;
|
||||
}
|
||||
}
|
||||
|
||||
value(): Json {
|
||||
this.skipWhitespace();
|
||||
const character = this.text[this.index];
|
||||
if (character === undefined) fail('invalid JSON: unexpected end of input');
|
||||
switch (character) {
|
||||
case '{':
|
||||
return this.object();
|
||||
case '[':
|
||||
return this.array();
|
||||
case '"':
|
||||
return this.string();
|
||||
case 't':
|
||||
this.literal('true');
|
||||
return true;
|
||||
case 'f':
|
||||
this.literal('false');
|
||||
return false;
|
||||
case 'n':
|
||||
this.literal('null');
|
||||
return null;
|
||||
default:
|
||||
return this.number();
|
||||
}
|
||||
}
|
||||
|
||||
private literal(word: string): void {
|
||||
if (!this.text.startsWith(word, this.index)) fail(`invalid JSON: expected ${word}`);
|
||||
this.index += word.length;
|
||||
}
|
||||
|
||||
private object(): Json {
|
||||
this.index += 1;
|
||||
const out: { [key: string]: Json } = {};
|
||||
this.skipWhitespace();
|
||||
if (this.text[this.index] === '}') {
|
||||
this.index += 1;
|
||||
return out;
|
||||
}
|
||||
for (;;) {
|
||||
this.skipWhitespace();
|
||||
if (this.text[this.index] !== '"') fail('invalid JSON: expected a key');
|
||||
const key = this.string();
|
||||
if (Object.prototype.hasOwnProperty.call(out, key)) {
|
||||
fail(`invalid JSON: duplicate key ${JSON.stringify(key)}`);
|
||||
}
|
||||
this.skipWhitespace();
|
||||
if (this.text[this.index] !== ':') fail('invalid JSON: expected :');
|
||||
this.index += 1;
|
||||
out[key] = this.value();
|
||||
this.skipWhitespace();
|
||||
const next = this.text[this.index];
|
||||
if (next === ',') {
|
||||
this.index += 1;
|
||||
continue;
|
||||
}
|
||||
if (next === '}') {
|
||||
this.index += 1;
|
||||
return out;
|
||||
}
|
||||
fail('invalid JSON: expected , or }');
|
||||
}
|
||||
}
|
||||
|
||||
private array(): Json {
|
||||
this.index += 1;
|
||||
const out: Json[] = [];
|
||||
this.skipWhitespace();
|
||||
if (this.text[this.index] === ']') {
|
||||
this.index += 1;
|
||||
return out;
|
||||
}
|
||||
for (;;) {
|
||||
out.push(this.value());
|
||||
this.skipWhitespace();
|
||||
const next = this.text[this.index];
|
||||
if (next === ',') {
|
||||
this.index += 1;
|
||||
continue;
|
||||
}
|
||||
if (next === ']') {
|
||||
this.index += 1;
|
||||
return out;
|
||||
}
|
||||
fail('invalid JSON: expected , or ]');
|
||||
}
|
||||
}
|
||||
|
||||
private string(): string {
|
||||
this.index += 1;
|
||||
let out = '';
|
||||
for (;;) {
|
||||
const character = this.text[this.index];
|
||||
if (character === undefined) fail('invalid JSON: unterminated string');
|
||||
this.index += 1;
|
||||
if (character === '"') return out;
|
||||
if (character === '\\') {
|
||||
const escape = this.text[this.index];
|
||||
this.index += 1;
|
||||
switch (escape) {
|
||||
case '"':
|
||||
case '\\':
|
||||
case '/':
|
||||
out += escape;
|
||||
break;
|
||||
case 'b':
|
||||
out += '\b';
|
||||
break;
|
||||
case 'f':
|
||||
out += '\f';
|
||||
break;
|
||||
case 'n':
|
||||
out += '\n';
|
||||
break;
|
||||
case 'r':
|
||||
out += '\r';
|
||||
break;
|
||||
case 't':
|
||||
out += '\t';
|
||||
break;
|
||||
case 'u': {
|
||||
const hex = this.text.slice(this.index, this.index + 4);
|
||||
if (!/^[0-9a-fA-F]{4}$/.test(hex)) fail('invalid JSON: bad \\u escape');
|
||||
out += String.fromCharCode(Number.parseInt(hex, 16));
|
||||
this.index += 4;
|
||||
break;
|
||||
}
|
||||
default:
|
||||
fail('invalid JSON: bad escape');
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (character.charCodeAt(0) < 0x20) fail('invalid JSON: control character in a string');
|
||||
out += character;
|
||||
}
|
||||
}
|
||||
|
||||
private number(): number {
|
||||
const match = /^-?(0|[1-9][0-9]*)(\.[0-9]+)?([eE][-+]?[0-9]+)?/.exec(
|
||||
this.text.slice(this.index),
|
||||
);
|
||||
if (!match) fail('invalid JSON: expected a value');
|
||||
this.index += match[0].length;
|
||||
const value = Number(match[0]);
|
||||
if (!Number.isFinite(value)) fail('invalid JSON: non-finite number');
|
||||
return value;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Refuses a domain payload that does not fit the bus envelope ceiling. `envelopeOverhead` is
|
||||
* what the surrounding envelope adds, so a payload that only fits without its envelope fails.
|
||||
*/
|
||||
export function requireEnvelopeFit(value: unknown, envelopeOverhead: number): number {
|
||||
const total = canonicalize(value).length + envelopeOverhead;
|
||||
if (total > MAX_ENVELOPE_BYTES) {
|
||||
fail(`envelope: ${total} bytes exceeds the ${MAX_ENVELOPE_BYTES}-byte maximum`);
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
/** Keys that belong to the bus and never to a domain body (ipc-v1 section 5). */
|
||||
export const BUS_ONLY_KEYS = [
|
||||
'callId',
|
||||
'deliveryId',
|
||||
'ownerId',
|
||||
'ownerIds',
|
||||
'deliveryIds',
|
||||
'requestDeliveryId',
|
||||
'expectedIncarnation',
|
||||
'serviceIncarnation',
|
||||
'connectionId',
|
||||
'topicSequence',
|
||||
'subscriptionId',
|
||||
] as const;
|
||||
|
||||
/** Fails if any bus-only key appears anywhere in `value`. */
|
||||
export function rejectBusIdentities(value: unknown): void {
|
||||
if (Array.isArray(value)) {
|
||||
for (const item of value) rejectBusIdentities(item);
|
||||
return;
|
||||
}
|
||||
if (value === null || typeof value !== 'object') return;
|
||||
for (const [key, item] of Object.entries(value as Record<string, unknown>)) {
|
||||
if ((BUS_ONLY_KEYS as readonly string[]).includes(key)) {
|
||||
fail(`canonical body: "${key}" is a bus identity and never part of a domain body`);
|
||||
}
|
||||
rejectBusIdentities(item);
|
||||
}
|
||||
}
|
||||
273
packages/session-types/src/checkpoint.ts
Normal file
273
packages/session-types/src/checkpoint.ts
Normal file
|
|
@ -0,0 +1,273 @@
|
|||
/**
|
||||
* `FLYSESS1`: the envelope layout of
|
||||
* `docs/design/session-framework/checkpoint-envelope-v1.md`.
|
||||
*
|
||||
* The layout half of the specification, not the store: writing generations, fsyncing and
|
||||
* committing a manifest belong to the STATE-01 slice. `FLYSIM01` is a different format with a
|
||||
* different magic and is not touched by any of this.
|
||||
*/
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
import { type Json, canonicalize, fail, parseStrict } from './canonical';
|
||||
import { requireUnique } from './reader';
|
||||
import { isId } from './scalar';
|
||||
|
||||
export const MAGIC = 'FLYSESS1';
|
||||
export const FOOTER_MAGIC = 'FLYSESSF';
|
||||
export const VERSION = 1;
|
||||
export const HEADER_BYTES = 32;
|
||||
export const TABLE_ENTRY_BYTES = 112;
|
||||
export const NAME_BYTES = 64;
|
||||
export const FOOTER_BYTES = 48;
|
||||
export const ALIGNMENT = 8;
|
||||
export const MAX_PAYLOADS = 64;
|
||||
|
||||
export interface PayloadEntry {
|
||||
name: string;
|
||||
offset: number;
|
||||
byteLength: number;
|
||||
digest: string;
|
||||
}
|
||||
|
||||
export interface Layout {
|
||||
manifestOffset: number;
|
||||
manifestBytes: number;
|
||||
tableOffset: number;
|
||||
entries: PayloadEntry[];
|
||||
footerOffset: number;
|
||||
totalBytes: number;
|
||||
}
|
||||
|
||||
export interface Envelope {
|
||||
manifest: Json;
|
||||
payloads: { name: string; bytes: Uint8Array }[];
|
||||
layout: Layout;
|
||||
}
|
||||
|
||||
function alignUp(value: number): number {
|
||||
return Math.ceil(value / ALIGNMENT) * ALIGNMENT;
|
||||
}
|
||||
|
||||
function sha256(bytes: Uint8Array): string {
|
||||
return createHash('sha256').update(bytes).digest('hex');
|
||||
}
|
||||
|
||||
export function layoutOf(
|
||||
manifest: unknown,
|
||||
payloads: readonly { name: string; bytes: Uint8Array }[],
|
||||
): Layout {
|
||||
if (payloads.length > MAX_PAYLOADS) fail('checkpoint envelope: at most 64 payloads');
|
||||
requireUnique(
|
||||
payloads.map((payload) => payload.name),
|
||||
'checkpoint envelope: payload names',
|
||||
);
|
||||
for (const payload of payloads) {
|
||||
if (!isId(payload.name)) {
|
||||
fail(`checkpoint envelope: payload name "${payload.name}" is not an Id`);
|
||||
}
|
||||
}
|
||||
const manifestBytes = new TextEncoder().encode(canonicalize(manifest)).length;
|
||||
const manifestOffset = HEADER_BYTES;
|
||||
const tableOffset = alignUp(manifestOffset + manifestBytes);
|
||||
let offset = alignUp(tableOffset + payloads.length * TABLE_ENTRY_BYTES);
|
||||
const entries: PayloadEntry[] = [];
|
||||
for (const payload of payloads) {
|
||||
entries.push({
|
||||
name: payload.name,
|
||||
offset,
|
||||
byteLength: payload.bytes.length,
|
||||
digest: sha256(payload.bytes),
|
||||
});
|
||||
offset = alignUp(offset + payload.bytes.length);
|
||||
}
|
||||
return {
|
||||
manifestOffset,
|
||||
manifestBytes,
|
||||
tableOffset,
|
||||
entries,
|
||||
footerOffset: offset,
|
||||
totalBytes: offset + FOOTER_BYTES,
|
||||
};
|
||||
}
|
||||
|
||||
export function encode(
|
||||
manifest: unknown,
|
||||
payloads: readonly { name: string; bytes: Uint8Array }[],
|
||||
): Uint8Array {
|
||||
const layout = layoutOf(manifest, payloads);
|
||||
const manifestText = new TextEncoder().encode(canonicalize(manifest));
|
||||
const out = Buffer.alloc(layout.footerOffset);
|
||||
out.write(MAGIC, 0, 'ascii');
|
||||
out.writeUInt32LE(VERSION, 8);
|
||||
out.writeUInt32LE(HEADER_BYTES, 12);
|
||||
out.writeUInt32LE(layout.manifestBytes, 16);
|
||||
out.writeUInt32LE(payloads.length, 20);
|
||||
out.writeUInt32LE(layout.tableOffset, 24);
|
||||
out.writeUInt32LE(0, 28);
|
||||
Buffer.from(manifestText).copy(out, layout.manifestOffset);
|
||||
layout.entries.forEach((entry, index) => {
|
||||
const base = layout.tableOffset + index * TABLE_ENTRY_BYTES;
|
||||
out.write(entry.name, base, 'ascii');
|
||||
out.writeBigUInt64LE(BigInt(entry.offset), base + NAME_BYTES);
|
||||
out.writeBigUInt64LE(BigInt(entry.byteLength), base + NAME_BYTES + 8);
|
||||
Buffer.from(entry.digest, 'hex').copy(out, base + NAME_BYTES + 16);
|
||||
});
|
||||
layout.entries.forEach((entry, index) => {
|
||||
Buffer.from((payloads[index] as { bytes: Uint8Array }).bytes).copy(out, entry.offset);
|
||||
});
|
||||
const footer = Buffer.alloc(FOOTER_BYTES);
|
||||
footer.writeBigUInt64LE(BigInt(layout.totalBytes), 0);
|
||||
Buffer.from(sha256(out), 'hex').copy(footer, 8);
|
||||
footer.write(FOOTER_MAGIC, 40, 'ascii');
|
||||
return Buffer.concat([out, footer]);
|
||||
}
|
||||
|
||||
/** Reads and fully validates one envelope. */
|
||||
export function decode(input: Uint8Array): Envelope {
|
||||
const bytes = Buffer.from(input);
|
||||
if (bytes.length < HEADER_BYTES + FOOTER_BYTES) {
|
||||
fail('checkpoint envelope: shorter than a header plus a footer');
|
||||
}
|
||||
if (bytes.subarray(0, 8).toString('ascii') !== MAGIC) {
|
||||
fail('checkpoint envelope: wrong magic (FLYSIM01 is a different format)');
|
||||
}
|
||||
if (bytes.readUInt32LE(8) !== VERSION) fail('checkpoint envelope: unsupported version');
|
||||
if (bytes.readUInt32LE(12) !== HEADER_BYTES) {
|
||||
fail('checkpoint envelope: headerBytes must be 32');
|
||||
}
|
||||
if (bytes.readUInt32LE(28) !== 0) {
|
||||
fail('checkpoint envelope: reserved header word must be zero');
|
||||
}
|
||||
const manifestBytes = bytes.readUInt32LE(16);
|
||||
const payloadCount = bytes.readUInt32LE(20);
|
||||
const tableOffset = bytes.readUInt32LE(24);
|
||||
if (payloadCount > MAX_PAYLOADS) fail('checkpoint envelope: at most 64 payloads');
|
||||
const footerOffset = bytes.length - FOOTER_BYTES;
|
||||
if (bytes.subarray(footerOffset + 40).toString('ascii') !== FOOTER_MAGIC) {
|
||||
fail('checkpoint envelope: missing footer magic');
|
||||
}
|
||||
if (bytes.readBigUInt64LE(footerOffset) !== BigInt(bytes.length)) {
|
||||
fail('checkpoint envelope: footer length does not match the file');
|
||||
}
|
||||
const recorded = bytes.subarray(footerOffset + 8, footerOffset + 40).toString('hex');
|
||||
if (recorded !== sha256(bytes.subarray(0, footerOffset))) {
|
||||
fail('checkpoint envelope: footer digest does not match the contents');
|
||||
}
|
||||
const manifestEnd = HEADER_BYTES + manifestBytes;
|
||||
if (manifestEnd > footerOffset) {
|
||||
fail('checkpoint envelope: manifest runs past the payload area');
|
||||
}
|
||||
const manifestSlice = bytes.subarray(HEADER_BYTES, manifestEnd);
|
||||
const manifest = parseStrict(manifestSlice);
|
||||
if (canonicalize(manifest) !== manifestSlice.toString('utf8')) {
|
||||
fail('checkpoint envelope: the manifest is not canonical JSON');
|
||||
}
|
||||
if (tableOffset !== alignUp(manifestEnd)) {
|
||||
fail('checkpoint envelope: the payload table is not at its laid-out offset');
|
||||
}
|
||||
const tableEnd = tableOffset + payloadCount * TABLE_ENTRY_BYTES;
|
||||
if (tableEnd > footerOffset) {
|
||||
fail('checkpoint envelope: the payload table runs past the payload area');
|
||||
}
|
||||
const entries: PayloadEntry[] = [];
|
||||
const payloads: { name: string; bytes: Uint8Array }[] = [];
|
||||
let previousEnd = alignUp(tableEnd);
|
||||
for (let index = 0; index < payloadCount; index += 1) {
|
||||
const base = tableOffset + index * TABLE_ENTRY_BYTES;
|
||||
const nameField = bytes.subarray(base, base + NAME_BYTES);
|
||||
const terminator = nameField.indexOf(0);
|
||||
const length = terminator === -1 ? NAME_BYTES : terminator;
|
||||
if (nameField.subarray(length).some((byte) => byte !== 0)) {
|
||||
fail('checkpoint envelope: a payload name has bytes after its terminator');
|
||||
}
|
||||
const name = nameField.subarray(0, length).toString('utf8');
|
||||
if (!isId(name)) fail(`checkpoint envelope: payload name "${name}" is not an Id`);
|
||||
const offset = Number(bytes.readBigUInt64LE(base + NAME_BYTES));
|
||||
const byteLength = Number(bytes.readBigUInt64LE(base + NAME_BYTES + 8));
|
||||
const digest = bytes.subarray(base + NAME_BYTES + 16, base + NAME_BYTES + 48).toString('hex');
|
||||
if (offset !== previousEnd) {
|
||||
fail(
|
||||
`checkpoint envelope: payload "${name}" starts at ${offset}, not at its aligned ${previousEnd}`,
|
||||
);
|
||||
}
|
||||
const end = offset + byteLength;
|
||||
if (end > footerOffset) {
|
||||
fail(`checkpoint envelope: payload "${name}" runs past the payload area`);
|
||||
}
|
||||
const payload = bytes.subarray(offset, end);
|
||||
if (sha256(payload) !== digest) {
|
||||
fail(`checkpoint envelope: payload "${name}" fails its digest`);
|
||||
}
|
||||
previousEnd = alignUp(end);
|
||||
entries.push({ name, offset, byteLength, digest });
|
||||
payloads.push({ name, bytes: Uint8Array.from(payload) });
|
||||
}
|
||||
requireUnique(
|
||||
entries.map((entry) => entry.name),
|
||||
'checkpoint envelope: payload names',
|
||||
);
|
||||
if (previousEnd !== footerOffset) {
|
||||
fail('checkpoint envelope: padding between the last payload and the footer');
|
||||
}
|
||||
return {
|
||||
manifest,
|
||||
payloads,
|
||||
layout: {
|
||||
manifestOffset: HEADER_BYTES,
|
||||
manifestBytes,
|
||||
tableOffset,
|
||||
entries,
|
||||
footerOffset,
|
||||
totalBytes: bytes.length,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** The manifest fields state-media-v1 section 4 requires. */
|
||||
export const REQUIRED_MANIFEST_FIELDS = [
|
||||
'envelopeVersion',
|
||||
'checkpointId',
|
||||
'sourceScope',
|
||||
'episodeId',
|
||||
'worldTime',
|
||||
'schedulerId',
|
||||
'compositionDigest',
|
||||
'portMap',
|
||||
'compatibility',
|
||||
'agents',
|
||||
'coordinator',
|
||||
'payloads',
|
||||
] as const;
|
||||
|
||||
/** Checks the required field set and that the manifest's payload table mirrors the envelope's. */
|
||||
export function validateManifest(envelope: Envelope): void {
|
||||
const manifest = envelope.manifest;
|
||||
if (manifest === null || typeof manifest !== 'object' || Array.isArray(manifest)) {
|
||||
fail('checkpoint manifest: must be an object');
|
||||
}
|
||||
const map = manifest as Record<string, Json>;
|
||||
for (const field of REQUIRED_MANIFEST_FIELDS) {
|
||||
if (!Object.prototype.hasOwnProperty.call(map, field)) {
|
||||
fail(`checkpoint manifest: missing "${field}"`);
|
||||
}
|
||||
}
|
||||
if (map.envelopeVersion !== VERSION) fail('checkpoint manifest: envelopeVersion must be 1');
|
||||
const listed = map.payloads;
|
||||
if (!Array.isArray(listed)) fail('checkpoint manifest: payloads must be an array');
|
||||
if (listed.length !== envelope.layout.entries.length) {
|
||||
fail('checkpoint manifest: payloads does not match the payload table');
|
||||
}
|
||||
listed.forEach((declared, index) => {
|
||||
const entry = envelope.layout.entries[index] as PayloadEntry;
|
||||
const record = declared as Record<string, Json>;
|
||||
if (record.name !== entry.name) {
|
||||
fail('checkpoint manifest: payload name does not match the table');
|
||||
}
|
||||
if (record.byteLength !== String(entry.byteLength)) {
|
||||
fail(`checkpoint manifest: payload "${entry.name}" byteLength does not match the table`);
|
||||
}
|
||||
if (record.digest !== entry.digest) {
|
||||
fail(`checkpoint manifest: payload "${entry.name}" digest does not match the table`);
|
||||
}
|
||||
});
|
||||
}
|
||||
114
packages/session-types/src/common.ts
Normal file
114
packages/session-types/src/common.ts
Normal file
|
|
@ -0,0 +1,114 @@
|
|||
/**
|
||||
* `Scope`, `SchemaRef`, `TypedValue`, the operation key and the canonical body
|
||||
* (ipc-v1 sections 2 and 5).
|
||||
*/
|
||||
import { canonicalize, digestOf, fail, rejectBusIdentities } from './canonical';
|
||||
import { Reader } from './reader';
|
||||
import {
|
||||
MAX_TYPED_VALUE_BYTES,
|
||||
type RationalNs,
|
||||
type Scope,
|
||||
type SchemaRef,
|
||||
type TypedValue,
|
||||
isId,
|
||||
isMethod,
|
||||
validateRational,
|
||||
} from './scalar';
|
||||
|
||||
export function readScope(value: unknown): Scope {
|
||||
const reader = new Reader(value, 'Scope');
|
||||
const scope: Scope = {
|
||||
sessionId: reader.id('sessionId'),
|
||||
epoch: reader.id('epoch'),
|
||||
step: reader.u64('step'),
|
||||
};
|
||||
reader.finish();
|
||||
return scope;
|
||||
}
|
||||
|
||||
export function readNullableScope(value: unknown): Scope | null {
|
||||
return value === null ? null : readScope(value);
|
||||
}
|
||||
|
||||
export function readSchemaRef(value: unknown): SchemaRef {
|
||||
const reader = new Reader(value, 'SchemaRef');
|
||||
const schema: SchemaRef = {
|
||||
id: reader.id('id'),
|
||||
version: reader.int('version', 1, 65_535),
|
||||
digest: reader.digest('digest'),
|
||||
};
|
||||
reader.finish();
|
||||
return schema;
|
||||
}
|
||||
|
||||
export function readTypedValue(value: unknown): TypedValue {
|
||||
const reader = new Reader(value, 'TypedValue');
|
||||
const typed: TypedValue = {
|
||||
schema: readSchemaRef(reader.value('schema')),
|
||||
value: reader.object('value'),
|
||||
};
|
||||
reader.finish();
|
||||
const length = canonicalize(typed).length;
|
||||
if (length > MAX_TYPED_VALUE_BYTES) {
|
||||
fail(
|
||||
`TypedValue: ${length} bytes of canonical JSON exceeds the ${MAX_TYPED_VALUE_BYTES}-byte limit`,
|
||||
);
|
||||
}
|
||||
return typed;
|
||||
}
|
||||
|
||||
export function readNullableTypedValue(value: unknown): TypedValue | null {
|
||||
return value === null ? null : readTypedValue(value);
|
||||
}
|
||||
|
||||
export { validateRational };
|
||||
|
||||
/** `(sessionId, epoch, step, method, workerId)`: the operation key of a step mutation. */
|
||||
export interface OperationKey {
|
||||
scope: Scope;
|
||||
method: string;
|
||||
workerId: string;
|
||||
}
|
||||
|
||||
export function operationKeyJson(key: OperationKey): Record<string, unknown> {
|
||||
if (!isMethod(key.method)) {
|
||||
fail('OperationKey: method must be 1..=128 printable ASCII characters');
|
||||
}
|
||||
if (!isId(key.workerId)) fail('OperationKey: workerId is not a valid id');
|
||||
return { scope: key.scope, method: key.method, workerId: key.workerId };
|
||||
}
|
||||
|
||||
export function operationKeyDigest(key: OperationKey): string {
|
||||
return digestOf(operationKeyJson(key));
|
||||
}
|
||||
|
||||
/** The canonical body of a domain operation: method, scope and validated params. */
|
||||
export function canonicalBody(
|
||||
method: string,
|
||||
scope: Scope | null,
|
||||
params: unknown,
|
||||
): Record<string, unknown> {
|
||||
if (!isMethod(method)) {
|
||||
fail('canonical body: method must be 1..=128 printable ASCII characters');
|
||||
}
|
||||
if (params === null || typeof params !== 'object' || Array.isArray(params)) {
|
||||
fail('canonical body: params must be an object');
|
||||
}
|
||||
rejectBusIdentities(params);
|
||||
return { method, scope, params };
|
||||
}
|
||||
|
||||
export function bodyDigest(method: string, scope: Scope | null, params: unknown): string {
|
||||
return digestOf(canonicalBody(method, scope, params));
|
||||
}
|
||||
|
||||
export function readRational(value: unknown): RationalNs {
|
||||
const reader = new Reader(value, 'RationalNs');
|
||||
const rational: RationalNs = {
|
||||
numerator: reader.u64('numerator'),
|
||||
denominator: reader.u64('denominator'),
|
||||
};
|
||||
reader.finish();
|
||||
validateRational(rational);
|
||||
return rational;
|
||||
}
|
||||
62
packages/session-types/src/fixtures.ts
Normal file
62
packages/session-types/src/fixtures.ts
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
/**
|
||||
* Loading the fixture corpus, which lives with the Rust crate:
|
||||
* `services/flysim/crates/fly-session-types/fixtures`.
|
||||
*
|
||||
* One corpus, two implementations. A case written once holds both languages to it.
|
||||
*/
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
import { type Json, fail, parseStrict } from './canonical';
|
||||
|
||||
const here = fileURLToPath(new URL('.', import.meta.url));
|
||||
|
||||
/** The fixture directory. */
|
||||
export const FIXTURE_DIR = join(
|
||||
here,
|
||||
'../../../services/flysim/crates/fly-session-types/fixtures',
|
||||
);
|
||||
|
||||
export function loadBytes(name: string): Uint8Array {
|
||||
return new Uint8Array(readFileSync(join(FIXTURE_DIR, name)));
|
||||
}
|
||||
|
||||
/** Reads one fixture file, parsed strictly. */
|
||||
export function load(name: string): Json {
|
||||
return parseStrict(loadBytes(name));
|
||||
}
|
||||
|
||||
export function section(file: Json, key: string): Json[] {
|
||||
const value = (file as Record<string, Json>)[key];
|
||||
if (!Array.isArray(value) || value.length === 0) {
|
||||
fail(`fixture: ${key} must be a nonempty array`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
export function cases(file: Json): Json[] {
|
||||
return section(file, 'cases');
|
||||
}
|
||||
|
||||
/** A string field of one case. */
|
||||
export function field(value: Json, key: string): string {
|
||||
const found = (value as Record<string, Json>)[key];
|
||||
if (typeof found !== 'string') fail(`fixture case: missing string field "${key}"`);
|
||||
return found;
|
||||
}
|
||||
|
||||
export function optionalField(value: Json, key: string): string | undefined {
|
||||
const found = (value as Record<string, Json>)[key];
|
||||
return typeof found === 'string' ? found : undefined;
|
||||
}
|
||||
|
||||
export function member(value: Json, key: string): Json {
|
||||
const found = (value as Record<string, Json>)[key];
|
||||
if (found === undefined) fail(`fixture case: missing field "${key}"`);
|
||||
return found;
|
||||
}
|
||||
|
||||
export function decodeBase64(text: string): Uint8Array {
|
||||
return new Uint8Array(Buffer.from(text, 'base64'));
|
||||
}
|
||||
19
packages/session-types/src/index.ts
Normal file
19
packages/session-types/src/index.ts
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
/**
|
||||
* `@flybrain/session-types`: the session framework contracts in TypeScript.
|
||||
*
|
||||
* The other half of `services/flysim/crates/fly-session-types`. Same rules, same canonical
|
||||
* JSON, same digests, same fixtures. Nothing here opens a socket: it reads, validates and
|
||||
* hashes payloads.
|
||||
*/
|
||||
export * from './canonical';
|
||||
export * from './scalar';
|
||||
export * from './reader';
|
||||
export * from './common';
|
||||
export * from './media';
|
||||
export * from './workers';
|
||||
export * from './rpc';
|
||||
export * from './publishing';
|
||||
export * from './trace';
|
||||
export * as seed from './seed';
|
||||
export * as checkpoint from './checkpoint';
|
||||
export * as fixtures from './fixtures';
|
||||
264
packages/session-types/src/media.ts
Normal file
264
packages/session-types/src/media.ts
Normal file
|
|
@ -0,0 +1,264 @@
|
|||
/** Native observation media (state-media-v1 section 2) and the `State.*` payloads (section 5). */
|
||||
import { fail } from './canonical';
|
||||
import { readRational, readScope } from './common';
|
||||
import { Reader, readArtifactRef, requireUnique, u64 } from './reader';
|
||||
import { type ArtifactRef, type Digest, type Id, type Scope, type U64, isDigest } from './scalar';
|
||||
|
||||
/** Max views per sensory input (workers-v1 section 1), and per descriptor list. */
|
||||
export const MAX_VIEWS = 8;
|
||||
export const MAX_VIEW_DIMENSION = 4096;
|
||||
export const MAX_PIXEL_ASPECT = 65_535;
|
||||
export const MAX_OBSERVATION_DELAY_STEPS = 8;
|
||||
export const MAX_SAMPLE_FRAMES = 192_000;
|
||||
/** Not a stated bound; this crate's choice, published in the schema set. */
|
||||
export const MAX_AUDIO_STREAMS = 8;
|
||||
|
||||
export interface ViewDescriptor {
|
||||
viewId: Id;
|
||||
width: number;
|
||||
height: number;
|
||||
format: 'rgba8';
|
||||
rowStride: number;
|
||||
pixelAspect: { numerator: number; denominator: number };
|
||||
observationDelaySteps: number;
|
||||
}
|
||||
|
||||
export interface ViewRef {
|
||||
viewId: Id;
|
||||
producedStep: U64;
|
||||
pixels: ArtifactRef;
|
||||
}
|
||||
|
||||
export interface AudioDescriptor {
|
||||
streamId: Id;
|
||||
sampleRate: number;
|
||||
channels: number;
|
||||
format: 'f32le-interleaved';
|
||||
}
|
||||
|
||||
export interface AudioRef {
|
||||
streamId: Id;
|
||||
firstSample: U64;
|
||||
sampleFrames: number;
|
||||
samples: ArtifactRef;
|
||||
discontinuity: boolean;
|
||||
}
|
||||
|
||||
export function readViewDescriptor(value: unknown): ViewDescriptor {
|
||||
const reader = new Reader(value, 'ViewDescriptor');
|
||||
const viewId = reader.id('viewId');
|
||||
const width = reader.int('width', 1, MAX_VIEW_DIMENSION);
|
||||
const height = reader.int('height', 1, MAX_VIEW_DIMENSION);
|
||||
const format = reader.constant('format', 'rgba8');
|
||||
const rowStride = reader.int('rowStride', 1, MAX_VIEW_DIMENSION * 4);
|
||||
const aspectReader = new Reader(reader.value('pixelAspect'), 'ViewDescriptor.pixelAspect');
|
||||
const pixelAspect = {
|
||||
numerator: aspectReader.int('numerator', 1, MAX_PIXEL_ASPECT),
|
||||
denominator: aspectReader.int('denominator', 1, MAX_PIXEL_ASPECT),
|
||||
};
|
||||
aspectReader.finish();
|
||||
const observationDelaySteps = reader.int(
|
||||
'observationDelaySteps',
|
||||
0,
|
||||
MAX_OBSERVATION_DELAY_STEPS,
|
||||
);
|
||||
reader.finish();
|
||||
if (rowStride !== width * 4) {
|
||||
fail('ViewDescriptor: rowStride must be exactly 4 x width (no padded rows in v1)');
|
||||
}
|
||||
return { viewId, width, height, format, rowStride, pixelAspect, observationDelaySteps };
|
||||
}
|
||||
|
||||
/** The exact byte length of one frame of this view. */
|
||||
export function frameBytes(descriptor: ViewDescriptor): number {
|
||||
return descriptor.rowStride * descriptor.height;
|
||||
}
|
||||
|
||||
/** `max(0, boundary - observationDelaySteps)` (state-media-v1 section 2). */
|
||||
export function requiredProducedStep(descriptor: ViewDescriptor, boundary: bigint): bigint {
|
||||
const delay = BigInt(descriptor.observationDelaySteps);
|
||||
return boundary > delay ? boundary - delay : 0n;
|
||||
}
|
||||
|
||||
export function readViewRef(value: unknown): ViewRef {
|
||||
const reader = new Reader(value, 'ViewRef');
|
||||
const view: ViewRef = {
|
||||
viewId: reader.id('viewId'),
|
||||
producedStep: reader.u64('producedStep'),
|
||||
pixels: readArtifactRef(reader.value('pixels')),
|
||||
};
|
||||
reader.finish();
|
||||
if (u64(view.pixels.byteLength) === 0n) {
|
||||
fail('ViewRef: pixels must have a positive byte length');
|
||||
}
|
||||
return view;
|
||||
}
|
||||
|
||||
export function readViewList(reader: Reader, key: string): ViewRef[] {
|
||||
const views = reader.list(key, 0, MAX_VIEWS, readViewRef);
|
||||
requireUnique(
|
||||
views.map((view) => view.viewId),
|
||||
key,
|
||||
);
|
||||
return views;
|
||||
}
|
||||
|
||||
export function readAudioDescriptor(value: unknown): AudioDescriptor {
|
||||
const reader = new Reader(value, 'AudioDescriptor');
|
||||
const descriptor: AudioDescriptor = {
|
||||
streamId: reader.id('streamId'),
|
||||
sampleRate: reader.int('sampleRate', 8_000, 192_000),
|
||||
channels: reader.int('channels', 1, 8),
|
||||
format: reader.constant('format', 'f32le-interleaved'),
|
||||
};
|
||||
reader.finish();
|
||||
return descriptor;
|
||||
}
|
||||
|
||||
export function readAudioRef(value: unknown): AudioRef {
|
||||
const reader = new Reader(value, 'AudioRef');
|
||||
const chunk: AudioRef = {
|
||||
streamId: reader.id('streamId'),
|
||||
firstSample: reader.u64('firstSample'),
|
||||
sampleFrames: reader.int('sampleFrames', 0, MAX_SAMPLE_FRAMES),
|
||||
samples: readArtifactRef(reader.value('samples')),
|
||||
discontinuity: reader.boolean('discontinuity'),
|
||||
};
|
||||
reader.finish();
|
||||
if (u64(chunk.firstSample) + BigInt(chunk.sampleFrames) > 18446744073709551615n) {
|
||||
fail('AudioRef: firstSample + sampleFrames overflows U64');
|
||||
}
|
||||
return chunk;
|
||||
}
|
||||
|
||||
export function readAudioList(reader: Reader, key: string): AudioRef[] {
|
||||
const audio = reader.list(key, 0, MAX_AUDIO_STREAMS, readAudioRef);
|
||||
requireUnique(
|
||||
audio.map((chunk) => chunk.streamId),
|
||||
key,
|
||||
);
|
||||
return audio;
|
||||
}
|
||||
|
||||
/** Byte shape and producing boundary against the descriptor that declared this view. */
|
||||
export function validateViewAgainst(
|
||||
view: ViewRef,
|
||||
descriptor: ViewDescriptor,
|
||||
boundary: bigint | null,
|
||||
): void {
|
||||
if (view.viewId !== descriptor.viewId) {
|
||||
fail(`ViewRef: viewId "${view.viewId}" does not match descriptor "${descriptor.viewId}"`);
|
||||
}
|
||||
if (u64(view.pixels.byteLength) !== BigInt(frameBytes(descriptor))) {
|
||||
fail(
|
||||
`ViewRef ${view.viewId}: artifact is ${view.pixels.byteLength} bytes, rowStride x height is ${frameBytes(descriptor)}`,
|
||||
);
|
||||
}
|
||||
if (boundary !== null) {
|
||||
const expected = requiredProducedStep(descriptor, boundary);
|
||||
if (u64(view.producedStep) !== expected) {
|
||||
fail(
|
||||
`ViewRef ${view.viewId}: producedStep ${view.producedStep} must be max(0, ${boundary} - ${descriptor.observationDelaySteps}) = ${expected}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export function validateAudioAgainst(chunk: AudioRef, descriptor: AudioDescriptor): void {
|
||||
if (chunk.streamId !== descriptor.streamId) {
|
||||
fail(`AudioRef: streamId "${chunk.streamId}" does not match the descriptor`);
|
||||
}
|
||||
const expected = BigInt(chunk.sampleFrames) * BigInt(descriptor.channels) * 4n;
|
||||
if (u64(chunk.samples.byteLength) !== expected) {
|
||||
fail(
|
||||
`AudioRef ${chunk.streamId}: artifact is ${chunk.samples.byteLength} bytes, sampleFrames x channels x 4 is ${expected}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------- State.*
|
||||
|
||||
export interface CaptureParams {
|
||||
checkpointId: Id;
|
||||
}
|
||||
|
||||
export interface CaptureResult {
|
||||
checkpointId: Id;
|
||||
boundary: U64;
|
||||
compatibilityDigest: Digest;
|
||||
payload: ArtifactRef;
|
||||
}
|
||||
|
||||
export interface StageRestoreParams {
|
||||
checkpointId: Id;
|
||||
sourceScope: Scope;
|
||||
compatibilityDigest: Digest;
|
||||
payload: ArtifactRef;
|
||||
}
|
||||
|
||||
export interface StageRestoreResult {
|
||||
checkpointId: Id;
|
||||
restoreToken: Id;
|
||||
}
|
||||
|
||||
export interface ActivateRestoreParams {
|
||||
restoreToken: Id;
|
||||
}
|
||||
|
||||
function checkpointPayload(reader: Reader, key: string): ArtifactRef {
|
||||
const reference = readArtifactRef(reader.value(key));
|
||||
if (!isDigest(reference.digest)) {
|
||||
fail(`${key}: a checkpoint payload must carry a content digest`);
|
||||
}
|
||||
return reference;
|
||||
}
|
||||
|
||||
export function readCaptureParams(value: unknown): CaptureParams {
|
||||
const reader = new Reader(value, 'CaptureParams');
|
||||
const params = { checkpointId: reader.id('checkpointId') };
|
||||
reader.finish();
|
||||
return params;
|
||||
}
|
||||
|
||||
export function readCaptureResult(value: unknown): CaptureResult {
|
||||
const reader = new Reader(value, 'CaptureResult');
|
||||
const result: CaptureResult = {
|
||||
checkpointId: reader.id('checkpointId'),
|
||||
boundary: reader.u64('boundary'),
|
||||
compatibilityDigest: reader.digest('compatibilityDigest'),
|
||||
payload: checkpointPayload(reader, 'payload'),
|
||||
};
|
||||
reader.finish();
|
||||
return result;
|
||||
}
|
||||
|
||||
export function readStageRestoreParams(value: unknown): StageRestoreParams {
|
||||
const reader = new Reader(value, 'StageRestoreParams');
|
||||
const params: StageRestoreParams = {
|
||||
checkpointId: reader.id('checkpointId'),
|
||||
sourceScope: readScope(reader.value('sourceScope')),
|
||||
compatibilityDigest: reader.digest('compatibilityDigest'),
|
||||
payload: checkpointPayload(reader, 'payload'),
|
||||
};
|
||||
reader.finish();
|
||||
return params;
|
||||
}
|
||||
|
||||
export function readStageRestoreResult(value: unknown): StageRestoreResult {
|
||||
const reader = new Reader(value, 'StageRestoreResult');
|
||||
const result: StageRestoreResult = {
|
||||
checkpointId: reader.id('checkpointId'),
|
||||
restoreToken: reader.id('restoreToken'),
|
||||
};
|
||||
reader.finish();
|
||||
return result;
|
||||
}
|
||||
|
||||
export function readActivateRestoreParams(value: unknown): ActivateRestoreParams {
|
||||
const reader = new Reader(value, 'ActivateRestoreParams');
|
||||
const params = { restoreToken: reader.id('restoreToken') };
|
||||
reader.finish();
|
||||
return params;
|
||||
}
|
||||
|
||||
export { readRational };
|
||||
221
packages/session-types/src/publishing.ts
Normal file
221
packages/session-types/src/publishing.ts
Normal file
|
|
@ -0,0 +1,221 @@
|
|||
/** The publication types of publishing-v1 section 3. */
|
||||
import { fail } from './canonical';
|
||||
import { readRational, readScope, readSchemaRef, readTypedValue, readNullableTypedValue } from './common';
|
||||
import {
|
||||
type AudioRef,
|
||||
MAX_VIEWS,
|
||||
type ViewRef,
|
||||
readAudioList,
|
||||
readViewList,
|
||||
} from './media';
|
||||
import { Reader, requireUnique, u64 } from './reader';
|
||||
import { type Digest, type Id, type RationalNs, type SchemaRef, type Scope, type TypedValue, type U64 } from './scalar';
|
||||
import {
|
||||
type AgentTelemetry,
|
||||
type AssetRef,
|
||||
type EnvironmentDescriptor,
|
||||
MAX_AGENTS,
|
||||
MAX_RATE_ROLES,
|
||||
type PortControl,
|
||||
findPort,
|
||||
readAgentTelemetry,
|
||||
readAssetRef,
|
||||
readEnvironmentDescriptor,
|
||||
readPortControl,
|
||||
validatePortControlAgainst,
|
||||
validateTelemetryRoles,
|
||||
} from './workers';
|
||||
|
||||
/** Not stated by a document; this crate's choices, published in the schema set. */
|
||||
export const MAX_SUPPORTED_STIMULI = 64;
|
||||
export const MAX_ASSETS = 64;
|
||||
export const MAX_SNAPSHOT_EVENTS = 64;
|
||||
|
||||
export interface AgentDescriptor {
|
||||
agentId: Id;
|
||||
portId: Id;
|
||||
profileDigest: Digest;
|
||||
datasetDigest: Digest;
|
||||
indexDigest: Digest;
|
||||
neuronCount: U64;
|
||||
rateRoles: Id[];
|
||||
supportedStimuli: Id[];
|
||||
}
|
||||
|
||||
export interface SessionDescriptor {
|
||||
sessionId: Id;
|
||||
revision: U64;
|
||||
compositionDigest: Digest;
|
||||
schedulerId: 'lockstep-v1';
|
||||
environment: EnvironmentDescriptor;
|
||||
taskSchema: SchemaRef;
|
||||
agents: AgentDescriptor[];
|
||||
assets: AssetRef[];
|
||||
}
|
||||
|
||||
export interface SnapshotAgent {
|
||||
agentId: Id;
|
||||
telemetry: AgentTelemetry;
|
||||
selectedDecision: TypedValue | null;
|
||||
appliedControls: PortControl | null;
|
||||
}
|
||||
|
||||
export interface CommittedSnapshot {
|
||||
descriptorRevision: U64;
|
||||
publisherIncarnation: Id;
|
||||
scope: Scope;
|
||||
episodeId: Id;
|
||||
sequence: U64;
|
||||
worldTime: RationalNs;
|
||||
agents: SnapshotAgent[];
|
||||
progress: TypedValue;
|
||||
media: { views: ViewRef[]; audio: AudioRef[] };
|
||||
eventIds: Id[];
|
||||
}
|
||||
|
||||
export function readSessionDescriptor(value: unknown): SessionDescriptor {
|
||||
const reader = new Reader(value, 'SessionDescriptor');
|
||||
const descriptor: SessionDescriptor = {
|
||||
sessionId: reader.id('sessionId'),
|
||||
revision: reader.u64('revision'),
|
||||
compositionDigest: reader.digest('compositionDigest'),
|
||||
schedulerId: reader.constant('schedulerId', 'lockstep-v1'),
|
||||
environment: readEnvironmentDescriptor(reader.value('environment')),
|
||||
taskSchema: readSchemaRef(reader.value('taskSchema')),
|
||||
agents: reader.list('agents', 1, MAX_AGENTS, (item) => {
|
||||
const agent = new Reader(item, 'SessionDescriptor.agents');
|
||||
const entry: AgentDescriptor = {
|
||||
agentId: agent.id('agentId'),
|
||||
portId: agent.id('portId'),
|
||||
profileDigest: agent.digest('profileDigest'),
|
||||
datasetDigest: agent.digest('datasetDigest'),
|
||||
indexDigest: agent.digest('indexDigest'),
|
||||
neuronCount: agent.u64('neuronCount'),
|
||||
rateRoles: agent.idList('rateRoles', 0, MAX_RATE_ROLES),
|
||||
supportedStimuli: agent.idList('supportedStimuli', 0, MAX_SUPPORTED_STIMULI),
|
||||
};
|
||||
agent.finish();
|
||||
requireUnique(entry.rateRoles, 'SessionDescriptor.agents rateRoles');
|
||||
requireUnique(entry.supportedStimuli, 'SessionDescriptor.agents supportedStimuli');
|
||||
return entry;
|
||||
}),
|
||||
assets: reader.list('assets', 0, MAX_ASSETS, readAssetRef),
|
||||
};
|
||||
reader.finish();
|
||||
requireUnique(
|
||||
descriptor.agents.map((agent) => agent.agentId),
|
||||
'SessionDescriptor.agents agentId',
|
||||
);
|
||||
requireUnique(
|
||||
descriptor.agents.map((agent) => agent.portId),
|
||||
'SessionDescriptor.agents portId',
|
||||
);
|
||||
requireUnique(
|
||||
descriptor.assets.map((asset) => asset.id),
|
||||
'SessionDescriptor.assets',
|
||||
);
|
||||
for (const agent of descriptor.agents) {
|
||||
if (!findPort(descriptor.environment, agent.portId)) {
|
||||
fail(
|
||||
`SessionDescriptor: agent "${agent.agentId}" is bound to port "${agent.portId}", which the environment does not declare`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return descriptor;
|
||||
}
|
||||
|
||||
export function readCommittedSnapshot(value: unknown): CommittedSnapshot {
|
||||
const reader = new Reader(value, 'CommittedSnapshot');
|
||||
const descriptorRevision = reader.u64('descriptorRevision');
|
||||
const publisherIncarnation = reader.id('publisherIncarnation');
|
||||
const scope = readScope(reader.value('scope'));
|
||||
const episodeId = reader.id('episodeId');
|
||||
const sequence = reader.u64('sequence');
|
||||
const worldTime = readRational(reader.value('worldTime'));
|
||||
const agents = reader.list('agents', 1, MAX_AGENTS, (item) => {
|
||||
const agent = new Reader(item, 'CommittedSnapshot.agents');
|
||||
const controls = agent.value('appliedControls');
|
||||
const entry: SnapshotAgent = {
|
||||
agentId: agent.id('agentId'),
|
||||
telemetry: readAgentTelemetry(agent.value('telemetry')),
|
||||
selectedDecision: readNullableTypedValue(agent.value('selectedDecision')),
|
||||
appliedControls: controls === null ? null : readPortControl(controls),
|
||||
};
|
||||
agent.finish();
|
||||
return entry;
|
||||
});
|
||||
const progress = readTypedValue(reader.value('progress'));
|
||||
const mediaReader = new Reader(reader.value('media'), 'CommittedSnapshot.media');
|
||||
const media = {
|
||||
views: readViewList(mediaReader, 'views'),
|
||||
audio: readAudioList(mediaReader, 'audio'),
|
||||
};
|
||||
mediaReader.finish();
|
||||
const eventIds = reader.idList('eventIds', 0, MAX_SNAPSHOT_EVENTS);
|
||||
reader.finish();
|
||||
requireUnique(
|
||||
agents.map((agent) => agent.agentId),
|
||||
'CommittedSnapshot.agents',
|
||||
);
|
||||
requireUnique(
|
||||
media.views.map((view) => view.viewId),
|
||||
'CommittedSnapshot.media.views',
|
||||
);
|
||||
requireUnique(eventIds, 'CommittedSnapshot.eventIds');
|
||||
if (media.views.length > MAX_VIEWS) fail('CommittedSnapshot: at most 8 views');
|
||||
const atBoundaryZero = u64(scope.step) === 0n;
|
||||
for (const agent of agents) {
|
||||
// "Decisions/controls describe the transition ending at that boundary, null at initial
|
||||
// boundary 0." (publishing-v1 section 3)
|
||||
if (atBoundaryZero && (agent.selectedDecision !== null || agent.appliedControls !== null)) {
|
||||
fail('CommittedSnapshot: at boundary 0 selectedDecision and appliedControls are null');
|
||||
}
|
||||
if (!atBoundaryZero && (agent.selectedDecision === null || agent.appliedControls === null)) {
|
||||
fail(
|
||||
'CommittedSnapshot: past boundary 0 every agent has a decision and applied controls',
|
||||
);
|
||||
}
|
||||
}
|
||||
return {
|
||||
descriptorRevision,
|
||||
publisherIncarnation,
|
||||
scope,
|
||||
episodeId,
|
||||
sequence,
|
||||
worldTime,
|
||||
agents,
|
||||
progress,
|
||||
media,
|
||||
eventIds,
|
||||
};
|
||||
}
|
||||
|
||||
/** Descriptor agreement: revision, session, agent set and each agent's assigned port. */
|
||||
export function validateSnapshotAgainst(
|
||||
snapshot: CommittedSnapshot,
|
||||
descriptor: SessionDescriptor,
|
||||
): void {
|
||||
if (snapshot.descriptorRevision !== descriptor.revision) {
|
||||
fail('CommittedSnapshot: descriptorRevision does not match the descriptor');
|
||||
}
|
||||
if (snapshot.scope.sessionId !== descriptor.sessionId) {
|
||||
fail('CommittedSnapshot: sessionId does not match the descriptor');
|
||||
}
|
||||
for (const agent of snapshot.agents) {
|
||||
const declared = descriptor.agents.find((candidate) => candidate.agentId === agent.agentId);
|
||||
if (!declared) {
|
||||
fail(`CommittedSnapshot: agent "${agent.agentId}" is not in the descriptor`);
|
||||
}
|
||||
validateTelemetryRoles(agent.telemetry, declared.rateRoles);
|
||||
if (agent.appliedControls !== null) {
|
||||
if (agent.appliedControls.portId !== declared.portId) {
|
||||
fail(
|
||||
`CommittedSnapshot: agent "${agent.agentId}" controls port "${agent.appliedControls.portId}", not its assigned "${declared.portId}"`,
|
||||
);
|
||||
}
|
||||
const port = findPort(descriptor.environment, declared.portId);
|
||||
if (!port) fail('CommittedSnapshot: assigned port is not declared');
|
||||
validatePortControlAgainst(agent.appliedControls, port.controls);
|
||||
}
|
||||
}
|
||||
}
|
||||
231
packages/session-types/src/reader.ts
Normal file
231
packages/session-types/src/reader.ts
Normal file
|
|
@ -0,0 +1,231 @@
|
|||
/**
|
||||
* Reading one JSON object field by field, then refusing any field that was not read.
|
||||
*
|
||||
* The Rust crate's `flybus::wire::Fields` does the same job; keeping the two shaped alike is
|
||||
* what lets the fixture corpus hold both languages to the same rules.
|
||||
*/
|
||||
import { ContractError, canonicalize, fail } from './canonical';
|
||||
import {
|
||||
type Digest,
|
||||
type Id,
|
||||
type U64,
|
||||
type ArtifactRef,
|
||||
isDigest,
|
||||
isId,
|
||||
parseU64,
|
||||
requireU64,
|
||||
} from './scalar';
|
||||
|
||||
export class Reader {
|
||||
private readonly map: Record<string, unknown>;
|
||||
private readonly seen = new Set<string>();
|
||||
|
||||
constructor(
|
||||
value: unknown,
|
||||
private readonly what: string,
|
||||
) {
|
||||
if (value === null || typeof value !== 'object' || Array.isArray(value)) {
|
||||
fail(`${what} must be an object`);
|
||||
}
|
||||
this.map = value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
value(key: string): unknown {
|
||||
this.seen.add(key);
|
||||
if (!Object.prototype.hasOwnProperty.call(this.map, key)) {
|
||||
fail(`${this.what}: missing field "${key}"`);
|
||||
}
|
||||
return this.map[key];
|
||||
}
|
||||
|
||||
string(key: string): string {
|
||||
const value = this.value(key);
|
||||
if (typeof value !== 'string') fail(`${this.what}: ${key} must be a string`);
|
||||
return value;
|
||||
}
|
||||
|
||||
id(key: string): Id {
|
||||
const value = this.string(key);
|
||||
if (!isId(value)) fail(`${this.what}: ${key} is not a valid id`);
|
||||
return value;
|
||||
}
|
||||
|
||||
nullableId(key: string): Id | null {
|
||||
const value = this.value(key);
|
||||
if (value === null) return null;
|
||||
return this.id(key);
|
||||
}
|
||||
|
||||
digest(key: string): Digest {
|
||||
const value = this.string(key);
|
||||
if (!isDigest(value)) fail(`${this.what}: ${key} must be 64 lowercase hex digits`);
|
||||
return value;
|
||||
}
|
||||
|
||||
u64(key: string): U64 {
|
||||
return requireU64(this.value(key), `${this.what}: ${key}`);
|
||||
}
|
||||
|
||||
int(key: string, low: number, high: number): number {
|
||||
const value = this.value(key);
|
||||
if (typeof value !== 'number' || !Number.isInteger(value) || value < low || value > high) {
|
||||
fail(`${this.what}: ${key} must be an integer in ${low}..=${high}`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
finite(key: string): number {
|
||||
const value = this.value(key);
|
||||
if (typeof value !== 'number' || !Number.isFinite(value)) {
|
||||
fail(`${this.what}: ${key} must be a finite JSON number`);
|
||||
}
|
||||
if (Number.isInteger(value) && Math.abs(value) > Number.MAX_SAFE_INTEGER) {
|
||||
fail(`${this.what}: ${key} is outside the exact double range`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
finiteIn(key: string, low: number, high: number): number {
|
||||
const value = this.finite(key);
|
||||
if (value < low || value > high) fail(`${this.what}: ${key} must be in [${low}, ${high}]`);
|
||||
return value;
|
||||
}
|
||||
|
||||
boolean(key: string): boolean {
|
||||
const value = this.value(key);
|
||||
if (typeof value !== 'boolean') fail(`${this.what}: ${key} must be a boolean`);
|
||||
return value;
|
||||
}
|
||||
|
||||
constantTrue(key: string): true {
|
||||
if (!this.boolean(key)) fail(`${this.what}: ${key} must be true`);
|
||||
return true;
|
||||
}
|
||||
|
||||
object(key: string): Record<string, unknown> {
|
||||
const value = this.value(key);
|
||||
if (value === null || typeof value !== 'object' || Array.isArray(value)) {
|
||||
fail(`${this.what}: ${key} must be an object`);
|
||||
}
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
array(key: string, low: number, high: number): unknown[] {
|
||||
const value = this.value(key);
|
||||
if (!Array.isArray(value) || value.length < low || value.length > high) {
|
||||
fail(`${this.what}: ${key} must be an array of ${low}..=${high} items`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
list<T>(key: string, low: number, high: number, read: (item: unknown) => T): T[] {
|
||||
return this.array(key, low, high).map((item, index) => {
|
||||
try {
|
||||
return read(item);
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
throw new ContractError(`${this.what}: ${key}[${index}]: ${message}`);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
idList(key: string, low: number, high: number): Id[] {
|
||||
return this.list(key, low, high, (item) => {
|
||||
if (!isId(item)) fail('every entry must be an id');
|
||||
return item;
|
||||
});
|
||||
}
|
||||
|
||||
enumeration<T extends string>(key: string, allowed: readonly T[]): T {
|
||||
const value = this.string(key);
|
||||
if (!(allowed as readonly string[]).includes(value)) {
|
||||
fail(`${this.what}: ${key} must be one of ${allowed.join(', ')}`);
|
||||
}
|
||||
return value as T;
|
||||
}
|
||||
|
||||
constant<T extends string>(key: string, expected: T): T {
|
||||
const value = this.string(key);
|
||||
if (value !== expected) fail(`${this.what}: ${key} must be "${expected}"`);
|
||||
return expected;
|
||||
}
|
||||
|
||||
boundedString(key: string, maxCodePoints: number): string {
|
||||
const value = this.string(key);
|
||||
if ([...value].length > maxCodePoints) {
|
||||
fail(`${this.what}: ${key} must be at most ${maxCodePoints} code points`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
nullableBoundedString(key: string, maxCodePoints: number): string | null {
|
||||
return this.value(key) === null ? null : this.boundedString(key, maxCodePoints);
|
||||
}
|
||||
|
||||
/** Refuses fields that were not read. */
|
||||
finish(): void {
|
||||
for (const key of Object.keys(this.map)) {
|
||||
if (!this.seen.has(key)) fail(`${this.what}: unknown field "${key}"`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Fails on the first repeated key, naming it. */
|
||||
export function requireUnique(keys: readonly string[], what: string): void {
|
||||
const seen = new Set<string>();
|
||||
for (const key of keys) {
|
||||
if (seen.has(key)) fail(`${what}: duplicate "${key}"`);
|
||||
seen.add(key);
|
||||
}
|
||||
}
|
||||
|
||||
/** Fails unless `actual` is exactly `expected`, in that order. */
|
||||
export function requireSameOrder(
|
||||
actual: readonly string[],
|
||||
expected: readonly string[],
|
||||
what: string,
|
||||
): void {
|
||||
if (actual.length !== expected.length || actual.some((key, index) => key !== expected[index])) {
|
||||
fail(
|
||||
`${what}: must list [${expected.join(', ')}] in that order, found [${actual.join(', ')}]`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** A bus `ArtifactRef`, read with the bus's own rules. */
|
||||
export function readArtifactRef(value: unknown): ArtifactRef {
|
||||
const reader = new Reader(value, 'ArtifactRef');
|
||||
const storeId = reader.id('storeId');
|
||||
const artifactId = reader.id('artifactId');
|
||||
const generation = reader.u64('generation');
|
||||
const byteLength = reader.u64('byteLength');
|
||||
const contentType = reader.string('contentType');
|
||||
const digestValue = reader.value('digest');
|
||||
reader.finish();
|
||||
if (contentType.length < 1 || contentType.length > 127) {
|
||||
fail('ArtifactRef: contentType must be 1..=127 printable ASCII characters');
|
||||
}
|
||||
if (digestValue !== null && !isDigest(digestValue)) {
|
||||
fail('ArtifactRef: digest must be null or 64 lowercase hex digits');
|
||||
}
|
||||
return {
|
||||
storeId,
|
||||
artifactId,
|
||||
generation,
|
||||
byteLength,
|
||||
contentType,
|
||||
digest: digestValue as ArtifactRef['digest'],
|
||||
};
|
||||
}
|
||||
|
||||
/** The canonical JSON byte length of a value. */
|
||||
export function canonicalLength(value: unknown): number {
|
||||
return canonicalize(value).length;
|
||||
}
|
||||
|
||||
/** `parseU64` that throws, for places that have already validated the string. */
|
||||
export function u64(value: U64): bigint {
|
||||
const parsed = parseU64(value);
|
||||
if (parsed === undefined) fail(`${value} is not a canonical U64 string`);
|
||||
return parsed;
|
||||
}
|
||||
140
packages/session-types/src/rpc.ts
Normal file
140
packages/session-types/src/rpc.ts
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
/** The domain request/reply envelope of ipc-v1 section 3 and the error codes of section 7. */
|
||||
import { fail, rejectBusIdentities } from './canonical';
|
||||
import { bodyDigest, readNullableScope } from './common';
|
||||
import { Reader } from './reader';
|
||||
import { type DomainRequestId, type Id, type Scope, domainRequestId } from './scalar';
|
||||
import { MAX_MESSAGE_CODE_POINTS } from './workers';
|
||||
|
||||
export const ERROR_CODES = [
|
||||
'INVALID_ARGUMENT',
|
||||
'UNSUPPORTED',
|
||||
'IDENTITY_MISMATCH',
|
||||
'STALE_EPOCH',
|
||||
'STALE_STEP',
|
||||
'FUTURE_STEP',
|
||||
'INVALID_PHASE',
|
||||
'CONFLICT',
|
||||
'IN_PROGRESS',
|
||||
'BUSY',
|
||||
'BUFFER_INVALID',
|
||||
'RESULT_EXPIRED',
|
||||
'INCOMPATIBLE_STATE',
|
||||
'BACKEND_FAILURE',
|
||||
'INTERNAL',
|
||||
] as const;
|
||||
export type ErrorCode = (typeof ERROR_CODES)[number];
|
||||
|
||||
export const MUTATION_CERTAINTIES = ['none', 'applied', 'unknown'] as const;
|
||||
export type MutationCertainty = (typeof MUTATION_CERTAINTIES)[number];
|
||||
|
||||
/** The codes raised strictly before any mutation, so their certainty is `none`. */
|
||||
export const BEFORE_MUTATION: readonly ErrorCode[] = [
|
||||
'INVALID_ARGUMENT',
|
||||
'UNSUPPORTED',
|
||||
'IDENTITY_MISMATCH',
|
||||
'STALE_EPOCH',
|
||||
'STALE_STEP',
|
||||
'FUTURE_STEP',
|
||||
'INVALID_PHASE',
|
||||
'CONFLICT',
|
||||
'IN_PROGRESS',
|
||||
'BUSY',
|
||||
'BUFFER_INVALID',
|
||||
'INCOMPATIBLE_STATE',
|
||||
];
|
||||
|
||||
export interface SessionRpcRequest {
|
||||
requestId: DomainRequestId;
|
||||
scope: Scope | null;
|
||||
params: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export interface SessionRpcSuccess {
|
||||
type: 'result';
|
||||
requestId: DomainRequestId;
|
||||
workerId: Id;
|
||||
incarnationId: Id;
|
||||
scope: Scope | null;
|
||||
result: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export interface SessionRpcFailure {
|
||||
type: 'error';
|
||||
requestId: DomainRequestId;
|
||||
workerId: Id;
|
||||
incarnationId: Id;
|
||||
scope: Scope | null;
|
||||
error: { code: ErrorCode; message: string; mutation: MutationCertainty };
|
||||
}
|
||||
|
||||
export type SessionRpcOutcome = SessionRpcSuccess | SessionRpcFailure;
|
||||
|
||||
export function readSessionRpcRequest(value: unknown): SessionRpcRequest {
|
||||
const reader = new Reader(value, 'SessionRpcRequest');
|
||||
const request: SessionRpcRequest = {
|
||||
requestId: domainRequestId(reader.string('requestId')),
|
||||
scope: readNullableScope(reader.value('scope')),
|
||||
params: reader.object('params'),
|
||||
};
|
||||
reader.finish();
|
||||
rejectBusIdentities(request.params);
|
||||
return request;
|
||||
}
|
||||
|
||||
/** The canonical body digest of this request under `method` (ipc-v1 section 5). */
|
||||
export function requestBodyDigest(request: SessionRpcRequest, method: string): string {
|
||||
return bodyDigest(method, request.scope, request.params);
|
||||
}
|
||||
|
||||
export function readSessionRpcSuccess(value: unknown): SessionRpcSuccess {
|
||||
const reader = new Reader(value, 'SessionRpcSuccess');
|
||||
const success: SessionRpcSuccess = {
|
||||
type: reader.constant('type', 'result'),
|
||||
requestId: domainRequestId(reader.string('requestId')),
|
||||
workerId: reader.id('workerId'),
|
||||
incarnationId: reader.id('incarnationId'),
|
||||
scope: readNullableScope(reader.value('scope')),
|
||||
result: reader.object('result'),
|
||||
};
|
||||
reader.finish();
|
||||
return success;
|
||||
}
|
||||
|
||||
export function readSessionRpcFailure(value: unknown): SessionRpcFailure {
|
||||
const reader = new Reader(value, 'SessionRpcFailure');
|
||||
const type = reader.constant('type', 'error');
|
||||
const requestId = domainRequestId(reader.string('requestId'));
|
||||
const workerId = reader.id('workerId');
|
||||
const incarnationId = reader.id('incarnationId');
|
||||
const scope = readNullableScope(reader.value('scope'));
|
||||
const errorReader = new Reader(reader.value('error'), 'SessionRpcFailure.error');
|
||||
const error = {
|
||||
code: errorReader.enumeration('code', ERROR_CODES),
|
||||
message: errorReader.boundedString('message', MAX_MESSAGE_CODE_POINTS),
|
||||
mutation: errorReader.enumeration('mutation', MUTATION_CERTAINTIES),
|
||||
};
|
||||
errorReader.finish();
|
||||
reader.finish();
|
||||
if (BEFORE_MUTATION.includes(error.code) && error.mutation !== 'none') {
|
||||
fail(`SessionRpcFailure: ${error.code} is raised before mutation, so mutation is "none"`);
|
||||
}
|
||||
return { type, requestId, workerId, incarnationId, scope, error };
|
||||
}
|
||||
|
||||
export function readSessionRpcOutcome(value: unknown): SessionRpcOutcome {
|
||||
const type = (value as { type?: unknown } | null)?.type;
|
||||
if (type === 'result') return readSessionRpcSuccess(value);
|
||||
if (type === 'error') return readSessionRpcFailure(value);
|
||||
return fail('SessionRpcOutcome: type must be "result" or "error"');
|
||||
}
|
||||
|
||||
/** Replies echo the original scope (ipc-v1 section 3). */
|
||||
export function echoes(outcome: SessionRpcOutcome, request: SessionRpcRequest): boolean {
|
||||
const sameScope =
|
||||
outcome.scope === null || request.scope === null
|
||||
? outcome.scope === request.scope
|
||||
: outcome.scope.sessionId === request.scope.sessionId &&
|
||||
outcome.scope.epoch === request.scope.epoch &&
|
||||
outcome.scope.step === request.scope.step;
|
||||
return outcome.requestId === request.requestId && sameScope;
|
||||
}
|
||||
262
packages/session-types/src/scalar.ts
Normal file
262
packages/session-types/src/scalar.ts
Normal file
|
|
@ -0,0 +1,262 @@
|
|||
/**
|
||||
* The domain scalars of ipc-v1 section 2, and the four identities that must never be confused.
|
||||
*
|
||||
* `Id`, `U64` and `Digest` are the bus encodings (bus-v1 section 3 defers to ipc-v1 for them),
|
||||
* and `tests/encodings.test.ts` pins the same edge cases the Rust crate pins.
|
||||
*/
|
||||
import { fail } from './canonical';
|
||||
|
||||
/** `^[a-z0-9][a-z0-9._-]{0,63}$`. */
|
||||
export type Id = string;
|
||||
/** `"0"` or `[1-9][0-9]*`, at most 18446744073709551615. A counter, never a JSON number. */
|
||||
export type U64 = string;
|
||||
/** 64 lowercase hexadecimal digits (SHA-256). */
|
||||
export type Digest = string;
|
||||
|
||||
const ID = /^[a-z0-9][a-z0-9._-]{0,63}$/;
|
||||
const U64_TEXT = /^(0|[1-9][0-9]*)$/;
|
||||
const DIGEST = /^[0-9a-f]{64}$/;
|
||||
/** The largest U64, as a bigint. */
|
||||
export const U64_MAX = 18446744073709551615n;
|
||||
|
||||
export function isId(value: unknown): value is Id {
|
||||
return typeof value === 'string' && ID.test(value);
|
||||
}
|
||||
|
||||
export function isDigest(value: unknown): value is Digest {
|
||||
return typeof value === 'string' && DIGEST.test(value);
|
||||
}
|
||||
|
||||
/** The value a `U64` string denotes, or `undefined` if it is not canonical. */
|
||||
export function parseU64(value: unknown): bigint | undefined {
|
||||
if (typeof value !== 'string' || !U64_TEXT.test(value)) return undefined;
|
||||
const parsed = BigInt(value);
|
||||
return parsed <= U64_MAX ? parsed : undefined;
|
||||
}
|
||||
|
||||
export function requireU64(value: unknown, what: string): U64 {
|
||||
if (parseU64(value) === undefined) fail(`${what} is not a canonical U64 string`);
|
||||
return value as U64;
|
||||
}
|
||||
|
||||
/** RPC method: 1..=128 printable ASCII characters (bus-v1 section 5). */
|
||||
export function isMethod(value: unknown): boolean {
|
||||
return (
|
||||
typeof value === 'string' &&
|
||||
value.length >= 1 &&
|
||||
value.length <= 128 &&
|
||||
[...value].every((character) => {
|
||||
const point = character.codePointAt(0) ?? 0;
|
||||
return point >= 0x20 && point <= 0x7e;
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// The four identities
|
||||
|
||||
declare const brand: unique symbol;
|
||||
/** A branded string: assignable only through its own parser. */
|
||||
type Branded<Name extends string> = string & { readonly [brand]: Name };
|
||||
|
||||
/** A bus RPC correlation id, `call-<U64>`. Not a domain operation id. */
|
||||
export type BusCallId = Branded<'BusCallId'>;
|
||||
/** A domain operation id, `req-<U64>`. A safe retry keeps it and gets a new `BusCallId`. */
|
||||
export type DomainRequestId = Branded<'DomainRequestId'>;
|
||||
/** A delivery (`dlv-<U64>`) or explicit-hold (`own-<U64>`) owner token. Connection-private. */
|
||||
export type OwnerToken = Branded<'OwnerToken'>;
|
||||
|
||||
export type OwnerKind = 'delivery' | 'hold';
|
||||
|
||||
function serial(prefix: string, value: unknown): bigint | undefined {
|
||||
if (typeof value !== 'string' || !value.startsWith(`${prefix}-`)) return undefined;
|
||||
return parseU64(value.slice(prefix.length + 1));
|
||||
}
|
||||
|
||||
export function isBusCallId(value: unknown): value is BusCallId {
|
||||
return serial('call', value) !== undefined;
|
||||
}
|
||||
|
||||
export function isDomainRequestId(value: unknown): value is DomainRequestId {
|
||||
return serial('req', value) !== undefined;
|
||||
}
|
||||
|
||||
export function ownerTokenKind(value: unknown): OwnerKind | undefined {
|
||||
if (serial('dlv', value) !== undefined) return 'delivery';
|
||||
if (serial('own', value) !== undefined) return 'hold';
|
||||
return undefined;
|
||||
}
|
||||
|
||||
export function busCallId(value: unknown): BusCallId {
|
||||
if (!isBusCallId(value)) fail('a bus callId must be canonical call-<U64>');
|
||||
return value;
|
||||
}
|
||||
|
||||
export function domainRequestId(value: unknown): DomainRequestId {
|
||||
if (!isDomainRequestId(value)) fail('a domain requestId must be canonical req-<U64>');
|
||||
return value;
|
||||
}
|
||||
|
||||
export function ownerToken(value: unknown): OwnerToken {
|
||||
if (ownerTokenKind(value) === undefined) {
|
||||
fail('an owner token must be canonical dlv-<U64> or own-<U64>');
|
||||
}
|
||||
return value as OwnerToken;
|
||||
}
|
||||
|
||||
/** The naming half of a bus `ArtifactRef`: what identifies the bytes. */
|
||||
export interface ArtifactIdentity {
|
||||
storeId: Id;
|
||||
artifactId: Id;
|
||||
generation: U64;
|
||||
}
|
||||
|
||||
/** A transient bus artifact reference (bus-v1 section 4). Never an `AssetRef`. */
|
||||
export interface ArtifactRef {
|
||||
storeId: Id;
|
||||
artifactId: Id;
|
||||
generation: U64;
|
||||
byteLength: U64;
|
||||
contentType: string;
|
||||
digest: Digest | null;
|
||||
}
|
||||
|
||||
export function artifactIdentity(reference: ArtifactRef): ArtifactIdentity {
|
||||
return {
|
||||
storeId: reference.storeId,
|
||||
artifactId: reference.artifactId,
|
||||
generation: reference.generation,
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// RationalNs
|
||||
|
||||
/** A nanosecond rational: reduced, positive denominator, zero encoded `0/1`. */
|
||||
export interface RationalNs {
|
||||
numerator: U64;
|
||||
denominator: U64;
|
||||
}
|
||||
|
||||
export const RATIONAL_ZERO: RationalNs = { numerator: '0', denominator: '1' };
|
||||
|
||||
function gcd(a: bigint, b: bigint): bigint {
|
||||
let left = a;
|
||||
let right = b;
|
||||
while (right !== 0n) {
|
||||
const rest = left % right;
|
||||
left = right;
|
||||
right = rest;
|
||||
}
|
||||
return left;
|
||||
}
|
||||
|
||||
function parts(value: RationalNs, what: string): [bigint, bigint] {
|
||||
const numerator = parseU64(value.numerator);
|
||||
const denominator = parseU64(value.denominator);
|
||||
if (numerator === undefined || denominator === undefined) {
|
||||
fail(`${what}: numerator and denominator are U64 strings`);
|
||||
}
|
||||
return [numerator, denominator];
|
||||
}
|
||||
|
||||
/** The canonical-form rules: positive denominator, `0/1` zero, reduced fraction. */
|
||||
export function validateRational(value: RationalNs, what = 'RationalNs'): void {
|
||||
const [numerator, denominator] = parts(value, what);
|
||||
if (denominator === 0n) fail(`${what}: denominator must be positive`);
|
||||
if (numerator === 0n && denominator !== 1n) fail(`${what}: zero is encoded 0/1`);
|
||||
if (numerator !== 0n && gcd(numerator, denominator) !== 1n) {
|
||||
fail(`${what}: fraction must be reduced`);
|
||||
}
|
||||
}
|
||||
|
||||
/** Reduces, then validates: the constructor for arithmetic results. */
|
||||
export function reduced(numerator: bigint, denominator: bigint): RationalNs {
|
||||
if (denominator <= 0n) fail('RationalNs: denominator must be positive');
|
||||
let n = numerator;
|
||||
let d = denominator;
|
||||
if (n === 0n) {
|
||||
d = 1n;
|
||||
} else {
|
||||
const divisor = gcd(n, d);
|
||||
n /= divisor;
|
||||
d /= divisor;
|
||||
}
|
||||
if (n > U64_MAX || d > U64_MAX) fail('RationalNs: reduced value does not fit U64');
|
||||
return { numerator: n.toString(), denominator: d.toString() };
|
||||
}
|
||||
|
||||
export function isRationalZero(value: RationalNs): boolean {
|
||||
return parseU64(value.numerator) === 0n;
|
||||
}
|
||||
|
||||
export function requirePositiveRational(value: RationalNs, what: string): void {
|
||||
if (isRationalZero(value)) fail(`${what}: duration must be positive`);
|
||||
}
|
||||
|
||||
export function addRational(left: RationalNs, right: RationalNs): RationalNs {
|
||||
const [ln, ld] = parts(left, 'RationalNs');
|
||||
const [rn, rd] = parts(right, 'RationalNs');
|
||||
return reduced(ln * rd + rn * ld, ld * rd);
|
||||
}
|
||||
|
||||
export function subtractRational(left: RationalNs, right: RationalNs): RationalNs {
|
||||
const [ln, ld] = parts(left, 'RationalNs');
|
||||
const [rn, rd] = parts(right, 'RationalNs');
|
||||
const a = ln * rd;
|
||||
const b = rn * ld;
|
||||
if (b > a) fail('RationalNs: subtraction would be negative');
|
||||
return reduced(a - b, ld * rd);
|
||||
}
|
||||
|
||||
export function multiplyRational(value: RationalNs, factor: bigint): RationalNs {
|
||||
const [numerator, denominator] = parts(value, 'RationalNs');
|
||||
return reduced(numerator * factor, denominator);
|
||||
}
|
||||
|
||||
export function compareRational(left: RationalNs, right: RationalNs): -1 | 0 | 1 {
|
||||
const [ln, ld] = parts(left, 'RationalNs');
|
||||
const [rn, rd] = parts(right, 'RationalNs');
|
||||
const a = ln * rd;
|
||||
const b = rn * ld;
|
||||
return a < b ? -1 : a > b ? 1 : 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* The step-v1 section 5 accumulator: `ticks = floor(value / tick)` and the remainder
|
||||
* `value - ticks * tick`, which is always `>= 0` and `< tick`.
|
||||
*/
|
||||
export function divideFloor(value: RationalNs, tick: RationalNs): { ticks: U64; remainder: RationalNs } {
|
||||
requirePositiveRational(tick, 'RationalNs divideFloor tick');
|
||||
const [vn, vd] = parts(value, 'RationalNs');
|
||||
const [tn, td] = parts(tick, 'RationalNs');
|
||||
const ticks = (vn * td) / (vd * tn);
|
||||
if (ticks > U64_MAX) fail('RationalNs: tick count does not fit U64');
|
||||
const remainder = subtractRational(value, multiplyRational(tick, ticks));
|
||||
return { ticks: ticks.toString(), remainder };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// Scope, SchemaRef, TypedValue
|
||||
|
||||
/** The simulation timeline identity. Never a bus route or store incarnation. */
|
||||
export interface Scope {
|
||||
sessionId: Id;
|
||||
epoch: Id;
|
||||
step: U64;
|
||||
}
|
||||
|
||||
export interface SchemaRef {
|
||||
id: Id;
|
||||
version: number;
|
||||
digest: Digest;
|
||||
}
|
||||
|
||||
/** A schema identity plus an object, capped at 32 KiB of canonical JSON. */
|
||||
export interface TypedValue {
|
||||
schema: SchemaRef;
|
||||
value: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** The canonical-JSON size limit of one `TypedValue`. */
|
||||
export const MAX_TYPED_VALUE_BYTES = 32 * 1024;
|
||||
57
packages/session-types/src/seed.ts
Normal file
57
packages/session-types/src/seed.ts
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
/**
|
||||
* `seed-derivation-v1`: independent per-agent seeds from one recorded master seed.
|
||||
*
|
||||
* The specification is `docs/design/session-framework/seed-derivation-v1.md`, and
|
||||
* `fixtures/seed-vectors.json` its test vectors, which the Rust crate reproduces.
|
||||
*/
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
import { fail } from './canonical';
|
||||
import { requireUnique } from './reader';
|
||||
import { isId, parseU64 } from './scalar';
|
||||
|
||||
export const ALGORITHM = 'seed-derivation-v1';
|
||||
export const PREFIX = 'flybrain/seed-derivation-v1';
|
||||
|
||||
/** The exact bytes hashed: prefix, master seed and agent id, each followed by one newline. */
|
||||
export function material(masterSeed: bigint, agentId: string): Uint8Array {
|
||||
if (!isId(agentId)) fail('seed derivation: agentId is not a valid id');
|
||||
if (masterSeed < 0n || masterSeed > 18446744073709551615n) {
|
||||
fail('seed derivation: the master seed is a U64');
|
||||
}
|
||||
return new TextEncoder().encode(`${PREFIX}\n${masterSeed.toString()}\n${agentId}\n`);
|
||||
}
|
||||
|
||||
export function materialDigest(masterSeed: bigint, agentId: string): string {
|
||||
return createHash('sha256').update(material(masterSeed, agentId)).digest('hex');
|
||||
}
|
||||
|
||||
/**
|
||||
* The signed 32-bit seed `Agent.Initialize` takes for `agentId`: the first nonzero big-endian
|
||||
* `u32` lane of the digest, as a two's-complement `i32`.
|
||||
*/
|
||||
export function agentSeed(masterSeed: bigint, agentId: string): number {
|
||||
let bytes = Buffer.from(material(masterSeed, agentId));
|
||||
for (let round = 0; round < 4; round += 1) {
|
||||
if (round > 0) bytes = Buffer.concat([bytes, Buffer.from(`${round}\n`, 'utf8')]);
|
||||
const digest = createHash('sha256').update(bytes).digest();
|
||||
for (let offset = 0; offset < digest.length; offset += 4) {
|
||||
const word = digest.readUInt32BE(offset);
|
||||
if (word !== 0) return word | 0;
|
||||
}
|
||||
}
|
||||
return fail('seed derivation: every lane of four digests was zero');
|
||||
}
|
||||
|
||||
/** The seeds of a whole composition, in the order the agent ids are given. */
|
||||
export function compositionSeeds(masterSeed: bigint, agentIds: readonly string[]): number[] {
|
||||
requireUnique(agentIds, 'seed derivation: agentIds');
|
||||
return agentIds.map((agentId) => agentSeed(masterSeed, agentId));
|
||||
}
|
||||
|
||||
/** Parses a master seed from its `U64` decimal string. */
|
||||
export function masterSeed(text: string): bigint {
|
||||
const parsed = parseU64(text);
|
||||
if (parsed === undefined) fail('seed derivation: the master seed is a canonical U64 string');
|
||||
return parsed;
|
||||
}
|
||||
268
packages/session-types/src/trace.ts
Normal file
268
packages/session-types/src/trace.ts
Normal file
|
|
@ -0,0 +1,268 @@
|
|||
/**
|
||||
* The trace format of step-v1 section 8, split into behaviour and operational metadata.
|
||||
*
|
||||
* `behaviourEquals` compares the first half only, which is the comparison section 8 asks for:
|
||||
* sequential, concurrent and reversed runs must agree "excluding wall time, request ids and
|
||||
* other explicitly operational metadata".
|
||||
*/
|
||||
import { canonicalize, digestOf, fail } from './canonical';
|
||||
import { readRational, readScope } from './common';
|
||||
import { MAX_VIEWS } from './media';
|
||||
import { Reader, requireUnique, u64 } from './reader';
|
||||
import {
|
||||
type BusCallId,
|
||||
type Digest,
|
||||
type DomainRequestId,
|
||||
type Id,
|
||||
type OwnerToken,
|
||||
type RationalNs,
|
||||
type Scope,
|
||||
type U64,
|
||||
busCallId,
|
||||
domainRequestId,
|
||||
ownerToken,
|
||||
} from './scalar';
|
||||
import { MAX_AGENTS, MAX_RATE_ROLES } from './workers';
|
||||
|
||||
export interface TraceAgent {
|
||||
agentId: Id;
|
||||
profileDigest: Digest;
|
||||
ticksAdvanced: U64;
|
||||
brainTicks: U64;
|
||||
remainder: RationalNs;
|
||||
decisionDigest: Digest;
|
||||
committedStep: U64;
|
||||
}
|
||||
|
||||
export interface TraceObservation {
|
||||
viewId: Id;
|
||||
producedStep: U64;
|
||||
}
|
||||
|
||||
export interface TraceBehaviour {
|
||||
scope: Scope;
|
||||
agents: TraceAgent[];
|
||||
batchId: Id;
|
||||
controlDigest: Digest;
|
||||
acknowledgedBoundary: U64;
|
||||
observationBoundaries: TraceObservation[];
|
||||
outcomeIds: Id[];
|
||||
eventIds: Id[];
|
||||
publishedBoundary: U64;
|
||||
}
|
||||
|
||||
export interface TraceRequest {
|
||||
agentId: Id;
|
||||
requestId: DomainRequestId;
|
||||
}
|
||||
|
||||
export interface TraceOperational {
|
||||
wallTimeNs: U64;
|
||||
prepareRequestIds: TraceRequest[];
|
||||
advanceRequestId: DomainRequestId;
|
||||
commitRequestIds: TraceRequest[];
|
||||
busCallIds: BusCallId[];
|
||||
deliveryIds: OwnerToken[];
|
||||
}
|
||||
|
||||
export interface TransitionTrace {
|
||||
behaviour: TraceBehaviour;
|
||||
operational: TraceOperational;
|
||||
}
|
||||
|
||||
export function readTraceBehaviour(value: unknown): TraceBehaviour {
|
||||
const reader = new Reader(value, 'TraceBehaviour');
|
||||
const scope = readScope(reader.value('scope'));
|
||||
const agents = reader.list('agents', 1, MAX_AGENTS, (item) => {
|
||||
const agent = new Reader(item, 'TraceBehaviour.agents');
|
||||
const entry: TraceAgent = {
|
||||
agentId: agent.id('agentId'),
|
||||
profileDigest: agent.digest('profileDigest'),
|
||||
ticksAdvanced: agent.u64('ticksAdvanced'),
|
||||
brainTicks: agent.u64('brainTicks'),
|
||||
remainder: readRational(agent.value('remainder')),
|
||||
decisionDigest: agent.digest('decisionDigest'),
|
||||
committedStep: agent.u64('committedStep'),
|
||||
};
|
||||
agent.finish();
|
||||
return entry;
|
||||
});
|
||||
const batchId = reader.id('batchId');
|
||||
const controlDigest = reader.digest('controlDigest');
|
||||
const acknowledgedBoundary = reader.u64('acknowledgedBoundary');
|
||||
const observationBoundaries = reader.list(
|
||||
'observationBoundaries',
|
||||
0,
|
||||
MAX_VIEWS * 2,
|
||||
(item) => {
|
||||
const observation = new Reader(item, 'TraceBehaviour.observationBoundaries');
|
||||
const entry: TraceObservation = {
|
||||
viewId: observation.id('viewId'),
|
||||
producedStep: observation.u64('producedStep'),
|
||||
};
|
||||
observation.finish();
|
||||
return entry;
|
||||
},
|
||||
);
|
||||
const outcomeIds = reader.idList('outcomeIds', 0, MAX_RATE_ROLES);
|
||||
const eventIds = reader.idList('eventIds', 0, MAX_RATE_ROLES);
|
||||
const publishedBoundary = reader.u64('publishedBoundary');
|
||||
reader.finish();
|
||||
|
||||
requireUnique(
|
||||
agents.map((agent) => agent.agentId),
|
||||
'TraceBehaviour.agents',
|
||||
);
|
||||
requireUnique(
|
||||
observationBoundaries.map((observation) => observation.viewId),
|
||||
'TraceBehaviour.observationBoundaries',
|
||||
);
|
||||
requireUnique(eventIds, 'TraceBehaviour.eventIds');
|
||||
requireUnique(outcomeIds, 'TraceBehaviour.outcomeIds');
|
||||
const next = u64(scope.step) + 1n;
|
||||
for (const agent of agents) {
|
||||
if (u64(agent.committedStep) !== next) {
|
||||
fail("TraceBehaviour: every commit acknowledgment is the transition's next boundary");
|
||||
}
|
||||
}
|
||||
if (u64(acknowledgedBoundary) !== next) {
|
||||
fail('TraceBehaviour: the acknowledged boundary is scope.step + 1');
|
||||
}
|
||||
if (u64(publishedBoundary) !== u64(acknowledgedBoundary)) {
|
||||
fail('TraceBehaviour: the published boundary is the boundary every agent committed');
|
||||
}
|
||||
return {
|
||||
scope,
|
||||
agents,
|
||||
batchId,
|
||||
controlDigest,
|
||||
acknowledgedBoundary,
|
||||
observationBoundaries,
|
||||
outcomeIds,
|
||||
eventIds,
|
||||
publishedBoundary,
|
||||
};
|
||||
}
|
||||
|
||||
export function readTraceOperational(value: unknown): TraceOperational {
|
||||
const reader = new Reader(value, 'TraceOperational');
|
||||
const readRequests = (item: unknown): TraceRequest => {
|
||||
const request = new Reader(item, 'TraceOperational request');
|
||||
const entry: TraceRequest = {
|
||||
agentId: request.id('agentId'),
|
||||
requestId: domainRequestId(request.string('requestId')),
|
||||
};
|
||||
request.finish();
|
||||
return entry;
|
||||
};
|
||||
const operational: TraceOperational = {
|
||||
wallTimeNs: reader.u64('wallTimeNs'),
|
||||
prepareRequestIds: reader.list('prepareRequestIds', 1, MAX_AGENTS, readRequests),
|
||||
advanceRequestId: domainRequestId(reader.string('advanceRequestId')),
|
||||
commitRequestIds: reader.list('commitRequestIds', 1, MAX_AGENTS, readRequests),
|
||||
busCallIds: reader.list('busCallIds', 0, 64, busCallId),
|
||||
deliveryIds: reader.list('deliveryIds', 0, 64, ownerToken),
|
||||
};
|
||||
reader.finish();
|
||||
requireUnique(
|
||||
operational.prepareRequestIds.map((request) => request.agentId),
|
||||
'TraceOperational.prepareRequestIds',
|
||||
);
|
||||
requireUnique(
|
||||
operational.commitRequestIds.map((request) => request.agentId),
|
||||
'TraceOperational.commitRequestIds',
|
||||
);
|
||||
requireUnique(operational.busCallIds, 'TraceOperational.busCallIds');
|
||||
requireUnique(operational.deliveryIds, 'TraceOperational.deliveryIds');
|
||||
return operational;
|
||||
}
|
||||
|
||||
export function readTransitionTrace(value: unknown): TransitionTrace {
|
||||
const reader = new Reader(value, 'TransitionTrace');
|
||||
const trace: TransitionTrace = {
|
||||
behaviour: readTraceBehaviour(reader.value('behaviour')),
|
||||
operational: readTraceOperational(reader.value('operational')),
|
||||
};
|
||||
reader.finish();
|
||||
const agents = trace.behaviour.agents.map((agent) => agent.agentId);
|
||||
for (const phase of [trace.operational.prepareRequestIds, trace.operational.commitRequestIds]) {
|
||||
for (const request of phase) {
|
||||
if (!agents.includes(request.agentId)) {
|
||||
fail(
|
||||
`TransitionTrace: request recorded for "${request.agentId}", which is not in the transition`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
return trace;
|
||||
}
|
||||
|
||||
/** Sorts the order-free collections, so completion order cannot change the comparison. */
|
||||
export function normalizeBehaviour(behaviour: TraceBehaviour): TraceBehaviour {
|
||||
return {
|
||||
...behaviour,
|
||||
agents: [...behaviour.agents].sort((left, right) =>
|
||||
left.agentId < right.agentId ? -1 : left.agentId > right.agentId ? 1 : 0,
|
||||
),
|
||||
observationBoundaries: [...behaviour.observationBoundaries].sort((left, right) =>
|
||||
left.viewId < right.viewId ? -1 : left.viewId > right.viewId ? 1 : 0,
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
export function behaviourDigest(behaviour: TraceBehaviour): string {
|
||||
return digestOf(normalizeBehaviour(behaviour));
|
||||
}
|
||||
|
||||
export function behaviourEquals(left: TransitionTrace, right: TransitionTrace): boolean {
|
||||
return (
|
||||
canonicalize(normalizeBehaviour(left.behaviour)) ===
|
||||
canonicalize(normalizeBehaviour(right.behaviour))
|
||||
);
|
||||
}
|
||||
|
||||
/** The behaviour fields that differ, named. Empty when `behaviourEquals` holds. */
|
||||
export function behaviourDiff(left: TransitionTrace, right: TransitionTrace): string[] {
|
||||
const a = normalizeBehaviour(left.behaviour);
|
||||
const b = normalizeBehaviour(right.behaviour);
|
||||
const out: string[] = [];
|
||||
const differs = (first: unknown, second: unknown): boolean =>
|
||||
canonicalize(first) !== canonicalize(second);
|
||||
if (differs(a.scope, b.scope)) out.push(`scope: ${canonicalize(a.scope)} vs ${canonicalize(b.scope)}`);
|
||||
if (a.batchId !== b.batchId) out.push(`batchId: ${a.batchId} vs ${b.batchId}`);
|
||||
if (a.controlDigest !== b.controlDigest) out.push('controlDigest differs');
|
||||
if (a.acknowledgedBoundary !== b.acknowledgedBoundary) {
|
||||
out.push(`acknowledgedBoundary: ${a.acknowledgedBoundary} vs ${b.acknowledgedBoundary}`);
|
||||
}
|
||||
if (a.publishedBoundary !== b.publishedBoundary) {
|
||||
out.push(`publishedBoundary: ${a.publishedBoundary} vs ${b.publishedBoundary}`);
|
||||
}
|
||||
if (differs(a.observationBoundaries, b.observationBoundaries)) {
|
||||
out.push('observationBoundaries differ');
|
||||
}
|
||||
if (differs(a.outcomeIds, b.outcomeIds)) out.push('outcomeIds differ');
|
||||
if (differs(a.eventIds, b.eventIds)) out.push('eventIds differ');
|
||||
const idsA = a.agents.map((agent) => agent.agentId);
|
||||
const idsB = b.agents.map((agent) => agent.agentId);
|
||||
if (differs(idsA, idsB)) {
|
||||
out.push(`agents: [${idsA.join(', ')}] vs [${idsB.join(', ')}]`);
|
||||
} else {
|
||||
a.agents.forEach((agent, index) => {
|
||||
if (differs(agent, b.agents[index])) {
|
||||
out.push(`agent ${agent.agentId}: behaviour differs`);
|
||||
}
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Two whole runs agree on behaviour, transition by transition. */
|
||||
export function runsEqual(
|
||||
left: readonly TransitionTrace[],
|
||||
right: readonly TransitionTrace[],
|
||||
): boolean {
|
||||
return (
|
||||
left.length === right.length &&
|
||||
left.every((trace, index) => behaviourEquals(trace, right[index] as TransitionTrace))
|
||||
);
|
||||
}
|
||||
996
packages/session-types/src/workers.ts
Normal file
996
packages/session-types/src/workers.ts
Normal file
|
|
@ -0,0 +1,996 @@
|
|||
/** The closed enums and method payloads of workers-v1, plus the `Worker.*` common methods. */
|
||||
import { fail } from './canonical';
|
||||
import { readRational, readScope } from './common';
|
||||
import {
|
||||
type AudioDescriptor,
|
||||
type AudioRef,
|
||||
MAX_VIEWS,
|
||||
type ViewDescriptor,
|
||||
type ViewRef,
|
||||
readAudioDescriptor,
|
||||
readAudioList,
|
||||
readViewDescriptor,
|
||||
readViewList,
|
||||
validateAudioAgainst,
|
||||
validateViewAgainst,
|
||||
} from './media';
|
||||
import { Reader, requireSameOrder, requireUnique, u64 } from './reader';
|
||||
import {
|
||||
type Digest,
|
||||
type DomainRequestId,
|
||||
type Id,
|
||||
type RationalNs,
|
||||
type SchemaRef,
|
||||
type Scope,
|
||||
type TypedValue,
|
||||
type U64,
|
||||
addRational,
|
||||
compareRational,
|
||||
domainRequestId,
|
||||
isRationalZero,
|
||||
requirePositiveRational,
|
||||
} from './scalar';
|
||||
import { readSchemaRef, readTypedValue, readNullableTypedValue } from './common';
|
||||
|
||||
export const MAX_AGENTS = 4;
|
||||
export const MAX_PORTS = 4;
|
||||
export const MAX_RATE_ROLES = 64;
|
||||
export const MAX_STIMULI = 64;
|
||||
export const MAX_REWARDS = 64;
|
||||
export const MAX_BUTTONS = 32;
|
||||
export const MAX_AXES = 16;
|
||||
export const MAX_ACKNOWLEDGE = 16;
|
||||
export const MAX_ENGINE_FRAME_LEN = 64;
|
||||
/** Not stated by a document; this crate's choice, published in the schema set. */
|
||||
export const MAX_CAPABILITIES = 32;
|
||||
export const MAX_SUPPORTED_MAJORS = 8;
|
||||
export const MAX_MESSAGE_CODE_POINTS = 512;
|
||||
|
||||
export const ROLES = ['agent', 'environment', 'coordinator'] as const;
|
||||
export type Role = (typeof ROLES)[number];
|
||||
|
||||
export const WORKER_STATES = [
|
||||
'uninitialized',
|
||||
'ready',
|
||||
'preparing',
|
||||
'prepared',
|
||||
'advancing',
|
||||
'committing',
|
||||
'capturing',
|
||||
'staged-restore',
|
||||
'restoring',
|
||||
'failed',
|
||||
'stopping',
|
||||
] as const;
|
||||
export type WorkerState = (typeof WORKER_STATES)[number];
|
||||
|
||||
export const RECOVERY = ['exact-checkpoint', 'episode-restart'] as const;
|
||||
export type Recovery = (typeof RECOVERY)[number];
|
||||
|
||||
export const DETERMINISM = ['fixed-build', 'unverified'] as const;
|
||||
export type Determinism = (typeof DETERMINISM)[number];
|
||||
|
||||
export const AXIS_RANGES = ['bipolar', 'unit'] as const;
|
||||
export type AxisRange = (typeof AXIS_RANGES)[number];
|
||||
|
||||
export function axisBounds(range: AxisRange): [number, number] {
|
||||
return range === 'bipolar' ? [-1, 1] : [0, 1];
|
||||
}
|
||||
|
||||
export interface AssetRef {
|
||||
id: Id;
|
||||
digest: Digest;
|
||||
byteLength: U64;
|
||||
format: Id;
|
||||
}
|
||||
|
||||
export interface SensoryInput {
|
||||
boundary: U64;
|
||||
views: ViewRef[];
|
||||
structured: TypedValue | null;
|
||||
}
|
||||
|
||||
export interface Stimulus {
|
||||
id: Id;
|
||||
kindId: Id;
|
||||
durationMs: number;
|
||||
}
|
||||
|
||||
export interface Reward {
|
||||
eventId: Id;
|
||||
ruleId: Id;
|
||||
value: number;
|
||||
}
|
||||
|
||||
export interface AgentTelemetry {
|
||||
brainTicks: U64;
|
||||
populationRateHz: number;
|
||||
rates: { roleId: Id; hz: number }[];
|
||||
learning: { enabled: boolean; updates: U64; changed: U64; signal: number };
|
||||
}
|
||||
|
||||
export function readAssetRef(value: unknown): AssetRef {
|
||||
const reader = new Reader(value, 'AssetRef');
|
||||
const asset: AssetRef = {
|
||||
id: reader.id('id'),
|
||||
digest: reader.digest('digest'),
|
||||
byteLength: reader.u64('byteLength'),
|
||||
format: reader.id('format'),
|
||||
};
|
||||
reader.finish();
|
||||
if (u64(asset.byteLength) === 0n) fail('AssetRef: byteLength must be positive');
|
||||
return asset;
|
||||
}
|
||||
|
||||
export function readSensoryInput(value: unknown): SensoryInput {
|
||||
const reader = new Reader(value, 'SensoryInput');
|
||||
const input: SensoryInput = {
|
||||
boundary: reader.u64('boundary'),
|
||||
views: readViewList(reader, 'views'),
|
||||
structured: readNullableTypedValue(reader.value('structured')),
|
||||
};
|
||||
reader.finish();
|
||||
for (const view of input.views) {
|
||||
if (u64(view.producedStep) > u64(input.boundary)) {
|
||||
fail(`SensoryInput: view "${view.viewId}" was produced after the observed boundary`);
|
||||
}
|
||||
}
|
||||
return input;
|
||||
}
|
||||
|
||||
/** Required sensory views against the descriptors that declared them. */
|
||||
export function validateSensoryInputAgainst(
|
||||
input: SensoryInput,
|
||||
descriptors: readonly ViewDescriptor[],
|
||||
): void {
|
||||
for (const view of input.views) {
|
||||
const descriptor = descriptors.find((candidate) => candidate.viewId === view.viewId);
|
||||
if (!descriptor) {
|
||||
fail(`SensoryInput: view "${view.viewId}" is not declared by the environment`);
|
||||
}
|
||||
validateViewAgainst(view, descriptor, u64(input.boundary));
|
||||
}
|
||||
}
|
||||
|
||||
/** A pixel-only profile rejects non-null structured input (workers-v1 section 1). */
|
||||
export function validateSensoryInputForProfile(
|
||||
input: SensoryInput,
|
||||
structuredSensing: boolean,
|
||||
): void {
|
||||
if (input.structured !== null && !structuredSensing) {
|
||||
fail('SensoryInput: a pixel-only profile rejects non-null structured input');
|
||||
}
|
||||
}
|
||||
|
||||
export function readStimulus(value: unknown): Stimulus {
|
||||
const reader = new Reader(value, 'Stimulus');
|
||||
const stimulus: Stimulus = {
|
||||
id: reader.id('id'),
|
||||
kindId: reader.id('kindId'),
|
||||
durationMs: reader.finite('durationMs'),
|
||||
};
|
||||
reader.finish();
|
||||
if (stimulus.durationMs <= 0) fail('Stimulus: durationMs must be finite and > 0');
|
||||
return stimulus;
|
||||
}
|
||||
|
||||
export function readReward(value: unknown): Reward {
|
||||
const reader = new Reader(value, 'Reward');
|
||||
const reward: Reward = {
|
||||
eventId: reader.id('eventId'),
|
||||
ruleId: reader.id('ruleId'),
|
||||
value: reader.finite('value'),
|
||||
};
|
||||
reader.finish();
|
||||
return reward;
|
||||
}
|
||||
|
||||
export function readStimulusList(reader: Reader, key: string): Stimulus[] {
|
||||
const items = reader.list(key, 0, MAX_STIMULI, readStimulus);
|
||||
requireUnique(
|
||||
items.map((item) => item.id),
|
||||
key,
|
||||
);
|
||||
return items;
|
||||
}
|
||||
|
||||
export function readRewardList(reader: Reader, key: string): Reward[] {
|
||||
const items = reader.list(key, 0, MAX_REWARDS, readReward);
|
||||
requireUnique(
|
||||
items.map((item) => item.eventId),
|
||||
key,
|
||||
);
|
||||
return items;
|
||||
}
|
||||
|
||||
export function readAgentTelemetry(value: unknown): AgentTelemetry {
|
||||
const reader = new Reader(value, 'AgentTelemetry');
|
||||
const brainTicks = reader.u64('brainTicks');
|
||||
const populationRateHz = reader.finiteIn('populationRateHz', 0, Number.MAX_VALUE);
|
||||
const rates = reader.list(('rates'), 0, MAX_RATE_ROLES, (item) => {
|
||||
const rate = new Reader(item, 'AgentTelemetry.rates');
|
||||
const entry = { roleId: rate.id('roleId'), hz: rate.finiteIn('hz', 0, Number.MAX_VALUE) };
|
||||
rate.finish();
|
||||
return entry;
|
||||
});
|
||||
const learningReader = new Reader(reader.value('learning'), 'AgentTelemetry.learning');
|
||||
const learning = {
|
||||
enabled: learningReader.boolean('enabled'),
|
||||
updates: learningReader.u64('updates'),
|
||||
changed: learningReader.u64('changed'),
|
||||
signal: learningReader.finite('signal'),
|
||||
};
|
||||
learningReader.finish();
|
||||
reader.finish();
|
||||
requireUnique(
|
||||
rates.map((rate) => rate.roleId),
|
||||
'AgentTelemetry.rates',
|
||||
);
|
||||
if (u64(learning.changed) > u64(learning.updates)) {
|
||||
fail('AgentTelemetry: learning.changed cannot exceed learning.updates');
|
||||
}
|
||||
return { brainTicks, populationRateHz, rates, learning };
|
||||
}
|
||||
|
||||
/** Rates are in profile-defined order (workers-v1 section 1). */
|
||||
export function validateTelemetryRoles(
|
||||
telemetry: AgentTelemetry,
|
||||
roleOrder: readonly string[],
|
||||
): void {
|
||||
requireSameOrder(
|
||||
telemetry.rates.map((rate) => rate.roleId),
|
||||
roleOrder,
|
||||
'AgentTelemetry.rates',
|
||||
);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------ agent methods
|
||||
|
||||
export interface AgentInitializeParams {
|
||||
agentId: Id;
|
||||
profile: AssetRef;
|
||||
seed: number;
|
||||
initialInput: SensoryInput;
|
||||
initialDecisionContext: TypedValue;
|
||||
workerThreads: number;
|
||||
}
|
||||
|
||||
export interface AgentInitializeResult {
|
||||
agentId: Id;
|
||||
profileDigest: Digest;
|
||||
tickDuration: RationalNs;
|
||||
warmupTicks: U64;
|
||||
committedStep: U64;
|
||||
decisionContextDigest: Digest;
|
||||
telemetry: AgentTelemetry;
|
||||
}
|
||||
|
||||
export interface PrepareParams {
|
||||
agentId: Id;
|
||||
profileDigest: Digest;
|
||||
interval: RationalNs;
|
||||
decisionContextDigest: Digest;
|
||||
preStepStimulations: Stimulus[];
|
||||
}
|
||||
|
||||
export interface PreparedDecision {
|
||||
agentId: Id;
|
||||
ticksAdvanced: U64;
|
||||
brainTicks: U64;
|
||||
remainder: RationalNs;
|
||||
decision: TypedValue;
|
||||
}
|
||||
|
||||
export interface CommitParams {
|
||||
agentId: Id;
|
||||
preparedRequestId: DomainRequestId;
|
||||
nextInput: SensoryInput;
|
||||
nextDecisionContext: TypedValue;
|
||||
rewards: Reward[];
|
||||
taskStimulations: Stimulus[];
|
||||
}
|
||||
|
||||
export interface AgentCommitResult {
|
||||
agentId: Id;
|
||||
committedStep: U64;
|
||||
decisionContextDigest: Digest;
|
||||
telemetry: AgentTelemetry;
|
||||
}
|
||||
|
||||
export function readAgentInitializeParams(value: unknown): AgentInitializeParams {
|
||||
const reader = new Reader(value, 'AgentInitializeParams');
|
||||
const params: AgentInitializeParams = {
|
||||
agentId: reader.id('agentId'),
|
||||
profile: readAssetRef(reader.value('profile')),
|
||||
seed: reader.int('seed', -2_147_483_648, 2_147_483_647),
|
||||
initialInput: readSensoryInput(reader.value('initialInput')),
|
||||
initialDecisionContext: readTypedValue(reader.value('initialDecisionContext')),
|
||||
workerThreads: reader.int('workerThreads', 1, 4_096),
|
||||
};
|
||||
reader.finish();
|
||||
return params;
|
||||
}
|
||||
|
||||
export function readAgentInitializeResult(value: unknown): AgentInitializeResult {
|
||||
const reader = new Reader(value, 'AgentInitializeResult');
|
||||
const result: AgentInitializeResult = {
|
||||
agentId: reader.id('agentId'),
|
||||
profileDigest: reader.digest('profileDigest'),
|
||||
tickDuration: readRational(reader.value('tickDuration')),
|
||||
warmupTicks: reader.u64('warmupTicks'),
|
||||
committedStep: reader.u64('committedStep'),
|
||||
decisionContextDigest: reader.digest('decisionContextDigest'),
|
||||
telemetry: readAgentTelemetry(reader.value('telemetry')),
|
||||
};
|
||||
reader.finish();
|
||||
requirePositiveRational(result.tickDuration, 'AgentInitializeResult.tickDuration');
|
||||
if (u64(result.committedStep) !== 0n) {
|
||||
fail('AgentInitializeResult: committedStep must be "0"');
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
export function readPrepareParams(value: unknown): PrepareParams {
|
||||
const reader = new Reader(value, 'PrepareParams');
|
||||
const params: PrepareParams = {
|
||||
agentId: reader.id('agentId'),
|
||||
profileDigest: reader.digest('profileDigest'),
|
||||
interval: readRational(reader.value('interval')),
|
||||
decisionContextDigest: reader.digest('decisionContextDigest'),
|
||||
preStepStimulations: readStimulusList(reader, 'preStepStimulations'),
|
||||
};
|
||||
reader.finish();
|
||||
requirePositiveRational(params.interval, 'PrepareParams.interval');
|
||||
return params;
|
||||
}
|
||||
|
||||
export function readPreparedDecision(value: unknown): PreparedDecision {
|
||||
const reader = new Reader(value, 'PreparedDecision');
|
||||
const decision: PreparedDecision = {
|
||||
agentId: reader.id('agentId'),
|
||||
ticksAdvanced: reader.u64('ticksAdvanced'),
|
||||
brainTicks: reader.u64('brainTicks'),
|
||||
remainder: readRational(reader.value('remainder')),
|
||||
decision: readTypedValue(reader.value('decision')),
|
||||
};
|
||||
reader.finish();
|
||||
if (u64(decision.ticksAdvanced) > u64(decision.brainTicks)) {
|
||||
fail('PreparedDecision: ticksAdvanced cannot exceed the total brainTicks');
|
||||
}
|
||||
return decision;
|
||||
}
|
||||
|
||||
/** The remainder is always `>= 0` and `< one model tick` (step-v1 section 5). */
|
||||
export function validateRemainder(decision: PreparedDecision, tickDuration: RationalNs): void {
|
||||
requirePositiveRational(tickDuration, 'tickDuration');
|
||||
if (compareRational(decision.remainder, tickDuration) >= 0) {
|
||||
fail('PreparedDecision: remainder must be less than one model tick');
|
||||
}
|
||||
}
|
||||
|
||||
export function readCommitParams(value: unknown): CommitParams {
|
||||
const reader = new Reader(value, 'CommitParams');
|
||||
const params: CommitParams = {
|
||||
agentId: reader.id('agentId'),
|
||||
preparedRequestId: domainRequestId(reader.string('preparedRequestId')),
|
||||
nextInput: readSensoryInput(reader.value('nextInput')),
|
||||
nextDecisionContext: readTypedValue(reader.value('nextDecisionContext')),
|
||||
rewards: readRewardList(reader, 'rewards'),
|
||||
taskStimulations: readStimulusList(reader, 'taskStimulations'),
|
||||
};
|
||||
reader.finish();
|
||||
return params;
|
||||
}
|
||||
|
||||
/** The commit of transition `k -> k+1` carries `scope.step = k` and the input for `k+1`. */
|
||||
export function validateCommitAgainstScope(params: CommitParams, scope: Scope): void {
|
||||
const expected = u64(scope.step) + 1n;
|
||||
if (u64(params.nextInput.boundary) !== expected) {
|
||||
fail(`CommitParams: nextInput.boundary must be ${expected} for scope.step ${scope.step}`);
|
||||
}
|
||||
}
|
||||
|
||||
export function readAgentCommitResult(value: unknown): AgentCommitResult {
|
||||
const reader = new Reader(value, 'AgentCommitResult');
|
||||
const result: AgentCommitResult = {
|
||||
agentId: reader.id('agentId'),
|
||||
committedStep: reader.u64('committedStep'),
|
||||
decisionContextDigest: reader.digest('decisionContextDigest'),
|
||||
telemetry: readAgentTelemetry(reader.value('telemetry')),
|
||||
};
|
||||
reader.finish();
|
||||
return result;
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------- environment methods
|
||||
|
||||
export interface AxisSchema {
|
||||
id: Id;
|
||||
range: AxisRange;
|
||||
neutral: number;
|
||||
}
|
||||
|
||||
export interface ControllerSchema {
|
||||
schema: SchemaRef;
|
||||
buttons: Id[];
|
||||
axes: AxisSchema[];
|
||||
}
|
||||
|
||||
export interface PortControl {
|
||||
portId: Id;
|
||||
buttons: { id: Id; down: boolean }[];
|
||||
axes: { id: Id; value: number }[];
|
||||
}
|
||||
|
||||
export interface PortDescriptor {
|
||||
portId: Id;
|
||||
controls: ControllerSchema;
|
||||
}
|
||||
|
||||
export interface EnvironmentDescriptor {
|
||||
backendDigest: Digest;
|
||||
contentDigest: Digest;
|
||||
configurationDigest: Digest;
|
||||
stepDuration: RationalNs;
|
||||
ports: PortDescriptor[];
|
||||
inspectionSchema: SchemaRef;
|
||||
views: ViewDescriptor[];
|
||||
audio: AudioDescriptor[];
|
||||
recovery: Recovery;
|
||||
determinism: Determinism;
|
||||
}
|
||||
|
||||
export interface WorldObservation {
|
||||
boundary: U64;
|
||||
worldTime: RationalNs;
|
||||
engineFrame: string | null;
|
||||
sensoryViews: ViewRef[];
|
||||
inspection: TypedValue;
|
||||
broadcastViews: ViewRef[];
|
||||
audio: AudioRef[];
|
||||
}
|
||||
|
||||
export interface EnvironmentInitializeParams {
|
||||
backendConfig: AssetRef;
|
||||
taskConfig: AssetRef;
|
||||
episodeId: Id;
|
||||
portBindings: { portId: Id; agentId: Id }[];
|
||||
}
|
||||
|
||||
export interface EnvironmentInitializeResult {
|
||||
descriptor: EnvironmentDescriptor;
|
||||
observation: WorldObservation;
|
||||
}
|
||||
|
||||
export interface AdvanceParams {
|
||||
batchId: Id;
|
||||
controls: PortControl[];
|
||||
}
|
||||
|
||||
export interface StepResult {
|
||||
batchId: Id;
|
||||
appliedFromStep: U64;
|
||||
nextStep: U64;
|
||||
appliedControlsDigest: Digest;
|
||||
observation: WorldObservation;
|
||||
}
|
||||
|
||||
export function readControllerSchema(value: unknown): ControllerSchema {
|
||||
const reader = new Reader(value, 'ControllerSchema');
|
||||
const controls: ControllerSchema = {
|
||||
schema: readSchemaRef(reader.value('schema')),
|
||||
buttons: reader.idList('buttons', 0, MAX_BUTTONS),
|
||||
axes: reader.list('axes', 0, MAX_AXES, (item) => {
|
||||
const axis = new Reader(item, 'ControllerSchema.axes');
|
||||
const id = axis.id('id');
|
||||
const range = axis.enumeration('range', AXIS_RANGES);
|
||||
const [low, high] = axisBounds(range);
|
||||
const neutral = axis.finiteIn('neutral', low, high);
|
||||
axis.finish();
|
||||
return { id, range, neutral };
|
||||
}),
|
||||
};
|
||||
reader.finish();
|
||||
requireUnique(controls.buttons, 'ControllerSchema.buttons');
|
||||
requireUnique(
|
||||
controls.axes.map((axis) => axis.id),
|
||||
'ControllerSchema.axes',
|
||||
);
|
||||
return controls;
|
||||
}
|
||||
|
||||
export function readPortControl(value: unknown): PortControl {
|
||||
const reader = new Reader(value, 'PortControl');
|
||||
const control: PortControl = {
|
||||
portId: reader.id('portId'),
|
||||
buttons: reader.list('buttons', 0, MAX_BUTTONS, (item) => {
|
||||
const button = new Reader(item, 'PortControl.buttons');
|
||||
const entry = { id: button.id('id'), down: button.boolean('down') };
|
||||
button.finish();
|
||||
return entry;
|
||||
}),
|
||||
axes: reader.list('axes', 0, MAX_AXES, (item) => {
|
||||
const axis = new Reader(item, 'PortControl.axes');
|
||||
const entry = { id: axis.id('id'), value: axis.finite('value') };
|
||||
axis.finish();
|
||||
return entry;
|
||||
}),
|
||||
};
|
||||
reader.finish();
|
||||
requireUnique(
|
||||
control.buttons.map((button) => button.id),
|
||||
'PortControl.buttons',
|
||||
);
|
||||
requireUnique(
|
||||
control.axes.map((axis) => axis.id),
|
||||
'PortControl.axes',
|
||||
);
|
||||
return control;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every declared button and axis, in descriptor order, in range. An out-of-range value is
|
||||
* refused, never clamped (workers-v1 section 3).
|
||||
*/
|
||||
export function validatePortControlAgainst(
|
||||
control: PortControl,
|
||||
controls: ControllerSchema,
|
||||
): void {
|
||||
requireSameOrder(
|
||||
control.buttons.map((button) => button.id),
|
||||
controls.buttons,
|
||||
'PortControl.buttons',
|
||||
);
|
||||
requireSameOrder(
|
||||
control.axes.map((axis) => axis.id),
|
||||
controls.axes.map((axis) => axis.id),
|
||||
'PortControl.axes',
|
||||
);
|
||||
control.axes.forEach((axis, index) => {
|
||||
const schema = controls.axes[index] as AxisSchema;
|
||||
const [low, high] = axisBounds(schema.range);
|
||||
if (axis.value < low || axis.value > high) {
|
||||
fail(
|
||||
`PortControl: axis "${axis.id}" value ${axis.value} is outside its ${schema.range} range and is refused, not clamped`,
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
export function readEnvironmentDescriptor(value: unknown): EnvironmentDescriptor {
|
||||
const reader = new Reader(value, 'EnvironmentDescriptor');
|
||||
const descriptor: EnvironmentDescriptor = {
|
||||
backendDigest: reader.digest('backendDigest'),
|
||||
contentDigest: reader.digest('contentDigest'),
|
||||
configurationDigest: reader.digest('configurationDigest'),
|
||||
stepDuration: readRational(reader.value('stepDuration')),
|
||||
ports: reader.list('ports', 1, MAX_PORTS, (item) => {
|
||||
const port = new Reader(item, 'EnvironmentDescriptor.ports');
|
||||
const entry = {
|
||||
portId: port.id('portId'),
|
||||
controls: readControllerSchema(port.value('controls')),
|
||||
};
|
||||
port.finish();
|
||||
return entry;
|
||||
}),
|
||||
inspectionSchema: readSchemaRef(reader.value('inspectionSchema')),
|
||||
views: reader.list('views', 0, MAX_VIEWS, readViewDescriptor),
|
||||
audio: reader.list('audio', 0, 8, readAudioDescriptor),
|
||||
recovery: reader.enumeration('recovery', RECOVERY),
|
||||
determinism: reader.enumeration('determinism', DETERMINISM),
|
||||
};
|
||||
reader.finish();
|
||||
requirePositiveRational(descriptor.stepDuration, 'EnvironmentDescriptor.stepDuration');
|
||||
requireUnique(
|
||||
descriptor.ports.map((port) => port.portId),
|
||||
'EnvironmentDescriptor.ports',
|
||||
);
|
||||
requireUnique(
|
||||
descriptor.views.map((view) => view.viewId),
|
||||
'EnvironmentDescriptor.views',
|
||||
);
|
||||
requireUnique(
|
||||
descriptor.audio.map((stream) => stream.streamId),
|
||||
'EnvironmentDescriptor.audio',
|
||||
);
|
||||
return descriptor;
|
||||
}
|
||||
|
||||
export function findPort(
|
||||
descriptor: EnvironmentDescriptor,
|
||||
portId: string,
|
||||
): PortDescriptor | undefined {
|
||||
return descriptor.ports.find((port) => port.portId === portId);
|
||||
}
|
||||
|
||||
/** One complete batch: every configured port once, in descriptor order. */
|
||||
export function validateBatch(
|
||||
descriptor: EnvironmentDescriptor,
|
||||
controls: readonly PortControl[],
|
||||
): void {
|
||||
requireSameOrder(
|
||||
controls.map((control) => control.portId),
|
||||
descriptor.ports.map((port) => port.portId),
|
||||
'Environment.Advance controls',
|
||||
);
|
||||
controls.forEach((control, index) => {
|
||||
validatePortControlAgainst(control, (descriptor.ports[index] as PortDescriptor).controls);
|
||||
});
|
||||
}
|
||||
|
||||
export function readWorldObservation(value: unknown): WorldObservation {
|
||||
const reader = new Reader(value, 'WorldObservation');
|
||||
const observation: WorldObservation = {
|
||||
boundary: reader.u64('boundary'),
|
||||
worldTime: readRational(reader.value('worldTime')),
|
||||
engineFrame: reader.nullableBoundedString('engineFrame', MAX_ENGINE_FRAME_LEN),
|
||||
sensoryViews: readViewList(reader, 'sensoryViews'),
|
||||
inspection: readTypedValue(reader.value('inspection')),
|
||||
broadcastViews: readViewList(reader, 'broadcastViews'),
|
||||
audio: readAudioList(reader, 'audio'),
|
||||
};
|
||||
reader.finish();
|
||||
for (const view of [...observation.sensoryViews, ...observation.broadcastViews]) {
|
||||
if (u64(view.producedStep) > u64(observation.boundary)) {
|
||||
fail('WorldObservation: a view cannot be produced after the boundary');
|
||||
}
|
||||
}
|
||||
return observation;
|
||||
}
|
||||
|
||||
export function validateObservationAgainst(
|
||||
observation: WorldObservation,
|
||||
descriptor: EnvironmentDescriptor,
|
||||
): void {
|
||||
const expected = descriptor.inspectionSchema;
|
||||
const found = observation.inspection.schema;
|
||||
if (
|
||||
found.id !== expected.id ||
|
||||
found.version !== expected.version ||
|
||||
found.digest !== expected.digest
|
||||
) {
|
||||
fail("WorldObservation: inspection must use the descriptor's inspectionSchema");
|
||||
}
|
||||
for (const view of [...observation.sensoryViews, ...observation.broadcastViews]) {
|
||||
const declared = descriptor.views.find((candidate) => candidate.viewId === view.viewId);
|
||||
if (!declared) {
|
||||
fail(`WorldObservation: view "${view.viewId}" is not declared by the descriptor`);
|
||||
}
|
||||
validateViewAgainst(view, declared, u64(observation.boundary));
|
||||
}
|
||||
for (const chunk of observation.audio) {
|
||||
const declared = descriptor.audio.find((candidate) => candidate.streamId === chunk.streamId);
|
||||
if (!declared) {
|
||||
fail(`WorldObservation: audio stream "${chunk.streamId}" is not declared by the descriptor`);
|
||||
}
|
||||
validateAudioAgainst(chunk, declared);
|
||||
}
|
||||
}
|
||||
|
||||
export function readEnvironmentInitializeParams(value: unknown): EnvironmentInitializeParams {
|
||||
const reader = new Reader(value, 'EnvironmentInitializeParams');
|
||||
const params: EnvironmentInitializeParams = {
|
||||
backendConfig: readAssetRef(reader.value('backendConfig')),
|
||||
taskConfig: readAssetRef(reader.value('taskConfig')),
|
||||
episodeId: reader.id('episodeId'),
|
||||
portBindings: reader.list('portBindings', 1, MAX_PORTS, (item) => {
|
||||
const binding = new Reader(item, 'EnvironmentInitializeParams.portBindings');
|
||||
const entry = { portId: binding.id('portId'), agentId: binding.id('agentId') };
|
||||
binding.finish();
|
||||
return entry;
|
||||
}),
|
||||
};
|
||||
reader.finish();
|
||||
requireUnique(
|
||||
params.portBindings.map((binding) => binding.portId),
|
||||
'EnvironmentInitializeParams.portBindings portId',
|
||||
);
|
||||
requireUnique(
|
||||
params.portBindings.map((binding) => binding.agentId),
|
||||
'EnvironmentInitializeParams.portBindings agentId',
|
||||
);
|
||||
return params;
|
||||
}
|
||||
|
||||
export function readEnvironmentInitializeResult(value: unknown): EnvironmentInitializeResult {
|
||||
const reader = new Reader(value, 'EnvironmentInitializeResult');
|
||||
const result: EnvironmentInitializeResult = {
|
||||
descriptor: readEnvironmentDescriptor(reader.value('descriptor')),
|
||||
observation: readWorldObservation(reader.value('observation')),
|
||||
};
|
||||
reader.finish();
|
||||
if (u64(result.observation.boundary) !== 0n) {
|
||||
fail('EnvironmentInitializeResult: the initial observation is boundary 0');
|
||||
}
|
||||
if (!isRationalZero(result.observation.worldTime)) {
|
||||
fail('EnvironmentInitializeResult: initial worldTime is zero (0/1)');
|
||||
}
|
||||
validateObservationAgainst(result.observation, result.descriptor);
|
||||
return result;
|
||||
}
|
||||
|
||||
export function readAdvanceParams(value: unknown): AdvanceParams {
|
||||
const reader = new Reader(value, 'AdvanceParams');
|
||||
const params: AdvanceParams = {
|
||||
batchId: reader.id('batchId'),
|
||||
controls: reader.list('controls', 1, MAX_PORTS, readPortControl),
|
||||
};
|
||||
reader.finish();
|
||||
requireUnique(
|
||||
params.controls.map((control) => control.portId),
|
||||
'AdvanceParams.controls',
|
||||
);
|
||||
return params;
|
||||
}
|
||||
|
||||
export function readStepResult(value: unknown): StepResult {
|
||||
const reader = new Reader(value, 'StepResult');
|
||||
const result: StepResult = {
|
||||
batchId: reader.id('batchId'),
|
||||
appliedFromStep: reader.u64('appliedFromStep'),
|
||||
nextStep: reader.u64('nextStep'),
|
||||
appliedControlsDigest: reader.digest('appliedControlsDigest'),
|
||||
observation: readWorldObservation(reader.value('observation')),
|
||||
};
|
||||
reader.finish();
|
||||
if (u64(result.nextStep) !== u64(result.appliedFromStep) + 1n) {
|
||||
fail('StepResult: nextStep must be appliedFromStep + 1; one result is one step');
|
||||
}
|
||||
if (u64(result.observation.boundary) !== u64(result.nextStep)) {
|
||||
fail('StepResult: the observation boundary must be nextStep');
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/** Exactly boundary `k+1`, with world time advanced by exactly one `stepDuration`. */
|
||||
export function validateStepResultAgainst(
|
||||
result: StepResult,
|
||||
descriptor: EnvironmentDescriptor,
|
||||
previous: WorldObservation,
|
||||
): void {
|
||||
validateObservationAgainst(result.observation, descriptor);
|
||||
const expected = addRational(previous.worldTime, descriptor.stepDuration);
|
||||
if (compareRational(result.observation.worldTime, expected) !== 0) {
|
||||
fail('StepResult: worldTime must advance by exactly one stepDuration');
|
||||
}
|
||||
if (u64(result.observation.boundary) !== u64(previous.boundary) + 1n) {
|
||||
fail('StepResult: the observation must be exactly the next boundary');
|
||||
}
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------------- common worker methods
|
||||
|
||||
export interface HelloParams {
|
||||
sessionId: Id;
|
||||
expectedWorkerId: Id;
|
||||
role: Role;
|
||||
supportedMajors: number[];
|
||||
}
|
||||
|
||||
export interface HelloResult {
|
||||
selectedMajor: 1;
|
||||
selectedMinor: 0;
|
||||
workerId: Id;
|
||||
incarnationId: Id;
|
||||
role: Role;
|
||||
buildDigest: Digest;
|
||||
contractDigest: Digest;
|
||||
capabilities: Id[];
|
||||
limits: { maxAgents: number; maxPorts: number };
|
||||
}
|
||||
|
||||
export interface StatusResult {
|
||||
state: WorkerState;
|
||||
currentScope: Scope | null;
|
||||
activeRequestId: DomainRequestId | null;
|
||||
lastCompletedRequestId: DomainRequestId | null;
|
||||
lastBatchId: Id | null;
|
||||
progressCounter: U64;
|
||||
}
|
||||
|
||||
export interface AcknowledgeParams {
|
||||
requestIds: DomainRequestId[];
|
||||
}
|
||||
|
||||
export interface AcknowledgeResult {
|
||||
acknowledged: DomainRequestId[];
|
||||
}
|
||||
|
||||
export interface ShutdownParams {
|
||||
reason: Id;
|
||||
}
|
||||
|
||||
export interface ShutdownResult {
|
||||
stopping: true;
|
||||
}
|
||||
|
||||
export interface TaskEvent {
|
||||
id: Id;
|
||||
kindId: Id;
|
||||
sourceStep: U64;
|
||||
agentId: Id | null;
|
||||
payload: TypedValue;
|
||||
}
|
||||
|
||||
export interface EpisodeRequest {
|
||||
kind: 'terminal';
|
||||
reason: Id;
|
||||
outcome: TypedValue;
|
||||
}
|
||||
|
||||
/** Required capabilities are agent-step-v1 and world-step-v1 for their roles. */
|
||||
export function requiredCapability(role: Role): string | null {
|
||||
if (role === 'agent') return 'agent-step-v1';
|
||||
if (role === 'environment') return 'world-step-v1';
|
||||
return null;
|
||||
}
|
||||
|
||||
export function readHelloParams(value: unknown): HelloParams {
|
||||
const reader = new Reader(value, 'HelloParams');
|
||||
const params: HelloParams = {
|
||||
sessionId: reader.id('sessionId'),
|
||||
expectedWorkerId: reader.id('expectedWorkerId'),
|
||||
role: reader.enumeration('role', ROLES),
|
||||
supportedMajors: reader.list('supportedMajors', 1, MAX_SUPPORTED_MAJORS, (item) => {
|
||||
if (typeof item !== 'number' || !Number.isInteger(item) || item < 1 || item > 65_535) {
|
||||
fail('every supported major must be an integer 1..=65535');
|
||||
}
|
||||
return item;
|
||||
}),
|
||||
};
|
||||
reader.finish();
|
||||
requireUnique(
|
||||
params.supportedMajors.map(String),
|
||||
'HelloParams.supportedMajors',
|
||||
);
|
||||
return params;
|
||||
}
|
||||
|
||||
export function readHelloResult(value: unknown): HelloResult {
|
||||
const reader = new Reader(value, 'HelloResult');
|
||||
const selectedMajor = reader.int('selectedMajor', 1, 1) as 1;
|
||||
const selectedMinor = reader.int('selectedMinor', 0, 0) as 0;
|
||||
const workerId = reader.id('workerId');
|
||||
const incarnationId = reader.id('incarnationId');
|
||||
const role = reader.enumeration('role', ROLES);
|
||||
const buildDigest = reader.digest('buildDigest');
|
||||
const contractDigest = reader.digest('contractDigest');
|
||||
const capabilities = reader.idList('capabilities', 0, MAX_CAPABILITIES);
|
||||
const limitsReader = new Reader(reader.value('limits'), 'HelloResult.limits');
|
||||
const limits = {
|
||||
maxAgents: limitsReader.int('maxAgents', 1, MAX_AGENTS),
|
||||
maxPorts: limitsReader.int('maxPorts', 1, MAX_PORTS),
|
||||
};
|
||||
limitsReader.finish();
|
||||
reader.finish();
|
||||
requireUnique(capabilities, 'HelloResult.capabilities');
|
||||
const required = requiredCapability(role);
|
||||
if (required !== null && !capabilities.includes(required)) {
|
||||
fail(`HelloResult: a ${role} worker must advertise ${required}`);
|
||||
}
|
||||
return {
|
||||
selectedMajor,
|
||||
selectedMinor,
|
||||
workerId,
|
||||
incarnationId,
|
||||
role,
|
||||
buildDigest,
|
||||
contractDigest,
|
||||
capabilities,
|
||||
limits,
|
||||
};
|
||||
}
|
||||
|
||||
export function readStatusResult(value: unknown): StatusResult {
|
||||
const reader = new Reader(value, 'StatusResult');
|
||||
const state = reader.enumeration('state', WORKER_STATES);
|
||||
const currentScopeValue = reader.value('currentScope');
|
||||
const currentScope = currentScopeValue === null ? null : readScope(currentScopeValue);
|
||||
const active = reader.value('activeRequestId');
|
||||
const completed = reader.value('lastCompletedRequestId');
|
||||
const result: StatusResult = {
|
||||
state,
|
||||
currentScope,
|
||||
activeRequestId: active === null ? null : domainRequestId(active),
|
||||
lastCompletedRequestId: completed === null ? null : domainRequestId(completed),
|
||||
lastBatchId: reader.nullableId('lastBatchId'),
|
||||
progressCounter: reader.u64('progressCounter'),
|
||||
};
|
||||
reader.finish();
|
||||
if (result.state === 'uninitialized' && result.currentScope !== null) {
|
||||
fail('StatusResult: an uninitialized worker has a null currentScope');
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
export function readAcknowledgeParams(value: unknown): AcknowledgeParams {
|
||||
const reader = new Reader(value, 'AcknowledgeParams');
|
||||
const params: AcknowledgeParams = {
|
||||
requestIds: reader.list('requestIds', 1, MAX_ACKNOWLEDGE, domainRequestId),
|
||||
};
|
||||
reader.finish();
|
||||
requireUnique(params.requestIds, 'AcknowledgeParams.requestIds');
|
||||
return params;
|
||||
}
|
||||
|
||||
export function readAcknowledgeResult(value: unknown): AcknowledgeResult {
|
||||
const reader = new Reader(value, 'AcknowledgeResult');
|
||||
const result: AcknowledgeResult = {
|
||||
acknowledged: reader.list('acknowledged', 0, MAX_ACKNOWLEDGE, domainRequestId),
|
||||
};
|
||||
reader.finish();
|
||||
requireUnique(result.acknowledged, 'AcknowledgeResult.acknowledged');
|
||||
return result;
|
||||
}
|
||||
|
||||
export function readShutdownParams(value: unknown): ShutdownParams {
|
||||
const reader = new Reader(value, 'ShutdownParams');
|
||||
const params = { reason: reader.id('reason') };
|
||||
reader.finish();
|
||||
return params;
|
||||
}
|
||||
|
||||
export function readShutdownResult(value: unknown): ShutdownResult {
|
||||
const reader = new Reader(value, 'ShutdownResult');
|
||||
const result: ShutdownResult = { stopping: reader.constantTrue('stopping') };
|
||||
reader.finish();
|
||||
return result;
|
||||
}
|
||||
|
||||
export function readTaskEvent(value: unknown): TaskEvent {
|
||||
const reader = new Reader(value, 'TaskEvent');
|
||||
const event: TaskEvent = {
|
||||
id: reader.id('id'),
|
||||
kindId: reader.id('kindId'),
|
||||
sourceStep: reader.u64('sourceStep'),
|
||||
agentId: reader.nullableId('agentId'),
|
||||
payload: readTypedValue(reader.value('payload')),
|
||||
};
|
||||
reader.finish();
|
||||
return event;
|
||||
}
|
||||
|
||||
export function readEpisodeRequest(value: unknown): EpisodeRequest {
|
||||
const reader = new Reader(value, 'EpisodeRequest');
|
||||
const request: EpisodeRequest = {
|
||||
kind: reader.constant('kind', 'terminal'),
|
||||
reason: reader.id('reason'),
|
||||
outcome: readTypedValue(reader.value('outcome')),
|
||||
};
|
||||
reader.finish();
|
||||
return request;
|
||||
}
|
||||
|
||||
export interface ActivateRestoreResult {
|
||||
committedStep: U64;
|
||||
checkpointId: Id;
|
||||
observation: WorldObservation | null;
|
||||
}
|
||||
|
||||
export function readActivateRestoreResult(value: unknown): ActivateRestoreResult {
|
||||
const reader = new Reader(value, 'ActivateRestoreResult');
|
||||
const observationValue = reader.value('observation');
|
||||
const result: ActivateRestoreResult = {
|
||||
committedStep: reader.u64('committedStep'),
|
||||
checkpointId: reader.id('checkpointId'),
|
||||
observation: observationValue === null ? null : readWorldObservation(observationValue),
|
||||
};
|
||||
reader.finish();
|
||||
if (
|
||||
result.observation !== null &&
|
||||
u64(result.observation.boundary) !== u64(result.committedStep)
|
||||
) {
|
||||
fail('ActivateRestoreResult: the observation boundary must be the committed step');
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/** An environment returns its restored observation; an agent returns null. */
|
||||
export function validateActivateRestoreForRole(result: ActivateRestoreResult, role: Role): void {
|
||||
if (role === 'environment' && result.observation === null) {
|
||||
fail('ActivateRestoreResult: an environment must return its restored observation');
|
||||
}
|
||||
if (role === 'agent' && result.observation !== null) {
|
||||
fail('ActivateRestoreResult: an agent returns a null observation');
|
||||
}
|
||||
}
|
||||
150
packages/session-types/tests/canonical.test.ts
Normal file
150
packages/session-types/tests/canonical.test.ts
Normal file
|
|
@ -0,0 +1,150 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
MAX_ENVELOPE_BYTES,
|
||||
canonicalize,
|
||||
digestOf,
|
||||
parseStrict,
|
||||
requireEnvelopeFit,
|
||||
} from '../src/canonical';
|
||||
import { bodyDigest, operationKeyDigest, readScope } from '../src/common';
|
||||
import * as fixtures from '../src/fixtures';
|
||||
|
||||
const ASTRAL = String.fromCodePoint(0x10400);
|
||||
const FULLWIDTH_A = String.fromCodePoint(0xff21);
|
||||
const E_ACUTE = String.fromCodePoint(0xe9);
|
||||
|
||||
test('object keys are sorted by UTF-16 code unit', () => {
|
||||
const value: Record<string, number> = { b: 1, a: 2, A: 3 };
|
||||
value[E_ACUTE] = 4;
|
||||
value[ASTRAL] = 5;
|
||||
value[FULLWIDTH_A] = 6;
|
||||
assert.equal(
|
||||
canonicalize(value),
|
||||
`{"A":3,"a":2,"b":1,"${E_ACUTE}":4,"${ASTRAL}":5,"${FULLWIDTH_A}":6}`,
|
||||
'an astral key, whose leading surrogate is D801, sorts before U+FF21',
|
||||
);
|
||||
});
|
||||
|
||||
test('numbers print the way ECMAScript prints them', () => {
|
||||
const file = fixtures.load('boundaries.json');
|
||||
for (const item of fixtures.section(file, 'doubles')) {
|
||||
const record = item as Record<string, unknown>;
|
||||
const value = record.value as number;
|
||||
if (record.accept === true) {
|
||||
assert.equal(canonicalize(value), record.canonical, `${value} prints as its canonical form`);
|
||||
} else {
|
||||
assert.throws(() => canonicalize(value), `${value} must be refused`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('strings are escaped the way JSON.stringify escapes them', () => {
|
||||
const bell = String.fromCharCode(7);
|
||||
const del = String.fromCharCode(0x7f);
|
||||
const value = { s: ['q"', 'b\\', 't\t', 'n\n', bell, del].join(' ') };
|
||||
assert.equal(
|
||||
canonicalize(value),
|
||||
JSON.stringify(value),
|
||||
'for a single-key object the two agree exactly, escape for escape',
|
||||
);
|
||||
assert.ok(canonicalize(value).includes('\\u0007'), 'a control character uses lowercase \\u');
|
||||
assert.ok(canonicalize(value).includes(del), 'DEL is not an escape in JSON');
|
||||
});
|
||||
|
||||
test('canonical form does not depend on the input formatting', () => {
|
||||
const compact = '{"b":[1,2,{"y":true,"x":null}],"a":"z"}';
|
||||
const pretty = '{\n "a" : "z",\n "b": [ 1, 2, { "x": null, "y": true } ]\n}';
|
||||
assert.equal(canonicalize(parseStrict(compact)), canonicalize(parseStrict(pretty)));
|
||||
assert.equal(digestOf(parseStrict(compact)), digestOf(parseStrict(pretty)));
|
||||
});
|
||||
|
||||
test('duplicate keys and invalid UTF-8 never parse', () => {
|
||||
assert.throws(() => parseStrict('{"a":1,"a":2}'), /duplicate key/);
|
||||
assert.throws(() => parseStrict('{"a":{"b":1,"b":2}}'), /duplicate key/);
|
||||
assert.throws(() => parseStrict(new Uint8Array([0x7b, 0x22, 0xff, 0x22, 0x7d])), /UTF-8/);
|
||||
assert.throws(() => parseStrict('{"a":1} {"b":2}'), /trailing data/);
|
||||
assert.throws(() => parseStrict('{"a":NaN}'));
|
||||
assert.throws(() => parseStrict('{"a":Infinity}'));
|
||||
assert.throws(() => parseStrict('{"a":1'));
|
||||
assert.throws(() => parseStrict(''));
|
||||
});
|
||||
|
||||
test('an envelope over 64 KiB is refused', () => {
|
||||
assert.throws(() => requireEnvelopeFit({ pad: 'a'.repeat(MAX_ENVELOPE_BYTES) }, 0));
|
||||
const small = { pad: 'a' };
|
||||
const length = canonicalize(small).length;
|
||||
assert.equal(requireEnvelopeFit(small, MAX_ENVELOPE_BYTES - length), MAX_ENVELOPE_BYTES);
|
||||
assert.throws(() => requireEnvelopeFit(small, MAX_ENVELOPE_BYTES - length + 1));
|
||||
});
|
||||
|
||||
test('operation keys match the fixture and separate the operations they should', () => {
|
||||
const file = fixtures.load('operations.json');
|
||||
const digests: [string, string][] = [];
|
||||
for (const item of fixtures.section(file, 'keys')) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const digest = operationKeyDigest({
|
||||
scope: readScope(fixtures.member(item, 'scope')),
|
||||
method: fixtures.field(item, 'method'),
|
||||
workerId: fixtures.field(item, 'workerId'),
|
||||
});
|
||||
assert.equal(digest, fixtures.field(item, 'digest'), `${name}: operation key digest`);
|
||||
digests.push([name, digest]);
|
||||
}
|
||||
digests.forEach(([name, digest], index) => {
|
||||
for (const [otherName, other] of digests.slice(index + 1)) {
|
||||
assert.notEqual(digest, other, `${name} and ${otherName} are different operations`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
test('canonical bodies match the fixture', () => {
|
||||
const file = fixtures.load('operations.json');
|
||||
for (const item of fixtures.section(file, 'bodies')) {
|
||||
const scopeValue = fixtures.member(item, 'scope');
|
||||
const digest = bodyDigest(
|
||||
fixtures.field(item, 'method'),
|
||||
scopeValue === null ? null : readScope(scopeValue),
|
||||
fixtures.member(item, 'params'),
|
||||
);
|
||||
assert.equal(digest, fixtures.field(item, 'digest'), fixtures.field(item, 'name'));
|
||||
}
|
||||
});
|
||||
|
||||
test('operation pairs agree with the fixture about sameness', () => {
|
||||
const file = fixtures.load('operations.json');
|
||||
for (const item of fixtures.section(file, 'pairs')) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const reason = fixtures.field(item, 'reason');
|
||||
const worker = fixtures.field(item, 'workerId');
|
||||
const rightWorker = fixtures.optionalField(item, 'rightWorkerId') ?? worker;
|
||||
const side = (key: string, workerId: string): [string, string] => {
|
||||
const value = fixtures.member(item, key);
|
||||
const method = fixtures.field(value, 'method');
|
||||
const scope = readScope(fixtures.member(value, 'scope'));
|
||||
const params = fixtures.member(value, 'params');
|
||||
return [operationKeyDigest({ scope, method, workerId }), bodyDigest(method, scope, params)];
|
||||
};
|
||||
const [leftKey, leftBody] = side('left', worker);
|
||||
const [rightKey, rightBody] = side('right', rightWorker);
|
||||
const record = item as Record<string, unknown>;
|
||||
assert.equal(leftKey === rightKey, record.sameKey, `${name}: key sameness. ${reason}`);
|
||||
assert.equal(leftBody === rightBody, record.sameBody, `${name}: body sameness. ${reason}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('a domain body can never carry a bus identity', () => {
|
||||
const file = fixtures.load('operations.json');
|
||||
for (const item of fixtures.section(file, 'rejected')) {
|
||||
assert.throws(
|
||||
() =>
|
||||
bodyDigest(
|
||||
fixtures.field(item, 'method'),
|
||||
readScope(fixtures.member(item, 'scope')),
|
||||
fixtures.member(item, 'params'),
|
||||
),
|
||||
`${fixtures.field(item, 'name')} must be refused`,
|
||||
);
|
||||
}
|
||||
});
|
||||
126
packages/session-types/tests/checkpoint.test.ts
Normal file
126
packages/session-types/tests/checkpoint.test.ts
Normal file
|
|
@ -0,0 +1,126 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { canonicalize, parseStrict } from '../src/canonical';
|
||||
import * as checkpoint from '../src/checkpoint';
|
||||
import * as fixtures from '../src/fixtures';
|
||||
|
||||
function envelopeBytes(): Uint8Array {
|
||||
const file = fixtures.load('checkpoint-envelope.json');
|
||||
const envelope = fixtures.member(file, 'envelope');
|
||||
return fixtures.decodeBase64(fixtures.field(envelope, 'base64'));
|
||||
}
|
||||
|
||||
test('the fixture envelope decodes to its recorded layout', () => {
|
||||
const file = fixtures.load('checkpoint-envelope.json');
|
||||
const bytes = envelopeBytes();
|
||||
const envelope = checkpoint.decode(bytes);
|
||||
checkpoint.validateManifest(envelope);
|
||||
|
||||
const layout = fixtures.member(fixtures.member(file, 'envelope'), 'layout') as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
assert.equal(Buffer.from(bytes.subarray(0, 8)).toString('ascii'), checkpoint.MAGIC);
|
||||
assert.equal(String(bytes.length), layout.totalBytes);
|
||||
assert.equal(String(envelope.layout.tableOffset), layout.tableOffset);
|
||||
assert.equal(envelope.layout.manifestBytes, layout.manifestBytes);
|
||||
const entries = layout.entries as Record<string, unknown>[];
|
||||
assert.equal(envelope.layout.entries.length, entries.length);
|
||||
envelope.layout.entries.forEach((entry, index) => {
|
||||
const recorded = entries[index]!;
|
||||
assert.equal(entry.name, recorded.name);
|
||||
assert.equal(String(entry.offset), recorded.offset);
|
||||
assert.equal(String(entry.byteLength), recorded.byteLength);
|
||||
assert.equal(entry.digest, recorded.digest);
|
||||
assert.equal(entry.offset % 8, 0, 'payloads start on an eight-byte boundary');
|
||||
});
|
||||
|
||||
for (const payload of fixtures.section(file, 'payloads')) {
|
||||
const name = fixtures.field(payload, 'name');
|
||||
const expected = fixtures.decodeBase64(fixtures.field(payload, 'base64'));
|
||||
const found = envelope.payloads.find((candidate) => candidate.name === name);
|
||||
assert.ok(found, `payload ${name} must be present`);
|
||||
assert.deepEqual(found.bytes, expected, `payload ${name} must come back byte for byte`);
|
||||
}
|
||||
assert.equal(canonicalize(envelope.manifest), canonicalize(fixtures.member(file, 'manifest')));
|
||||
});
|
||||
|
||||
test('every recorded corruption is refused', () => {
|
||||
const file = fixtures.load('checkpoint-envelope.json');
|
||||
const bytes = envelopeBytes();
|
||||
for (const item of fixtures.section(file, 'corruption')) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const offset = (item as Record<string, unknown>).offset as number;
|
||||
const corrupted = Uint8Array.from(bytes);
|
||||
corrupted[offset] = (corrupted[offset]! ^ 0x01) & 0xff;
|
||||
assert.throws(() => checkpoint.decode(corrupted), `${name} must be refused`);
|
||||
}
|
||||
assert.throws(() => checkpoint.decode(bytes.subarray(0, bytes.length - 1)));
|
||||
assert.throws(() => checkpoint.decode(bytes.subarray(0, 8)));
|
||||
});
|
||||
|
||||
test('a FLYSIM01 envelope is not read as a session checkpoint', () => {
|
||||
const manifest = Buffer.from('{"schemaVersion":2,"chunks":["agent"]}', 'utf8');
|
||||
const length = Buffer.alloc(4);
|
||||
length.writeUInt32LE(manifest.length, 0);
|
||||
const chunkLength = Buffer.alloc(4);
|
||||
chunkLength.writeUInt32LE(64, 0);
|
||||
const legacy = Buffer.concat([
|
||||
Buffer.from('FLYSIM01', 'ascii'),
|
||||
length,
|
||||
manifest,
|
||||
chunkLength,
|
||||
Buffer.alloc(64),
|
||||
Buffer.alloc(4), // the CRC32 footer
|
||||
]);
|
||||
assert.throws(() => checkpoint.decode(legacy), /magic/);
|
||||
});
|
||||
|
||||
test('the layout is deterministic and the manifest is canonical', () => {
|
||||
const manifest = parseStrict('{"b":2,"a":1}');
|
||||
const payloads = [
|
||||
{ name: 'one', bytes: new TextEncoder().encode('first') },
|
||||
{ name: 'two', bytes: new Uint8Array(9) },
|
||||
];
|
||||
const bytes = checkpoint.encode(manifest, payloads);
|
||||
assert.deepEqual(bytes, checkpoint.encode(manifest, payloads));
|
||||
const envelope = checkpoint.decode(bytes);
|
||||
const start = checkpoint.HEADER_BYTES;
|
||||
const end = start + envelope.layout.manifestBytes;
|
||||
assert.equal(Buffer.from(bytes.subarray(start, end)).toString('utf8'), '{"a":1,"b":2}');
|
||||
assert.throws(() =>
|
||||
checkpoint.encode(manifest, [
|
||||
{ name: 'one', bytes: new Uint8Array() },
|
||||
{ name: 'one', bytes: new Uint8Array() },
|
||||
]),
|
||||
);
|
||||
assert.throws(() => checkpoint.encode(manifest, [{ name: 'One', bytes: new Uint8Array() }]));
|
||||
});
|
||||
|
||||
test('an envelope written here is read by the same rules the Rust crate wrote its fixture with', () => {
|
||||
const file = fixtures.load('checkpoint-envelope.json');
|
||||
const manifest = fixtures.member(file, 'manifest');
|
||||
const payloads = fixtures
|
||||
.section(file, 'payloads')
|
||||
.map((payload) => ({
|
||||
name: fixtures.field(payload, 'name'),
|
||||
bytes: fixtures.decodeBase64(fixtures.field(payload, 'base64')),
|
||||
}));
|
||||
assert.deepEqual(
|
||||
Uint8Array.from(checkpoint.encode(manifest, payloads)),
|
||||
Uint8Array.from(envelopeBytes()),
|
||||
'the two implementations produce the same bytes for the same inputs',
|
||||
);
|
||||
});
|
||||
|
||||
test('a manifest missing a required field is not a complete checkpoint', () => {
|
||||
const file = fixtures.load('checkpoint-envelope.json');
|
||||
const full = fixtures.member(file, 'manifest') as Record<string, unknown>;
|
||||
for (const field of checkpoint.REQUIRED_MANIFEST_FIELDS) {
|
||||
const manifest = { ...full };
|
||||
delete manifest[field];
|
||||
const envelope = checkpoint.decode(checkpoint.encode(manifest, []));
|
||||
assert.throws(() => checkpoint.validateManifest(envelope), `without ${field}`);
|
||||
}
|
||||
});
|
||||
85
packages/session-types/tests/descriptor-checks.test.ts
Normal file
85
packages/session-types/tests/descriptor-checks.test.ts
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import * as fixtures from '../src/fixtures';
|
||||
import { readCommittedSnapshot, readSessionDescriptor, validateSnapshotAgainst } from '../src/publishing';
|
||||
import {
|
||||
findPort,
|
||||
readEnvironmentDescriptor,
|
||||
readPortControl,
|
||||
readSensoryInput,
|
||||
readStepResult,
|
||||
readWorldObservation,
|
||||
validateBatch,
|
||||
validateObservationAgainst,
|
||||
validatePortControlAgainst,
|
||||
validateSensoryInputAgainst,
|
||||
validateStepResultAgainst,
|
||||
} from '../src/workers';
|
||||
import { requiredProducedStep, frameBytes } from '../src/media';
|
||||
|
||||
test('every descriptor check lands the way the fixture says', () => {
|
||||
const file = fixtures.load('descriptor-checks.json');
|
||||
const descriptor = readEnvironmentDescriptor(fixtures.member(file, 'descriptor'));
|
||||
const delayed = readEnvironmentDescriptor(fixtures.member(file, 'delayedDescriptor'));
|
||||
const session = readSessionDescriptor(fixtures.member(file, 'sessionDescriptor'));
|
||||
const previous = readWorldObservation(fixtures.member(file, 'stepResultPrevious'));
|
||||
|
||||
for (const item of fixtures.cases(file)) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const kind = fixtures.field(item, 'kind');
|
||||
const reason = fixtures.optionalField(item, 'reason') ?? '';
|
||||
const expectAccept = fixtures.field(item, 'expect') === 'accept';
|
||||
const value = fixtures.member(item, 'value');
|
||||
const attempt = () => {
|
||||
switch (kind) {
|
||||
case 'portControl': {
|
||||
const control = readPortControl(value);
|
||||
const port = findPort(descriptor, control.portId);
|
||||
if (!port) throw new Error('no such port');
|
||||
validatePortControlAgainst(control, port.controls);
|
||||
return;
|
||||
}
|
||||
case 'advanceControls': {
|
||||
const controls = (value as unknown[]).map(readPortControl);
|
||||
validateBatch(descriptor, controls);
|
||||
return;
|
||||
}
|
||||
case 'sensoryInput':
|
||||
validateSensoryInputAgainst(readSensoryInput(value), descriptor.views);
|
||||
return;
|
||||
case 'sensoryInputDelayed':
|
||||
validateSensoryInputAgainst(readSensoryInput(value), delayed.views);
|
||||
return;
|
||||
case 'worldObservation':
|
||||
validateObservationAgainst(readWorldObservation(value), descriptor);
|
||||
return;
|
||||
case 'stepResult':
|
||||
validateStepResultAgainst(readStepResult(value), descriptor, previous);
|
||||
return;
|
||||
case 'snapshot':
|
||||
validateSnapshotAgainst(readCommittedSnapshot(value), session);
|
||||
return;
|
||||
default:
|
||||
throw new Error(`unknown descriptor check kind "${kind}"`);
|
||||
}
|
||||
};
|
||||
if (expectAccept) {
|
||||
assert.doesNotThrow(attempt, `${name} must be accepted. ${reason}`);
|
||||
} else {
|
||||
assert.throws(attempt, `${name} must be refused. ${reason}`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('the required producing boundary saturates at zero', () => {
|
||||
const file = fixtures.load('descriptor-checks.json');
|
||||
const delayed = readEnvironmentDescriptor(fixtures.member(file, 'delayedDescriptor'));
|
||||
const view = delayed.views[0]!;
|
||||
assert.equal(view.observationDelaySteps, 2);
|
||||
assert.equal(requiredProducedStep(view, 0n), 0n);
|
||||
assert.equal(requiredProducedStep(view, 1n), 0n);
|
||||
assert.equal(requiredProducedStep(view, 2n), 0n);
|
||||
assert.equal(requiredProducedStep(view, 3n), 1n);
|
||||
assert.equal(frameBytes(view), 160 * 4 * 144);
|
||||
});
|
||||
70
packages/session-types/tests/encodings.test.ts
Normal file
70
packages/session-types/tests/encodings.test.ts
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import * as fixtures from '../src/fixtures';
|
||||
import { readArtifactRef } from '../src/reader';
|
||||
import {
|
||||
artifactIdentity,
|
||||
isBusCallId,
|
||||
isDigest,
|
||||
isDomainRequestId,
|
||||
isId,
|
||||
ownerTokenKind,
|
||||
parseU64,
|
||||
} from '../src/scalar';
|
||||
import { readAssetRef } from '../src/workers';
|
||||
|
||||
test('U64 boundaries reject from the fixture', () => {
|
||||
const file = fixtures.load('boundaries.json');
|
||||
for (const item of fixtures.section(file, 'u64')) {
|
||||
const record = item as Record<string, unknown>;
|
||||
assert.equal(
|
||||
parseU64(record.text) !== undefined,
|
||||
record.accept,
|
||||
`${String(record.text)}: ${String(record.reason)}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('the Id and Digest encodings are the ones the bus uses', () => {
|
||||
for (const id of ['a', 'fly-a', '0', 'a.b_c-d', 'a'.repeat(64)]) {
|
||||
assert.ok(isId(id), `${id} is an Id`);
|
||||
}
|
||||
for (const id of ['', 'A', '-a', '.a', 'a b', 'fly/a', 'a'.repeat(65)]) {
|
||||
assert.ok(!isId(id), `${id} is not an Id`);
|
||||
}
|
||||
assert.ok(isDigest('a'.repeat(64)));
|
||||
assert.ok(!isDigest('A'.repeat(64)), 'digests are lowercase');
|
||||
assert.ok(!isDigest('a'.repeat(63)), 'digests are 64 hex digits');
|
||||
assert.ok(!isDigest('g'.repeat(64)), 'digests are hexadecimal');
|
||||
});
|
||||
|
||||
test('the four identities never accept each other spellings', () => {
|
||||
const file = fixtures.load('identities.json');
|
||||
for (const item of fixtures.cases(file)) {
|
||||
const record = item as Record<string, unknown>;
|
||||
const text = record.text as string;
|
||||
assert.equal(isBusCallId(text), record.busCallId, `busCallId ${text}`);
|
||||
assert.equal(isDomainRequestId(text), record.domainRequestId, `domainRequestId ${text}`);
|
||||
assert.equal(ownerTokenKind(text) ?? null, record.ownerToken, `owner token ${text}`);
|
||||
const accepted = [
|
||||
isBusCallId(text),
|
||||
isDomainRequestId(text),
|
||||
ownerTokenKind(text) !== undefined,
|
||||
].filter(Boolean).length;
|
||||
assert.ok(accepted <= 1, `${text} is accepted by more than one identity type`);
|
||||
}
|
||||
});
|
||||
|
||||
test('an artifact identity is the naming half of an ArtifactRef, and an asset is neither', () => {
|
||||
const file = fixtures.load('identities.json');
|
||||
const artifact = fixtures.member(file, 'artifact');
|
||||
const reference = readArtifactRef(fixtures.member(artifact, 'ref'));
|
||||
assert.deepEqual(artifactIdentity(reference), fixtures.member(artifact, 'identity'));
|
||||
const asset = readAssetRef(fixtures.member(file, 'asset'));
|
||||
assert.notEqual(
|
||||
asset.id,
|
||||
reference.artifactId,
|
||||
'the fixture asset and artifact are deliberately different things',
|
||||
);
|
||||
});
|
||||
136
packages/session-types/tests/payloads.test.ts
Normal file
136
packages/session-types/tests/payloads.test.ts
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { canonicalize, parseStrict, requireEnvelopeFit, sha256Hex } from '../src/canonical';
|
||||
import { MAX_TYPED_VALUE_BYTES } from '../src/scalar';
|
||||
import { readTypedValue } from '../src/common';
|
||||
import * as fixtures from '../src/fixtures';
|
||||
import { READERS, roundTrip } from './readers';
|
||||
|
||||
test('every valid case round trips and canonicalizes to its recorded bytes', () => {
|
||||
const file = fixtures.load('valid.json');
|
||||
const cases = fixtures.cases(file);
|
||||
for (const item of cases) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const typeName = fixtures.field(item, 'type');
|
||||
const value = fixtures.member(item, 'value');
|
||||
let written: unknown;
|
||||
try {
|
||||
written = roundTrip(typeName, value);
|
||||
} catch (error) {
|
||||
assert.fail(`${name} (${typeName}) must be accepted: ${String(error)}`);
|
||||
}
|
||||
const canonicalIn = canonicalize(value);
|
||||
assert.equal(
|
||||
canonicalize(written),
|
||||
canonicalIn,
|
||||
`${name}: reading and writing must preserve every field`,
|
||||
);
|
||||
assert.equal(canonicalIn, fixtures.field(item, 'canonical'), `${name}: canonical JSON`);
|
||||
assert.equal(sha256Hex(canonicalIn), fixtures.field(item, 'digest'), `${name}: digest`);
|
||||
}
|
||||
assert.ok(cases.length >= 70, 'the valid fixture should stay broad');
|
||||
});
|
||||
|
||||
test('every type this package reads appears in the valid fixture', () => {
|
||||
const covered = fixtures
|
||||
.cases(fixtures.load('valid.json'))
|
||||
.map((item) => fixtures.field(item, 'type'));
|
||||
const missing = Object.keys(READERS).filter((typeName) => !covered.includes(typeName));
|
||||
assert.deepEqual(missing, [], 'every readable type needs at least one accepted fixture');
|
||||
});
|
||||
|
||||
test('every invalid case is refused', () => {
|
||||
const file = fixtures.load('invalid.json');
|
||||
const cases = fixtures.cases(file);
|
||||
for (const item of cases) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const typeName = fixtures.field(item, 'type');
|
||||
const reason = fixtures.field(item, 'reason');
|
||||
assert.throws(
|
||||
() => roundTrip(typeName, fixtures.member(item, 'value')),
|
||||
`${name} (${typeName}) must be refused: ${reason}`,
|
||||
);
|
||||
}
|
||||
assert.ok(cases.length >= 80, 'the invalid fixture should stay broad');
|
||||
});
|
||||
|
||||
test('every raw byte case is refused before or during validation', () => {
|
||||
for (const item of fixtures.cases(fixtures.load('raw.json'))) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const typeName = fixtures.field(item, 'type');
|
||||
const reason = fixtures.field(item, 'reason');
|
||||
const bytes = fixtures.decodeBase64(fixtures.field(item, 'base64'));
|
||||
// parseStrict is the only door into the readers, so a byte sequence that does not parse
|
||||
// never reaches validation.
|
||||
assert.throws(
|
||||
() => roundTrip(typeName, parseStrict(bytes)),
|
||||
`${name} must be refused: ${reason}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('generated boundary cases land on the right side of every limit', () => {
|
||||
const file = fixtures.load('generated.json');
|
||||
const padSchema = fixtures.member(file, 'padSchema');
|
||||
for (const item of fixtures.cases(file)) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const kind = fixtures.field(item, 'kind');
|
||||
const expectAccept = fixtures.field(item, 'expect') === 'accept';
|
||||
const record = item as Record<string, unknown>;
|
||||
const attempt = () => {
|
||||
switch (kind) {
|
||||
case 'padded-typed-value': {
|
||||
const pad = 'a'.repeat(record.padCharacters as number);
|
||||
readTypedValue({ schema: padSchema, value: { pad } });
|
||||
return;
|
||||
}
|
||||
case 'padded-request': {
|
||||
const pad = 'a'.repeat(record.padCharacters as number);
|
||||
const total = record.envelopeTotal as number;
|
||||
const body = roundTrip('SessionRpcRequest', {
|
||||
requestId: 'req-1',
|
||||
scope: null,
|
||||
params: { pad },
|
||||
});
|
||||
requireEnvelopeFit(body, total - canonicalize(body).length);
|
||||
return;
|
||||
}
|
||||
case 'error-message':
|
||||
case 'error-message-astral': {
|
||||
const character = kind === 'error-message' ? 'x' : '\u{10400}';
|
||||
const message = character.repeat(record.codePoints as number);
|
||||
roundTrip('SessionRpcFailure', {
|
||||
type: 'error',
|
||||
requestId: 'req-41',
|
||||
workerId: 'fly-a',
|
||||
incarnationId: 'inc-1',
|
||||
scope: null,
|
||||
error: { code: 'INTERNAL', message, mutation: 'unknown' },
|
||||
});
|
||||
return;
|
||||
}
|
||||
default:
|
||||
throw new Error(`unknown generated case kind "${kind}"`);
|
||||
}
|
||||
};
|
||||
if (expectAccept) {
|
||||
assert.doesNotThrow(attempt, `${name} must be accepted`);
|
||||
} else {
|
||||
assert.throws(attempt, `${name} must be refused`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('a typed value at the cap is accepted and one byte more is not', () => {
|
||||
const schema = { id: 'pad.v1', version: 1, digest: sha256Hex('pad.v1') };
|
||||
const overhead = canonicalize({ schema, value: { pad: '' } }).length;
|
||||
const atCap = readTypedValue({
|
||||
schema,
|
||||
value: { pad: 'a'.repeat(MAX_TYPED_VALUE_BYTES - overhead) },
|
||||
});
|
||||
assert.equal(canonicalize(atCap).length, MAX_TYPED_VALUE_BYTES);
|
||||
assert.throws(() =>
|
||||
readTypedValue({ schema, value: { pad: 'a'.repeat(MAX_TYPED_VALUE_BYTES - overhead + 1) } }),
|
||||
);
|
||||
});
|
||||
98
packages/session-types/tests/rational.test.ts
Normal file
98
packages/session-types/tests/rational.test.ts
Normal file
|
|
@ -0,0 +1,98 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { readRational } from '../src/common';
|
||||
import * as fixtures from '../src/fixtures';
|
||||
import {
|
||||
RATIONAL_ZERO,
|
||||
U64_MAX,
|
||||
addRational,
|
||||
compareRational,
|
||||
divideFloor,
|
||||
multiplyRational,
|
||||
reduced,
|
||||
requirePositiveRational,
|
||||
subtractRational,
|
||||
validateRational,
|
||||
} from '../src/scalar';
|
||||
|
||||
test('the accumulator produces the fixture tick counts and remainders', () => {
|
||||
const file = fixtures.load('rational.json');
|
||||
for (const item of fixtures.section(file, 'accumulator')) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const step = readRational(fixtures.member(item, 'stepDuration'));
|
||||
const tick = readRational(fixtures.member(item, 'tickDuration'));
|
||||
let accumulator = RATIONAL_ZERO;
|
||||
let total = 0n;
|
||||
fixtures.section(item, 'steps').forEach((expected, index) => {
|
||||
accumulator = addRational(accumulator, step);
|
||||
const { ticks, remainder } = divideFloor(accumulator, tick);
|
||||
accumulator = remainder;
|
||||
total += BigInt(ticks);
|
||||
assert.equal(ticks, fixtures.field(expected, 'ticks'), `${name}: ticks at step ${index}`);
|
||||
assert.deepEqual(
|
||||
remainder,
|
||||
readRational(fixtures.member(expected, 'remainder')),
|
||||
`${name}: remainder at step ${index}`,
|
||||
);
|
||||
assert.ok(compareRational(remainder, tick) < 0, `${name}: remainder below one tick`);
|
||||
});
|
||||
assert.equal(total.toString(), fixtures.field(item, 'totalTicks'), `${name}: total ticks`);
|
||||
}
|
||||
});
|
||||
|
||||
test('checked arithmetic reduces or refuses', () => {
|
||||
const file = fixtures.load('rational.json');
|
||||
for (const item of fixtures.section(file, 'add')) {
|
||||
const left = readRational(fixtures.member(item, 'a'));
|
||||
const right = readRational(fixtures.member(item, 'b'));
|
||||
const record = item as Record<string, unknown>;
|
||||
if (record.sum !== undefined) {
|
||||
assert.deepEqual(addRational(left, right), readRational(record.sum as never));
|
||||
} else {
|
||||
assert.throws(() => addRational(left, right));
|
||||
}
|
||||
}
|
||||
for (const item of fixtures.section(file, 'subtract')) {
|
||||
const left = readRational(fixtures.member(item, 'a'));
|
||||
const right = readRational(fixtures.member(item, 'b'));
|
||||
const record = item as Record<string, unknown>;
|
||||
if (record.difference !== undefined) {
|
||||
assert.deepEqual(subtractRational(left, right), readRational(record.difference as never));
|
||||
} else {
|
||||
assert.throws(() => subtractRational(left, right));
|
||||
}
|
||||
}
|
||||
for (const item of fixtures.section(file, 'multiply')) {
|
||||
const value = readRational(fixtures.member(item, 'a'));
|
||||
const factor = BigInt(fixtures.field(item, 'k'));
|
||||
const record = item as Record<string, unknown>;
|
||||
if (record.product !== undefined) {
|
||||
assert.deepEqual(multiplyRational(value, factor), readRational(record.product as never));
|
||||
} else {
|
||||
assert.throws(() => multiplyRational(value, factor));
|
||||
}
|
||||
}
|
||||
for (const item of fixtures.section(file, 'compare')) {
|
||||
const left = readRational(fixtures.member(item, 'a'));
|
||||
const right = readRational(fixtures.member(item, 'b'));
|
||||
const expected = { less: -1, equal: 0, greater: 1 }[fixtures.field(item, 'ordering')];
|
||||
assert.equal(compareRational(left, right), expected);
|
||||
}
|
||||
});
|
||||
|
||||
test('zero has exactly one encoding and durations must be positive', () => {
|
||||
validateRational(RATIONAL_ZERO);
|
||||
assert.throws(() => validateRational({ numerator: '0', denominator: '2' }), /0\/1/);
|
||||
assert.throws(() => validateRational({ numerator: '1', denominator: '0' }), /positive/);
|
||||
assert.throws(() => validateRational({ numerator: '2', denominator: '4' }), /reduced/);
|
||||
assert.throws(() => requirePositiveRational(RATIONAL_ZERO, 'worldTime'));
|
||||
assert.throws(() => divideFloor(RATIONAL_ZERO, RATIONAL_ZERO), /positive/);
|
||||
});
|
||||
|
||||
test('reduction refuses a result that does not fit U64', () => {
|
||||
const big = { numerator: U64_MAX.toString(), denominator: '1' };
|
||||
assert.throws(() => multiplyRational(big, 2n), /does not fit U64/);
|
||||
assert.throws(() => addRational(big, big), /does not fit U64/);
|
||||
assert.deepEqual(reduced(U64_MAX * 2n, 2n), big);
|
||||
});
|
||||
122
packages/session-types/tests/readers.ts
Normal file
122
packages/session-types/tests/readers.ts
Normal file
|
|
@ -0,0 +1,122 @@
|
|||
/** One place that knows how to read every type named by a fixture. */
|
||||
import type { Json } from '../src/canonical';
|
||||
import {
|
||||
readRational,
|
||||
readSchemaRef,
|
||||
readScope,
|
||||
readTypedValue,
|
||||
} from '../src/common';
|
||||
import {
|
||||
readActivateRestoreParams,
|
||||
readAudioDescriptor,
|
||||
readAudioRef,
|
||||
readCaptureParams,
|
||||
readCaptureResult,
|
||||
readStageRestoreParams,
|
||||
readStageRestoreResult,
|
||||
readViewDescriptor,
|
||||
readViewRef,
|
||||
} from '../src/media';
|
||||
import { readCommittedSnapshot, readSessionDescriptor } from '../src/publishing';
|
||||
import {
|
||||
readSessionRpcFailure,
|
||||
readSessionRpcRequest,
|
||||
readSessionRpcSuccess,
|
||||
} from '../src/rpc';
|
||||
import {
|
||||
readTraceBehaviour,
|
||||
readTraceOperational,
|
||||
readTransitionTrace,
|
||||
} from '../src/trace';
|
||||
import {
|
||||
readAcknowledgeParams,
|
||||
readAcknowledgeResult,
|
||||
readActivateRestoreResult,
|
||||
readAdvanceParams,
|
||||
readAgentCommitResult,
|
||||
readAgentInitializeParams,
|
||||
readAgentInitializeResult,
|
||||
readAgentTelemetry,
|
||||
readAssetRef,
|
||||
readCommitParams,
|
||||
readControllerSchema,
|
||||
readEnvironmentDescriptor,
|
||||
readEnvironmentInitializeParams,
|
||||
readEnvironmentInitializeResult,
|
||||
readEpisodeRequest,
|
||||
readHelloParams,
|
||||
readHelloResult,
|
||||
readPortControl,
|
||||
readPrepareParams,
|
||||
readPreparedDecision,
|
||||
readReward,
|
||||
readSensoryInput,
|
||||
readShutdownParams,
|
||||
readShutdownResult,
|
||||
readStatusResult,
|
||||
readStepResult,
|
||||
readStimulus,
|
||||
readTaskEvent,
|
||||
readWorldObservation,
|
||||
} from '../src/workers';
|
||||
|
||||
/** Every type the fixtures name, and the reader that validates it. */
|
||||
export const READERS: Record<string, (value: unknown) => unknown> = {
|
||||
Scope: readScope,
|
||||
RationalNs: readRational,
|
||||
SchemaRef: readSchemaRef,
|
||||
TypedValue: readTypedValue,
|
||||
SessionRpcRequest: readSessionRpcRequest,
|
||||
SessionRpcSuccess: readSessionRpcSuccess,
|
||||
SessionRpcFailure: readSessionRpcFailure,
|
||||
AssetRef: readAssetRef,
|
||||
SensoryInput: readSensoryInput,
|
||||
Stimulus: readStimulus,
|
||||
Reward: readReward,
|
||||
AgentTelemetry: readAgentTelemetry,
|
||||
AgentInitializeParams: readAgentInitializeParams,
|
||||
AgentInitializeResult: readAgentInitializeResult,
|
||||
PrepareParams: readPrepareParams,
|
||||
PreparedDecision: readPreparedDecision,
|
||||
CommitParams: readCommitParams,
|
||||
AgentCommitResult: readAgentCommitResult,
|
||||
ControllerSchema: readControllerSchema,
|
||||
PortControl: readPortControl,
|
||||
EnvironmentDescriptor: readEnvironmentDescriptor,
|
||||
EnvironmentInitializeParams: readEnvironmentInitializeParams,
|
||||
EnvironmentInitializeResult: readEnvironmentInitializeResult,
|
||||
WorldObservation: readWorldObservation,
|
||||
AdvanceParams: readAdvanceParams,
|
||||
StepResult: readStepResult,
|
||||
HelloParams: readHelloParams,
|
||||
HelloResult: readHelloResult,
|
||||
StatusResult: readStatusResult,
|
||||
AcknowledgeParams: readAcknowledgeParams,
|
||||
AcknowledgeResult: readAcknowledgeResult,
|
||||
ShutdownParams: readShutdownParams,
|
||||
ShutdownResult: readShutdownResult,
|
||||
TaskEvent: readTaskEvent,
|
||||
EpisodeRequest: readEpisodeRequest,
|
||||
ViewDescriptor: readViewDescriptor,
|
||||
ViewRef: readViewRef,
|
||||
AudioDescriptor: readAudioDescriptor,
|
||||
AudioRef: readAudioRef,
|
||||
CaptureParams: readCaptureParams,
|
||||
CaptureResult: readCaptureResult,
|
||||
StageRestoreParams: readStageRestoreParams,
|
||||
StageRestoreResult: readStageRestoreResult,
|
||||
ActivateRestoreParams: readActivateRestoreParams,
|
||||
ActivateRestoreResult: readActivateRestoreResult,
|
||||
SessionDescriptor: readSessionDescriptor,
|
||||
CommittedSnapshot: readCommittedSnapshot,
|
||||
TraceBehaviour: readTraceBehaviour,
|
||||
TraceOperational: readTraceOperational,
|
||||
TransitionTrace: readTransitionTrace,
|
||||
};
|
||||
|
||||
/** Reads the value as `typeName` and hands back what the reader reconstructed. */
|
||||
export function roundTrip(typeName: string, value: Json): unknown {
|
||||
const reader = READERS[typeName];
|
||||
if (!reader) throw new Error(`no fixture reader for type "${typeName}"`);
|
||||
return reader(value);
|
||||
}
|
||||
100
packages/session-types/tests/schema.test.ts
Normal file
100
packages/session-types/tests/schema.test.ts
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { canonicalize, digestOf, parseStrict, sha256Hex } from '../src/canonical';
|
||||
import * as fixtures from '../src/fixtures';
|
||||
|
||||
/**
|
||||
* The schema set is generated by the Rust crate; this side hashes the checked-in file with its
|
||||
* own canonical JSON and digest, which is the cross-language half of `contractDigest`.
|
||||
*/
|
||||
test('the contract digest is the digest of the checked-in schema set', () => {
|
||||
const bytes = fixtures.loadBytes('schema-set.json');
|
||||
const text = Buffer.from(bytes).toString('utf8');
|
||||
assert.ok(text.endsWith('\n'), 'the file is the canonical set plus one newline');
|
||||
const canonical = text.slice(0, -1);
|
||||
const recorded = fixtures.load('contract-digest.json') as Record<string, unknown>;
|
||||
assert.equal(sha256Hex(canonical), recorded.contractDigest);
|
||||
assert.equal(canonical.length, recorded.schemaSetBytes);
|
||||
// ... and the file really is canonical JSON: reserializing it changes nothing.
|
||||
assert.equal(canonicalize(parseStrict(canonical)), canonical);
|
||||
assert.equal(digestOf(parseStrict(canonical)), recorded.contractDigest);
|
||||
});
|
||||
|
||||
test('the contract digest survives reformatting and changes when a schema changes', () => {
|
||||
const set = parseStrict(
|
||||
Buffer.from(fixtures.loadBytes('schema-set.json')).toString('utf8').slice(0, -1),
|
||||
) as Record<string, unknown>;
|
||||
const recorded = fixtures.load('contract-digest.json') as Record<string, unknown>;
|
||||
const pretty = parseStrict(JSON.stringify(set, null, 4));
|
||||
assert.equal(digestOf(pretty), recorded.contractDigest, 'pretty printing is not a change');
|
||||
|
||||
const types = set.types as Record<string, unknown>[];
|
||||
const renamed = structuredClone(set);
|
||||
((renamed.types as Record<string, unknown>[])[0]!.fields as Record<string, unknown>[])[0]!.name =
|
||||
'sessionIdentifier';
|
||||
assert.notEqual(digestOf(renamed), recorded.contractDigest, 'a renamed field is a change');
|
||||
|
||||
const widened = structuredClone(set);
|
||||
for (const limit of widened.limits as Record<string, unknown>[]) {
|
||||
if (limit.name === 'maxAgents') limit.value = 8;
|
||||
}
|
||||
assert.notEqual(digestOf(widened), recorded.contractDigest, 'a widened bound is a change');
|
||||
|
||||
const dropped = structuredClone(set);
|
||||
(dropped.types as unknown[]).pop();
|
||||
assert.notEqual(digestOf(dropped), recorded.contractDigest, 'a dropped type is a change');
|
||||
assert.ok(types.length >= 50, 'the schema set should stay broad');
|
||||
});
|
||||
|
||||
test('the schema set publishes the limits this package enforces', async () => {
|
||||
const set = parseStrict(
|
||||
Buffer.from(fixtures.loadBytes('schema-set.json')).toString('utf8').slice(0, -1),
|
||||
) as Record<string, unknown>;
|
||||
const limits = new Map(
|
||||
(set.limits as Record<string, unknown>[]).map((limit) => [
|
||||
limit.name as string,
|
||||
limit.value as number,
|
||||
]),
|
||||
);
|
||||
const workers = await import('../src/workers');
|
||||
const media = await import('../src/media');
|
||||
const scalar = await import('../src/scalar');
|
||||
const canonical = await import('../src/canonical');
|
||||
assert.equal(limits.get('maxAgents'), workers.MAX_AGENTS);
|
||||
assert.equal(limits.get('maxPorts'), workers.MAX_PORTS);
|
||||
assert.equal(limits.get('maxRateRoles'), workers.MAX_RATE_ROLES);
|
||||
assert.equal(limits.get('maxStimuliPerOperation'), workers.MAX_STIMULI);
|
||||
assert.equal(limits.get('maxRewardsPerOperation'), workers.MAX_REWARDS);
|
||||
assert.equal(limits.get('maxButtons'), workers.MAX_BUTTONS);
|
||||
assert.equal(limits.get('maxAxes'), workers.MAX_AXES);
|
||||
assert.equal(limits.get('maxAcknowledge'), workers.MAX_ACKNOWLEDGE);
|
||||
assert.equal(limits.get('maxMessageCodePoints'), workers.MAX_MESSAGE_CODE_POINTS);
|
||||
assert.equal(limits.get('maxViews'), media.MAX_VIEWS);
|
||||
assert.equal(limits.get('maxViewDimension'), media.MAX_VIEW_DIMENSION);
|
||||
assert.equal(limits.get('maxSampleFrames'), media.MAX_SAMPLE_FRAMES);
|
||||
assert.equal(limits.get('maxAudioStreams'), media.MAX_AUDIO_STREAMS);
|
||||
assert.equal(limits.get('maxTypedValueBytes'), scalar.MAX_TYPED_VALUE_BYTES);
|
||||
assert.equal(limits.get('maxEnvelopeBytes'), canonical.MAX_ENVELOPE_BYTES);
|
||||
});
|
||||
|
||||
test('the closed enums this package knows are the ones the schema set declares', async () => {
|
||||
const set = parseStrict(
|
||||
Buffer.from(fixtures.loadBytes('schema-set.json')).toString('utf8').slice(0, -1),
|
||||
) as Record<string, unknown>;
|
||||
const enums = new Map(
|
||||
(set.enums as Record<string, unknown>[]).map((entry) => [
|
||||
entry.name as string,
|
||||
entry.members as string[],
|
||||
]),
|
||||
);
|
||||
const workers = await import('../src/workers');
|
||||
const rpc = await import('../src/rpc');
|
||||
assert.deepEqual(enums.get('ErrorCode'), [...rpc.ERROR_CODES]);
|
||||
assert.deepEqual(enums.get('MutationCertainty'), [...rpc.MUTATION_CERTAINTIES]);
|
||||
assert.deepEqual(enums.get('Role'), [...workers.ROLES]);
|
||||
assert.deepEqual(enums.get('WorkerState'), [...workers.WORKER_STATES]);
|
||||
assert.deepEqual(enums.get('Recovery'), [...workers.RECOVERY]);
|
||||
assert.deepEqual(enums.get('Determinism'), [...workers.DETERMINISM]);
|
||||
assert.deepEqual(enums.get('AxisRange'), [...workers.AXIS_RANGES]);
|
||||
});
|
||||
61
packages/session-types/tests/seeds.test.ts
Normal file
61
packages/session-types/tests/seeds.test.ts
Normal file
|
|
@ -0,0 +1,61 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import * as fixtures from '../src/fixtures';
|
||||
import * as seed from '../src/seed';
|
||||
|
||||
test('every vector derives its recorded seed', () => {
|
||||
const file = fixtures.load('seed-vectors.json');
|
||||
assert.equal((file as Record<string, unknown>).algorithm, seed.ALGORITHM);
|
||||
const vectors = fixtures.section(file, 'vectors');
|
||||
for (const item of vectors) {
|
||||
const master = seed.masterSeed(fixtures.field(item, 'masterSeed'));
|
||||
const agentId = fixtures.field(item, 'agentId');
|
||||
assert.equal(
|
||||
Buffer.from(seed.material(master, agentId)).toString('utf8'),
|
||||
fixtures.field(item, 'material'),
|
||||
'the hashed material is part of the specification',
|
||||
);
|
||||
assert.equal(seed.materialDigest(master, agentId), fixtures.field(item, 'materialDigest'));
|
||||
assert.equal(
|
||||
seed.agentSeed(master, agentId),
|
||||
(item as Record<string, unknown>).seed,
|
||||
`seed for ${agentId} under master ${master}`,
|
||||
);
|
||||
}
|
||||
assert.ok(vectors.length >= 20, 'keep the vector table broad');
|
||||
});
|
||||
|
||||
test('one composition gets independent seeds', () => {
|
||||
const file = fixtures.load('seed-vectors.json');
|
||||
const composition = fixtures.member(file, 'composition');
|
||||
const master = seed.masterSeed(fixtures.field(composition, 'masterSeed'));
|
||||
const ids = fixtures.section(composition, 'agentIds') as string[];
|
||||
const seeds = seed.compositionSeeds(master, ids);
|
||||
assert.deepEqual(seeds, fixtures.section(composition, 'seeds'));
|
||||
assert.equal(new Set(seeds).size, seeds.length, 'per-agent seeds are independent');
|
||||
assert.ok(
|
||||
seeds.every((value) => value !== 0),
|
||||
'a zero seed would stall an xorshift generator',
|
||||
);
|
||||
});
|
||||
|
||||
test('a different master seed or agent id derives a different seed', () => {
|
||||
assert.notEqual(seed.agentSeed(0n, 'fly-a'), seed.agentSeed(1n, 'fly-a'));
|
||||
assert.notEqual(seed.agentSeed(0n, 'fly-a'), seed.agentSeed(0n, 'fly-b'));
|
||||
assert.equal(seed.agentSeed(7n, 'fly-a'), seed.agentSeed(7n, 'fly-a'));
|
||||
});
|
||||
|
||||
test('invalid inputs are refused rather than normalized', () => {
|
||||
const file = fixtures.load('seed-vectors.json');
|
||||
for (const item of fixtures.section(file, 'invalid')) {
|
||||
const master = seed.masterSeed(fixtures.field(item, 'masterSeed'));
|
||||
const agentId = fixtures.optionalField(item, 'agentId');
|
||||
if (agentId !== undefined) {
|
||||
assert.throws(() => seed.agentSeed(master, agentId), `${agentId} must be refused`);
|
||||
} else {
|
||||
const ids = fixtures.section(item, 'agentIds') as string[];
|
||||
assert.throws(() => seed.compositionSeeds(master, ids));
|
||||
}
|
||||
}
|
||||
});
|
||||
80
packages/session-types/tests/traces.test.ts
Normal file
80
packages/session-types/tests/traces.test.ts
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import * as fixtures from '../src/fixtures';
|
||||
import {
|
||||
behaviourDiff,
|
||||
behaviourDigest,
|
||||
behaviourEquals,
|
||||
readTransitionTrace,
|
||||
runsEqual,
|
||||
} from '../src/trace';
|
||||
|
||||
test('every variant compares the way the fixture says', () => {
|
||||
const file = fixtures.load('traces.json');
|
||||
const baseline = readTransitionTrace(fixtures.member(file, 'baseline'));
|
||||
for (const item of fixtures.section(file, 'variants')) {
|
||||
const name = fixtures.field(item, 'name');
|
||||
const variant = readTransitionTrace(fixtures.member(item, 'trace'));
|
||||
const expected = (item as Record<string, unknown>).behaviourEquals as boolean;
|
||||
const diff = behaviourDiff(baseline, variant);
|
||||
assert.equal(
|
||||
behaviourEquals(baseline, variant),
|
||||
expected,
|
||||
`${name}: behaviour equality. differences: ${JSON.stringify(diff)}`,
|
||||
);
|
||||
assert.equal(diff.length === 0, expected, `${name}: the diff is empty exactly when equal`);
|
||||
const needle = fixtures.optionalField(item, 'diffContains');
|
||||
if (needle !== undefined) {
|
||||
assert.ok(
|
||||
diff.some((line) => line.includes(needle)),
|
||||
`${name}: the diff should name ${needle}, got ${JSON.stringify(diff)}`,
|
||||
);
|
||||
}
|
||||
if (expected) {
|
||||
assert.equal(
|
||||
behaviourDigest(baseline.behaviour),
|
||||
behaviourDigest(variant.behaviour),
|
||||
`${name}: equal behaviour has one digest`,
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('a whole run compares transition by transition', () => {
|
||||
const file = fixtures.load('traces.json');
|
||||
const baseline = readTransitionTrace(fixtures.member(file, 'baseline'));
|
||||
const variants = fixtures.section(file, 'variants');
|
||||
const reversed = readTransitionTrace(fixtures.member(variants[0] as never, 'trace'));
|
||||
const changed = readTransitionTrace(
|
||||
fixtures.member(
|
||||
variants.find((item) => fixtures.field(item, 'name') === 'one extra neural tick') as never,
|
||||
'trace',
|
||||
),
|
||||
);
|
||||
assert.ok(runsEqual([baseline, baseline], [reversed, baseline]));
|
||||
assert.ok(!runsEqual([baseline], [changed]));
|
||||
assert.ok(!runsEqual([baseline], [baseline, baseline]));
|
||||
});
|
||||
|
||||
test('operational metadata is recorded and excluded', () => {
|
||||
const file = fixtures.load('traces.json');
|
||||
const baseline = readTransitionTrace(fixtures.member(file, 'baseline'));
|
||||
assert.equal(baseline.operational.busCallIds.length, 3);
|
||||
assert.equal(baseline.operational.prepareRequestIds.length, 2);
|
||||
assert.equal(baseline.operational.deliveryIds.length, 2);
|
||||
const retried = readTransitionTrace(
|
||||
fixtures.member(
|
||||
fixtures
|
||||
.section(file, 'variants')
|
||||
.find(
|
||||
(item) =>
|
||||
fixtures.field(item, 'name') ===
|
||||
'a safe retry with fresh bus callIds, delivery ids and wall time',
|
||||
) as never,
|
||||
'trace',
|
||||
),
|
||||
);
|
||||
assert.notDeepEqual(baseline.operational, retried.operational);
|
||||
assert.ok(behaviourEquals(baseline, retried));
|
||||
});
|
||||
19
packages/session-types/tsconfig.json
Normal file
19
packages/session-types/tsconfig.json
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"useDefineForClassFields": true,
|
||||
"module": "ESNext",
|
||||
"lib": ["ES2022", "DOM", "DOM.Iterable", "WebWorker"],
|
||||
"types": ["node"],
|
||||
"skipLibCheck": true,
|
||||
"moduleResolution": "Bundler",
|
||||
"isolatedModules": true,
|
||||
"moduleDetection": "force",
|
||||
"noEmit": true,
|
||||
"strict": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"resolveJsonModule": true
|
||||
},
|
||||
"include": ["src", "tests"]
|
||||
}
|
||||
21
services/flysim/Cargo.lock
generated
21
services/flysim/Cargo.lock
generated
|
|
@ -416,6 +416,27 @@ dependencies = [
|
|||
"serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fly-session"
|
||||
version = "0.1.1"
|
||||
dependencies = [
|
||||
"fly-session-types",
|
||||
"flybus",
|
||||
"serde_json",
|
||||
"tempfile",
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fly-session-types"
|
||||
version = "0.1.1"
|
||||
dependencies = [
|
||||
"flybus",
|
||||
"ryu-js",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "flybrain-core"
|
||||
version = "0.1.1"
|
||||
|
|
|
|||
|
|
@ -1,6 +1,13 @@
|
|||
[workspace]
|
||||
resolver = "3"
|
||||
members = ["crates/flybrain-core", "crates/flybrain-gb", "crates/flybus", "crates/flysim"]
|
||||
members = [
|
||||
"crates/fly-session",
|
||||
"crates/fly-session-types",
|
||||
"crates/flybrain-core",
|
||||
"crates/flybrain-gb",
|
||||
"crates/flybus",
|
||||
"crates/flysim",
|
||||
]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.1.1"
|
||||
|
|
|
|||
16
services/flysim/crates/fly-session-types/Cargo.toml
Normal file
16
services/flysim/crates/fly-session-types/Cargo.toml
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
[package]
|
||||
name = "fly-session-types"
|
||||
version.workspace = true
|
||||
edition = "2024"
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
publish = false
|
||||
description = "Session domain scalars, closed enums, method payloads, canonical JSON digests and the step trace format (session-framework CONTRACT-01)."
|
||||
|
||||
[dependencies]
|
||||
# The bus owns the Id/U64/Digest encodings, strict JSON and ArtifactRef; this crate reuses
|
||||
# them rather than forking their semantics.
|
||||
flybus = { path = "../flybus" }
|
||||
ryu-js.workspace = true
|
||||
serde_json = { workspace = true }
|
||||
sha2 = { workspace = true }
|
||||
103
services/flysim/crates/fly-session-types/README.md
Normal file
103
services/flysim/crates/fly-session-types/README.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
# fly-session-types
|
||||
|
||||
The executable schemas of the session framework: domain scalars, closed enums, method
|
||||
payloads, canonical JSON, canonical digests and the step trace format.
|
||||
|
||||
This crate is CONTRACT-01 of
|
||||
[`docs/design/session-framework/implementation.md`](../../../../docs/design/session-framework/implementation.md).
|
||||
It holds no transport, no worker, no coordinator and no store; it never opens a socket or a
|
||||
file other than its own fixtures. The bus owns the wire
|
||||
([`flybus`](../flybus)), and this crate owns what the messages mean.
|
||||
|
||||
## Layout
|
||||
|
||||
| Module | Contents |
|
||||
| --- | --- |
|
||||
| `scalar` | `Scope`, `RationalNs`, `SchemaRef`, `TypedValue`, the `DomainType` trait, and `BusCallId` / `DomainRequestId` / `ArtifactIdentity` / `OwnerToken` |
|
||||
| `canonical` | RFC 8785 canonical JSON, SHA-256 digests, `OperationKey`, canonical bodies, the 64-KiB envelope check |
|
||||
| `rpc` | `SessionRpcRequest`, `SessionRpcSuccess`, `SessionRpcFailure`, `ErrorCode`, `MutationCertainty` |
|
||||
| `workers` | The closed enums and every Agent/Environment/Worker method payload of workers-v1 |
|
||||
| `media` | `ViewDescriptor`, `ViewRef`, `AudioDescriptor`, `AudioRef` and the `State.*` payloads |
|
||||
| `publishing` | `SessionDescriptor` and `CommittedSnapshot` |
|
||||
| `trace` | `TraceBehaviour`, `TraceOperational`, `TransitionTrace` and the behaviour comparator |
|
||||
| `schema` | The canonical schema set and `contract_digest()` |
|
||||
| `seed` | `seed-derivation-v1` |
|
||||
| `checkpoint` | The `FLYSESS1` envelope layout |
|
||||
| `fixtures` | Loading `fixtures/`, shared with `packages/session-types` |
|
||||
|
||||
`Id`, `U64` and `Digest` are the bus encodings: `scalar` calls into `flybus::wire` instead of
|
||||
restating them, and `tests/encodings.rs` pins that the two agree for every edge case.
|
||||
|
||||
## Reading and validating
|
||||
|
||||
Every type implements `DomainType`:
|
||||
|
||||
```rust
|
||||
use fly_session_types::scalar::{DomainType, Scope};
|
||||
|
||||
let scope = Scope::from_json(&value)?; // reads, refusing unknown fields, then validates
|
||||
scope.validate()?; // the cross-field rules, re-runnable
|
||||
let json = scope.to_json(); // the canonical shape
|
||||
```
|
||||
|
||||
Rules that need another value in hand are separate, because a payload cannot check them alone:
|
||||
|
||||
```rust
|
||||
control.validate_against(&port.controls)?; // complete batch, descriptor order, ranges
|
||||
input.validate_against(&descriptor.views)?; // max(0, boundary - observationDelaySteps)
|
||||
result.validate_against(&descriptor, &previous)?; // exactly one stepDuration of world time
|
||||
snapshot.validate_against(&session_descriptor)?; // revision, agent set, assigned ports
|
||||
telemetry.validate_against_roles(&profile_roles)?; // rates in profile-defined order
|
||||
```
|
||||
|
||||
## Digests
|
||||
|
||||
- `contract_digest()` is the SHA-256 of the canonical schema set (`schema::schema_set()`),
|
||||
which is a declaration: type names, JSON field names, kinds, bounds and closed enums.
|
||||
Reformatting this crate cannot change it; changing a field or a bound does.
|
||||
- `canonical::body_digest(method, scope, params)` is the comparison ipc-v1 section 5 uses to
|
||||
tell a safe replay from a `CONFLICT`. It refuses a body that carries a bus identity.
|
||||
- `OperationKey` is `(sessionId, epoch, step, method, workerId)`, and deliberately not the
|
||||
request id: a changed id for an existing key is the conflict to detect.
|
||||
|
||||
## Fixtures
|
||||
|
||||
`fixtures/` is loaded by these tests and by `packages/session-types`, so a case is written
|
||||
once and holds both languages to it.
|
||||
|
||||
| File | Contents |
|
||||
| --- | --- |
|
||||
| `valid.json` | Payloads every implementation accepts, with their canonical JSON and digest |
|
||||
| `invalid.json` | Payloads every implementation refuses, each with the rule it breaks |
|
||||
| `raw.json` | Byte sequences refused before validation: duplicate keys, invalid UTF-8, `NaN`, trailing data |
|
||||
| `generated.json` | Recipes for payloads too large to store: the 32-KiB and 64-KiB boundaries, 512-code-point messages |
|
||||
| `boundaries.json` | The `U64` decimal-string and double boundaries |
|
||||
| `rational.json` | Checked rational arithmetic and the 16, 17, 17 tick accumulator |
|
||||
| `identities.json` | Which of the four identity types accepts which spelling |
|
||||
| `descriptor-checks.json` | Rules that need a descriptor: batches, delays, byte shapes, descriptor agreement |
|
||||
| `operations.json` | Operation keys, canonical bodies and the pairs that are or are not the same operation |
|
||||
| `traces.json` | A baseline transition and the variants that must or must not compare equal |
|
||||
| `schema-set.json`, `contract-digest.json` | The canonical schema set and its digest |
|
||||
| `seed-vectors.json` | `seed-derivation-v1` test vectors |
|
||||
| `checkpoint-envelope.json` | One `FLYSESS1` envelope, its layout and the corruptions a reader refuses |
|
||||
|
||||
The derived files (`schema-set.json`, `contract-digest.json`, the `canonical`/`digest` fields
|
||||
of `valid.json`, the digests in `operations.json`, `seed-vectors.json` and
|
||||
`checkpoint-envelope.json`) come from
|
||||
`cargo run -p fly-session-types --example update_fixtures`;
|
||||
`tests/schema_set.rs` fails if the checked-in files are stale.
|
||||
|
||||
## Tests
|
||||
|
||||
```sh
|
||||
cargo test -p fly-session-types
|
||||
cargo clippy -p fly-session-types --all-targets
|
||||
```
|
||||
|
||||
## Bounds this crate chose
|
||||
|
||||
Every bound in the schema set names its source. Six are marked `crate` because no document
|
||||
states them: `maxAudioStreams` (8), `maxCapabilities` (32), `maxSupportedMajors` (8),
|
||||
`maxSupportedStimuli` (64), `maxAssets` (64) and `maxSnapshotEvents` (64). They exist so an
|
||||
unbounded array cannot fill an envelope, and they are in the digest, so widening one is a
|
||||
contract change rather than a quiet edit.
|
||||
|
|
@ -0,0 +1,290 @@
|
|||
//! Regenerates the derived fixture files.
|
||||
//!
|
||||
//! `cargo run -p fly-session-types --example update_fixtures`. `tests/fixtures_current.rs`
|
||||
//! fails if the checked-in files differ from what this writes, so the digests in the
|
||||
//! fixtures can never drift from the code that produced them.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use fly_session_types::scalar::{DomainType, Scope};
|
||||
use fly_session_types::{canonical, checkpoint, fixtures, schema, seed};
|
||||
use serde_json::{Map, Value, json};
|
||||
|
||||
fn main() {
|
||||
let dir = fixtures::dir();
|
||||
for (name, contents) in derived() {
|
||||
let path = dir.join(&name);
|
||||
std::fs::write(&path, contents).expect("write fixture");
|
||||
println!("wrote {}", path.display());
|
||||
}
|
||||
}
|
||||
|
||||
/// Every derived fixture, as `(file name, exact bytes)`.
|
||||
pub fn derived() -> Vec<(String, String)> {
|
||||
vec![
|
||||
("schema-set.json".to_owned(), schema_set()),
|
||||
("contract-digest.json".to_owned(), contract_digest()),
|
||||
("valid.json".to_owned(), valid()),
|
||||
("operations.json".to_owned(), operations()),
|
||||
("seed-vectors.json".to_owned(), seed_vectors()),
|
||||
("checkpoint-envelope.json".to_owned(), checkpoint_envelope()),
|
||||
]
|
||||
}
|
||||
|
||||
fn write(value: &Value) -> String {
|
||||
let mut text = serde_json::to_string_pretty(value).expect("serializable");
|
||||
text.push('\n');
|
||||
text
|
||||
}
|
||||
|
||||
fn schema_set() -> String {
|
||||
// The rendered set is itself canonical JSON, so the file the TypeScript package hashes is
|
||||
// byte for byte what the digest was taken over.
|
||||
let mut text = schema::schema_set_json().expect("canonicalizable");
|
||||
text.push('\n');
|
||||
text
|
||||
}
|
||||
|
||||
fn contract_digest() -> String {
|
||||
let set = schema::schema_set_json().expect("canonicalizable");
|
||||
write(&json!({
|
||||
"description": "contractDigest is the SHA-256 of the canonical schema set in schema-set.json.",
|
||||
"contractDigest": schema::contract_digest(),
|
||||
"schemaSetVersion": schema::SCHEMA_SET_VERSION,
|
||||
"schemaSetBytes": set.len(),
|
||||
"types": schema::SCHEMAS.len(),
|
||||
"enums": schema::ENUMS.len(),
|
||||
"limits": schema::LIMITS.len(),
|
||||
}))
|
||||
}
|
||||
|
||||
fn valid() -> String {
|
||||
let mut file = fixtures::load("valid.json").expect("valid.json");
|
||||
let cases = file
|
||||
.get_mut("cases")
|
||||
.and_then(Value::as_array_mut)
|
||||
.expect("cases");
|
||||
for case in cases.iter_mut() {
|
||||
let value = case.get("value").expect("value").clone();
|
||||
let canonical = canonical::canonicalize(&value).expect("canonicalizable");
|
||||
let digest = canonical::sha256_hex(canonical.as_bytes());
|
||||
let map = case.as_object_mut().expect("case object");
|
||||
map.insert("canonical".to_owned(), Value::String(canonical));
|
||||
map.insert("digest".to_owned(), Value::String(digest));
|
||||
}
|
||||
write(&file)
|
||||
}
|
||||
|
||||
fn operations() -> String {
|
||||
let mut file = fixtures::load("operations.json").expect("operations.json");
|
||||
let scope_of = |case: &Value| -> Option<Scope> {
|
||||
match case.get("scope") {
|
||||
Some(Value::Null) | None => None,
|
||||
Some(v) => Some(Scope::from_json(v).expect("scope")),
|
||||
}
|
||||
};
|
||||
for key in file
|
||||
.get_mut("keys")
|
||||
.and_then(Value::as_array_mut)
|
||||
.expect("keys")
|
||||
{
|
||||
let scope = scope_of(key).expect("an operation key has a scope");
|
||||
let method = key.get("method").and_then(Value::as_str).expect("method");
|
||||
let worker = key.get("workerId").and_then(Value::as_str).expect("workerId");
|
||||
let digest = canonical::OperationKey::new(scope, method, worker)
|
||||
.expect("valid key")
|
||||
.digest()
|
||||
.expect("digest");
|
||||
key.as_object_mut()
|
||||
.expect("object")
|
||||
.insert("digest".to_owned(), Value::String(digest));
|
||||
}
|
||||
for body in file
|
||||
.get_mut("bodies")
|
||||
.and_then(Value::as_array_mut)
|
||||
.expect("bodies")
|
||||
{
|
||||
let scope = scope_of(body);
|
||||
let method = body.get("method").and_then(Value::as_str).expect("method");
|
||||
let params = body.get("params").expect("params").clone();
|
||||
let digest =
|
||||
canonical::body_digest(method, scope.as_ref(), ¶ms).expect("canonical body");
|
||||
body.as_object_mut()
|
||||
.expect("object")
|
||||
.insert("digest".to_owned(), Value::String(digest));
|
||||
}
|
||||
write(&file)
|
||||
}
|
||||
|
||||
fn seed_vectors() -> String {
|
||||
let master_seeds: [u64; 5] = [0, 1, 42, 9_223_372_036_854_775_808, u64::MAX];
|
||||
let agents = ["fly-a", "fly-b", "fly-c", "fly-d"];
|
||||
let mut vectors = Vec::new();
|
||||
for master in master_seeds {
|
||||
for agent in agents {
|
||||
let material = seed::material(master, agent).expect("material");
|
||||
vectors.push(json!({
|
||||
"masterSeed": master.to_string(),
|
||||
"agentId": agent,
|
||||
"material": String::from_utf8(material).expect("utf-8"),
|
||||
"materialDigest": seed::material_digest(master, agent).expect("digest"),
|
||||
"seed": seed::agent_seed(master, agent).expect("seed"),
|
||||
}));
|
||||
}
|
||||
}
|
||||
let composition: Vec<Value> = seed::composition_seeds(
|
||||
42,
|
||||
&agents.iter().map(|a| (*a).to_owned()).collect::<Vec<_>>(),
|
||||
)
|
||||
.expect("composition")
|
||||
.into_iter()
|
||||
.map(Value::from)
|
||||
.collect();
|
||||
write(&json!({
|
||||
"description": "seed-derivation-v1 test vectors. Both languages must reproduce every seed.",
|
||||
"algorithm": seed::ALGORITHM,
|
||||
"prefix": seed::PREFIX,
|
||||
"materialTemplate": "<prefix>\\n<masterSeed>\\n<agentId>\\n",
|
||||
"rule": "SHA-256 of the material, read as eight big-endian u32 lanes; the first nonzero lane is the seed as a two's-complement i32.",
|
||||
"vectors": vectors,
|
||||
"composition": {
|
||||
"masterSeed": "42",
|
||||
"agentIds": agents,
|
||||
"seeds": composition,
|
||||
"reason": "independent per-agent seeds from one recorded master seed and stable agent ids",
|
||||
},
|
||||
"invalid": [
|
||||
{"masterSeed": "0", "agentId": "Fly-A", "reason": "an agent id is an Id: lowercase"},
|
||||
{"masterSeed": "0", "agentId": "", "reason": "an agent id is 1..=64 characters"},
|
||||
{"masterSeed": "0", "agentIds": ["fly-a", "fly-a"],
|
||||
"reason": "a composition with a repeated agent id is refused rather than silently sharing a seed"},
|
||||
],
|
||||
}))
|
||||
}
|
||||
|
||||
fn checkpoint_envelope() -> String {
|
||||
let scope = Scope::new("demo", "epoch-1", 42).expect("scope");
|
||||
let manifest = json!({
|
||||
"envelopeVersion": checkpoint::VERSION,
|
||||
"checkpointId": "ckpt-1",
|
||||
"sourceScope": scope.to_json(),
|
||||
"episodeId": "episode-1",
|
||||
"worldTime": {"numerator": "700000000", "denominator": "1"},
|
||||
"schedulerId": "lockstep-v1",
|
||||
"compositionDigest": canonical::sha256_hex(b"composition"),
|
||||
"portMap": [{"portId": "port-1", "agentId": "fly-a"}],
|
||||
"compatibility": {
|
||||
"backendDigest": canonical::sha256_hex(b"backend"),
|
||||
"contentDigest": canonical::sha256_hex(b"content"),
|
||||
"patchDigest": canonical::sha256_hex(b"patch"),
|
||||
"controllerDigest": canonical::sha256_hex(b"controller"),
|
||||
"parserDigest": canonical::sha256_hex(b"parser"),
|
||||
"stateFormatId": "flysess-1",
|
||||
},
|
||||
"agents": [{
|
||||
"agentId": "fly-a",
|
||||
"profileDigest": canonical::sha256_hex(b"profile"),
|
||||
"datasetDigest": canonical::sha256_hex(b"fafb-v783"),
|
||||
"modelVersion": "lif-1ms-f64-v2",
|
||||
"plasticityVersion": "fly-kc-mbon-rstdp-v2",
|
||||
"seed": seed::agent_seed(42, "fly-a").expect("seed"),
|
||||
"brainTicks": "2534",
|
||||
"remainder": {"numerator": "1000000", "denominator": "3"},
|
||||
"payload": "agent-fly-a",
|
||||
}],
|
||||
"coordinator": {
|
||||
"taskLedger": "task-ledger",
|
||||
"priorInspection": "prior-inspection",
|
||||
"executorState": [{"agentId": "fly-a", "payload": "executor-fly-a"}],
|
||||
"admissionState": null,
|
||||
"eventWatermarks": {"lastEventId": "evt-1", "lastOrdinal": "7"},
|
||||
},
|
||||
"helperState": [],
|
||||
"payloads": payload_table(),
|
||||
});
|
||||
let bytes = checkpoint::encode(&manifest, &payloads()).expect("encode");
|
||||
let envelope = checkpoint::decode(&bytes).expect("decode");
|
||||
let entries: Vec<Value> = envelope
|
||||
.layout
|
||||
.entries
|
||||
.iter()
|
||||
.map(|entry| {
|
||||
json!({
|
||||
"name": entry.name,
|
||||
"offset": entry.offset.to_string(),
|
||||
"byteLength": entry.byte_length.to_string(),
|
||||
"digest": checkpoint::hex(&entry.digest),
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
let first_payload = envelope.layout.entries[0].offset;
|
||||
write(&json!({
|
||||
"description": "One FLYSESS1 envelope, its layout and the corruptions a reader must refuse.",
|
||||
"magic": "FLYSESS1",
|
||||
"footerMagic": "FLYSESSF",
|
||||
"version": checkpoint::VERSION,
|
||||
"manifest": manifest,
|
||||
"payloads": payloads()
|
||||
.iter()
|
||||
.map(|(name, bytes)| json!({"name": name, "base64": fixtures::encode_base64(bytes)}))
|
||||
.collect::<Vec<_>>(),
|
||||
"envelope": {
|
||||
"base64": fixtures::encode_base64(&bytes),
|
||||
"byteLength": bytes.len(),
|
||||
"layout": {
|
||||
"headerBytes": checkpoint::HEADER_BYTES,
|
||||
"manifestOffset": envelope.layout.manifest_offset.to_string(),
|
||||
"manifestBytes": envelope.layout.manifest_bytes,
|
||||
"tableOffset": envelope.layout.table_offset.to_string(),
|
||||
"tableEntryBytes": checkpoint::TABLE_ENTRY_BYTES,
|
||||
"entries": entries,
|
||||
"footerOffset": envelope.layout.footer_offset.to_string(),
|
||||
"footerBytes": checkpoint::FOOTER_BYTES,
|
||||
"totalBytes": envelope.layout.total_bytes.to_string(),
|
||||
},
|
||||
},
|
||||
"corruption": [
|
||||
{"name": "a flipped magic byte", "offset": 0, "reason": "wrong magic"},
|
||||
{"name": "an unsupported version", "offset": 8, "reason": "unsupported version"},
|
||||
{"name": "a flipped manifest byte", "offset": checkpoint::HEADER_BYTES,
|
||||
"reason": "the footer digest covers the manifest"},
|
||||
{"name": "a flipped payload byte", "offset": first_payload,
|
||||
"reason": "every payload carries its own digest"},
|
||||
{"name": "a flipped footer digest byte", "offset": bytes.len() - 40,
|
||||
"reason": "the footer digest must match the contents"},
|
||||
{"name": "a flipped footer magic byte", "offset": bytes.len() - 8,
|
||||
"reason": "a truncated file cannot look complete"},
|
||||
],
|
||||
}))
|
||||
}
|
||||
|
||||
fn payloads() -> Vec<(String, Vec<u8>)> {
|
||||
vec![
|
||||
("agent-fly-a".to_owned(), b"agent state bytes".to_vec()),
|
||||
("executor-fly-a".to_owned(), b"executor state".to_vec()),
|
||||
("task-ledger".to_owned(), b"{\"rank\":10}".to_vec()),
|
||||
("prior-inspection".to_owned(), b"{\"map\":40}".to_vec()),
|
||||
("world".to_owned(), vec![0u8; 64]),
|
||||
]
|
||||
}
|
||||
|
||||
fn payload_table() -> Value {
|
||||
let mut out = Vec::new();
|
||||
for (name, bytes) in payloads() {
|
||||
let mut entry = Map::new();
|
||||
entry.insert("name".to_owned(), Value::String(name));
|
||||
entry.insert(
|
||||
"byteLength".to_owned(),
|
||||
Value::String(bytes.len().to_string()),
|
||||
);
|
||||
entry.insert(
|
||||
"digest".to_owned(),
|
||||
Value::String(canonical::sha256_hex(&bytes)),
|
||||
);
|
||||
out.push(Value::Object(entry));
|
||||
}
|
||||
// A BTreeMap would sort the payload names; the table order is the write order, which is
|
||||
// what the envelope records.
|
||||
let _: BTreeMap<(), ()> = BTreeMap::new();
|
||||
Value::Array(out)
|
||||
}
|
||||
|
|
@ -0,0 +1,177 @@
|
|||
{
|
||||
"description": "The U64 decimal-string and double boundaries, shared by both languages.",
|
||||
"u64": [
|
||||
{
|
||||
"text": "0",
|
||||
"accept": true,
|
||||
"reason": "zero is \"0\""
|
||||
},
|
||||
{
|
||||
"text": "1",
|
||||
"accept": true,
|
||||
"reason": ""
|
||||
},
|
||||
{
|
||||
"text": "18446744073709551615",
|
||||
"accept": true,
|
||||
"reason": "the U64 maximum"
|
||||
},
|
||||
{
|
||||
"text": "18446744073709551616",
|
||||
"accept": false,
|
||||
"reason": "one past the maximum"
|
||||
},
|
||||
{
|
||||
"text": "184467440737095516150",
|
||||
"accept": false,
|
||||
"reason": "far past the maximum"
|
||||
},
|
||||
{
|
||||
"text": "00",
|
||||
"accept": false,
|
||||
"reason": "no leading zeros"
|
||||
},
|
||||
{
|
||||
"text": "01",
|
||||
"accept": false,
|
||||
"reason": "no leading zeros"
|
||||
},
|
||||
{
|
||||
"text": "",
|
||||
"accept": false,
|
||||
"reason": "empty"
|
||||
},
|
||||
{
|
||||
"text": "-1",
|
||||
"accept": false,
|
||||
"reason": "unsigned"
|
||||
},
|
||||
{
|
||||
"text": "+1",
|
||||
"accept": false,
|
||||
"reason": "no sign"
|
||||
},
|
||||
{
|
||||
"text": "1.0",
|
||||
"accept": false,
|
||||
"reason": "integers only"
|
||||
},
|
||||
{
|
||||
"text": "1e3",
|
||||
"accept": false,
|
||||
"reason": "decimal digits only"
|
||||
},
|
||||
{
|
||||
"text": " 1",
|
||||
"accept": false,
|
||||
"reason": "no whitespace"
|
||||
},
|
||||
{
|
||||
"text": "1 ",
|
||||
"accept": false,
|
||||
"reason": "no whitespace"
|
||||
},
|
||||
{
|
||||
"text": "0x10",
|
||||
"accept": false,
|
||||
"reason": "decimal only"
|
||||
},
|
||||
{
|
||||
"text": "9007199254740993",
|
||||
"accept": true,
|
||||
"reason": "a U64 string keeps precision a double would lose"
|
||||
}
|
||||
],
|
||||
"doubles": [
|
||||
{
|
||||
"value": 0.0,
|
||||
"canonical": "0",
|
||||
"accept": true,
|
||||
"reason": ""
|
||||
},
|
||||
{
|
||||
"value": -0.0,
|
||||
"canonical": "0",
|
||||
"accept": true,
|
||||
"reason": "JSON.stringify prints negative zero as 0"
|
||||
},
|
||||
{
|
||||
"value": 1.0,
|
||||
"canonical": "1",
|
||||
"accept": true,
|
||||
"reason": "an integral double prints without a fraction"
|
||||
},
|
||||
{
|
||||
"value": -17.0,
|
||||
"canonical": "-17",
|
||||
"accept": true,
|
||||
"reason": ""
|
||||
},
|
||||
{
|
||||
"value": 0.1,
|
||||
"canonical": "0.1",
|
||||
"accept": true,
|
||||
"reason": "the shortest round-tripping form"
|
||||
},
|
||||
{
|
||||
"value": 0.30000000000000004,
|
||||
"canonical": "0.30000000000000004",
|
||||
"accept": true,
|
||||
"reason": "shortest round-tripping form, not a rounded one"
|
||||
},
|
||||
{
|
||||
"value": 1e-07,
|
||||
"canonical": "1e-7",
|
||||
"accept": true,
|
||||
"reason": "ECMAScript switches to exponent notation below 1e-6"
|
||||
},
|
||||
{
|
||||
"value": 5e-324,
|
||||
"canonical": "5e-324",
|
||||
"accept": true,
|
||||
"reason": "the smallest subnormal double"
|
||||
},
|
||||
{
|
||||
"value": 1234.5678,
|
||||
"canonical": "1234.5678",
|
||||
"accept": true,
|
||||
"reason": "a fractional value of any magnitude is canonicalizable"
|
||||
},
|
||||
{
|
||||
"value": 9007199254740991,
|
||||
"canonical": "9007199254740991",
|
||||
"accept": true,
|
||||
"reason": "the largest exactly representable integer"
|
||||
},
|
||||
{
|
||||
"value": -9007199254740991,
|
||||
"canonical": "-9007199254740991",
|
||||
"accept": true,
|
||||
"reason": ""
|
||||
},
|
||||
{
|
||||
"value": 9007199254740993,
|
||||
"canonical": null,
|
||||
"accept": false,
|
||||
"reason": "an integral value past the exact range; counters are U64 strings"
|
||||
},
|
||||
{
|
||||
"value": 1e+21,
|
||||
"canonical": null,
|
||||
"accept": false,
|
||||
"reason": "integral and far past the exact range; JSON.parse cannot tell it from the same digits written out"
|
||||
},
|
||||
{
|
||||
"value": 1.7976931348623157e+308,
|
||||
"canonical": null,
|
||||
"accept": false,
|
||||
"reason": "integral and far past the exact range"
|
||||
},
|
||||
{
|
||||
"value": 1.5e+20,
|
||||
"canonical": null,
|
||||
"accept": false,
|
||||
"reason": "1.5e20 is integral as a double, so it falls under the same rule as 1e21"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,195 @@
|
|||
{
|
||||
"description": "One FLYSESS1 envelope, its layout and the corruptions a reader must refuse.",
|
||||
"magic": "FLYSESS1",
|
||||
"footerMagic": "FLYSESSF",
|
||||
"version": 1,
|
||||
"manifest": {
|
||||
"envelopeVersion": 1,
|
||||
"checkpointId": "ckpt-1",
|
||||
"sourceScope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "42"
|
||||
},
|
||||
"episodeId": "episode-1",
|
||||
"worldTime": {
|
||||
"numerator": "700000000",
|
||||
"denominator": "1"
|
||||
},
|
||||
"schedulerId": "lockstep-v1",
|
||||
"compositionDigest": "730d725c8a59d3a7303def2bed041a577edb4255aabd4889ce12918311d952f0",
|
||||
"portMap": [
|
||||
{
|
||||
"portId": "port-1",
|
||||
"agentId": "fly-a"
|
||||
}
|
||||
],
|
||||
"compatibility": {
|
||||
"backendDigest": "10e08a419e850eba1ebba18fdd28eb7ec1b7e8baa9bcc3b973e2b8891ec726be",
|
||||
"contentDigest": "ed7002b439e9ac845f22357d822bac1444730fbdb6016d3ec9432297b9ec9f73",
|
||||
"patchDigest": "a4895eb44afc336fecbba6e520cd67e178dace0276655d102fceffa8e5f70570",
|
||||
"controllerDigest": "c1472135b14c77c8bef98e73f70208325fa0dcf1e6bd668ae9b31a9cea295fe7",
|
||||
"parserDigest": "b17d45121150928f2146af49e195eff1eef5d67325be273a733fb74acadaa342",
|
||||
"stateFormatId": "flysess-1"
|
||||
},
|
||||
"agents": [
|
||||
{
|
||||
"agentId": "fly-a",
|
||||
"profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0",
|
||||
"datasetDigest": "6c0af1f0784ef63a393ee77d614e8246c625051360f3f1a48838374c5d355b52",
|
||||
"modelVersion": "lif-1ms-f64-v2",
|
||||
"plasticityVersion": "fly-kc-mbon-rstdp-v2",
|
||||
"seed": -184946063,
|
||||
"brainTicks": "2534",
|
||||
"remainder": {
|
||||
"numerator": "1000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"payload": "agent-fly-a"
|
||||
}
|
||||
],
|
||||
"coordinator": {
|
||||
"taskLedger": "task-ledger",
|
||||
"priorInspection": "prior-inspection",
|
||||
"executorState": [
|
||||
{
|
||||
"agentId": "fly-a",
|
||||
"payload": "executor-fly-a"
|
||||
}
|
||||
],
|
||||
"admissionState": null,
|
||||
"eventWatermarks": {
|
||||
"lastEventId": "evt-1",
|
||||
"lastOrdinal": "7"
|
||||
}
|
||||
},
|
||||
"helperState": [],
|
||||
"payloads": [
|
||||
{
|
||||
"name": "agent-fly-a",
|
||||
"byteLength": "17",
|
||||
"digest": "1321dffb0cdc6f9092cbf7fa2a5fc68bbed12c993d5ad398264012810ce9bf93"
|
||||
},
|
||||
{
|
||||
"name": "executor-fly-a",
|
||||
"byteLength": "14",
|
||||
"digest": "3aee60df7e29efeba7f5f99fc5867647b36aebff1d5d3c838dbff323122e6462"
|
||||
},
|
||||
{
|
||||
"name": "task-ledger",
|
||||
"byteLength": "11",
|
||||
"digest": "40b00ed2bbba901d68205ff71b04a44b9ee53c51cb3109aa2ceaa44f1c45727e"
|
||||
},
|
||||
{
|
||||
"name": "prior-inspection",
|
||||
"byteLength": "10",
|
||||
"digest": "2c13b7b4d9a9916801ab9191c314f31b045e9b9c5b669a6c0474f0217ef75bf5"
|
||||
},
|
||||
{
|
||||
"name": "world",
|
||||
"byteLength": "64",
|
||||
"digest": "f5a5fd42d16a20302798ef6ed309979b43003d2320d9f0e8ea9831a92759fb4b"
|
||||
}
|
||||
]
|
||||
},
|
||||
"payloads": [
|
||||
{
|
||||
"name": "agent-fly-a",
|
||||
"base64": "YWdlbnQgc3RhdGUgYnl0ZXM="
|
||||
},
|
||||
{
|
||||
"name": "executor-fly-a",
|
||||
"base64": "ZXhlY3V0b3Igc3RhdGU="
|
||||
},
|
||||
{
|
||||
"name": "task-ledger",
|
||||
"base64": "eyJyYW5rIjoxMH0="
|
||||
},
|
||||
{
|
||||
"name": "prior-inspection",
|
||||
"base64": "eyJtYXAiOjQwfQ=="
|
||||
},
|
||||
{
|
||||
"name": "world",
|
||||
"base64": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="
|
||||
}
|
||||
],
|
||||
"envelope": {
|
||||
"base64": "RkxZU0VTUzEBAAAAIAAAAAAIAAAFAAAAIAgAAAAAAAB7ImFnZW50cyI6W3siYWdlbnRJZCI6ImZseS1hIiwiYnJhaW5UaWNrcyI6IjI1MzQiLCJkYXRhc2V0RGlnZXN0IjoiNmMwYWYxZjA3ODRlZjYzYTM5M2VlNzdkNjE0ZTgyNDZjNjI1MDUxMzYwZjNmMWE0ODgzODM3NGM1ZDM1NWI1MiIsIm1vZGVsVmVyc2lvbiI6ImxpZi0xbXMtZjY0LXYyIiwicGF5bG9hZCI6ImFnZW50LWZseS1hIiwicGxhc3RpY2l0eVZlcnNpb24iOiJmbHkta2MtbWJvbi1yc3RkcC12MiIsInByb2ZpbGVEaWdlc3QiOiIxOTAwZWFiNmMwMjg0ODNkNzEyNjU5OWVlNmY1MGRlMGQyNzkwN2I1YzY1ZmE5MDUyNDU4MGI0YjBmOTg1MmIwIiwicmVtYWluZGVyIjp7ImRlbm9taW5hdG9yIjoiMyIsIm51bWVyYXRvciI6IjEwMDAwMDAifSwic2VlZCI6LTE4NDk0NjA2M31dLCJjaGVja3BvaW50SWQiOiJja3B0LTEiLCJjb21wYXRpYmlsaXR5Ijp7ImJhY2tlbmREaWdlc3QiOiIxMGUwOGE0MTllODUwZWJhMWViYmExOGZkZDI4ZWI3ZWMxYjdlOGJhYTliY2MzYjk3M2UyYjg4OTFlYzcyNmJlIiwiY29udGVudERpZ2VzdCI6ImVkNzAwMmI0MzllOWFjODQ1ZjIyMzU3ZDgyMmJhYzE0NDQ3MzBmYmRiNjAxNmQzZWM5NDMyMjk3YjllYzlmNzMiLCJjb250cm9sbGVyRGlnZXN0IjoiYzE0NzIxMzViMTRjNzdjOGJlZjk4ZTczZjcwMjA4MzI1ZmEwZGNmMWU2YmQ2NjhhZTliMzFhOWNlYTI5NWZlNyIsInBhcnNlckRpZ2VzdCI6ImIxN2Q0NTEyMTE1MDkyOGYyMTQ2YWY0OWUxOTVlZmYxZWVmNWQ2NzMyNWJlMjczYTczM2ZiNzRhY2FkYWEzNDIiLCJwYXRjaERpZ2VzdCI6ImE0ODk1ZWI0NGFmYzMzNmZlY2JiYTZlNTIwY2Q2N2UxNzhkYWNlMDI3NjY1NWQxMDJmY2VmZmE4ZTVmNzA1NzAiLCJzdGF0ZUZvcm1hdElkIjoiZmx5c2Vzcy0xIn0sImNvbXBvc2l0aW9uRGlnZXN0IjoiNzMwZDcyNWM4YTU5ZDNhNzMwM2RlZjJiZWQwNDFhNTc3ZWRiNDI1NWFhYmQ0ODg5Y2UxMjkxODMxMWQ5NTJmMCIsImNvb3JkaW5hdG9yIjp7ImFkbWlzc2lvblN0YXRlIjpudWxsLCJldmVudFdhdGVybWFya3MiOnsibGFzdEV2ZW50SWQiOiJldnQtMSIsImxhc3RPcmRpbmFsIjoiNyJ9LCJleGVjdXRvclN0YXRlIjpbeyJhZ2VudElkIjoiZmx5LWEiLCJwYXlsb2FkIjoiZXhlY3V0b3ItZmx5LWEifV0sInByaW9ySW5zcGVjdGlvbiI6InByaW9yLWluc3BlY3Rpb24iLCJ0YXNrTGVkZ2VyIjoidGFzay1sZWRnZXIifSwiZW52ZWxvcGVWZXJzaW9uIjoxLCJlcGlzb2RlSWQiOiJlcGlzb2RlLTEiLCJoZWxwZXJTdGF0ZSI6W10sInBheWxvYWRzIjpbeyJieXRlTGVuZ3RoIjoiMTciLCJkaWdlc3QiOiIxMzIxZGZmYjBjZGM2ZjkwOTJjYmY3ZmEyYTVmYzY4YmJlZDEyYzk5M2Q1YWQzOTgyNjQwMTI4MTBjZTliZjkzIiwibmFtZSI6ImFnZW50LWZseS1hIn0seyJieXRlTGVuZ3RoIjoiMTQiLCJkaWdlc3QiOiIzYWVlNjBkZjdlMjllZmViYTdmNWY5OWZjNTg2NzY0N2IzNmFlYmZmMWQ1ZDNjODM4ZGJmZjMyMzEyMmU2NDYyIiwibmFtZSI6ImV4ZWN1dG9yLWZseS1hIn0seyJieXRlTGVuZ3RoIjoiMTEiLCJkaWdlc3QiOiI0MGIwMGVkMmJiYmE5MDFkNjgyMDVmZjcxYjA0YTQ0YjllZTUzYzUxY2IzMTA5YWEyY2VhYTQ0ZjFjNDU3MjdlIiwibmFtZSI6InRhc2stbGVkZ2VyIn0seyJieXRlTGVuZ3RoIjoiMTAiLCJkaWdlc3QiOiIyYzEzYjdiNGQ5YTk5MTY4MDFhYjkxOTFjMzE0ZjMxYjA0NWU5YjljNWI2NjlhNmMwNDc0ZjAyMTdlZjc1YmY1IiwibmFtZSI6InByaW9yLWluc3BlY3Rpb24ifSx7ImJ5dGVMZW5ndGgiOiI2NCIsImRpZ2VzdCI6ImY1YTVmZDQyZDE2YTIwMzAyNzk4ZWY2ZWQzMDk5NzliNDMwMDNkMjMyMGQ5ZjBlOGVhOTgzMWE5Mjc1OWZiNGIiLCJuYW1lIjoid29ybGQifV0sInBvcnRNYXAiOlt7ImFnZW50SWQiOiJmbHktYSIsInBvcnRJZCI6InBvcnQtMSJ9XSwic2NoZWR1bGVySWQiOiJsb2Nrc3RlcC12MSIsInNvdXJjZVNjb3BlIjp7ImVwb2NoIjoiZXBvY2gtMSIsInNlc3Npb25JZCI6ImRlbW8iLCJzdGVwIjoiNDIifSwid29ybGRUaW1lIjp7ImRlbm9taW5hdG9yIjoiMSIsIm51bWVyYXRvciI6IjcwMDAwMDAwMCJ9fWFnZW50LWZseS1hAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQCgAAAAAAABEAAAAAAAAAEyHf+wzcb5CSy/f6Kl/Gi77RLJk9WtOYJkASgQzpv5NleGVjdXRvci1mbHktYQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAaAoAAAAAAAAOAAAAAAAAADruYN9+Ke/rp/X5n8WGdkezauv/HV08g42/8yMSLmRidGFzay1sZWRnZXIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHgKAAAAAAAACwAAAAAAAABAsA7Su7qQHWggX/cbBKRLnuU8UcsxCaos6qRPHEVyfnByaW9yLWluc3BlY3Rpb24AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACICgAAAAAAAAoAAAAAAAAALBO3tNmpkWgBq5GRwxTzGwRem5xbZppsBHTwIX73W/V3b3JsZAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAmAoAAAAAAABAAAAAAAAAAPWl/ULRaiAwJ5jvbtMJl5tDAD0jINnw6OqYMaknWftLYWdlbnQgc3RhdGUgYnl0ZXMAAAAAAAAAZXhlY3V0b3Igc3RhdGUAAHsicmFuayI6MTB9AAAAAAB7Im1hcCI6NDB9AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgLAAAAAAAAq++fEx+FvDZho/eB4imbENN4HZrGNC2OCAsI7/gp9r5GTFlTRVNTRg==",
|
||||
"byteLength": 2824,
|
||||
"layout": {
|
||||
"headerBytes": 32,
|
||||
"manifestOffset": "32",
|
||||
"manifestBytes": 2048,
|
||||
"tableOffset": "2080",
|
||||
"tableEntryBytes": 112,
|
||||
"entries": [
|
||||
{
|
||||
"name": "agent-fly-a",
|
||||
"offset": "2640",
|
||||
"byteLength": "17",
|
||||
"digest": "1321dffb0cdc6f9092cbf7fa2a5fc68bbed12c993d5ad398264012810ce9bf93"
|
||||
},
|
||||
{
|
||||
"name": "executor-fly-a",
|
||||
"offset": "2664",
|
||||
"byteLength": "14",
|
||||
"digest": "3aee60df7e29efeba7f5f99fc5867647b36aebff1d5d3c838dbff323122e6462"
|
||||
},
|
||||
{
|
||||
"name": "task-ledger",
|
||||
"offset": "2680",
|
||||
"byteLength": "11",
|
||||
"digest": "40b00ed2bbba901d68205ff71b04a44b9ee53c51cb3109aa2ceaa44f1c45727e"
|
||||
},
|
||||
{
|
||||
"name": "prior-inspection",
|
||||
"offset": "2696",
|
||||
"byteLength": "10",
|
||||
"digest": "2c13b7b4d9a9916801ab9191c314f31b045e9b9c5b669a6c0474f0217ef75bf5"
|
||||
},
|
||||
{
|
||||
"name": "world",
|
||||
"offset": "2712",
|
||||
"byteLength": "64",
|
||||
"digest": "f5a5fd42d16a20302798ef6ed309979b43003d2320d9f0e8ea9831a92759fb4b"
|
||||
}
|
||||
],
|
||||
"footerOffset": "2776",
|
||||
"footerBytes": 48,
|
||||
"totalBytes": "2824"
|
||||
}
|
||||
},
|
||||
"corruption": [
|
||||
{
|
||||
"name": "a flipped magic byte",
|
||||
"offset": 0,
|
||||
"reason": "wrong magic"
|
||||
},
|
||||
{
|
||||
"name": "an unsupported version",
|
||||
"offset": 8,
|
||||
"reason": "unsupported version"
|
||||
},
|
||||
{
|
||||
"name": "a flipped manifest byte",
|
||||
"offset": 32,
|
||||
"reason": "the footer digest covers the manifest"
|
||||
},
|
||||
{
|
||||
"name": "a flipped payload byte",
|
||||
"offset": 2640,
|
||||
"reason": "every payload carries its own digest"
|
||||
},
|
||||
{
|
||||
"name": "a flipped footer digest byte",
|
||||
"offset": 2784,
|
||||
"reason": "the footer digest must match the contents"
|
||||
},
|
||||
{
|
||||
"name": "a flipped footer magic byte",
|
||||
"offset": 2816,
|
||||
"reason": "a truncated file cannot look complete"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
{
|
||||
"description": "contractDigest is the SHA-256 of the canonical schema set in schema-set.json.",
|
||||
"contractDigest": "7932aef30c4d2d16e428081affc4e0ad187987f5b361138d553e54fd7f843b50",
|
||||
"schemaSetVersion": 1,
|
||||
"schemaSetBytes": 26685,
|
||||
"types": 53,
|
||||
"enums": 11,
|
||||
"limits": 25
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,74 @@
|
|||
{
|
||||
"description": "Boundary cases both languages build from a recipe, because the payload is too large to store.",
|
||||
"padSchema": {
|
||||
"id": "pad.v1",
|
||||
"version": 1,
|
||||
"digest": "018689202f154300eeda48ccee8cc021c36672df03f7404097613f5898fdfdcc"
|
||||
},
|
||||
"cases": [
|
||||
{
|
||||
"name": "typed value exactly at the 32 KiB cap",
|
||||
"kind": "padded-typed-value",
|
||||
"padCharacters": 32635,
|
||||
"expect": "accept",
|
||||
"reason": "32768 bytes of canonical JSON is the limit, not one byte less"
|
||||
},
|
||||
{
|
||||
"name": "typed value one byte over the cap",
|
||||
"kind": "padded-typed-value",
|
||||
"padCharacters": 32636,
|
||||
"expect": "reject",
|
||||
"reason": "a TypedValue is at most 32 KiB of canonical JSON"
|
||||
},
|
||||
{
|
||||
"name": "request that fills the 64 KiB envelope exactly",
|
||||
"kind": "padded-request",
|
||||
"padCharacters": 60000,
|
||||
"expect": "accept",
|
||||
"reason": "65536 bytes including the envelope wrapper is admissible",
|
||||
"envelopeTotal": 65536
|
||||
},
|
||||
{
|
||||
"name": "request one byte past the envelope ceiling",
|
||||
"kind": "padded-request",
|
||||
"padCharacters": 60000,
|
||||
"expect": "reject",
|
||||
"reason": "the complete envelope must fit Flybus's 64-KiB maximum",
|
||||
"envelopeTotal": 65537
|
||||
},
|
||||
{
|
||||
"name": "error message of 512 code points",
|
||||
"kind": "error-message",
|
||||
"codePoints": 512,
|
||||
"expect": "accept",
|
||||
"reason": "messages are <= 512 code points"
|
||||
},
|
||||
{
|
||||
"name": "error message of 513 code points",
|
||||
"kind": "error-message",
|
||||
"codePoints": 513,
|
||||
"expect": "reject",
|
||||
"reason": "messages are <= 512 code points"
|
||||
},
|
||||
{
|
||||
"name": "error message of 512 astral code points",
|
||||
"kind": "error-message-astral",
|
||||
"codePoints": 512,
|
||||
"expect": "accept",
|
||||
"reason": "the bound counts code points, not UTF-16 units or bytes"
|
||||
},
|
||||
{
|
||||
"name": "error message of 513 astral code points",
|
||||
"kind": "error-message-astral",
|
||||
"codePoints": 513,
|
||||
"expect": "reject",
|
||||
"reason": "the bound counts code points, not UTF-16 units or bytes"
|
||||
}
|
||||
],
|
||||
"recipes": {
|
||||
"padded-typed-value": "a TypedValue whose schema is padSchema and whose value is {\"pad\": <padCharacters> 'a' characters}; validate it",
|
||||
"padded-request": "a SessionRpcRequest req-1 with scope null and params {\"pad\": <padCharacters> 'a' characters}; canonicalize it, then require the envelope to fit with an overhead of envelopeTotal minus that canonical length",
|
||||
"error-message": "a SessionRpcFailure with code INTERNAL, mutation unknown and a message of <codePoints> 'x' characters",
|
||||
"error-message-astral": "the same failure with a message of <codePoints> repetitions of U+10400, one code point and two UTF-16 units each"
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,111 @@
|
|||
{
|
||||
"description": "Bus callId, domain requestId, artifact identity and delivery/hold owner tokens are four types, not four spellings of one string.",
|
||||
"cases": [
|
||||
{
|
||||
"text": "call-0",
|
||||
"busCallId": true,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "call-102",
|
||||
"busCallId": true,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "req-41",
|
||||
"busCallId": false,
|
||||
"domainRequestId": true,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "dlv-7",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": "delivery"
|
||||
},
|
||||
{
|
||||
"text": "own-9",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": "hold"
|
||||
},
|
||||
{
|
||||
"text": "req-041",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "call-",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "call",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "callid-1",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "request-1",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "REQ-1",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "dlv-07",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "sub-1",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
},
|
||||
{
|
||||
"text": "svc-1",
|
||||
"busCallId": false,
|
||||
"domainRequestId": false,
|
||||
"ownerToken": null
|
||||
}
|
||||
],
|
||||
"artifact": {
|
||||
"ref": {
|
||||
"storeId": "store-1",
|
||||
"artifactId": "frame-1",
|
||||
"generation": "1",
|
||||
"byteLength": "92160",
|
||||
"contentType": "image/x-rgba8",
|
||||
"digest": null
|
||||
},
|
||||
"identity": {
|
||||
"storeId": "store-1",
|
||||
"artifactId": "frame-1",
|
||||
"generation": "1"
|
||||
},
|
||||
"reason": "the identity is the naming half of an ArtifactRef; byteLength, contentType and digest are not identity, and an AssetRef is not an artifact at all"
|
||||
},
|
||||
"asset": {
|
||||
"id": "profile-fly-a",
|
||||
"digest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0",
|
||||
"byteLength": "4096",
|
||||
"format": "flyprofile"
|
||||
}
|
||||
}
|
||||
3900
services/flysim/crates/fly-session-types/fixtures/invalid.json
Normal file
3900
services/flysim/crates/fly-session-types/fixtures/invalid.json
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,667 @@
|
|||
{
|
||||
"description": "ipc-v1 section 5: the operation key of a step mutation and the canonical body a duplicate is compared against.",
|
||||
"keys": [
|
||||
{
|
||||
"name": "Agent.Prepare on fly-a at step 41",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"method": "Agent.Prepare",
|
||||
"workerId": "fly-a",
|
||||
"digest": "eda73d62bb5d616052997ab5e9d2c4ba5872fdefb44cf706910a18a24d39225f"
|
||||
},
|
||||
{
|
||||
"name": "Agent.Prepare on fly-b at step 41",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"method": "Agent.Prepare",
|
||||
"workerId": "fly-b",
|
||||
"digest": "5a7fd15da236f3fd6343698a8cc5c4f14fa7d84683a819d5a28904d90f700c4e"
|
||||
},
|
||||
{
|
||||
"name": "Agent.Commit on fly-a at step 41",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"method": "Agent.Commit",
|
||||
"workerId": "fly-a",
|
||||
"digest": "2a89a120e5594895d024b4ffc2a7d225bd27322cfd345ebd78e80ed7b271c680"
|
||||
},
|
||||
{
|
||||
"name": "Agent.Prepare on fly-a at step 42",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "42"
|
||||
},
|
||||
"method": "Agent.Prepare",
|
||||
"workerId": "fly-a",
|
||||
"digest": "5bb184cab6c3c2d7567a7979d860a7e9b26ea52bdc92b139053b0a2f81a9ed4d"
|
||||
},
|
||||
{
|
||||
"name": "Agent.Prepare on fly-a in another epoch",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-2",
|
||||
"step": "41"
|
||||
},
|
||||
"method": "Agent.Prepare",
|
||||
"workerId": "fly-a",
|
||||
"digest": "fa1d9cf3936aaffe3cedd0e586d7763efca2d250e89dd85b849cf35fbd91a477"
|
||||
},
|
||||
{
|
||||
"name": "Environment.Advance at step 41",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"method": "Environment.Advance",
|
||||
"workerId": "world",
|
||||
"digest": "c89c390aa78ef2895d9e2b86a6000160923dc54e3be75529e0289cd18b9088b1"
|
||||
}
|
||||
],
|
||||
"bodies": [
|
||||
{
|
||||
"name": "Agent.Prepare body",
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"agentId": "fly-a",
|
||||
"profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0",
|
||||
"interval": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4",
|
||||
"preStepStimulations": [
|
||||
{
|
||||
"id": "stim-1",
|
||||
"kindId": "sugar",
|
||||
"durationMs": 50.0
|
||||
}
|
||||
]
|
||||
},
|
||||
"digest": "74c9d5f27b2cb06922cc3ca13d87c67faea0a6c73aefbbdb21ba8f52c6d33f60"
|
||||
},
|
||||
{
|
||||
"name": "Environment.Advance body",
|
||||
"method": "Environment.Advance",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"batchId": "batch-41",
|
||||
"controls": [
|
||||
{
|
||||
"portId": "port-1",
|
||||
"buttons": [
|
||||
{
|
||||
"id": "a",
|
||||
"down": true
|
||||
},
|
||||
{
|
||||
"id": "b",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "start",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "select",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "up",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "down",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "left",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "right",
|
||||
"down": false
|
||||
}
|
||||
],
|
||||
"axes": [
|
||||
{
|
||||
"id": "stick-x",
|
||||
"value": 0.0
|
||||
},
|
||||
{
|
||||
"id": "trigger",
|
||||
"value": 0.0
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"digest": "5aaae9c083336460a2ce34fb50e191f7108444a953f09b952e97a48b7962ee79"
|
||||
},
|
||||
{
|
||||
"name": "Worker.Hello body with no scope",
|
||||
"method": "Worker.Hello",
|
||||
"scope": null,
|
||||
"params": {
|
||||
"sessionId": "demo",
|
||||
"expectedWorkerId": "fly-a",
|
||||
"role": "agent",
|
||||
"supportedMajors": [
|
||||
1
|
||||
]
|
||||
},
|
||||
"digest": "0c07677a45178d8680dddbeec56d4e09f6ccbfe5fe77d9d4ff2fb07b369febaf"
|
||||
}
|
||||
],
|
||||
"pairs": [
|
||||
{
|
||||
"name": "the same operation retried on a new bus call",
|
||||
"left": {
|
||||
"method": "Environment.Advance",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"batchId": "batch-41",
|
||||
"controls": [
|
||||
{
|
||||
"portId": "port-1",
|
||||
"buttons": [
|
||||
{
|
||||
"id": "a",
|
||||
"down": true
|
||||
},
|
||||
{
|
||||
"id": "b",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "start",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "select",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "up",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "down",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "left",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "right",
|
||||
"down": false
|
||||
}
|
||||
],
|
||||
"axes": [
|
||||
{
|
||||
"id": "stick-x",
|
||||
"value": 0.0
|
||||
},
|
||||
{
|
||||
"id": "trigger",
|
||||
"value": 0.0
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"right": {
|
||||
"method": "Environment.Advance",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"batchId": "batch-41",
|
||||
"controls": [
|
||||
{
|
||||
"portId": "port-1",
|
||||
"buttons": [
|
||||
{
|
||||
"id": "a",
|
||||
"down": true
|
||||
},
|
||||
{
|
||||
"id": "b",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "start",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "select",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "up",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "down",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "left",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "right",
|
||||
"down": false
|
||||
}
|
||||
],
|
||||
"axes": [
|
||||
{
|
||||
"id": "stick-x",
|
||||
"value": 0.0
|
||||
},
|
||||
{
|
||||
"id": "trigger",
|
||||
"value": 0.0
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"workerId": "world",
|
||||
"sameKey": true,
|
||||
"sameBody": true,
|
||||
"reason": "a retry reuses the requestId and body; only the callId changes, and the callId is not in either digest"
|
||||
},
|
||||
{
|
||||
"name": "the same batch id with altered controls",
|
||||
"left": {
|
||||
"method": "Environment.Advance",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"batchId": "batch-41",
|
||||
"controls": [
|
||||
{
|
||||
"portId": "port-1",
|
||||
"buttons": [
|
||||
{
|
||||
"id": "a",
|
||||
"down": true
|
||||
},
|
||||
{
|
||||
"id": "b",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "start",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "select",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "up",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "down",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "left",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "right",
|
||||
"down": false
|
||||
}
|
||||
],
|
||||
"axes": [
|
||||
{
|
||||
"id": "stick-x",
|
||||
"value": 0.0
|
||||
},
|
||||
{
|
||||
"id": "trigger",
|
||||
"value": 0.0
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"right": {
|
||||
"method": "Environment.Advance",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"batchId": "batch-41",
|
||||
"controls": [
|
||||
{
|
||||
"portId": "port-1",
|
||||
"buttons": [
|
||||
{
|
||||
"id": "a",
|
||||
"down": true
|
||||
},
|
||||
{
|
||||
"id": "b",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "start",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "select",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "up",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "down",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "left",
|
||||
"down": false
|
||||
},
|
||||
{
|
||||
"id": "right",
|
||||
"down": false
|
||||
}
|
||||
],
|
||||
"axes": [
|
||||
{
|
||||
"id": "stick-x",
|
||||
"value": 1.0
|
||||
},
|
||||
{
|
||||
"id": "trigger",
|
||||
"value": 0.0
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"workerId": "world",
|
||||
"sameKey": true,
|
||||
"sameBody": false,
|
||||
"reason": "same key, changed body: CONFLICT, never a second world mutation"
|
||||
},
|
||||
{
|
||||
"name": "params written with their keys in another order",
|
||||
"left": {
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"agentId": "fly-a",
|
||||
"profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0",
|
||||
"interval": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4",
|
||||
"preStepStimulations": [
|
||||
{
|
||||
"id": "stim-1",
|
||||
"kindId": "sugar",
|
||||
"durationMs": 50.0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"right": {
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"preStepStimulations": [
|
||||
{
|
||||
"id": "stim-1",
|
||||
"kindId": "sugar",
|
||||
"durationMs": 50.0
|
||||
}
|
||||
],
|
||||
"decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4",
|
||||
"interval": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0",
|
||||
"agentId": "fly-a"
|
||||
}
|
||||
},
|
||||
"workerId": "fly-a",
|
||||
"sameKey": true,
|
||||
"sameBody": true,
|
||||
"reason": "RFC 8785 sorts keys, so serialization order is not a body change"
|
||||
},
|
||||
{
|
||||
"name": "the same body one step later",
|
||||
"left": {
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"agentId": "fly-a",
|
||||
"profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0",
|
||||
"interval": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4",
|
||||
"preStepStimulations": [
|
||||
{
|
||||
"id": "stim-1",
|
||||
"kindId": "sugar",
|
||||
"durationMs": 50.0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"right": {
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "42"
|
||||
},
|
||||
"params": {
|
||||
"agentId": "fly-a",
|
||||
"profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0",
|
||||
"interval": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4",
|
||||
"preStepStimulations": [
|
||||
{
|
||||
"id": "stim-1",
|
||||
"kindId": "sugar",
|
||||
"durationMs": 50.0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"workerId": "fly-a",
|
||||
"sameKey": false,
|
||||
"sameBody": false,
|
||||
"reason": "the step is part of both the key and the body"
|
||||
},
|
||||
{
|
||||
"name": "the same body on another worker",
|
||||
"left": {
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"agentId": "fly-a",
|
||||
"profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0",
|
||||
"interval": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4",
|
||||
"preStepStimulations": [
|
||||
{
|
||||
"id": "stim-1",
|
||||
"kindId": "sugar",
|
||||
"durationMs": 50.0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"right": {
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"agentId": "fly-a",
|
||||
"profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0",
|
||||
"interval": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4",
|
||||
"preStepStimulations": [
|
||||
{
|
||||
"id": "stim-1",
|
||||
"kindId": "sugar",
|
||||
"durationMs": 50.0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"workerId": "fly-a",
|
||||
"rightWorkerId": "fly-b",
|
||||
"sameKey": false,
|
||||
"sameBody": true,
|
||||
"reason": "the worker is part of the key, not of the body"
|
||||
}
|
||||
],
|
||||
"rejected": [
|
||||
{
|
||||
"name": "a body carrying a bus callId",
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"callId": "call-2",
|
||||
"agentId": "fly-a"
|
||||
},
|
||||
"reason": "the canonical body excludes bus callIds"
|
||||
},
|
||||
{
|
||||
"name": "a body carrying a delivery id",
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"deliveryId": "dlv-7"
|
||||
},
|
||||
"reason": "the canonical body excludes deliveryIds"
|
||||
},
|
||||
{
|
||||
"name": "a body carrying an owner token",
|
||||
"method": "Agent.Commit",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"nextInput": {
|
||||
"frame": {
|
||||
"ownerId": "own-3"
|
||||
}
|
||||
}
|
||||
},
|
||||
"reason": "the canonical body excludes owner tokens"
|
||||
},
|
||||
{
|
||||
"name": "a body carrying a pinned service incarnation",
|
||||
"method": "Agent.Prepare",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {
|
||||
"expectedIncarnation": "svc-1"
|
||||
},
|
||||
"reason": "route pinning is transport state, not domain state"
|
||||
},
|
||||
{
|
||||
"name": "an empty method",
|
||||
"method": "",
|
||||
"scope": {
|
||||
"sessionId": "demo",
|
||||
"epoch": "epoch-1",
|
||||
"step": "41"
|
||||
},
|
||||
"params": {},
|
||||
"reason": "methods are 1..=128 printable ASCII characters"
|
||||
}
|
||||
]
|
||||
}
|
||||
273
services/flysim/crates/fly-session-types/fixtures/rational.json
Normal file
273
services/flysim/crates/fly-session-types/fixtures/rational.json
Normal file
|
|
@ -0,0 +1,273 @@
|
|||
{
|
||||
"description": "Checked rational arithmetic and the step-v1 section 5 tick accumulator.",
|
||||
"accumulator": [
|
||||
{
|
||||
"name": "a synthetic 60 Hz environment on a 1 ms model tick",
|
||||
"stepDuration": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"tickDuration": {
|
||||
"numerator": "1000000",
|
||||
"denominator": "1"
|
||||
},
|
||||
"steps": [
|
||||
{
|
||||
"ticks": "16",
|
||||
"remainder": {
|
||||
"numerator": "2000000",
|
||||
"denominator": "3"
|
||||
}
|
||||
},
|
||||
{
|
||||
"ticks": "17",
|
||||
"remainder": {
|
||||
"numerator": "1000000",
|
||||
"denominator": "3"
|
||||
}
|
||||
},
|
||||
{
|
||||
"ticks": "17",
|
||||
"remainder": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
}
|
||||
}
|
||||
],
|
||||
"totalTicks": "50",
|
||||
"reason": "16, 17, 17 and a remainder of zero after three steps"
|
||||
},
|
||||
{
|
||||
"name": "a whole millisecond cadence never accumulates a remainder",
|
||||
"stepDuration": {
|
||||
"numerator": "16000000",
|
||||
"denominator": "1"
|
||||
},
|
||||
"tickDuration": {
|
||||
"numerator": "1000000",
|
||||
"denominator": "1"
|
||||
},
|
||||
"steps": [
|
||||
{
|
||||
"ticks": "16",
|
||||
"remainder": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
}
|
||||
},
|
||||
{
|
||||
"ticks": "16",
|
||||
"remainder": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
}
|
||||
}
|
||||
],
|
||||
"totalTicks": "32",
|
||||
"reason": ""
|
||||
},
|
||||
{
|
||||
"name": "a step shorter than one tick advances nothing and keeps the remainder",
|
||||
"stepDuration": {
|
||||
"numerator": "500000",
|
||||
"denominator": "1"
|
||||
},
|
||||
"tickDuration": {
|
||||
"numerator": "1000000",
|
||||
"denominator": "1"
|
||||
},
|
||||
"steps": [
|
||||
{
|
||||
"ticks": "0",
|
||||
"remainder": {
|
||||
"numerator": "500000",
|
||||
"denominator": "1"
|
||||
}
|
||||
},
|
||||
{
|
||||
"ticks": "1",
|
||||
"remainder": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
}
|
||||
}
|
||||
],
|
||||
"totalTicks": "1",
|
||||
"reason": "the fractional period is carried, not rounded"
|
||||
}
|
||||
],
|
||||
"add": [
|
||||
{
|
||||
"a": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"sum": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
}
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "1",
|
||||
"denominator": "3"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "1",
|
||||
"denominator": "6"
|
||||
},
|
||||
"sum": {
|
||||
"numerator": "1",
|
||||
"denominator": "2"
|
||||
}
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "18446744073709551615",
|
||||
"denominator": "1"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "1",
|
||||
"denominator": "2"
|
||||
},
|
||||
"error": "reduced value does not fit U64"
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "18446744073709551615",
|
||||
"denominator": "18446744073709551614"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "18446744073709551613",
|
||||
"denominator": "18446744073709551611"
|
||||
},
|
||||
"error": "the cross-multiplied numerators sum past 2^128",
|
||||
"reason": "two reduced fractions near the U64 maximum: every multiplication fits u128, their sum does not, and the arithmetic must refuse rather than wrap"
|
||||
}
|
||||
],
|
||||
"subtract": [
|
||||
{
|
||||
"a": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"difference": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
}
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "1",
|
||||
"denominator": "2"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "1",
|
||||
"denominator": "3"
|
||||
},
|
||||
"difference": {
|
||||
"numerator": "1",
|
||||
"denominator": "6"
|
||||
}
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "1",
|
||||
"denominator": "2"
|
||||
},
|
||||
"error": "subtraction would be negative"
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "18446744073709551615",
|
||||
"denominator": "18446744073709551614"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "1",
|
||||
"denominator": "18446744073709551611"
|
||||
},
|
||||
"error": "the reduced difference does not fit U64",
|
||||
"reason": "large denominators reach the reduction limit rather than the subtraction one"
|
||||
}
|
||||
],
|
||||
"multiply": [
|
||||
{
|
||||
"a": {
|
||||
"numerator": "1000000",
|
||||
"denominator": "1"
|
||||
},
|
||||
"k": "17",
|
||||
"product": {
|
||||
"numerator": "17000000",
|
||||
"denominator": "1"
|
||||
}
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
},
|
||||
"k": "1000",
|
||||
"product": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
}
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "18446744073709551615",
|
||||
"denominator": "1"
|
||||
},
|
||||
"k": "2",
|
||||
"error": "reduced value does not fit U64"
|
||||
}
|
||||
],
|
||||
"compare": [
|
||||
{
|
||||
"a": {
|
||||
"numerator": "0",
|
||||
"denominator": "1"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "1000000",
|
||||
"denominator": "1"
|
||||
},
|
||||
"ordering": "less"
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "1",
|
||||
"denominator": "3"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "1",
|
||||
"denominator": "3"
|
||||
},
|
||||
"ordering": "equal",
|
||||
"reason": "equal values compare equal; an unreduced 2/6 never reaches a comparison, because it never parses"
|
||||
},
|
||||
{
|
||||
"a": {
|
||||
"numerator": "50000000",
|
||||
"denominator": "3"
|
||||
},
|
||||
"b": {
|
||||
"numerator": "1000000",
|
||||
"denominator": "1"
|
||||
},
|
||||
"ordering": "greater"
|
||||
}
|
||||
]
|
||||
}
|
||||
77
services/flysim/crates/fly-session-types/fixtures/raw.json
Normal file
77
services/flysim/crates/fly-session-types/fixtures/raw.json
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
{
|
||||
"description": "Byte sequences every implementation must refuse before validation.",
|
||||
"cases": [
|
||||
{
|
||||
"name": "duplicate key at the top level",
|
||||
"type": "Scope",
|
||||
"base64": "eyJzZXNzaW9uSWQiOiJkZW1vIiwiZXBvY2giOiJlcG9jaC0xIiwic3RlcCI6IjEiLCJzdGVwIjoiMiJ9",
|
||||
"reason": "duplicate JSON keys are refused at any depth"
|
||||
},
|
||||
{
|
||||
"name": "duplicate key inside a nested object",
|
||||
"type": "TypedValue",
|
||||
"base64": "eyJzY2hlbWEiOnsiaWQiOiJhLnYxIiwidmVyc2lvbiI6MSwiZGlnZXN0IjoiYmJhNzk1MWMwNDY2NGNkNDliYjZlZWQxZGE2ZGMxYTRlMTdlOTg5ZTc4N2JmMmU3MmY1OTMzYTVjNjE3MTM3MiJ9LCJ2YWx1ZSI6eyJ4IjoxLCJ4IjoyfX0=",
|
||||
"reason": "duplicate JSON keys are refused at any depth"
|
||||
},
|
||||
{
|
||||
"name": "invalid UTF-8 in a string",
|
||||
"type": "Scope",
|
||||
"base64": "eyJzZXNzaW9uSWQiOiJkZf9tbyIsImVwb2NoIjoiZXBvY2gtMSIsInN0ZXAiOiIxIn0=",
|
||||
"reason": "the envelope is UTF-8"
|
||||
},
|
||||
{
|
||||
"name": "invalid UTF-8 in a key",
|
||||
"type": "Scope",
|
||||
"base64": "eyJzZXNzaW9u/0lkIjoiZGVtbyIsImVwb2NoIjoiZXBvY2gtMSIsInN0ZXAiOiIxIn0=",
|
||||
"reason": "the envelope is UTF-8"
|
||||
},
|
||||
{
|
||||
"name": "NaN literal",
|
||||
"type": "Stimulus",
|
||||
"base64": "eyJpZCI6InN0aW0tMSIsImtpbmRJZCI6InN1Z2FyIiwiZHVyYXRpb25NcyI6TmFOfQ==",
|
||||
"reason": "NaN and Infinity are not JSON"
|
||||
},
|
||||
{
|
||||
"name": "Infinity literal",
|
||||
"type": "Stimulus",
|
||||
"base64": "eyJpZCI6InN0aW0tMSIsImtpbmRJZCI6InN1Z2FyIiwiZHVyYXRpb25NcyI6SW5maW5pdHl9",
|
||||
"reason": "NaN and Infinity are not JSON"
|
||||
},
|
||||
{
|
||||
"name": "number that overflows a double",
|
||||
"type": "Stimulus",
|
||||
"base64": "eyJpZCI6InN0aW0tMSIsImtpbmRJZCI6InN1Z2FyIiwiZHVyYXRpb25NcyI6MWU5OTl9",
|
||||
"reason": "a non-finite number never survives parsing"
|
||||
},
|
||||
{
|
||||
"name": "integer past the exact double range",
|
||||
"type": "Stimulus",
|
||||
"base64": "eyJpZCI6InN0aW0tMSIsImtpbmRJZCI6InN1Z2FyIiwiZHVyYXRpb25NcyI6OTAwNzE5OTI1NDc0MDk5M30=",
|
||||
"reason": "canonical JSON cannot encode it exactly; counters are U64 strings"
|
||||
},
|
||||
{
|
||||
"name": "trailing data after the object",
|
||||
"type": "Scope",
|
||||
"base64": "eyJzZXNzaW9uSWQiOiJkZW1vIiwiZXBvY2giOiJlcG9jaC0xIiwic3RlcCI6IjEifSB7fQ==",
|
||||
"reason": "one value per payload"
|
||||
},
|
||||
{
|
||||
"name": "truncated object",
|
||||
"type": "Scope",
|
||||
"base64": "eyJzZXNzaW9uSWQiOiJkZW1vIiwiZXBvY2giOiJlcG9jaC0xIg==",
|
||||
"reason": "partial JSON is refused"
|
||||
},
|
||||
{
|
||||
"name": "empty payload",
|
||||
"type": "Scope",
|
||||
"base64": "",
|
||||
"reason": "an empty payload is not a JSON object"
|
||||
},
|
||||
{
|
||||
"name": "a bare array",
|
||||
"type": "Scope",
|
||||
"base64": "W10=",
|
||||
"reason": "a payload is an object"
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,185 @@
|
|||
{
|
||||
"description": "seed-derivation-v1 test vectors. Both languages must reproduce every seed.",
|
||||
"algorithm": "seed-derivation-v1",
|
||||
"prefix": "flybrain/seed-derivation-v1",
|
||||
"materialTemplate": "<prefix>\\n<masterSeed>\\n<agentId>\\n",
|
||||
"rule": "SHA-256 of the material, read as eight big-endian u32 lanes; the first nonzero lane is the seed as a two's-complement i32.",
|
||||
"vectors": [
|
||||
{
|
||||
"masterSeed": "0",
|
||||
"agentId": "fly-a",
|
||||
"material": "flybrain/seed-derivation-v1\n0\nfly-a\n",
|
||||
"materialDigest": "6cf7c34a422f4cdd890c64d0ef5a072e4f6ddc2ee18e83ee24a5f2214f4960d1",
|
||||
"seed": 1828176714
|
||||
},
|
||||
{
|
||||
"masterSeed": "0",
|
||||
"agentId": "fly-b",
|
||||
"material": "flybrain/seed-derivation-v1\n0\nfly-b\n",
|
||||
"materialDigest": "48a52f4069cfca001ba5ee6ae870dccb5a5bd59d61babf333a01749b6a8f4025",
|
||||
"seed": 1218785088
|
||||
},
|
||||
{
|
||||
"masterSeed": "0",
|
||||
"agentId": "fly-c",
|
||||
"material": "flybrain/seed-derivation-v1\n0\nfly-c\n",
|
||||
"materialDigest": "53a83c2d0cfce5a3356aab3b0b73a270201a2dff7816168a4167c71a32d863fe",
|
||||
"seed": 1403534381
|
||||
},
|
||||
{
|
||||
"masterSeed": "0",
|
||||
"agentId": "fly-d",
|
||||
"material": "flybrain/seed-derivation-v1\n0\nfly-d\n",
|
||||
"materialDigest": "f202513f32fe6070eee65ad72027490f9218b86dc8cf74e4e3563f02d532ea94",
|
||||
"seed": -234729153
|
||||
},
|
||||
{
|
||||
"masterSeed": "1",
|
||||
"agentId": "fly-a",
|
||||
"material": "flybrain/seed-derivation-v1\n1\nfly-a\n",
|
||||
"materialDigest": "6eab9d6d6d002ffc0e2cdf2d64dd8226f90c391e45641512d14edbb244225698",
|
||||
"seed": 1856740717
|
||||
},
|
||||
{
|
||||
"masterSeed": "1",
|
||||
"agentId": "fly-b",
|
||||
"material": "flybrain/seed-derivation-v1\n1\nfly-b\n",
|
||||
"materialDigest": "00115cb341d839490c5485462fd7eedd8b6baeda3748054fcba0d1558e471a6c",
|
||||
"seed": 1137843
|
||||
},
|
||||
{
|
||||
"masterSeed": "1",
|
||||
"agentId": "fly-c",
|
||||
"material": "flybrain/seed-derivation-v1\n1\nfly-c\n",
|
||||
"materialDigest": "af97482074d6d36e5f5920aaa1d87f02365045f3712300dbd243584f2cb4a64c",
|
||||
"seed": -1349040096
|
||||
},
|
||||
{
|
||||
"masterSeed": "1",
|
||||
"agentId": "fly-d",
|
||||
"material": "flybrain/seed-derivation-v1\n1\nfly-d\n",
|
||||
"materialDigest": "1081b24880040dcc4d442f88451513e4fbcf8dae9ccb79329dd289fabbc7938b",
|
||||
"seed": 276935240
|
||||
},
|
||||
{
|
||||
"masterSeed": "42",
|
||||
"agentId": "fly-a",
|
||||
"material": "flybrain/seed-derivation-v1\n42\nfly-a\n",
|
||||
"materialDigest": "f4f9f271d53e94c38c6260b99840513f9eb70e5bb442d63345e217185e6ba9f6",
|
||||
"seed": -184946063
|
||||
},
|
||||
{
|
||||
"masterSeed": "42",
|
||||
"agentId": "fly-b",
|
||||
"material": "flybrain/seed-derivation-v1\n42\nfly-b\n",
|
||||
"materialDigest": "1feb285e834138ec823ffea696fbe5fb3c211f349ce17140e25286d06c56ece8",
|
||||
"seed": 535504990
|
||||
},
|
||||
{
|
||||
"masterSeed": "42",
|
||||
"agentId": "fly-c",
|
||||
"material": "flybrain/seed-derivation-v1\n42\nfly-c\n",
|
||||
"materialDigest": "b5a251ed4521f91eab1d8a6813880794c5c6706eafc137d8ebc169f743c60eed",
|
||||
"seed": -1247653395
|
||||
},
|
||||
{
|
||||
"masterSeed": "42",
|
||||
"agentId": "fly-d",
|
||||
"material": "flybrain/seed-derivation-v1\n42\nfly-d\n",
|
||||
"materialDigest": "e36d6e40f27a4ffe473351d4bd01154da2efb3c3038d88f3aa48f77963d465c6",
|
||||
"seed": -479367616
|
||||
},
|
||||
{
|
||||
"masterSeed": "9223372036854775808",
|
||||
"agentId": "fly-a",
|
||||
"material": "flybrain/seed-derivation-v1\n9223372036854775808\nfly-a\n",
|
||||
"materialDigest": "d59f513b6fdce1646ed13907302e0cb3ca4696738dd7df38e9581397508ec933",
|
||||
"seed": -710979269
|
||||
},
|
||||
{
|
||||
"masterSeed": "9223372036854775808",
|
||||
"agentId": "fly-b",
|
||||
"material": "flybrain/seed-derivation-v1\n9223372036854775808\nfly-b\n",
|
||||
"materialDigest": "c8de8ca0279829c438fa19c07f05768cfe050e8c21bdaa0fe3c68f8048b4e1a3",
|
||||
"seed": -924939104
|
||||
},
|
||||
{
|
||||
"masterSeed": "9223372036854775808",
|
||||
"agentId": "fly-c",
|
||||
"material": "flybrain/seed-derivation-v1\n9223372036854775808\nfly-c\n",
|
||||
"materialDigest": "354ad1ec388aa7be6136151d0f6281cc17279b62e95d458c96f392aaac40ee62",
|
||||
"seed": 894095852
|
||||
},
|
||||
{
|
||||
"masterSeed": "9223372036854775808",
|
||||
"agentId": "fly-d",
|
||||
"material": "flybrain/seed-derivation-v1\n9223372036854775808\nfly-d\n",
|
||||
"materialDigest": "58e61deb4777b7702bdcd7edf5a5bcd0d51273699cc1051c7fbc9a5ffedd9e15",
|
||||
"seed": 1491475947
|
||||
},
|
||||
{
|
||||
"masterSeed": "18446744073709551615",
|
||||
"agentId": "fly-a",
|
||||
"material": "flybrain/seed-derivation-v1\n18446744073709551615\nfly-a\n",
|
||||
"materialDigest": "f888eb95bcab276936fce5f72df3a0741e36da26722f43cc2a974932dfd42cd1",
|
||||
"seed": -125244523
|
||||
},
|
||||
{
|
||||
"masterSeed": "18446744073709551615",
|
||||
"agentId": "fly-b",
|
||||
"material": "flybrain/seed-derivation-v1\n18446744073709551615\nfly-b\n",
|
||||
"materialDigest": "d343df049e8ef71617e9af7a321cf498d61083c50229f2d6852077670c86acbd",
|
||||
"seed": -750526716
|
||||
},
|
||||
{
|
||||
"masterSeed": "18446744073709551615",
|
||||
"agentId": "fly-c",
|
||||
"material": "flybrain/seed-derivation-v1\n18446744073709551615\nfly-c\n",
|
||||
"materialDigest": "88016b220245b431a4eb510d31b643475bb2496130b65ae81b165acd5a899292",
|
||||
"seed": -2013172958
|
||||
},
|
||||
{
|
||||
"masterSeed": "18446744073709551615",
|
||||
"agentId": "fly-d",
|
||||
"material": "flybrain/seed-derivation-v1\n18446744073709551615\nfly-d\n",
|
||||
"materialDigest": "d9b5bee718c87599c9ef0b3c16f361dadf714a33f7dd1209a21108f4df700b16",
|
||||
"seed": -642400537
|
||||
}
|
||||
],
|
||||
"composition": {
|
||||
"masterSeed": "42",
|
||||
"agentIds": [
|
||||
"fly-a",
|
||||
"fly-b",
|
||||
"fly-c",
|
||||
"fly-d"
|
||||
],
|
||||
"seeds": [
|
||||
-184946063,
|
||||
535504990,
|
||||
-1247653395,
|
||||
-479367616
|
||||
],
|
||||
"reason": "independent per-agent seeds from one recorded master seed and stable agent ids"
|
||||
},
|
||||
"invalid": [
|
||||
{
|
||||
"masterSeed": "0",
|
||||
"agentId": "Fly-A",
|
||||
"reason": "an agent id is an Id: lowercase"
|
||||
},
|
||||
{
|
||||
"masterSeed": "0",
|
||||
"agentId": "",
|
||||
"reason": "an agent id is 1..=64 characters"
|
||||
},
|
||||
{
|
||||
"masterSeed": "0",
|
||||
"agentIds": [
|
||||
"fly-a",
|
||||
"fly-a"
|
||||
],
|
||||
"reason": "a composition with a repeated agent id is refused rather than silently sharing a seed"
|
||||
}
|
||||
]
|
||||
}
|
||||
1141
services/flysim/crates/fly-session-types/fixtures/traces.json
Normal file
1141
services/flysim/crates/fly-session-types/fixtures/traces.json
Normal file
File diff suppressed because it is too large
Load diff
2787
services/flysim/crates/fly-session-types/fixtures/valid.json
Normal file
2787
services/flysim/crates/fly-session-types/fixtures/valid.json
Normal file
File diff suppressed because it is too large
Load diff
275
services/flysim/crates/fly-session-types/src/canonical.rs
Normal file
275
services/flysim/crates/fly-session-types/src/canonical.rs
Normal file
|
|
@ -0,0 +1,275 @@
|
|||
//! Canonical JSON (RFC 8785) and the digest rules of ipc-v1 section 5.
|
||||
//!
|
||||
//! One serialization, two languages: keys sorted by UTF-16 code unit, numbers printed by the
|
||||
//! ECMAScript `Number::toString` algorithm (so a JavaScript `JSON.stringify` over the same
|
||||
//! sorted tree produces the same bytes), strings escaped the way `JSON.stringify` escapes
|
||||
//! them, no insignificant whitespace. A digest is the SHA-256 of those bytes, lowercase hex.
|
||||
//!
|
||||
//! A JSON number is canonicalizable when it is finite and, if it is integral, no larger in
|
||||
//! magnitude than 2^53-1. Integers past that range are refused rather than rounded: every
|
||||
//! counter and clock in these contracts is a `U64` decimal string, so a large JSON number is
|
||||
//! a schema error. The rule is stated on the value, not on how it was written, because
|
||||
//! `JSON.parse` cannot tell `1e21` from `1000000000000000000000`, and two implementations
|
||||
//! that disagree about one number do not agree about any digest.
|
||||
|
||||
use serde_json::{Number, Value};
|
||||
use sha2::{Digest as _, Sha256};
|
||||
|
||||
use crate::scalar::{Result, Scope, err, wire_err};
|
||||
|
||||
/// The largest integer a double represents exactly.
|
||||
pub const MAX_EXACT_INTEGER: i64 = 9_007_199_254_740_991;
|
||||
|
||||
/// The bus envelope ceiling every domain message must also fit (bus-v1 section 4).
|
||||
pub const MAX_ENVELOPE_BYTES: usize = flybus::wire::MAX_ENVELOPE_BYTES;
|
||||
|
||||
/// The `f64` a JSON number denotes, or `None` if it is not canonicalizable: not finite, or an
|
||||
/// integral value outside the exactly representable integer range.
|
||||
pub fn finite_double(n: &Number) -> Option<f64> {
|
||||
let value = n.as_f64().filter(|v| v.is_finite())?;
|
||||
if value.fract() == 0.0 && value.abs() > MAX_EXACT_INTEGER as f64 {
|
||||
return None;
|
||||
}
|
||||
Some(value)
|
||||
}
|
||||
|
||||
/// `String(number)` for a finite double, the ECMAScript algorithm RFC 8785 requires.
|
||||
fn number_to_string(value: f64) -> String {
|
||||
if value == 0.0 {
|
||||
// Covers -0.0, which `JSON.stringify` prints as "0".
|
||||
return "0".to_owned();
|
||||
}
|
||||
let mut buffer = ryu_js::Buffer::new();
|
||||
buffer.format(value).to_owned()
|
||||
}
|
||||
|
||||
/// Escapes one string the way `JSON.stringify` does.
|
||||
fn write_string(out: &mut String, s: &str) {
|
||||
out.push('"');
|
||||
for c in s.chars() {
|
||||
match c {
|
||||
'"' => out.push_str("\\\""),
|
||||
'\\' => out.push_str("\\\\"),
|
||||
'\u{08}' => out.push_str("\\b"),
|
||||
'\u{09}' => out.push_str("\\t"),
|
||||
'\u{0a}' => out.push_str("\\n"),
|
||||
'\u{0c}' => out.push_str("\\f"),
|
||||
'\u{0d}' => out.push_str("\\r"),
|
||||
c if (c as u32) < 0x20 => out.push_str(&format!("\\u{:04x}", c as u32)),
|
||||
c => out.push(c),
|
||||
}
|
||||
}
|
||||
out.push('"');
|
||||
}
|
||||
|
||||
/// Sorts object keys by UTF-16 code unit, as RFC 8785 section 3.2.3 specifies.
|
||||
fn utf16_key(key: &str) -> Vec<u16> {
|
||||
key.encode_utf16().collect()
|
||||
}
|
||||
|
||||
/// The canonical JSON text of `value`.
|
||||
pub fn canonicalize(value: &Value) -> Result<String> {
|
||||
let mut out = String::new();
|
||||
write_value(&mut out, value)?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// The canonical JSON bytes of `value`.
|
||||
pub fn canonical_bytes(value: &Value) -> Result<Vec<u8>> {
|
||||
canonicalize(value).map(String::into_bytes)
|
||||
}
|
||||
|
||||
fn write_value(out: &mut String, value: &Value) -> Result<()> {
|
||||
match value {
|
||||
Value::Null => out.push_str("null"),
|
||||
Value::Bool(true) => out.push_str("true"),
|
||||
Value::Bool(false) => out.push_str("false"),
|
||||
Value::Number(n) => {
|
||||
let d = finite_double(n).ok_or_else(|| {
|
||||
wire_err(format!(
|
||||
"canonical JSON: {n} is not a finite number in the exact double range"
|
||||
))
|
||||
})?;
|
||||
out.push_str(&number_to_string(d));
|
||||
}
|
||||
Value::String(s) => write_string(out, s),
|
||||
Value::Array(items) => {
|
||||
out.push('[');
|
||||
for (i, item) in items.iter().enumerate() {
|
||||
if i > 0 {
|
||||
out.push(',');
|
||||
}
|
||||
write_value(out, item)?;
|
||||
}
|
||||
out.push(']');
|
||||
}
|
||||
Value::Object(map) => {
|
||||
let mut keys: Vec<&String> = map.keys().collect();
|
||||
keys.sort_by_cached_key(|k| utf16_key(k));
|
||||
out.push('{');
|
||||
for (i, key) in keys.iter().enumerate() {
|
||||
if i > 0 {
|
||||
out.push(',');
|
||||
}
|
||||
write_string(out, key);
|
||||
out.push(':');
|
||||
write_value(out, &map[key.as_str()])?;
|
||||
}
|
||||
out.push('}');
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Lowercase hex SHA-256.
|
||||
pub fn sha256_hex(bytes: &[u8]) -> String {
|
||||
let digest = Sha256::digest(bytes);
|
||||
let mut out = String::with_capacity(64);
|
||||
for byte in digest {
|
||||
out.push_str(&format!("{byte:02x}"));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The canonical digest of a JSON value: SHA-256 over its canonical JSON bytes.
|
||||
pub fn digest_of(value: &Value) -> Result<String> {
|
||||
canonical_bytes(value).map(|bytes| sha256_hex(&bytes))
|
||||
}
|
||||
|
||||
/// Parses JSON strictly: duplicate keys at any depth, invalid UTF-8, non-finite numbers and
|
||||
/// trailing bytes are refused. The bus reader, reused so both layers agree byte for byte.
|
||||
pub fn parse_strict(bytes: &[u8]) -> Result<Value> {
|
||||
flybus::wire::parse_json_strict(bytes).map_err(|e| wire_err(e.0))
|
||||
}
|
||||
|
||||
/// Refuses a domain payload that does not fit the bus envelope ceiling.
|
||||
///
|
||||
/// The check is on canonical bytes, and the caller passes the overhead the surrounding
|
||||
/// envelope adds, so a payload that only fits without its envelope still fails.
|
||||
pub fn require_envelope_fit(value: &Value, envelope_overhead: usize) -> Result<usize> {
|
||||
let len = canonicalize(value)?.len();
|
||||
let total = len + envelope_overhead;
|
||||
if total > MAX_ENVELOPE_BYTES {
|
||||
return err(format!(
|
||||
"envelope: {total} bytes exceeds the {MAX_ENVELOPE_BYTES}-byte maximum"
|
||||
));
|
||||
}
|
||||
Ok(total)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// Operation keys and canonical bodies
|
||||
|
||||
/// The keys that belong to the bus, never to a domain body (ipc-v1 section 5: the canonical
|
||||
/// body "excludes changing bus callIds, deliveryIds and owner tokens").
|
||||
pub const BUS_ONLY_KEYS: &[&str] = &[
|
||||
"callId",
|
||||
"deliveryId",
|
||||
"ownerId",
|
||||
"ownerIds",
|
||||
"deliveryIds",
|
||||
"requestDeliveryId",
|
||||
"expectedIncarnation",
|
||||
"serviceIncarnation",
|
||||
"connectionId",
|
||||
"topicSequence",
|
||||
"subscriptionId",
|
||||
];
|
||||
|
||||
/// Fails if any bus-only key appears anywhere in `value`.
|
||||
pub fn reject_bus_identities(value: &Value) -> Result<()> {
|
||||
match value {
|
||||
Value::Object(map) => {
|
||||
for (key, inner) in map {
|
||||
if BUS_ONLY_KEYS.contains(&key.as_str()) {
|
||||
return err(format!(
|
||||
"canonical body: {key:?} is a bus identity and never part of a domain body"
|
||||
));
|
||||
}
|
||||
reject_bus_identities(inner)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
Value::Array(items) => {
|
||||
for item in items {
|
||||
reject_bus_identities(item)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
_ => Ok(()),
|
||||
}
|
||||
}
|
||||
|
||||
/// `(sessionId, epoch, step, method, workerId)`: the operation key of a step mutation.
|
||||
///
|
||||
/// There is at most one Prepare, Commit or Advance for one key (ipc-v1 section 5). The key
|
||||
/// deliberately does not contain the requestId: a changed id for an existing key is CONFLICT,
|
||||
/// which can only be detected if the key is the same.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
|
||||
pub struct OperationKey {
|
||||
pub scope: Scope,
|
||||
pub method: String,
|
||||
pub worker_id: String,
|
||||
}
|
||||
|
||||
impl OperationKey {
|
||||
pub fn new(scope: Scope, method: &str, worker_id: &str) -> Result<OperationKey> {
|
||||
let key = OperationKey {
|
||||
scope,
|
||||
method: method.to_owned(),
|
||||
worker_id: worker_id.to_owned(),
|
||||
};
|
||||
key.validate()?;
|
||||
Ok(key)
|
||||
}
|
||||
|
||||
pub fn validate(&self) -> Result<()> {
|
||||
use crate::scalar::DomainType;
|
||||
self.scope.validate()?;
|
||||
if !flybus::wire::is_method(&self.method) {
|
||||
return err("OperationKey: method must be 1..=128 printable ASCII characters");
|
||||
}
|
||||
if !crate::scalar::is_id(&self.worker_id) {
|
||||
return err("OperationKey: workerId is not a valid id");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn to_json(&self) -> Value {
|
||||
use crate::scalar::DomainType;
|
||||
crate::scalar::obj(vec![
|
||||
("scope", self.scope.to_json()),
|
||||
("method", self.method.clone().into()),
|
||||
("workerId", self.worker_id.clone().into()),
|
||||
])
|
||||
}
|
||||
|
||||
/// The canonical digest of the key, for a deduplication table that stores digests.
|
||||
pub fn digest(&self) -> Result<String> {
|
||||
digest_of(&self.to_json())
|
||||
}
|
||||
}
|
||||
|
||||
/// The canonical body of a domain operation: method, scope and validated params.
|
||||
///
|
||||
/// Two calls of the same operation key whose body digests differ are CONFLICT; two calls with
|
||||
/// the same digest are the same operation, whatever bus callId carried them.
|
||||
pub fn canonical_body(method: &str, scope: Option<&Scope>, params: &Value) -> Result<Value> {
|
||||
if !flybus::wire::is_method(method) {
|
||||
return err("canonical body: method must be 1..=128 printable ASCII characters");
|
||||
}
|
||||
if !params.is_object() {
|
||||
return err("canonical body: params must be an object");
|
||||
}
|
||||
reject_bus_identities(params)?;
|
||||
Ok(crate::scalar::obj(vec![
|
||||
("method", method.into()),
|
||||
("scope", Scope::nullable_to_json(scope)),
|
||||
("params", params.clone()),
|
||||
]))
|
||||
}
|
||||
|
||||
/// The canonical body digest of a domain operation.
|
||||
pub fn body_digest(method: &str, scope: Option<&Scope>, params: &Value) -> Result<String> {
|
||||
digest_of(&canonical_body(method, scope, params)?)
|
||||
}
|
||||
368
services/flysim/crates/fly-session-types/src/checkpoint.rs
Normal file
368
services/flysim/crates/fly-session-types/src/checkpoint.rs
Normal file
|
|
@ -0,0 +1,368 @@
|
|||
//! `FLYSESS1`: the envelope layout of `docs/design/session-framework/checkpoint-envelope-v1.md`.
|
||||
//!
|
||||
//! This is the layout half of the specification, not the store: it lays out a header, a
|
||||
//! canonical-JSON manifest, a payload table and the payload bytes, and it reads one back.
|
||||
//! Writing generations, fsyncing and committing a manifest belong to the STATE-01 store slice.
|
||||
//! `FLYSIM01` is a different format with a different magic and is not touched by any of this.
|
||||
|
||||
use serde_json::Value;
|
||||
use sha2::{Digest as _, Sha256};
|
||||
|
||||
use crate::canonical;
|
||||
use crate::scalar::{Result, err, is_id};
|
||||
|
||||
/// Envelope magic. Eight ASCII bytes, distinct from `FLYSIM01`.
|
||||
pub const MAGIC: &[u8; 8] = b"FLYSESS1";
|
||||
/// Footer magic, so a truncated file cannot look complete.
|
||||
pub const FOOTER_MAGIC: &[u8; 8] = b"FLYSESSF";
|
||||
/// Envelope version, in the header and in the manifest.
|
||||
pub const VERSION: u32 = 1;
|
||||
/// Fixed header size in bytes.
|
||||
pub const HEADER_BYTES: usize = 32;
|
||||
/// One payload table entry: a 64-byte name field, offset, length and a 32-byte digest.
|
||||
pub const TABLE_ENTRY_BYTES: usize = 112;
|
||||
/// Payload name field width.
|
||||
pub const NAME_BYTES: usize = 64;
|
||||
/// Footer size in bytes: total length, whole-prefix digest and the footer magic.
|
||||
pub const FOOTER_BYTES: usize = 48;
|
||||
/// Payloads start on an eight-byte boundary.
|
||||
pub const ALIGNMENT: u64 = 8;
|
||||
/// Payloads per envelope.
|
||||
pub const MAX_PAYLOADS: usize = 64;
|
||||
|
||||
/// One payload's table entry.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct PayloadEntry {
|
||||
/// An `Id`: the new envelope widens the historical letters-only chunk name deliberately,
|
||||
/// which is why it is a new version and not an extension of `FLYSIM01`.
|
||||
pub name: String,
|
||||
pub offset: u64,
|
||||
pub byte_length: u64,
|
||||
/// SHA-256 of exactly `byte_length` bytes at `offset`.
|
||||
pub digest: [u8; 32],
|
||||
}
|
||||
|
||||
/// A laid-out envelope: where everything is, before any bytes are written.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct Layout {
|
||||
pub manifest_offset: u64,
|
||||
pub manifest_bytes: u32,
|
||||
pub table_offset: u64,
|
||||
pub entries: Vec<PayloadEntry>,
|
||||
pub footer_offset: u64,
|
||||
pub total_bytes: u64,
|
||||
}
|
||||
|
||||
fn align_up(value: u64) -> u64 {
|
||||
value.div_ceil(ALIGNMENT) * ALIGNMENT
|
||||
}
|
||||
|
||||
/// Lays out the envelope for one manifest and a list of `(name, bytes)` payloads.
|
||||
pub fn layout(manifest: &Value, payloads: &[(String, Vec<u8>)]) -> Result<Layout> {
|
||||
if payloads.len() > MAX_PAYLOADS {
|
||||
return err("checkpoint envelope: at most 64 payloads");
|
||||
}
|
||||
crate::scalar::require_unique(
|
||||
payloads.iter().map(|(name, _)| name.as_str()),
|
||||
"checkpoint envelope: payload names",
|
||||
)?;
|
||||
for (name, _) in payloads {
|
||||
if !is_id(name) {
|
||||
return err(format!(
|
||||
"checkpoint envelope: payload name {name:?} is not an Id"
|
||||
));
|
||||
}
|
||||
}
|
||||
let manifest_text = canonical::canonicalize(manifest)?;
|
||||
let manifest_bytes = u32::try_from(manifest_text.len())
|
||||
.map_err(|_| crate::scalar::wire_err("checkpoint envelope: manifest is too large"))?;
|
||||
let manifest_offset = HEADER_BYTES as u64;
|
||||
let table_offset = align_up(manifest_offset + u64::from(manifest_bytes));
|
||||
let mut offset = align_up(table_offset + (payloads.len() * TABLE_ENTRY_BYTES) as u64);
|
||||
let mut entries = Vec::with_capacity(payloads.len());
|
||||
for (name, bytes) in payloads {
|
||||
entries.push(PayloadEntry {
|
||||
name: name.clone(),
|
||||
offset,
|
||||
byte_length: bytes.len() as u64,
|
||||
digest: Sha256::digest(bytes).into(),
|
||||
});
|
||||
offset = align_up(offset + bytes.len() as u64);
|
||||
}
|
||||
Ok(Layout {
|
||||
manifest_offset,
|
||||
manifest_bytes,
|
||||
table_offset,
|
||||
entries,
|
||||
footer_offset: offset,
|
||||
total_bytes: offset + FOOTER_BYTES as u64,
|
||||
})
|
||||
}
|
||||
|
||||
/// Writes one envelope: header, manifest, payload table, payloads, footer.
|
||||
pub fn encode(manifest: &Value, payloads: &[(String, Vec<u8>)]) -> Result<Vec<u8>> {
|
||||
let layout = layout(manifest, payloads)?;
|
||||
let manifest_text = canonical::canonicalize(manifest)?;
|
||||
let mut out = vec![0u8; layout.footer_offset as usize];
|
||||
out[0..8].copy_from_slice(MAGIC);
|
||||
out[8..12].copy_from_slice(&VERSION.to_le_bytes());
|
||||
out[12..16].copy_from_slice(&(HEADER_BYTES as u32).to_le_bytes());
|
||||
out[16..20].copy_from_slice(&layout.manifest_bytes.to_le_bytes());
|
||||
out[20..24].copy_from_slice(&(payloads.len() as u32).to_le_bytes());
|
||||
out[24..28].copy_from_slice(&(layout.table_offset as u32).to_le_bytes());
|
||||
out[28..32].copy_from_slice(&0u32.to_le_bytes());
|
||||
let manifest_start = layout.manifest_offset as usize;
|
||||
out[manifest_start..manifest_start + manifest_text.len()]
|
||||
.copy_from_slice(manifest_text.as_bytes());
|
||||
for (index, entry) in layout.entries.iter().enumerate() {
|
||||
let base = layout.table_offset as usize + index * TABLE_ENTRY_BYTES;
|
||||
out[base..base + entry.name.len()].copy_from_slice(entry.name.as_bytes());
|
||||
let numbers = base + NAME_BYTES;
|
||||
out[numbers..numbers + 8].copy_from_slice(&entry.offset.to_le_bytes());
|
||||
out[numbers + 8..numbers + 16].copy_from_slice(&entry.byte_length.to_le_bytes());
|
||||
out[numbers + 16..numbers + 48].copy_from_slice(&entry.digest);
|
||||
}
|
||||
for (entry, (_, bytes)) in layout.entries.iter().zip(payloads) {
|
||||
let start = entry.offset as usize;
|
||||
out[start..start + bytes.len()].copy_from_slice(bytes);
|
||||
}
|
||||
let digest: [u8; 32] = Sha256::digest(&out).into();
|
||||
out.extend_from_slice(&layout.total_bytes.to_le_bytes());
|
||||
out.extend_from_slice(&digest);
|
||||
out.extend_from_slice(FOOTER_MAGIC);
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// A decoded envelope.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct Envelope {
|
||||
pub manifest: Value,
|
||||
pub payloads: Vec<(String, Vec<u8>)>,
|
||||
pub layout: Layout,
|
||||
}
|
||||
|
||||
impl Envelope {
|
||||
pub fn payload(&self, name: &str) -> Option<&[u8]> {
|
||||
self.payloads
|
||||
.iter()
|
||||
.find(|(key, _)| key == name)
|
||||
.map(|(_, bytes)| bytes.as_slice())
|
||||
}
|
||||
}
|
||||
|
||||
fn u32_at(bytes: &[u8], offset: usize) -> u32 {
|
||||
u32::from_le_bytes([
|
||||
bytes[offset],
|
||||
bytes[offset + 1],
|
||||
bytes[offset + 2],
|
||||
bytes[offset + 3],
|
||||
])
|
||||
}
|
||||
|
||||
fn u64_at(bytes: &[u8], offset: usize) -> u64 {
|
||||
let mut buf = [0u8; 8];
|
||||
buf.copy_from_slice(&bytes[offset..offset + 8]);
|
||||
u64::from_le_bytes(buf)
|
||||
}
|
||||
|
||||
/// Reads and fully validates one envelope: magic, version, footer digest, table ordering,
|
||||
/// alignment, bounds and every payload digest.
|
||||
pub fn decode(bytes: &[u8]) -> Result<Envelope> {
|
||||
if bytes.len() < HEADER_BYTES + FOOTER_BYTES {
|
||||
return err("checkpoint envelope: shorter than a header plus a footer");
|
||||
}
|
||||
if &bytes[0..8] != MAGIC {
|
||||
return err("checkpoint envelope: wrong magic (FLYSIM01 is a different format)");
|
||||
}
|
||||
if u32_at(bytes, 8) != VERSION {
|
||||
return err("checkpoint envelope: unsupported version");
|
||||
}
|
||||
if u32_at(bytes, 12) as usize != HEADER_BYTES {
|
||||
return err("checkpoint envelope: headerBytes must be 32");
|
||||
}
|
||||
if u32_at(bytes, 28) != 0 {
|
||||
return err("checkpoint envelope: reserved header word must be zero");
|
||||
}
|
||||
let manifest_bytes = u32_at(bytes, 16) as usize;
|
||||
let payload_count = u32_at(bytes, 20) as usize;
|
||||
let table_offset = u32_at(bytes, 24) as u64;
|
||||
if payload_count > MAX_PAYLOADS {
|
||||
return err("checkpoint envelope: at most 64 payloads");
|
||||
}
|
||||
let footer_offset = bytes.len() - FOOTER_BYTES;
|
||||
if &bytes[footer_offset + 40..] != FOOTER_MAGIC {
|
||||
return err("checkpoint envelope: missing footer magic");
|
||||
}
|
||||
if u64_at(bytes, footer_offset) != bytes.len() as u64 {
|
||||
return err("checkpoint envelope: footer length does not match the file");
|
||||
}
|
||||
let recorded = &bytes[footer_offset + 8..footer_offset + 40];
|
||||
let computed: [u8; 32] = Sha256::digest(&bytes[..footer_offset]).into();
|
||||
if recorded != computed {
|
||||
return err("checkpoint envelope: footer digest does not match the contents");
|
||||
}
|
||||
let manifest_start = HEADER_BYTES;
|
||||
let manifest_end = manifest_start + manifest_bytes;
|
||||
if manifest_end > footer_offset {
|
||||
return err("checkpoint envelope: manifest runs past the payload area");
|
||||
}
|
||||
let manifest = canonical::parse_strict(&bytes[manifest_start..manifest_end])?;
|
||||
let canonical_manifest = canonical::canonicalize(&manifest)?;
|
||||
if canonical_manifest.as_bytes() != &bytes[manifest_start..manifest_end] {
|
||||
return err("checkpoint envelope: the manifest is not canonical JSON");
|
||||
}
|
||||
if table_offset != align_up(manifest_end as u64) {
|
||||
return err("checkpoint envelope: the payload table is not at its laid-out offset");
|
||||
}
|
||||
let table_end = table_offset as usize + payload_count * TABLE_ENTRY_BYTES;
|
||||
if table_end > footer_offset {
|
||||
return err("checkpoint envelope: the payload table runs past the payload area");
|
||||
}
|
||||
let mut entries = Vec::with_capacity(payload_count);
|
||||
let mut payloads = Vec::with_capacity(payload_count);
|
||||
let mut previous_end = align_up(table_end as u64);
|
||||
for index in 0..payload_count {
|
||||
let base = table_offset as usize + index * TABLE_ENTRY_BYTES;
|
||||
let name_field = &bytes[base..base + NAME_BYTES];
|
||||
let length = name_field
|
||||
.iter()
|
||||
.position(|b| *b == 0)
|
||||
.unwrap_or(NAME_BYTES);
|
||||
if name_field[length..].iter().any(|b| *b != 0) {
|
||||
return err("checkpoint envelope: a payload name has bytes after its terminator");
|
||||
}
|
||||
let name = std::str::from_utf8(&name_field[..length])
|
||||
.map_err(|_| crate::scalar::wire_err("checkpoint envelope: payload name is not UTF-8"))?
|
||||
.to_owned();
|
||||
if !is_id(&name) {
|
||||
return err(format!(
|
||||
"checkpoint envelope: payload name {name:?} is not an Id"
|
||||
));
|
||||
}
|
||||
let numbers = base + NAME_BYTES;
|
||||
let offset = u64_at(bytes, numbers);
|
||||
let byte_length = u64_at(bytes, numbers + 8);
|
||||
let mut digest = [0u8; 32];
|
||||
digest.copy_from_slice(&bytes[numbers + 16..numbers + 48]);
|
||||
if offset != previous_end {
|
||||
return err(format!(
|
||||
"checkpoint envelope: payload {name:?} starts at {offset}, not at its aligned {previous_end}"
|
||||
));
|
||||
}
|
||||
let end = offset
|
||||
.checked_add(byte_length)
|
||||
.ok_or_else(|| crate::scalar::wire_err("checkpoint envelope: payload overflows"))?;
|
||||
if end > footer_offset as u64 {
|
||||
return err(format!(
|
||||
"checkpoint envelope: payload {name:?} runs past the payload area"
|
||||
));
|
||||
}
|
||||
let payload = bytes[offset as usize..end as usize].to_vec();
|
||||
let computed: [u8; 32] = Sha256::digest(&payload).into();
|
||||
if computed != digest {
|
||||
return err(format!(
|
||||
"checkpoint envelope: payload {name:?} fails its digest"
|
||||
));
|
||||
}
|
||||
previous_end = align_up(end);
|
||||
entries.push(PayloadEntry {
|
||||
name: name.clone(),
|
||||
offset,
|
||||
byte_length,
|
||||
digest,
|
||||
});
|
||||
payloads.push((name, payload));
|
||||
}
|
||||
crate::scalar::require_unique(
|
||||
entries.iter().map(|e| e.name.as_str()),
|
||||
"checkpoint envelope: payload names",
|
||||
)?;
|
||||
if previous_end != footer_offset as u64 {
|
||||
return err("checkpoint envelope: padding between the last payload and the footer");
|
||||
}
|
||||
Ok(Envelope {
|
||||
manifest,
|
||||
layout: Layout {
|
||||
manifest_offset: manifest_start as u64,
|
||||
manifest_bytes: manifest_bytes as u32,
|
||||
table_offset,
|
||||
entries,
|
||||
footer_offset: footer_offset as u64,
|
||||
total_bytes: bytes.len() as u64,
|
||||
},
|
||||
payloads,
|
||||
})
|
||||
}
|
||||
|
||||
/// The manifest fields state-media-v1 section 4 requires, checked as a set: a manifest that
|
||||
/// omits one of them is not a complete checkpoint.
|
||||
pub const REQUIRED_MANIFEST_FIELDS: &[&str] = &[
|
||||
"envelopeVersion",
|
||||
"checkpointId",
|
||||
"sourceScope",
|
||||
"episodeId",
|
||||
"worldTime",
|
||||
"schedulerId",
|
||||
"compositionDigest",
|
||||
"portMap",
|
||||
"compatibility",
|
||||
"agents",
|
||||
"coordinator",
|
||||
"payloads",
|
||||
];
|
||||
|
||||
/// Checks the manifest's required field set and that its payload table mirrors the envelope's.
|
||||
pub fn validate_manifest(envelope: &Envelope) -> Result<()> {
|
||||
let map = envelope
|
||||
.manifest
|
||||
.as_object()
|
||||
.ok_or_else(|| crate::scalar::wire_err("checkpoint manifest: must be an object"))?;
|
||||
for field in REQUIRED_MANIFEST_FIELDS {
|
||||
if !map.contains_key(*field) {
|
||||
return err(format!("checkpoint manifest: missing {field:?}"));
|
||||
}
|
||||
}
|
||||
if map.get("envelopeVersion").and_then(Value::as_u64) != Some(u64::from(VERSION)) {
|
||||
return err("checkpoint manifest: envelopeVersion must be 1");
|
||||
}
|
||||
let listed = map
|
||||
.get("payloads")
|
||||
.and_then(Value::as_array)
|
||||
.ok_or_else(|| crate::scalar::wire_err("checkpoint manifest: payloads must be an array"))?;
|
||||
if listed.len() != envelope.layout.entries.len() {
|
||||
return err("checkpoint manifest: payloads does not match the payload table");
|
||||
}
|
||||
for (declared, entry) in listed.iter().zip(&envelope.layout.entries) {
|
||||
let name = declared.get("name").and_then(Value::as_str);
|
||||
let length = declared
|
||||
.get("byteLength")
|
||||
.and_then(Value::as_str)
|
||||
.and_then(crate::scalar::parse_u64);
|
||||
let digest = declared.get("digest").and_then(Value::as_str);
|
||||
if name != Some(entry.name.as_str()) {
|
||||
return err("checkpoint manifest: payload name does not match the table");
|
||||
}
|
||||
if length != Some(entry.byte_length) {
|
||||
return err(format!(
|
||||
"checkpoint manifest: payload {:?} byteLength does not match the table",
|
||||
entry.name
|
||||
));
|
||||
}
|
||||
if digest != Some(hex(&entry.digest).as_str()) {
|
||||
return err(format!(
|
||||
"checkpoint manifest: payload {:?} digest does not match the table",
|
||||
entry.name
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Lowercase hex of a raw digest, the form the manifest records.
|
||||
pub fn hex(bytes: &[u8]) -> String {
|
||||
let mut out = String::with_capacity(bytes.len() * 2);
|
||||
for byte in bytes {
|
||||
out.push_str(&format!("{byte:02x}"));
|
||||
}
|
||||
out
|
||||
}
|
||||
108
services/flysim/crates/fly-session-types/src/fixtures.rs
Normal file
108
services/flysim/crates/fly-session-types/src/fixtures.rs
Normal file
|
|
@ -0,0 +1,108 @@
|
|||
//! Loading the crate's `fixtures/` directory.
|
||||
//!
|
||||
//! The same files are read by the Rust tests and by `packages/session-types`, so a case only
|
||||
//! has to be written once to hold both languages to it.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::canonical;
|
||||
use crate::scalar::{Result, err, wire_err};
|
||||
|
||||
/// The crate's `fixtures/` directory.
|
||||
pub fn dir() -> PathBuf {
|
||||
Path::new(env!("CARGO_MANIFEST_DIR")).join("fixtures")
|
||||
}
|
||||
|
||||
/// Reads one fixture file, parsed strictly.
|
||||
pub fn load(name: &str) -> Result<Value> {
|
||||
let path = dir().join(name);
|
||||
let bytes =
|
||||
std::fs::read(&path).map_err(|e| wire_err(format!("fixture {}: {e}", path.display())))?;
|
||||
canonical::parse_strict(&bytes)
|
||||
}
|
||||
|
||||
/// Reads one fixture file as raw bytes, for the cases that are deliberately not valid JSON.
|
||||
pub fn load_bytes(name: &str) -> Result<Vec<u8>> {
|
||||
let path = dir().join(name);
|
||||
std::fs::read(&path).map_err(|e| wire_err(format!("fixture {}: {e}", path.display())))
|
||||
}
|
||||
|
||||
/// The `cases` array of a fixture file.
|
||||
pub fn cases(file: &Value) -> Result<&Vec<Value>> {
|
||||
match file.get("cases").and_then(Value::as_array) {
|
||||
Some(cases) if !cases.is_empty() => Ok(cases),
|
||||
_ => err("fixture: cases must be a nonempty array"),
|
||||
}
|
||||
}
|
||||
|
||||
/// A string field of one case.
|
||||
pub fn field<'a>(case: &'a Value, key: &str) -> Result<&'a str> {
|
||||
case.get(key)
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| wire_err(format!("fixture case: missing string field {key:?}")))
|
||||
}
|
||||
|
||||
/// Decodes the `base64` field of a case that carries raw bytes.
|
||||
pub fn base64(case: &Value, key: &str) -> Result<Vec<u8>> {
|
||||
decode_base64(field(case, key)?)
|
||||
}
|
||||
|
||||
/// Standard base64 with padding. Small and local: the crate has no base64 dependency and the
|
||||
/// fixtures only carry a few hundred bytes.
|
||||
pub fn decode_base64(text: &str) -> Result<Vec<u8>> {
|
||||
const ALPHABET: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
|
||||
let bytes = text.as_bytes();
|
||||
if !bytes.len().is_multiple_of(4) {
|
||||
return err("base64: length must be a multiple of 4");
|
||||
}
|
||||
let mut out = Vec::with_capacity(bytes.len() / 4 * 3);
|
||||
for quad in bytes.chunks_exact(4) {
|
||||
let mut buffer = 0u32;
|
||||
let mut keep = 3;
|
||||
for (index, byte) in quad.iter().enumerate() {
|
||||
let value = if *byte == b'=' {
|
||||
if index < 2 {
|
||||
return err("base64: misplaced padding");
|
||||
}
|
||||
keep -= 1;
|
||||
0
|
||||
} else {
|
||||
ALPHABET
|
||||
.iter()
|
||||
.position(|c| c == byte)
|
||||
.ok_or_else(|| wire_err("base64: invalid character"))? as u32
|
||||
};
|
||||
buffer = (buffer << 6) | value;
|
||||
}
|
||||
let triple = buffer.to_be_bytes();
|
||||
out.extend_from_slice(&triple[1..1 + keep]);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Standard base64 with padding, for generating fixtures.
|
||||
pub fn encode_base64(bytes: &[u8]) -> String {
|
||||
const ALPHABET: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
|
||||
let mut out = String::with_capacity(bytes.len().div_ceil(3) * 4);
|
||||
for chunk in bytes.chunks(3) {
|
||||
let mut buffer = [0u8; 3];
|
||||
buffer[..chunk.len()].copy_from_slice(chunk);
|
||||
let value = u32::from_be_bytes([0, buffer[0], buffer[1], buffer[2]]);
|
||||
let indexes = [
|
||||
(value >> 18) & 0x3f,
|
||||
(value >> 12) & 0x3f,
|
||||
(value >> 6) & 0x3f,
|
||||
value & 0x3f,
|
||||
];
|
||||
for (position, index) in indexes.iter().enumerate() {
|
||||
if position <= chunk.len() {
|
||||
out.push(ALPHABET[*index as usize] as char);
|
||||
} else {
|
||||
out.push('=');
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
56
services/flysim/crates/fly-session-types/src/lib.rs
Normal file
56
services/flysim/crates/fly-session-types/src/lib.rs
Normal file
|
|
@ -0,0 +1,56 @@
|
|||
//! `fly-session-types`: the executable schemas of the session framework (CONTRACT-01).
|
||||
//!
|
||||
//! What this crate is:
|
||||
//!
|
||||
//! - the domain scalars of [ipc-v1] section 2 ([`scalar`]), reusing the bus's `Id`, `U64` and
|
||||
//! `Digest` encodings rather than restating them;
|
||||
//! the domain request/reply envelope and error codes of sections 3 and 7 ([`rpc`]);
|
||||
//! - the closed enums and method payloads of [workers-v1] ([`workers`]), the native media
|
||||
//! and State.* payloads of [state-media-v1] ([`media`]), and the publication types of
|
||||
//! [publishing-v1] ([`publishing`]);
|
||||
//! - canonical JSON (RFC 8785), canonical digests, the operation key and the canonical body
|
||||
//! rules of ipc-v1 section 5 ([`canonical`]);
|
||||
//! - the documented canonical schema set and `contractDigest` ([`schema`]);
|
||||
//! - the trace format of [step-v1] section 8, with behaviour separated from operational
|
||||
//! metadata and a comparator over behaviour alone ([`trace`]);
|
||||
//! - `seed-derivation-v1` ([`seed`]) and the `FLYSESS1` checkpoint envelope layout
|
||||
//! ([`checkpoint`]), the two specifications CONTRACT-01 has to settle before the real-agent
|
||||
//! and store slices.
|
||||
//!
|
||||
//! What it is not: a transport, a worker, a coordinator or a store. It holds no Game Boy FFI,
|
||||
//! no Melee parser and no console-specific state, and it never reaches the network.
|
||||
//!
|
||||
//! Every type implements [`scalar::DomainType`]: `from_json` reads and validates, `to_json`
|
||||
//! writes the canonical shape, and `validate` re-checks the rules that span fields. Reading
|
||||
//! refuses unknown fields, so a payload with a misspelled required field fails instead of
|
||||
//! silently defaulting.
|
||||
//!
|
||||
//! [ipc-v1]: ../../../../docs/design/session-framework/ipc-v1.md
|
||||
//! [workers-v1]: ../../../../docs/design/session-framework/workers-v1.md
|
||||
//! [state-media-v1]: ../../../../docs/design/session-framework/state-media-v1.md
|
||||
//! [publishing-v1]: ../../../../docs/design/session-framework/publishing-v1.md
|
||||
//! [step-v1]: ../../../../docs/design/session-framework/step-v1.md
|
||||
|
||||
pub mod canonical;
|
||||
pub mod checkpoint;
|
||||
pub mod fixtures;
|
||||
pub mod media;
|
||||
pub mod publishing;
|
||||
pub mod rpc;
|
||||
pub mod scalar;
|
||||
pub mod schema;
|
||||
pub mod seed;
|
||||
pub mod trace;
|
||||
pub mod workers;
|
||||
|
||||
pub use canonical::{OperationKey, body_digest, canonicalize, digest_of};
|
||||
pub use scalar::{
|
||||
ArtifactIdentity, BusCallId, DomainRequestId, DomainType, OwnerKind, OwnerToken, RationalNs,
|
||||
SchemaRef, Scope, TypedValue,
|
||||
};
|
||||
pub use schema::contract_digest;
|
||||
pub use trace::{TraceBehaviour, TraceOperational, TransitionTrace};
|
||||
|
||||
/// The bus `ArtifactRef` these contracts reference. Re-exported so a consumer does not have
|
||||
/// to decide whether the domain has its own copy: it does not.
|
||||
pub use flybus::wire::ArtifactRef;
|
||||
716
services/flysim/crates/fly-session-types/src/media.rs
Normal file
716
services/flysim/crates/fly-session-types/src/media.rs
Normal file
|
|
@ -0,0 +1,716 @@
|
|||
//! Native observation media (state-media-v1 section 2) and the State.* payloads (section 5).
|
||||
//!
|
||||
//! Descriptors carry the shape; refs carry one produced object. Both are validated against
|
||||
//! the descriptor, because a ref on its own cannot know its own row stride: use
|
||||
//! [`ViewRef::validate_against`] and [`AudioRef::validate_against`] wherever the descriptor
|
||||
//! is in hand.
|
||||
|
||||
use flybus::wire::{ArtifactRef, Fields};
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::scalar::{
|
||||
DomainType, RationalNs, Result, Scope, constant, err, finite_in, is_digest, is_id, list, obj,
|
||||
require_unique, u64_json,
|
||||
};
|
||||
|
||||
/// Max views per sensory input (workers-v1 section 1). The same bound applies to a
|
||||
/// descriptor's view list and to an observation's view lists: a descriptor that declared more
|
||||
/// views than one sensory input can carry could not be satisfied.
|
||||
pub const MAX_VIEWS: usize = 8;
|
||||
/// View dimensions are integers 1..=4096 (state-media-v1 section 2).
|
||||
pub const MAX_VIEW_DIMENSION: u64 = 4096;
|
||||
/// Pixel aspect numerator/denominator are positive integers <=65535.
|
||||
pub const MAX_PIXEL_ASPECT: u64 = 65_535;
|
||||
/// observationDelaySteps is an integer 0..=8.
|
||||
pub const MAX_OBSERVATION_DELAY_STEPS: u64 = 8;
|
||||
/// sampleFrames is 0..=192000 per chunk; sampleRate is 8000..=192000.
|
||||
pub const MAX_SAMPLE_FRAMES: u64 = 192_000;
|
||||
/// Audio streams per descriptor. Not a stated bound: chosen so an envelope cannot be filled
|
||||
/// with descriptors, and recorded in the schema set so it cannot drift silently.
|
||||
pub const MAX_AUDIO_STREAMS: usize = 8;
|
||||
|
||||
/// `ViewDescriptor`: the fixed shape of one native view.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct ViewDescriptor {
|
||||
pub view_id: String,
|
||||
pub width: u64,
|
||||
pub height: u64,
|
||||
pub row_stride: u64,
|
||||
pub pixel_aspect_numerator: u64,
|
||||
pub pixel_aspect_denominator: u64,
|
||||
pub observation_delay_steps: u64,
|
||||
}
|
||||
|
||||
impl ViewDescriptor {
|
||||
/// The exact byte length of one frame of this view.
|
||||
pub fn frame_bytes(&self) -> u64 {
|
||||
self.row_stride * self.height
|
||||
}
|
||||
|
||||
/// The producing boundary a required sensory view must have at `boundary`
|
||||
/// (state-media-v1 section 2): `max(0, boundary - observationDelaySteps)`.
|
||||
pub fn required_produced_step(&self, boundary: u64) -> u64 {
|
||||
boundary.saturating_sub(self.observation_delay_steps)
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for ViewDescriptor {
|
||||
const TYPE_NAME: &'static str = "ViewDescriptor";
|
||||
|
||||
fn from_json(value: &Value) -> Result<ViewDescriptor> {
|
||||
let mut f = Fields::new(value, "ViewDescriptor")?;
|
||||
let view_id = f.id("viewId")?;
|
||||
let width = f.int("width", 1, MAX_VIEW_DIMENSION)?;
|
||||
let height = f.int("height", 1, MAX_VIEW_DIMENSION)?;
|
||||
constant(&mut f, "format", "rgba8")?;
|
||||
let row_stride = f.int("rowStride", 1, MAX_VIEW_DIMENSION * 4)?;
|
||||
let aspect = f.value("pixelAspect")?;
|
||||
let (pixel_aspect_numerator, pixel_aspect_denominator) = {
|
||||
let mut a = Fields::new(aspect, "ViewDescriptor.pixelAspect")?;
|
||||
let n = a.int("numerator", 1, MAX_PIXEL_ASPECT)?;
|
||||
let d = a.int("denominator", 1, MAX_PIXEL_ASPECT)?;
|
||||
a.finish()?;
|
||||
(n, d)
|
||||
};
|
||||
let observation_delay_steps =
|
||||
f.int("observationDelaySteps", 0, MAX_OBSERVATION_DELAY_STEPS)?;
|
||||
f.finish()?;
|
||||
let d = ViewDescriptor {
|
||||
view_id,
|
||||
width,
|
||||
height,
|
||||
row_stride,
|
||||
pixel_aspect_numerator,
|
||||
pixel_aspect_denominator,
|
||||
observation_delay_steps,
|
||||
};
|
||||
d.validate()?;
|
||||
Ok(d)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("viewId", self.view_id.clone().into()),
|
||||
("width", Value::from(self.width)),
|
||||
("height", Value::from(self.height)),
|
||||
("format", "rgba8".into()),
|
||||
("rowStride", Value::from(self.row_stride)),
|
||||
(
|
||||
"pixelAspect",
|
||||
obj(vec![
|
||||
("numerator", Value::from(self.pixel_aspect_numerator)),
|
||||
("denominator", Value::from(self.pixel_aspect_denominator)),
|
||||
]),
|
||||
),
|
||||
(
|
||||
"observationDelaySteps",
|
||||
Value::from(self.observation_delay_steps),
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.view_id) {
|
||||
return err("ViewDescriptor: viewId is not a valid id");
|
||||
}
|
||||
if !(1..=MAX_VIEW_DIMENSION).contains(&self.width)
|
||||
|| !(1..=MAX_VIEW_DIMENSION).contains(&self.height)
|
||||
{
|
||||
return err("ViewDescriptor: width and height must be integers 1..=4096");
|
||||
}
|
||||
if self.row_stride != self.width * 4 {
|
||||
return err(
|
||||
"ViewDescriptor: rowStride must be exactly 4 x width (no padded rows in v1)",
|
||||
);
|
||||
}
|
||||
if !(1..=MAX_PIXEL_ASPECT).contains(&self.pixel_aspect_numerator)
|
||||
|| !(1..=MAX_PIXEL_ASPECT).contains(&self.pixel_aspect_denominator)
|
||||
{
|
||||
return err("ViewDescriptor: pixelAspect parts must be positive integers <=65535");
|
||||
}
|
||||
if self.observation_delay_steps > MAX_OBSERVATION_DELAY_STEPS {
|
||||
return err("ViewDescriptor: observationDelaySteps must be 0..=8");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// `ViewRef`: one produced frame of one view.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct ViewRef {
|
||||
pub view_id: String,
|
||||
pub produced_step: u64,
|
||||
pub pixels: ArtifactRef,
|
||||
}
|
||||
|
||||
impl ViewRef {
|
||||
/// Byte shape and producing boundary against the descriptor that declared this view.
|
||||
///
|
||||
/// `boundary` is the observation's boundary; a required sensory view must have been
|
||||
/// produced at exactly `max(0, boundary - observationDelaySteps)`.
|
||||
pub fn validate_against(
|
||||
&self,
|
||||
descriptor: &ViewDescriptor,
|
||||
boundary: Option<u64>,
|
||||
) -> Result<()> {
|
||||
if self.view_id != descriptor.view_id {
|
||||
return err(format!(
|
||||
"ViewRef: viewId {:?} does not match descriptor {:?}",
|
||||
self.view_id, descriptor.view_id
|
||||
));
|
||||
}
|
||||
if self.pixels.byte_length != descriptor.frame_bytes() {
|
||||
return err(format!(
|
||||
"ViewRef {}: artifact is {} bytes, rowStride x height is {}",
|
||||
self.view_id,
|
||||
self.pixels.byte_length,
|
||||
descriptor.frame_bytes()
|
||||
));
|
||||
}
|
||||
if let Some(boundary) = boundary {
|
||||
let expected = descriptor.required_produced_step(boundary);
|
||||
if self.produced_step != expected {
|
||||
return err(format!(
|
||||
"ViewRef {}: producedStep {} must be max(0, {boundary} - {}) = {expected}",
|
||||
self.view_id, self.produced_step, descriptor.observation_delay_steps
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for ViewRef {
|
||||
const TYPE_NAME: &'static str = "ViewRef";
|
||||
|
||||
fn from_json(value: &Value) -> Result<ViewRef> {
|
||||
let mut f = Fields::new(value, "ViewRef")?;
|
||||
let view_id = f.id("viewId")?;
|
||||
let produced_step = f.u64_string("producedStep")?;
|
||||
let pixels = ArtifactRef::from_json(f.value("pixels")?)?;
|
||||
f.finish()?;
|
||||
let r = ViewRef {
|
||||
view_id,
|
||||
produced_step,
|
||||
pixels,
|
||||
};
|
||||
r.validate()?;
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("viewId", self.view_id.clone().into()),
|
||||
("producedStep", u64_json(self.produced_step)),
|
||||
("pixels", self.pixels.to_json()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.view_id) {
|
||||
return err("ViewRef: viewId is not a valid id");
|
||||
}
|
||||
if self.pixels.byte_length == 0 {
|
||||
return err("ViewRef: pixels must have a positive byte length");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// `AudioDescriptor`: one native audio stream's shape.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct AudioDescriptor {
|
||||
pub stream_id: String,
|
||||
pub sample_rate: u64,
|
||||
pub channels: u64,
|
||||
}
|
||||
|
||||
impl AudioDescriptor {
|
||||
/// The exact byte length of `frames` interleaved f32 frames.
|
||||
pub fn chunk_bytes(&self, frames: u64) -> u64 {
|
||||
frames * self.channels * 4
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for AudioDescriptor {
|
||||
const TYPE_NAME: &'static str = "AudioDescriptor";
|
||||
|
||||
fn from_json(value: &Value) -> Result<AudioDescriptor> {
|
||||
let mut f = Fields::new(value, "AudioDescriptor")?;
|
||||
let stream_id = f.id("streamId")?;
|
||||
let sample_rate = f.int("sampleRate", 8_000, 192_000)?;
|
||||
let channels = f.int("channels", 1, 8)?;
|
||||
constant(&mut f, "format", "f32le-interleaved")?;
|
||||
f.finish()?;
|
||||
let d = AudioDescriptor {
|
||||
stream_id,
|
||||
sample_rate,
|
||||
channels,
|
||||
};
|
||||
d.validate()?;
|
||||
Ok(d)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("streamId", self.stream_id.clone().into()),
|
||||
("sampleRate", Value::from(self.sample_rate)),
|
||||
("channels", Value::from(self.channels)),
|
||||
("format", "f32le-interleaved".into()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.stream_id) {
|
||||
return err("AudioDescriptor: streamId is not a valid id");
|
||||
}
|
||||
if !(8_000..=192_000).contains(&self.sample_rate) {
|
||||
return err("AudioDescriptor: sampleRate must be an integer 8000..=192000");
|
||||
}
|
||||
if !(1..=8).contains(&self.channels) {
|
||||
return err("AudioDescriptor: channels must be an integer 1..=8");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// `AudioRef`: one produced chunk of one audio stream.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct AudioRef {
|
||||
pub stream_id: String,
|
||||
pub first_sample: u64,
|
||||
pub sample_frames: u64,
|
||||
pub samples: ArtifactRef,
|
||||
pub discontinuity: bool,
|
||||
}
|
||||
|
||||
impl AudioRef {
|
||||
/// Byte shape against the descriptor that declared this stream.
|
||||
pub fn validate_against(&self, descriptor: &AudioDescriptor) -> Result<()> {
|
||||
if self.stream_id != descriptor.stream_id {
|
||||
return err(format!(
|
||||
"AudioRef: streamId {:?} does not match descriptor {:?}",
|
||||
self.stream_id, descriptor.stream_id
|
||||
));
|
||||
}
|
||||
let expected = descriptor.chunk_bytes(self.sample_frames);
|
||||
if self.samples.byte_length != expected {
|
||||
return err(format!(
|
||||
"AudioRef {}: artifact is {} bytes, sampleFrames x channels x 4 is {expected}",
|
||||
self.stream_id, self.samples.byte_length
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Within an epoch chunks cannot overlap or go backwards (state-media-v1 section 2).
|
||||
pub fn follows(&self, previous: &AudioRef) -> Result<()> {
|
||||
if self.stream_id != previous.stream_id {
|
||||
return err("AudioRef: chunks of different streams are not ordered against each other");
|
||||
}
|
||||
let expected = previous.first_sample + previous.sample_frames;
|
||||
if self.first_sample < expected {
|
||||
return err(format!(
|
||||
"AudioRef {}: firstSample {} overlaps the previous chunk, which ends at {expected}",
|
||||
self.stream_id, self.first_sample
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for AudioRef {
|
||||
const TYPE_NAME: &'static str = "AudioRef";
|
||||
|
||||
fn from_json(value: &Value) -> Result<AudioRef> {
|
||||
let mut f = Fields::new(value, "AudioRef")?;
|
||||
let stream_id = f.id("streamId")?;
|
||||
let first_sample = f.u64_string("firstSample")?;
|
||||
let sample_frames = f.int("sampleFrames", 0, MAX_SAMPLE_FRAMES)?;
|
||||
let samples = ArtifactRef::from_json(f.value("samples")?)?;
|
||||
let discontinuity = f.boolean("discontinuity")?;
|
||||
f.finish()?;
|
||||
let r = AudioRef {
|
||||
stream_id,
|
||||
first_sample,
|
||||
sample_frames,
|
||||
samples,
|
||||
discontinuity,
|
||||
};
|
||||
r.validate()?;
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("streamId", self.stream_id.clone().into()),
|
||||
("firstSample", u64_json(self.first_sample)),
|
||||
("sampleFrames", Value::from(self.sample_frames)),
|
||||
("samples", self.samples.to_json()),
|
||||
("discontinuity", Value::Bool(self.discontinuity)),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.stream_id) {
|
||||
return err("AudioRef: streamId is not a valid id");
|
||||
}
|
||||
if self.sample_frames > MAX_SAMPLE_FRAMES {
|
||||
return err("AudioRef: sampleFrames must be an integer 0..=192000");
|
||||
}
|
||||
if self.first_sample.checked_add(self.sample_frames).is_none() {
|
||||
return err("AudioRef: firstSample + sampleFrames overflows U64");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads a bounded, unique-by-`viewId` list of view refs.
|
||||
pub fn view_list(f: &mut Fields<'_>, key: &'static str) -> Result<Vec<ViewRef>> {
|
||||
let views = list(f, key, 0, MAX_VIEWS, ViewRef::from_json)?;
|
||||
require_unique(views.iter().map(|v| v.view_id.as_str()), key)?;
|
||||
Ok(views)
|
||||
}
|
||||
|
||||
/// Reads a bounded, unique-by-`streamId` list of audio refs.
|
||||
pub fn audio_list(f: &mut Fields<'_>, key: &'static str) -> Result<Vec<AudioRef>> {
|
||||
let audio = list(f, key, 0, MAX_AUDIO_STREAMS, AudioRef::from_json)?;
|
||||
require_unique(audio.iter().map(|a| a.stream_id.as_str()), key)?;
|
||||
Ok(audio)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// State.* payloads (state-media-v1 section 5)
|
||||
|
||||
/// A checkpoint payload artifact: the digest is mandatory on checkpoint payloads
|
||||
/// (state-media-v1 section 1).
|
||||
fn checkpoint_payload(f: &mut Fields<'_>, key: &'static str) -> Result<ArtifactRef> {
|
||||
let reference = ArtifactRef::from_json(f.value(key)?)?;
|
||||
match &reference.digest {
|
||||
Some(d) if is_digest(d) => Ok(reference),
|
||||
_ => err(format!(
|
||||
"{key}: a checkpoint payload must carry a content digest"
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
/// `State.Capture` params.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct CaptureParams {
|
||||
pub checkpoint_id: String,
|
||||
}
|
||||
|
||||
impl DomainType for CaptureParams {
|
||||
const TYPE_NAME: &'static str = "CaptureParams";
|
||||
|
||||
fn from_json(value: &Value) -> Result<CaptureParams> {
|
||||
let mut f = Fields::new(value, "CaptureParams")?;
|
||||
let checkpoint_id = f.id("checkpointId")?;
|
||||
f.finish()?;
|
||||
let p = CaptureParams { checkpoint_id };
|
||||
p.validate()?;
|
||||
Ok(p)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![("checkpointId", self.checkpoint_id.clone().into())])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.checkpoint_id) {
|
||||
return err("CaptureParams: checkpointId is not a valid id");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// `State.Capture` result.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct CaptureResult {
|
||||
pub checkpoint_id: String,
|
||||
pub boundary: u64,
|
||||
pub compatibility_digest: String,
|
||||
pub payload: ArtifactRef,
|
||||
}
|
||||
|
||||
impl DomainType for CaptureResult {
|
||||
const TYPE_NAME: &'static str = "CaptureResult";
|
||||
|
||||
fn from_json(value: &Value) -> Result<CaptureResult> {
|
||||
let mut f = Fields::new(value, "CaptureResult")?;
|
||||
let checkpoint_id = f.id("checkpointId")?;
|
||||
let boundary = f.u64_string("boundary")?;
|
||||
let compatibility_digest = f.string("compatibilityDigest")?.to_owned();
|
||||
let payload = checkpoint_payload(&mut f, "payload")?;
|
||||
f.finish()?;
|
||||
let r = CaptureResult {
|
||||
checkpoint_id,
|
||||
boundary,
|
||||
compatibility_digest,
|
||||
payload,
|
||||
};
|
||||
r.validate()?;
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("checkpointId", self.checkpoint_id.clone().into()),
|
||||
("boundary", u64_json(self.boundary)),
|
||||
(
|
||||
"compatibilityDigest",
|
||||
self.compatibility_digest.clone().into(),
|
||||
),
|
||||
("payload", self.payload.to_json()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.checkpoint_id) {
|
||||
return err("CaptureResult: checkpointId is not a valid id");
|
||||
}
|
||||
if !is_digest(&self.compatibility_digest) {
|
||||
return err("CaptureResult: compatibilityDigest must be 64 lowercase hex digits");
|
||||
}
|
||||
match &self.payload.digest {
|
||||
Some(d) if is_digest(d) => Ok(()),
|
||||
_ => err("CaptureResult: a checkpoint payload must carry a content digest"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `State.StageRestore` params. The scope is the source boundary, under a proposed new epoch.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct StageRestoreParams {
|
||||
pub checkpoint_id: String,
|
||||
pub source_scope: Scope,
|
||||
pub compatibility_digest: String,
|
||||
pub payload: ArtifactRef,
|
||||
}
|
||||
|
||||
impl DomainType for StageRestoreParams {
|
||||
const TYPE_NAME: &'static str = "StageRestoreParams";
|
||||
|
||||
fn from_json(value: &Value) -> Result<StageRestoreParams> {
|
||||
let mut f = Fields::new(value, "StageRestoreParams")?;
|
||||
let checkpoint_id = f.id("checkpointId")?;
|
||||
let source_scope = Scope::from_json(f.value("sourceScope")?)?;
|
||||
let compatibility_digest = f.string("compatibilityDigest")?.to_owned();
|
||||
let payload = checkpoint_payload(&mut f, "payload")?;
|
||||
f.finish()?;
|
||||
let p = StageRestoreParams {
|
||||
checkpoint_id,
|
||||
source_scope,
|
||||
compatibility_digest,
|
||||
payload,
|
||||
};
|
||||
p.validate()?;
|
||||
Ok(p)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("checkpointId", self.checkpoint_id.clone().into()),
|
||||
("sourceScope", self.source_scope.to_json()),
|
||||
(
|
||||
"compatibilityDigest",
|
||||
self.compatibility_digest.clone().into(),
|
||||
),
|
||||
("payload", self.payload.to_json()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.checkpoint_id) {
|
||||
return err("StageRestoreParams: checkpointId is not a valid id");
|
||||
}
|
||||
self.source_scope.validate()?;
|
||||
if !is_digest(&self.compatibility_digest) {
|
||||
return err("StageRestoreParams: compatibilityDigest must be 64 lowercase hex digits");
|
||||
}
|
||||
match &self.payload.digest {
|
||||
Some(d) if is_digest(d) => Ok(()),
|
||||
_ => err("StageRestoreParams: a checkpoint payload must carry a content digest"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `State.StageRestore` result.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct StageRestoreResult {
|
||||
pub checkpoint_id: String,
|
||||
pub restore_token: String,
|
||||
}
|
||||
|
||||
impl DomainType for StageRestoreResult {
|
||||
const TYPE_NAME: &'static str = "StageRestoreResult";
|
||||
|
||||
fn from_json(value: &Value) -> Result<StageRestoreResult> {
|
||||
let mut f = Fields::new(value, "StageRestoreResult")?;
|
||||
let checkpoint_id = f.id("checkpointId")?;
|
||||
let restore_token = f.id("restoreToken")?;
|
||||
f.finish()?;
|
||||
let r = StageRestoreResult {
|
||||
checkpoint_id,
|
||||
restore_token,
|
||||
};
|
||||
r.validate()?;
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("checkpointId", self.checkpoint_id.clone().into()),
|
||||
("restoreToken", self.restore_token.clone().into()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.checkpoint_id) || !is_id(&self.restore_token) {
|
||||
return err("StageRestoreResult: checkpointId and restoreToken must be valid ids");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// `State.ActivateRestore` params.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct ActivateRestoreParams {
|
||||
pub restore_token: String,
|
||||
}
|
||||
|
||||
impl DomainType for ActivateRestoreParams {
|
||||
const TYPE_NAME: &'static str = "ActivateRestoreParams";
|
||||
|
||||
fn from_json(value: &Value) -> Result<ActivateRestoreParams> {
|
||||
let mut f = Fields::new(value, "ActivateRestoreParams")?;
|
||||
let restore_token = f.id("restoreToken")?;
|
||||
f.finish()?;
|
||||
let p = ActivateRestoreParams { restore_token };
|
||||
p.validate()?;
|
||||
Ok(p)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![("restoreToken", self.restore_token.clone().into())])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.restore_token) {
|
||||
return err("ActivateRestoreParams: restoreToken is not a valid id");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// `State.ActivateRestore` result. The observation is required from an environment and null
|
||||
/// from an agent (state-media-v1 section 5); which one applies is the caller's role, so the
|
||||
/// role-specific check is [`ActivateRestoreResult::validate_for_role`].
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct ActivateRestoreResult {
|
||||
pub committed_step: u64,
|
||||
pub checkpoint_id: String,
|
||||
pub observation: Option<crate::workers::WorldObservation>,
|
||||
}
|
||||
|
||||
impl ActivateRestoreResult {
|
||||
pub fn validate_for_role(&self, role: crate::workers::Role) -> Result<()> {
|
||||
self.validate()?;
|
||||
match (role, &self.observation) {
|
||||
(crate::workers::Role::Environment, None) => {
|
||||
err("ActivateRestoreResult: an environment must return its restored observation")
|
||||
}
|
||||
(crate::workers::Role::Agent, Some(_)) => {
|
||||
err("ActivateRestoreResult: an agent returns a null observation")
|
||||
}
|
||||
_ => Ok(()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for ActivateRestoreResult {
|
||||
const TYPE_NAME: &'static str = "ActivateRestoreResult";
|
||||
|
||||
fn from_json(value: &Value) -> Result<ActivateRestoreResult> {
|
||||
let mut f = Fields::new(value, "ActivateRestoreResult")?;
|
||||
let committed_step = f.u64_string("committedStep")?;
|
||||
let checkpoint_id = f.id("checkpointId")?;
|
||||
let observation = match f.value("observation")? {
|
||||
Value::Null => None,
|
||||
v => Some(crate::workers::WorldObservation::from_json(v)?),
|
||||
};
|
||||
f.finish()?;
|
||||
let r = ActivateRestoreResult {
|
||||
committed_step,
|
||||
checkpoint_id,
|
||||
observation,
|
||||
};
|
||||
r.validate()?;
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("committedStep", u64_json(self.committed_step)),
|
||||
("checkpointId", self.checkpoint_id.clone().into()),
|
||||
(
|
||||
"observation",
|
||||
self.observation
|
||||
.as_ref()
|
||||
.map_or(Value::Null, |o| o.to_json()),
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.checkpoint_id) {
|
||||
return err("ActivateRestoreResult: checkpointId is not a valid id");
|
||||
}
|
||||
if let Some(observation) = &self.observation {
|
||||
observation.validate()?;
|
||||
if observation.boundary != self.committed_step {
|
||||
return err(
|
||||
"ActivateRestoreResult: the observation boundary must be the committed step",
|
||||
);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// The pixel aspect of a view as a rational, for presentation.
|
||||
pub fn pixel_aspect(descriptor: &ViewDescriptor) -> Result<RationalNs> {
|
||||
RationalNs::reduced(
|
||||
u128::from(descriptor.pixel_aspect_numerator),
|
||||
u128::from(descriptor.pixel_aspect_denominator),
|
||||
)
|
||||
}
|
||||
|
||||
/// Audio presentation timestamp, `firstSample / sampleRate` seconds, as a checked rational.
|
||||
pub fn audio_pts(reference: &AudioRef, descriptor: &AudioDescriptor) -> Result<RationalNs> {
|
||||
RationalNs::reduced(
|
||||
u128::from(reference.first_sample),
|
||||
u128::from(descriptor.sample_rate),
|
||||
)
|
||||
}
|
||||
|
||||
/// Samples must be finite f32 (state-media-v1 section 2). The bytes live in an artifact, so
|
||||
/// this is the check a reader runs over a mapped chunk.
|
||||
pub fn require_finite_samples(bytes: &[u8]) -> Result<()> {
|
||||
if !bytes.len().is_multiple_of(4) {
|
||||
return err("audio chunk: length must be a multiple of 4");
|
||||
}
|
||||
for (index, chunk) in bytes.chunks_exact(4).enumerate() {
|
||||
let sample = f32::from_le_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]);
|
||||
if !sample.is_finite() {
|
||||
return err(format!("audio chunk: sample {index} is not finite"));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A unit-range helper for presentation code that needs the neutral-in-range rule.
|
||||
pub fn require_unit(f: &mut Fields<'_>, key: &'static str) -> Result<f64> {
|
||||
finite_in(f, key, 0.0, 1.0)
|
||||
}
|
||||
450
services/flysim/crates/fly-session-types/src/publishing.rs
Normal file
450
services/flysim/crates/fly-session-types/src/publishing.rs
Normal file
|
|
@ -0,0 +1,450 @@
|
|||
//! The publication types of publishing-v1 section 3.
|
||||
//!
|
||||
//! A descriptor changes rarely and a snapshot changes every boundary; both are published on
|
||||
//! the same bus, and a snapshot names the descriptor revision it was shaped by.
|
||||
|
||||
use flybus::wire::Fields;
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::media::{AudioRef, MAX_VIEWS, ViewRef, audio_list, view_list};
|
||||
use crate::scalar::{
|
||||
DomainType, RationalNs, Result, SchemaRef, Scope, TypedValue, constant, err, id_list,
|
||||
is_digest, is_id, list, obj, require_unique, u64_json,
|
||||
};
|
||||
use crate::workers::{
|
||||
AgentTelemetry, AssetRef, EnvironmentDescriptor, MAX_AGENTS, MAX_RATE_ROLES, PortControl,
|
||||
};
|
||||
|
||||
/// Declared stimulus kinds per agent. Not a stated bound; recorded in the schema set.
|
||||
pub const MAX_SUPPORTED_STIMULI: usize = 64;
|
||||
/// Installed assets in one descriptor. Not a stated bound; recorded in the schema set.
|
||||
pub const MAX_ASSETS: usize = 64;
|
||||
/// Scoped event ids in one snapshot. Not a stated bound; recorded in the schema set.
|
||||
pub const MAX_SNAPSHOT_EVENTS: usize = 64;
|
||||
|
||||
/// One agent's place in the composition.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct AgentDescriptor {
|
||||
pub agent_id: String,
|
||||
pub port_id: String,
|
||||
pub profile_digest: String,
|
||||
pub dataset_digest: String,
|
||||
pub index_digest: String,
|
||||
pub neuron_count: u64,
|
||||
pub rate_roles: Vec<String>,
|
||||
pub supported_stimuli: Vec<String>,
|
||||
}
|
||||
|
||||
/// `SessionDescriptor`: the framework shape of one running session.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct SessionDescriptor {
|
||||
pub session_id: String,
|
||||
pub revision: u64,
|
||||
pub composition_digest: String,
|
||||
pub environment: EnvironmentDescriptor,
|
||||
pub task_schema: SchemaRef,
|
||||
pub agents: Vec<AgentDescriptor>,
|
||||
pub assets: Vec<AssetRef>,
|
||||
}
|
||||
|
||||
impl DomainType for SessionDescriptor {
|
||||
const TYPE_NAME: &'static str = "SessionDescriptor";
|
||||
|
||||
fn from_json(value: &Value) -> Result<SessionDescriptor> {
|
||||
let mut f = Fields::new(value, "SessionDescriptor")?;
|
||||
let session_id = f.id("sessionId")?;
|
||||
let revision = f.u64_string("revision")?;
|
||||
let composition_digest = f.string("compositionDigest")?.to_owned();
|
||||
constant(&mut f, "schedulerId", "lockstep-v1")?;
|
||||
let environment = EnvironmentDescriptor::from_json(f.value("environment")?)?;
|
||||
let task_schema = SchemaRef::from_json(f.value("taskSchema")?)?;
|
||||
let agents = list(&mut f, "agents", 1, MAX_AGENTS, |v| {
|
||||
let mut a = Fields::new(v, "SessionDescriptor.agents")?;
|
||||
let agent_id = a.id("agentId")?;
|
||||
let port_id = a.id("portId")?;
|
||||
let profile_digest = a.string("profileDigest")?.to_owned();
|
||||
let dataset_digest = a.string("datasetDigest")?.to_owned();
|
||||
let index_digest = a.string("indexDigest")?.to_owned();
|
||||
let neuron_count = a.u64_string("neuronCount")?;
|
||||
let rate_roles = id_list(&mut a, "rateRoles", 0, MAX_RATE_ROLES)?;
|
||||
let supported_stimuli = id_list(&mut a, "supportedStimuli", 0, MAX_SUPPORTED_STIMULI)?;
|
||||
a.finish()?;
|
||||
Ok(AgentDescriptor {
|
||||
agent_id,
|
||||
port_id,
|
||||
profile_digest,
|
||||
dataset_digest,
|
||||
index_digest,
|
||||
neuron_count,
|
||||
rate_roles,
|
||||
supported_stimuli,
|
||||
})
|
||||
})?;
|
||||
let assets = list(&mut f, "assets", 0, MAX_ASSETS, AssetRef::from_json)?;
|
||||
f.finish()?;
|
||||
let d = SessionDescriptor {
|
||||
session_id,
|
||||
revision,
|
||||
composition_digest,
|
||||
environment,
|
||||
task_schema,
|
||||
agents,
|
||||
assets,
|
||||
};
|
||||
d.validate()?;
|
||||
Ok(d)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("sessionId", self.session_id.clone().into()),
|
||||
("revision", u64_json(self.revision)),
|
||||
("compositionDigest", self.composition_digest.clone().into()),
|
||||
("schedulerId", "lockstep-v1".into()),
|
||||
("environment", self.environment.to_json()),
|
||||
("taskSchema", self.task_schema.to_json()),
|
||||
(
|
||||
"agents",
|
||||
Value::Array(
|
||||
self.agents
|
||||
.iter()
|
||||
.map(|a| {
|
||||
obj(vec![
|
||||
("agentId", a.agent_id.clone().into()),
|
||||
("portId", a.port_id.clone().into()),
|
||||
("profileDigest", a.profile_digest.clone().into()),
|
||||
("datasetDigest", a.dataset_digest.clone().into()),
|
||||
("indexDigest", a.index_digest.clone().into()),
|
||||
("neuronCount", u64_json(a.neuron_count)),
|
||||
(
|
||||
"rateRoles",
|
||||
Value::Array(
|
||||
a.rate_roles.iter().map(|r| r.clone().into()).collect(),
|
||||
),
|
||||
),
|
||||
(
|
||||
"supportedStimuli",
|
||||
Value::Array(
|
||||
a.supported_stimuli
|
||||
.iter()
|
||||
.map(|s| s.clone().into())
|
||||
.collect(),
|
||||
),
|
||||
),
|
||||
])
|
||||
})
|
||||
.collect(),
|
||||
),
|
||||
),
|
||||
(
|
||||
"assets",
|
||||
Value::Array(self.assets.iter().map(AssetRef::to_json).collect()),
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.session_id) {
|
||||
return err("SessionDescriptor: sessionId is not a valid id");
|
||||
}
|
||||
if !is_digest(&self.composition_digest) {
|
||||
return err("SessionDescriptor: compositionDigest must be 64 lowercase hex digits");
|
||||
}
|
||||
self.environment.validate()?;
|
||||
self.task_schema.validate()?;
|
||||
if self.agents.is_empty() || self.agents.len() > MAX_AGENTS {
|
||||
return err("SessionDescriptor: 1..=4 agents in the first composition");
|
||||
}
|
||||
require_unique(
|
||||
self.agents.iter().map(|a| a.agent_id.as_str()),
|
||||
"SessionDescriptor.agents agentId",
|
||||
)?;
|
||||
require_unique(
|
||||
self.agents.iter().map(|a| a.port_id.as_str()),
|
||||
"SessionDescriptor.agents portId",
|
||||
)?;
|
||||
for agent in &self.agents {
|
||||
if !is_id(&agent.agent_id) || !is_id(&agent.port_id) {
|
||||
return err("SessionDescriptor: agentId and portId must be valid ids");
|
||||
}
|
||||
for (what, digest) in [
|
||||
("profileDigest", &agent.profile_digest),
|
||||
("datasetDigest", &agent.dataset_digest),
|
||||
("indexDigest", &agent.index_digest),
|
||||
] {
|
||||
if !is_digest(digest) {
|
||||
return err(format!(
|
||||
"SessionDescriptor: agent {what} must be 64 lowercase hex digits"
|
||||
));
|
||||
}
|
||||
}
|
||||
if agent.rate_roles.len() > MAX_RATE_ROLES {
|
||||
return err("SessionDescriptor: at most 64 rate roles per agent");
|
||||
}
|
||||
require_unique(
|
||||
agent.rate_roles.iter().map(String::as_str),
|
||||
"SessionDescriptor.agents rateRoles",
|
||||
)?;
|
||||
require_unique(
|
||||
agent.supported_stimuli.iter().map(String::as_str),
|
||||
"SessionDescriptor.agents supportedStimuli",
|
||||
)?;
|
||||
if self.environment.port(&agent.port_id).is_none() {
|
||||
return err(format!(
|
||||
"SessionDescriptor: agent {:?} is bound to port {:?}, which the environment does not declare",
|
||||
agent.agent_id, agent.port_id
|
||||
));
|
||||
}
|
||||
}
|
||||
require_unique(
|
||||
self.assets.iter().map(|a| a.id.as_str()),
|
||||
"SessionDescriptor.assets",
|
||||
)?;
|
||||
for asset in &self.assets {
|
||||
asset.validate()?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// One agent's committed values in a snapshot.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct SnapshotAgent {
|
||||
pub agent_id: String,
|
||||
pub telemetry: AgentTelemetry,
|
||||
pub selected_decision: Option<TypedValue>,
|
||||
pub applied_controls: Option<PortControl>,
|
||||
}
|
||||
|
||||
/// `CommittedSnapshot`: the values of one committed boundary.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct CommittedSnapshot {
|
||||
pub descriptor_revision: u64,
|
||||
pub publisher_incarnation: String,
|
||||
pub scope: Scope,
|
||||
pub episode_id: String,
|
||||
pub sequence: u64,
|
||||
pub world_time: RationalNs,
|
||||
pub agents: Vec<SnapshotAgent>,
|
||||
pub progress: TypedValue,
|
||||
pub views: Vec<ViewRef>,
|
||||
pub audio: Vec<AudioRef>,
|
||||
pub event_ids: Vec<String>,
|
||||
}
|
||||
|
||||
impl CommittedSnapshot {
|
||||
/// Descriptor agreement: the revision, the agent set and the port each control names.
|
||||
pub fn validate_against(&self, descriptor: &SessionDescriptor) -> Result<()> {
|
||||
self.validate()?;
|
||||
if self.descriptor_revision != descriptor.revision {
|
||||
return err("CommittedSnapshot: descriptorRevision does not match the descriptor");
|
||||
}
|
||||
if self.scope.session_id != descriptor.session_id {
|
||||
return err("CommittedSnapshot: sessionId does not match the descriptor");
|
||||
}
|
||||
for agent in &self.agents {
|
||||
let declared = descriptor
|
||||
.agents
|
||||
.iter()
|
||||
.find(|a| a.agent_id == agent.agent_id)
|
||||
.ok_or_else(|| {
|
||||
crate::scalar::wire_err(format!(
|
||||
"CommittedSnapshot: agent {:?} is not in the descriptor",
|
||||
agent.agent_id
|
||||
))
|
||||
})?;
|
||||
agent
|
||||
.telemetry
|
||||
.validate_against_roles(&declared.rate_roles)?;
|
||||
if let Some(controls) = &agent.applied_controls {
|
||||
if controls.port_id != declared.port_id {
|
||||
return err(format!(
|
||||
"CommittedSnapshot: agent {:?} controls port {:?}, not its assigned {:?}",
|
||||
agent.agent_id, controls.port_id, declared.port_id
|
||||
));
|
||||
}
|
||||
let port = descriptor
|
||||
.environment
|
||||
.port(&declared.port_id)
|
||||
.ok_or_else(|| {
|
||||
crate::scalar::wire_err("CommittedSnapshot: assigned port is not declared")
|
||||
})?;
|
||||
controls.validate_against(&port.controls)?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for CommittedSnapshot {
|
||||
const TYPE_NAME: &'static str = "CommittedSnapshot";
|
||||
|
||||
fn from_json(value: &Value) -> Result<CommittedSnapshot> {
|
||||
let mut f = Fields::new(value, "CommittedSnapshot")?;
|
||||
let descriptor_revision = f.u64_string("descriptorRevision")?;
|
||||
let publisher_incarnation = f.id("publisherIncarnation")?;
|
||||
let scope = Scope::from_json(f.value("scope")?)?;
|
||||
let episode_id = f.id("episodeId")?;
|
||||
let sequence = f.u64_string("sequence")?;
|
||||
let world_time = RationalNs::from_json(f.value("worldTime")?)?;
|
||||
let agents = list(&mut f, "agents", 1, MAX_AGENTS, |v| {
|
||||
let mut a = Fields::new(v, "CommittedSnapshot.agents")?;
|
||||
let agent_id = a.id("agentId")?;
|
||||
let telemetry = AgentTelemetry::from_json(a.value("telemetry")?)?;
|
||||
let selected_decision = TypedValue::nullable_from_json(a.value("selectedDecision")?)?;
|
||||
let applied_controls = match a.value("appliedControls")? {
|
||||
Value::Null => None,
|
||||
v => Some(PortControl::from_json(v)?),
|
||||
};
|
||||
a.finish()?;
|
||||
Ok(SnapshotAgent {
|
||||
agent_id,
|
||||
telemetry,
|
||||
selected_decision,
|
||||
applied_controls,
|
||||
})
|
||||
})?;
|
||||
let progress = TypedValue::from_json(f.value("progress")?)?;
|
||||
let (views, audio) = {
|
||||
let v = f.value("media")?;
|
||||
let mut m = Fields::new(v, "CommittedSnapshot.media")?;
|
||||
let views = view_list(&mut m, "views")?;
|
||||
let audio = audio_list(&mut m, "audio")?;
|
||||
m.finish()?;
|
||||
(views, audio)
|
||||
};
|
||||
let event_ids = id_list(&mut f, "eventIds", 0, MAX_SNAPSHOT_EVENTS)?;
|
||||
f.finish()?;
|
||||
let s = CommittedSnapshot {
|
||||
descriptor_revision,
|
||||
publisher_incarnation,
|
||||
scope,
|
||||
episode_id,
|
||||
sequence,
|
||||
world_time,
|
||||
agents,
|
||||
progress,
|
||||
views,
|
||||
audio,
|
||||
event_ids,
|
||||
};
|
||||
s.validate()?;
|
||||
Ok(s)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("descriptorRevision", u64_json(self.descriptor_revision)),
|
||||
(
|
||||
"publisherIncarnation",
|
||||
self.publisher_incarnation.clone().into(),
|
||||
),
|
||||
("scope", self.scope.to_json()),
|
||||
("episodeId", self.episode_id.clone().into()),
|
||||
("sequence", u64_json(self.sequence)),
|
||||
("worldTime", self.world_time.to_json()),
|
||||
(
|
||||
"agents",
|
||||
Value::Array(
|
||||
self.agents
|
||||
.iter()
|
||||
.map(|a| {
|
||||
obj(vec![
|
||||
("agentId", a.agent_id.clone().into()),
|
||||
("telemetry", a.telemetry.to_json()),
|
||||
(
|
||||
"selectedDecision",
|
||||
TypedValue::nullable_to_json(a.selected_decision.as_ref()),
|
||||
),
|
||||
(
|
||||
"appliedControls",
|
||||
a.applied_controls
|
||||
.as_ref()
|
||||
.map_or(Value::Null, PortControl::to_json),
|
||||
),
|
||||
])
|
||||
})
|
||||
.collect(),
|
||||
),
|
||||
),
|
||||
("progress", self.progress.to_json()),
|
||||
(
|
||||
"media",
|
||||
obj(vec![
|
||||
(
|
||||
"views",
|
||||
Value::Array(self.views.iter().map(ViewRef::to_json).collect()),
|
||||
),
|
||||
(
|
||||
"audio",
|
||||
Value::Array(self.audio.iter().map(AudioRef::to_json).collect()),
|
||||
),
|
||||
]),
|
||||
),
|
||||
(
|
||||
"eventIds",
|
||||
Value::Array(self.event_ids.iter().map(|e| e.clone().into()).collect()),
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.publisher_incarnation) || !is_id(&self.episode_id) {
|
||||
return err("CommittedSnapshot: publisherIncarnation and episodeId must be valid ids");
|
||||
}
|
||||
self.scope.validate()?;
|
||||
self.world_time.validate()?;
|
||||
if self.agents.is_empty() || self.agents.len() > MAX_AGENTS {
|
||||
return err("CommittedSnapshot: 1..=4 agents");
|
||||
}
|
||||
require_unique(
|
||||
self.agents.iter().map(|a| a.agent_id.as_str()),
|
||||
"CommittedSnapshot.agents",
|
||||
)?;
|
||||
for agent in &self.agents {
|
||||
if !is_id(&agent.agent_id) {
|
||||
return err("CommittedSnapshot: agentId is not a valid id");
|
||||
}
|
||||
agent.telemetry.validate()?;
|
||||
if let Some(decision) = &agent.selected_decision {
|
||||
decision.validate()?;
|
||||
}
|
||||
if let Some(controls) = &agent.applied_controls {
|
||||
controls.validate()?;
|
||||
}
|
||||
// "Decisions/controls describe the transition ending at that boundary, null at
|
||||
// initial boundary 0." (publishing-v1 section 3)
|
||||
if self.scope.step == 0
|
||||
&& (agent.selected_decision.is_some() || agent.applied_controls.is_some())
|
||||
{
|
||||
return err(
|
||||
"CommittedSnapshot: at boundary 0 selectedDecision and appliedControls are null",
|
||||
);
|
||||
}
|
||||
if self.scope.step > 0
|
||||
&& (agent.selected_decision.is_none() || agent.applied_controls.is_none())
|
||||
{
|
||||
return err(
|
||||
"CommittedSnapshot: past boundary 0 every agent has a decision and applied controls",
|
||||
);
|
||||
}
|
||||
}
|
||||
self.progress.validate()?;
|
||||
if self.views.len() > MAX_VIEWS {
|
||||
return err("CommittedSnapshot: at most 8 views");
|
||||
}
|
||||
require_unique(
|
||||
self.views.iter().map(|v| v.view_id.as_str()),
|
||||
"CommittedSnapshot.media.views",
|
||||
)?;
|
||||
require_unique(
|
||||
self.audio.iter().map(|a| a.stream_id.as_str()),
|
||||
"CommittedSnapshot.media.audio",
|
||||
)?;
|
||||
require_unique(
|
||||
self.event_ids.iter().map(String::as_str),
|
||||
"CommittedSnapshot.eventIds",
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
420
services/flysim/crates/fly-session-types/src/rpc.rs
Normal file
420
services/flysim/crates/fly-session-types/src/rpc.rs
Normal file
|
|
@ -0,0 +1,420 @@
|
|||
//! The domain request/reply envelope of ipc-v1 section 3 and the error codes of section 7.
|
||||
//!
|
||||
//! A domain reply is the `outcome` object inside a bus `rpc.result`. Bus route or admission
|
||||
//! failure is not one of these: it never reaches a handler, so it cannot carry a mutation
|
||||
//! certainty.
|
||||
|
||||
use flybus::wire::Fields;
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::canonical;
|
||||
use crate::scalar::{
|
||||
DomainRequestId, DomainType, Result, Scope, bounded_string, constant, enumeration, err, is_id,
|
||||
obj,
|
||||
};
|
||||
use crate::workers::MAX_MESSAGE_CODE_POINTS;
|
||||
|
||||
/// The domain error codes of ipc-v1 section 7.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
|
||||
pub enum ErrorCode {
|
||||
/// Invalid schema/range, before mutation.
|
||||
InvalidArgument,
|
||||
/// Missing method or capability.
|
||||
Unsupported,
|
||||
/// Wrong session/profile/port/build/asset identity.
|
||||
IdentityMismatch,
|
||||
StaleEpoch,
|
||||
StaleStep,
|
||||
FutureStep,
|
||||
/// Wrong worker phase.
|
||||
InvalidPhase,
|
||||
/// Existing logical operation with a changed id or body.
|
||||
Conflict,
|
||||
/// The original operation is still executing; this duplicate bus call started no work.
|
||||
InProgress,
|
||||
/// Domain capacity unavailable before admission.
|
||||
Busy,
|
||||
/// Missing, unowned or mismatched artifact, or an invalid media shape.
|
||||
BufferInvalid,
|
||||
/// Safe replay is no longer available; never recompute to replace it.
|
||||
ResultExpired,
|
||||
/// Restore validation failed before activation.
|
||||
IncompatibleState,
|
||||
BackendFailure,
|
||||
Internal,
|
||||
}
|
||||
|
||||
impl ErrorCode {
|
||||
pub const ALL: &'static [&'static str] = &[
|
||||
"INVALID_ARGUMENT",
|
||||
"UNSUPPORTED",
|
||||
"IDENTITY_MISMATCH",
|
||||
"STALE_EPOCH",
|
||||
"STALE_STEP",
|
||||
"FUTURE_STEP",
|
||||
"INVALID_PHASE",
|
||||
"CONFLICT",
|
||||
"IN_PROGRESS",
|
||||
"BUSY",
|
||||
"BUFFER_INVALID",
|
||||
"RESULT_EXPIRED",
|
||||
"INCOMPATIBLE_STATE",
|
||||
"BACKEND_FAILURE",
|
||||
"INTERNAL",
|
||||
];
|
||||
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
ErrorCode::InvalidArgument => "INVALID_ARGUMENT",
|
||||
ErrorCode::Unsupported => "UNSUPPORTED",
|
||||
ErrorCode::IdentityMismatch => "IDENTITY_MISMATCH",
|
||||
ErrorCode::StaleEpoch => "STALE_EPOCH",
|
||||
ErrorCode::StaleStep => "STALE_STEP",
|
||||
ErrorCode::FutureStep => "FUTURE_STEP",
|
||||
ErrorCode::InvalidPhase => "INVALID_PHASE",
|
||||
ErrorCode::Conflict => "CONFLICT",
|
||||
ErrorCode::InProgress => "IN_PROGRESS",
|
||||
ErrorCode::Busy => "BUSY",
|
||||
ErrorCode::BufferInvalid => "BUFFER_INVALID",
|
||||
ErrorCode::ResultExpired => "RESULT_EXPIRED",
|
||||
ErrorCode::IncompatibleState => "INCOMPATIBLE_STATE",
|
||||
ErrorCode::BackendFailure => "BACKEND_FAILURE",
|
||||
ErrorCode::Internal => "INTERNAL",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse(s: &str) -> Result<ErrorCode> {
|
||||
Ok(match s {
|
||||
"INVALID_ARGUMENT" => ErrorCode::InvalidArgument,
|
||||
"UNSUPPORTED" => ErrorCode::Unsupported,
|
||||
"IDENTITY_MISMATCH" => ErrorCode::IdentityMismatch,
|
||||
"STALE_EPOCH" => ErrorCode::StaleEpoch,
|
||||
"STALE_STEP" => ErrorCode::StaleStep,
|
||||
"FUTURE_STEP" => ErrorCode::FutureStep,
|
||||
"INVALID_PHASE" => ErrorCode::InvalidPhase,
|
||||
"CONFLICT" => ErrorCode::Conflict,
|
||||
"IN_PROGRESS" => ErrorCode::InProgress,
|
||||
"BUSY" => ErrorCode::Busy,
|
||||
"BUFFER_INVALID" => ErrorCode::BufferInvalid,
|
||||
"RESULT_EXPIRED" => ErrorCode::ResultExpired,
|
||||
"INCOMPATIBLE_STATE" => ErrorCode::IncompatibleState,
|
||||
"BACKEND_FAILURE" => ErrorCode::BackendFailure,
|
||||
"INTERNAL" => ErrorCode::Internal,
|
||||
_ => return err("code is not one of the fifteen domain error codes"),
|
||||
})
|
||||
}
|
||||
|
||||
/// The codes that are raised strictly before any mutation, so their certainty is `none`.
|
||||
pub fn is_before_mutation(self) -> bool {
|
||||
matches!(
|
||||
self,
|
||||
ErrorCode::InvalidArgument
|
||||
| ErrorCode::Unsupported
|
||||
| ErrorCode::IdentityMismatch
|
||||
| ErrorCode::StaleEpoch
|
||||
| ErrorCode::StaleStep
|
||||
| ErrorCode::FutureStep
|
||||
| ErrorCode::InvalidPhase
|
||||
| ErrorCode::Conflict
|
||||
| ErrorCode::InProgress
|
||||
| ErrorCode::Busy
|
||||
| ErrorCode::BufferInvalid
|
||||
| ErrorCode::IncompatibleState
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// How certain the responder is that the operation mutated state.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
|
||||
pub enum MutationCertainty {
|
||||
/// Nothing was applied.
|
||||
None,
|
||||
/// The mutation completed.
|
||||
Applied,
|
||||
/// Completion is not established. "Errors after partial mutation use unknown unless
|
||||
/// completion is established." (ipc-v1 section 7)
|
||||
Unknown,
|
||||
}
|
||||
|
||||
impl MutationCertainty {
|
||||
pub const ALL: &'static [&'static str] = &["none", "applied", "unknown"];
|
||||
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
MutationCertainty::None => "none",
|
||||
MutationCertainty::Applied => "applied",
|
||||
MutationCertainty::Unknown => "unknown",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse(s: &str) -> Result<MutationCertainty> {
|
||||
match s {
|
||||
"none" => Ok(MutationCertainty::None),
|
||||
"applied" => Ok(MutationCertainty::Applied),
|
||||
"unknown" => Ok(MutationCertainty::Unknown),
|
||||
_ => err("mutation must be none, applied or unknown"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `SessionRpcRequest`: one domain operation, independent of the bus callId that carries it.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct SessionRpcRequest {
|
||||
pub request_id: DomainRequestId,
|
||||
pub scope: Option<Scope>,
|
||||
pub params: Value,
|
||||
}
|
||||
|
||||
impl SessionRpcRequest {
|
||||
/// The canonical body digest of this request under `method` (ipc-v1 section 5).
|
||||
pub fn body_digest(&self, method: &str) -> Result<String> {
|
||||
canonical::body_digest(method, self.scope.as_ref(), &self.params)
|
||||
}
|
||||
|
||||
/// The operation key of a step mutation issued by `worker_id` under `method`. Lifecycle
|
||||
/// calls with a null scope have no step operation key.
|
||||
pub fn operation_key(&self, method: &str, worker_id: &str) -> Result<canonical::OperationKey> {
|
||||
let scope = self
|
||||
.scope
|
||||
.clone()
|
||||
.ok_or_else(|| crate::scalar::wire_err("operation key: a step mutation has a scope"))?;
|
||||
canonical::OperationKey::new(scope, method, worker_id)
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for SessionRpcRequest {
|
||||
const TYPE_NAME: &'static str = "SessionRpcRequest";
|
||||
|
||||
fn from_json(value: &Value) -> Result<SessionRpcRequest> {
|
||||
let mut f = Fields::new(value, "SessionRpcRequest")?;
|
||||
let request_id = DomainRequestId::read(&mut f, "requestId")?;
|
||||
let scope = Scope::nullable_from_json(f.value("scope")?)?;
|
||||
let params = f.object("params")?.clone();
|
||||
f.finish()?;
|
||||
let r = SessionRpcRequest {
|
||||
request_id,
|
||||
scope,
|
||||
params: Value::Object(params),
|
||||
};
|
||||
r.validate()?;
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("requestId", self.request_id.to_json()),
|
||||
("scope", Scope::nullable_to_json(self.scope.as_ref())),
|
||||
("params", self.params.clone()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !self.params.is_object() {
|
||||
return err("SessionRpcRequest: params must be an object");
|
||||
}
|
||||
if let Some(scope) = &self.scope {
|
||||
scope.validate()?;
|
||||
}
|
||||
canonical::reject_bus_identities(&self.params)
|
||||
}
|
||||
}
|
||||
|
||||
/// `SessionRpcSuccess`: a terminal domain success, echoing the request scope.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct SessionRpcSuccess {
|
||||
pub request_id: DomainRequestId,
|
||||
pub worker_id: String,
|
||||
pub incarnation_id: String,
|
||||
pub scope: Option<Scope>,
|
||||
pub result: Value,
|
||||
}
|
||||
|
||||
impl DomainType for SessionRpcSuccess {
|
||||
const TYPE_NAME: &'static str = "SessionRpcSuccess";
|
||||
|
||||
fn from_json(value: &Value) -> Result<SessionRpcSuccess> {
|
||||
let mut f = Fields::new(value, "SessionRpcSuccess")?;
|
||||
constant(&mut f, "type", "result")?;
|
||||
let request_id = DomainRequestId::read(&mut f, "requestId")?;
|
||||
let worker_id = f.id("workerId")?;
|
||||
let incarnation_id = f.id("incarnationId")?;
|
||||
let scope = Scope::nullable_from_json(f.value("scope")?)?;
|
||||
let result = f.object("result")?.clone();
|
||||
f.finish()?;
|
||||
let s = SessionRpcSuccess {
|
||||
request_id,
|
||||
worker_id,
|
||||
incarnation_id,
|
||||
scope,
|
||||
result: Value::Object(result),
|
||||
};
|
||||
s.validate()?;
|
||||
Ok(s)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("type", "result".into()),
|
||||
("requestId", self.request_id.to_json()),
|
||||
("workerId", self.worker_id.clone().into()),
|
||||
("incarnationId", self.incarnation_id.clone().into()),
|
||||
("scope", Scope::nullable_to_json(self.scope.as_ref())),
|
||||
("result", self.result.clone()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.worker_id) || !is_id(&self.incarnation_id) {
|
||||
return err("SessionRpcSuccess: workerId and incarnationId must be valid ids");
|
||||
}
|
||||
if !self.result.is_object() {
|
||||
return err("SessionRpcSuccess: result must be an object");
|
||||
}
|
||||
if let Some(scope) = &self.scope {
|
||||
scope.validate()?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// `SessionRpcFailure`: a terminal domain error with an explicit mutation certainty.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct SessionRpcFailure {
|
||||
pub request_id: DomainRequestId,
|
||||
pub worker_id: String,
|
||||
pub incarnation_id: String,
|
||||
pub scope: Option<Scope>,
|
||||
pub code: ErrorCode,
|
||||
pub message: String,
|
||||
pub mutation: MutationCertainty,
|
||||
}
|
||||
|
||||
impl DomainType for SessionRpcFailure {
|
||||
const TYPE_NAME: &'static str = "SessionRpcFailure";
|
||||
|
||||
fn from_json(value: &Value) -> Result<SessionRpcFailure> {
|
||||
let mut f = Fields::new(value, "SessionRpcFailure")?;
|
||||
constant(&mut f, "type", "error")?;
|
||||
let request_id = DomainRequestId::read(&mut f, "requestId")?;
|
||||
let worker_id = f.id("workerId")?;
|
||||
let incarnation_id = f.id("incarnationId")?;
|
||||
let scope = Scope::nullable_from_json(f.value("scope")?)?;
|
||||
let (code, message, mutation) = {
|
||||
let v = f.value("error")?;
|
||||
let mut e = Fields::new(v, "SessionRpcFailure.error")?;
|
||||
let code = ErrorCode::parse(&enumeration(&mut e, "code", ErrorCode::ALL)?)?;
|
||||
let message = bounded_string(&mut e, "message", MAX_MESSAGE_CODE_POINTS)?;
|
||||
let mutation = MutationCertainty::parse(&enumeration(
|
||||
&mut e,
|
||||
"mutation",
|
||||
MutationCertainty::ALL,
|
||||
)?)?;
|
||||
e.finish()?;
|
||||
(code, message, mutation)
|
||||
};
|
||||
f.finish()?;
|
||||
let failure = SessionRpcFailure {
|
||||
request_id,
|
||||
worker_id,
|
||||
incarnation_id,
|
||||
scope,
|
||||
code,
|
||||
message,
|
||||
mutation,
|
||||
};
|
||||
failure.validate()?;
|
||||
Ok(failure)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("type", "error".into()),
|
||||
("requestId", self.request_id.to_json()),
|
||||
("workerId", self.worker_id.clone().into()),
|
||||
("incarnationId", self.incarnation_id.clone().into()),
|
||||
("scope", Scope::nullable_to_json(self.scope.as_ref())),
|
||||
(
|
||||
"error",
|
||||
obj(vec![
|
||||
("code", self.code.as_str().into()),
|
||||
("message", self.message.clone().into()),
|
||||
("mutation", self.mutation.as_str().into()),
|
||||
]),
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.worker_id) || !is_id(&self.incarnation_id) {
|
||||
return err("SessionRpcFailure: workerId and incarnationId must be valid ids");
|
||||
}
|
||||
if self.message.chars().count() > MAX_MESSAGE_CODE_POINTS {
|
||||
return err("SessionRpcFailure: message is at most 512 code points");
|
||||
}
|
||||
if self.code.is_before_mutation() && self.mutation != MutationCertainty::None {
|
||||
return err(format!(
|
||||
"SessionRpcFailure: {} is raised before mutation, so mutation is \"none\"",
|
||||
self.code.as_str()
|
||||
));
|
||||
}
|
||||
if let Some(scope) = &self.scope {
|
||||
scope.validate()?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// A terminal domain outcome: success or failure.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub enum SessionRpcOutcome {
|
||||
Success(SessionRpcSuccess),
|
||||
Failure(SessionRpcFailure),
|
||||
}
|
||||
|
||||
impl SessionRpcOutcome {
|
||||
pub fn request_id(&self) -> &DomainRequestId {
|
||||
match self {
|
||||
SessionRpcOutcome::Success(s) => &s.request_id,
|
||||
SessionRpcOutcome::Failure(f) => &f.request_id,
|
||||
}
|
||||
}
|
||||
|
||||
/// Replies echo the original scope (ipc-v1 section 3).
|
||||
pub fn echoes(&self, request: &SessionRpcRequest) -> bool {
|
||||
let scope = match self {
|
||||
SessionRpcOutcome::Success(s) => &s.scope,
|
||||
SessionRpcOutcome::Failure(f) => &f.scope,
|
||||
};
|
||||
self.request_id() == &request.request_id && scope == &request.scope
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for SessionRpcOutcome {
|
||||
const TYPE_NAME: &'static str = "SessionRpcOutcome";
|
||||
|
||||
fn from_json(value: &Value) -> Result<SessionRpcOutcome> {
|
||||
let kind = value
|
||||
.get("type")
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| crate::scalar::wire_err("SessionRpcOutcome: missing type"))?;
|
||||
match kind {
|
||||
"result" => SessionRpcSuccess::from_json(value).map(SessionRpcOutcome::Success),
|
||||
"error" => SessionRpcFailure::from_json(value).map(SessionRpcOutcome::Failure),
|
||||
_ => err("SessionRpcOutcome: type must be \"result\" or \"error\""),
|
||||
}
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
match self {
|
||||
SessionRpcOutcome::Success(s) => s.to_json(),
|
||||
SessionRpcOutcome::Failure(f) => f.to_json(),
|
||||
}
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
match self {
|
||||
SessionRpcOutcome::Success(s) => s.validate(),
|
||||
SessionRpcOutcome::Failure(f) => f.validate(),
|
||||
}
|
||||
}
|
||||
}
|
||||
739
services/flysim/crates/fly-session-types/src/scalar.rs
Normal file
739
services/flysim/crates/fly-session-types/src/scalar.rs
Normal file
|
|
@ -0,0 +1,739 @@
|
|||
//! The domain scalars of ipc-v1 section 2, and the four identities that must never be confused.
|
||||
//!
|
||||
//! `Id`, `U64` and `Digest` are the bus encodings: this module calls straight into
|
||||
//! [`flybus::wire`] instead of restating the regular expressions, and
|
||||
//! `tests/encodings.rs` pins that the two agree. Everything else here is domain-only:
|
||||
//! `Scope`, `RationalNs` (reduced, positive denominator, zero as `0/1`, checked arithmetic),
|
||||
//! `SchemaRef` and `TypedValue` with its 32-KiB canonical-JSON cap.
|
||||
|
||||
use std::cmp::Ordering;
|
||||
|
||||
use flybus::wire::{self, Fields, WireError};
|
||||
use serde_json::{Map, Value};
|
||||
|
||||
use crate::canonical;
|
||||
|
||||
pub type Result<T> = std::result::Result<T, WireError>;
|
||||
|
||||
/// Every parsed domain type re-validates itself, so a value built in Rust and a value read
|
||||
/// from JSON are held to the same rules.
|
||||
pub trait DomainType: Sized {
|
||||
/// The name this type has in the canonical schema set.
|
||||
const TYPE_NAME: &'static str;
|
||||
|
||||
/// Reads and validates one JSON value. Unknown fields are refused.
|
||||
fn from_json(value: &Value) -> Result<Self>;
|
||||
|
||||
/// The canonical JSON shape of this value.
|
||||
fn to_json(&self) -> Value;
|
||||
|
||||
/// The rules that are not expressible as one field read: ranges that depend on another
|
||||
/// field, uniqueness, ordering and size caps.
|
||||
fn validate(&self) -> Result<()>;
|
||||
}
|
||||
|
||||
pub fn err<T>(message: impl Into<String>) -> Result<T> {
|
||||
Err(WireError(message.into()))
|
||||
}
|
||||
|
||||
pub fn wire_err(message: impl Into<String>) -> WireError {
|
||||
WireError(message.into())
|
||||
}
|
||||
|
||||
pub(crate) fn obj(pairs: Vec<(&str, Value)>) -> Value {
|
||||
let mut map = Map::new();
|
||||
for (key, value) in pairs {
|
||||
map.insert(key.to_owned(), value);
|
||||
}
|
||||
Value::Object(map)
|
||||
}
|
||||
|
||||
/// A `U64` field: the decimal string encoding, never a JSON number.
|
||||
pub fn u64_json(n: u64) -> Value {
|
||||
Value::String(n.to_string())
|
||||
}
|
||||
|
||||
/// `Id`: `^[a-z0-9][a-z0-9._-]{0,63}$`, exactly the bus encoding.
|
||||
pub fn is_id(s: &str) -> bool {
|
||||
wire::is_id(s)
|
||||
}
|
||||
|
||||
/// `Digest`: 64 lowercase hexadecimal digits, exactly the bus encoding.
|
||||
pub fn is_digest(s: &str) -> bool {
|
||||
wire::is_digest(s)
|
||||
}
|
||||
|
||||
/// `U64`: `"0"` or `[1-9][0-9]*` up to `u64::MAX`, exactly the bus encoding.
|
||||
pub fn parse_u64(s: &str) -> Option<u64> {
|
||||
wire::parse_u64(s)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// Field readers the bus reader does not have
|
||||
|
||||
/// A finite JSON number. NaN and infinities never survive strict parsing; this also refuses
|
||||
/// integers outside the exactly representable double range, which canonical JSON cannot encode.
|
||||
pub fn finite(f: &mut Fields<'_>, key: &'static str) -> Result<f64> {
|
||||
let value = f.value(key)?;
|
||||
match value {
|
||||
Value::Number(n) => canonical::finite_double(n)
|
||||
.ok_or_else(|| wire_err(format!("{key} must be a finite JSON number"))),
|
||||
_ => err(format!("{key} must be a finite JSON number")),
|
||||
}
|
||||
}
|
||||
|
||||
/// A finite JSON number inside `lo..=hi`, refused rather than clamped.
|
||||
pub fn finite_in(f: &mut Fields<'_>, key: &'static str, lo: f64, hi: f64) -> Result<f64> {
|
||||
let n = finite(f, key)?;
|
||||
if n < lo || n > hi {
|
||||
return err(format!("{key} must be in [{lo}, {hi}]"));
|
||||
}
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// A JSON integer in `i32` range, the seed encoding Agent.Initialize uses.
|
||||
pub fn i32_field(f: &mut Fields<'_>, key: &'static str) -> Result<i32> {
|
||||
let value = f.value(key)?;
|
||||
match value.as_i64() {
|
||||
Some(n) if i64::from(i32::MIN) <= n && n <= i64::from(i32::MAX) => Ok(n as i32),
|
||||
_ => err(format!("{key} must be a signed 32-bit integer")),
|
||||
}
|
||||
}
|
||||
|
||||
/// One member of a closed string enum.
|
||||
pub fn enumeration(f: &mut Fields<'_>, key: &'static str, allowed: &[&str]) -> Result<String> {
|
||||
let s = f.string(key)?;
|
||||
if allowed.contains(&s) {
|
||||
Ok(s.to_owned())
|
||||
} else {
|
||||
err(format!("{key} must be one of {}", allowed.join(", ")))
|
||||
}
|
||||
}
|
||||
|
||||
/// A string constant: a field whose only legal value is `expected`.
|
||||
pub fn constant(f: &mut Fields<'_>, key: &'static str, expected: &str) -> Result<()> {
|
||||
let s = f.string(key)?;
|
||||
if s == expected {
|
||||
Ok(())
|
||||
} else {
|
||||
err(format!("{key} must be {expected:?}"))
|
||||
}
|
||||
}
|
||||
|
||||
/// A `true` constant.
|
||||
pub fn constant_true(f: &mut Fields<'_>, key: &'static str) -> Result<()> {
|
||||
if f.boolean(key)? {
|
||||
Ok(())
|
||||
} else {
|
||||
err(format!("{key} must be true"))
|
||||
}
|
||||
}
|
||||
|
||||
/// A string of at most `max` Unicode code points.
|
||||
pub fn bounded_string(f: &mut Fields<'_>, key: &'static str, max: usize) -> Result<String> {
|
||||
let s = f.string(key)?;
|
||||
if s.chars().count() > max {
|
||||
return err(format!("{key} must be at most {max} code points"));
|
||||
}
|
||||
Ok(s.to_owned())
|
||||
}
|
||||
|
||||
/// `null`, or a string of at most `max` code points.
|
||||
pub fn nullable_bounded_string(
|
||||
f: &mut Fields<'_>,
|
||||
key: &'static str,
|
||||
max: usize,
|
||||
) -> Result<Option<String>> {
|
||||
match f.value(key)? {
|
||||
Value::Null => Ok(None),
|
||||
_ => bounded_string(f, key, max).map(Some),
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads an array of `lo..=hi` items through `read`, keeping the supplied order.
|
||||
pub fn list<T>(
|
||||
f: &mut Fields<'_>,
|
||||
key: &'static str,
|
||||
lo: usize,
|
||||
hi: usize,
|
||||
read: impl Fn(&Value) -> Result<T>,
|
||||
) -> Result<Vec<T>> {
|
||||
let items = f.array(key, lo, hi)?;
|
||||
let mut out = Vec::with_capacity(items.len());
|
||||
for item in items {
|
||||
out.push(read(item).map_err(|e| wire_err(format!("{key}: {e}")))?);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// An array of `lo..=hi` `Id`s.
|
||||
pub fn id_list(f: &mut Fields<'_>, key: &'static str, lo: usize, hi: usize) -> Result<Vec<String>> {
|
||||
list(f, key, lo, hi, |v| match v.as_str() {
|
||||
Some(s) if is_id(s) => Ok(s.to_owned()),
|
||||
_ => err("every entry must be an id"),
|
||||
})
|
||||
}
|
||||
|
||||
/// Fails on the first repeated key, naming it.
|
||||
pub fn require_unique<'a>(keys: impl IntoIterator<Item = &'a str>, what: &str) -> Result<()> {
|
||||
let mut seen: Vec<&str> = Vec::new();
|
||||
for key in keys {
|
||||
if seen.contains(&key) {
|
||||
return err(format!("{what}: duplicate {key:?}"));
|
||||
}
|
||||
seen.push(key);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Fails unless `actual` is exactly `expected`, in that order: descriptor order is part of
|
||||
/// the contract, not a set membership test.
|
||||
pub fn require_same_order<'a>(
|
||||
actual: impl IntoIterator<Item = &'a str>,
|
||||
expected: impl IntoIterator<Item = &'a str>,
|
||||
what: &str,
|
||||
) -> Result<()> {
|
||||
let actual: Vec<&str> = actual.into_iter().collect();
|
||||
let expected: Vec<&str> = expected.into_iter().collect();
|
||||
if actual != expected {
|
||||
return err(format!(
|
||||
"{what}: must list [{}] in that order, found [{}]",
|
||||
expected.join(", "),
|
||||
actual.join(", ")
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// Scope
|
||||
|
||||
/// `Scope`: the simulation timeline identity. Never the bus route or store incarnation.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
|
||||
pub struct Scope {
|
||||
pub session_id: String,
|
||||
pub epoch: String,
|
||||
pub step: u64,
|
||||
}
|
||||
|
||||
impl Scope {
|
||||
pub fn new(session_id: &str, epoch: &str, step: u64) -> Result<Scope> {
|
||||
let scope = Scope {
|
||||
session_id: session_id.to_owned(),
|
||||
epoch: epoch.to_owned(),
|
||||
step,
|
||||
};
|
||||
scope.validate()?;
|
||||
Ok(scope)
|
||||
}
|
||||
|
||||
/// `null`, or a scope.
|
||||
pub fn nullable_from_json(value: &Value) -> Result<Option<Scope>> {
|
||||
match value {
|
||||
Value::Null => Ok(None),
|
||||
_ => Scope::from_json(value).map(Some),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn nullable_to_json(scope: Option<&Scope>) -> Value {
|
||||
scope.map_or(Value::Null, Scope::to_json)
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for Scope {
|
||||
const TYPE_NAME: &'static str = "Scope";
|
||||
|
||||
fn from_json(value: &Value) -> Result<Scope> {
|
||||
let mut f = Fields::new(value, "Scope")?;
|
||||
let session_id = f.id("sessionId")?;
|
||||
let epoch = f.id("epoch")?;
|
||||
let step = f.u64_string("step")?;
|
||||
f.finish()?;
|
||||
let scope = Scope {
|
||||
session_id,
|
||||
epoch,
|
||||
step,
|
||||
};
|
||||
scope.validate()?;
|
||||
Ok(scope)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("sessionId", self.session_id.clone().into()),
|
||||
("epoch", self.epoch.clone().into()),
|
||||
("step", u64_json(self.step)),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.session_id) {
|
||||
return err("Scope: sessionId is not a valid id");
|
||||
}
|
||||
if !is_id(&self.epoch) {
|
||||
return err("Scope: epoch is not a valid id");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// RationalNs
|
||||
|
||||
/// A nanosecond rational: reduced, positive denominator, zero encoded `0/1`.
|
||||
///
|
||||
/// ipc-v1 section 2: "Fractions are reduced, denominators positive, durations positive; zero
|
||||
/// is encoded 0/1. Arithmetic is checked." Durations are checked with
|
||||
/// [`RationalNs::require_positive`] by the fields that are durations; `worldTime` and a tick
|
||||
/// remainder are legitimately zero.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
|
||||
pub struct RationalNs {
|
||||
pub numerator: u64,
|
||||
pub denominator: u64,
|
||||
}
|
||||
|
||||
fn gcd(a: u64, b: u64) -> u64 {
|
||||
let (mut a, mut b) = (a, b);
|
||||
while b != 0 {
|
||||
let t = a % b;
|
||||
a = b;
|
||||
b = t;
|
||||
}
|
||||
a
|
||||
}
|
||||
|
||||
/// One checked `u64` x `u64` product. The product itself always fits `u128`; the function
|
||||
/// exists so every multiplication in the arithmetic below goes through one checked path.
|
||||
fn mul(a: u64, b: u64) -> Result<u128> {
|
||||
u128::from(a)
|
||||
.checked_mul(u128::from(b))
|
||||
.ok_or_else(|| wire_err("RationalNs: multiplication overflowed"))
|
||||
}
|
||||
|
||||
fn gcd128(a: u128, b: u128) -> u128 {
|
||||
let (mut a, mut b) = (a, b);
|
||||
while b != 0 {
|
||||
let t = a % b;
|
||||
a = b;
|
||||
b = t;
|
||||
}
|
||||
a
|
||||
}
|
||||
|
||||
impl RationalNs {
|
||||
pub const ZERO: RationalNs = RationalNs {
|
||||
numerator: 0,
|
||||
denominator: 1,
|
||||
};
|
||||
|
||||
/// Exactly the supplied pair, which must already be in canonical form.
|
||||
pub fn new(numerator: u64, denominator: u64) -> Result<RationalNs> {
|
||||
let r = RationalNs {
|
||||
numerator,
|
||||
denominator,
|
||||
};
|
||||
r.validate()?;
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
/// Reduces first, then validates: the constructor for arithmetic results.
|
||||
pub fn reduced(numerator: u128, denominator: u128) -> Result<RationalNs> {
|
||||
if denominator == 0 {
|
||||
return err("RationalNs: denominator must be positive");
|
||||
}
|
||||
let (n, d) = if numerator == 0 {
|
||||
(0u128, 1u128)
|
||||
} else {
|
||||
let g = gcd128(numerator, denominator);
|
||||
(numerator / g, denominator / g)
|
||||
};
|
||||
if n > u128::from(u64::MAX) || d > u128::from(u64::MAX) {
|
||||
return err("RationalNs: reduced value does not fit U64");
|
||||
}
|
||||
RationalNs::new(n as u64, d as u64)
|
||||
}
|
||||
|
||||
pub fn is_zero(&self) -> bool {
|
||||
self.numerator == 0
|
||||
}
|
||||
|
||||
/// Durations must be positive (ipc-v1 section 2).
|
||||
pub fn require_positive(&self, what: &str) -> Result<()> {
|
||||
if self.is_zero() {
|
||||
return err(format!("{what}: duration must be positive"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Cross-multiplication of two `U64` pairs fits `u128`, but their *sum* does not: two
|
||||
/// reduced fractions near the `U64` maximum add to about 2^129. Every step is checked, as
|
||||
/// ipc-v1 section 2 requires; nothing here may wrap in release and panic in debug.
|
||||
pub fn checked_add(&self, other: &RationalNs) -> Result<RationalNs> {
|
||||
let left = mul(self.numerator, other.denominator)?;
|
||||
let right = mul(other.numerator, self.denominator)?;
|
||||
let n = left
|
||||
.checked_add(right)
|
||||
.ok_or_else(|| wire_err("RationalNs: addition overflowed"))?;
|
||||
let d = mul(self.denominator, other.denominator)?;
|
||||
RationalNs::reduced(n, d)
|
||||
}
|
||||
|
||||
pub fn checked_sub(&self, other: &RationalNs) -> Result<RationalNs> {
|
||||
let left = mul(self.numerator, other.denominator)?;
|
||||
let right = mul(other.numerator, self.denominator)?;
|
||||
if right > left {
|
||||
return err("RationalNs: subtraction would be negative");
|
||||
}
|
||||
let d = mul(self.denominator, other.denominator)?;
|
||||
RationalNs::reduced(left - right, d)
|
||||
}
|
||||
|
||||
pub fn checked_mul_u64(&self, k: u64) -> Result<RationalNs> {
|
||||
let n = u128::from(self.numerator)
|
||||
.checked_mul(u128::from(k))
|
||||
.ok_or_else(|| wire_err("RationalNs: multiplication overflowed"))?;
|
||||
RationalNs::reduced(n, u128::from(self.denominator))
|
||||
}
|
||||
|
||||
/// The step-v1 section 5 accumulator: `ticks = floor(self / tick)` and the remainder
|
||||
/// `self - ticks * tick`, which is always `>= 0` and `< tick`.
|
||||
pub fn divide_floor(&self, tick: &RationalNs) -> Result<(u64, RationalNs)> {
|
||||
tick.require_positive("RationalNs::divide_floor tick")?;
|
||||
let n = mul(self.numerator, tick.denominator)?;
|
||||
let d = mul(self.denominator, tick.numerator)?;
|
||||
let ticks = n / d;
|
||||
if ticks > u128::from(u64::MAX) {
|
||||
return err("RationalNs: tick count does not fit U64");
|
||||
}
|
||||
let ticks = ticks as u64;
|
||||
let remainder = self.checked_sub(&tick.checked_mul_u64(ticks)?)?;
|
||||
Ok((ticks, remainder))
|
||||
}
|
||||
}
|
||||
|
||||
impl PartialOrd for RationalNs {
|
||||
fn partial_cmp(&self, other: &RationalNs) -> Option<Ordering> {
|
||||
Some(self.cmp(other))
|
||||
}
|
||||
}
|
||||
|
||||
impl Ord for RationalNs {
|
||||
fn cmp(&self, other: &RationalNs) -> Ordering {
|
||||
let left = u128::from(self.numerator) * u128::from(other.denominator);
|
||||
let right = u128::from(other.numerator) * u128::from(self.denominator);
|
||||
left.cmp(&right)
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for RationalNs {
|
||||
const TYPE_NAME: &'static str = "RationalNs";
|
||||
|
||||
fn from_json(value: &Value) -> Result<RationalNs> {
|
||||
let mut f = Fields::new(value, "RationalNs")?;
|
||||
let numerator = f.u64_string("numerator")?;
|
||||
let denominator = f.u64_string("denominator")?;
|
||||
f.finish()?;
|
||||
RationalNs::new(numerator, denominator)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("numerator", u64_json(self.numerator)),
|
||||
("denominator", u64_json(self.denominator)),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if self.denominator == 0 {
|
||||
return err("RationalNs: denominator must be positive");
|
||||
}
|
||||
if self.numerator == 0 && self.denominator != 1 {
|
||||
return err("RationalNs: zero is encoded 0/1");
|
||||
}
|
||||
if self.numerator != 0 && gcd(self.numerator, self.denominator) != 1 {
|
||||
return err("RationalNs: fraction must be reduced");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// SchemaRef and TypedValue
|
||||
|
||||
/// `SchemaRef`: the identity of a registered typed payload schema. Version is 1..=65535.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
|
||||
pub struct SchemaRef {
|
||||
pub id: String,
|
||||
pub version: u16,
|
||||
pub digest: String,
|
||||
}
|
||||
|
||||
impl SchemaRef {
|
||||
pub fn new(id: &str, version: u16, digest: &str) -> Result<SchemaRef> {
|
||||
let r = SchemaRef {
|
||||
id: id.to_owned(),
|
||||
version,
|
||||
digest: digest.to_owned(),
|
||||
};
|
||||
r.validate()?;
|
||||
Ok(r)
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for SchemaRef {
|
||||
const TYPE_NAME: &'static str = "SchemaRef";
|
||||
|
||||
fn from_json(value: &Value) -> Result<SchemaRef> {
|
||||
let mut f = Fields::new(value, "SchemaRef")?;
|
||||
let id = f.id("id")?;
|
||||
let version = f.int("version", 1, 65_535)? as u16;
|
||||
let digest = f.string("digest")?.to_owned();
|
||||
f.finish()?;
|
||||
let r = SchemaRef {
|
||||
id,
|
||||
version,
|
||||
digest,
|
||||
};
|
||||
r.validate()?;
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("id", self.id.clone().into()),
|
||||
("version", Value::from(u64::from(self.version))),
|
||||
("digest", self.digest.clone().into()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.id) {
|
||||
return err("SchemaRef: id is not a valid id");
|
||||
}
|
||||
if self.version == 0 {
|
||||
return err("SchemaRef: version must be 1..=65535");
|
||||
}
|
||||
if !is_digest(&self.digest) {
|
||||
return err("SchemaRef: digest must be 64 lowercase hex digits");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// The canonical-JSON size limit of one `TypedValue` (ipc-v1 section 2, workers-v1 section 1).
|
||||
pub const MAX_TYPED_VALUE_BYTES: usize = 32 * 1024;
|
||||
|
||||
/// `TypedValue`: a schema identity plus an object, capped at 32 KiB of canonical JSON.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct TypedValue {
|
||||
pub schema: SchemaRef,
|
||||
pub value: Value,
|
||||
}
|
||||
|
||||
impl TypedValue {
|
||||
pub fn new(schema: SchemaRef, value: Value) -> Result<TypedValue> {
|
||||
let t = TypedValue { schema, value };
|
||||
t.validate()?;
|
||||
Ok(t)
|
||||
}
|
||||
|
||||
pub fn nullable_from_json(value: &Value) -> Result<Option<TypedValue>> {
|
||||
match value {
|
||||
Value::Null => Ok(None),
|
||||
_ => TypedValue::from_json(value).map(Some),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn nullable_to_json(value: Option<&TypedValue>) -> Value {
|
||||
value.map_or(Value::Null, TypedValue::to_json)
|
||||
}
|
||||
|
||||
/// The canonical JSON byte length of the whole typed value.
|
||||
pub fn canonical_len(&self) -> Result<usize> {
|
||||
canonical::canonicalize(&self.to_json()).map(|s| s.len())
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for TypedValue {
|
||||
const TYPE_NAME: &'static str = "TypedValue";
|
||||
|
||||
fn from_json(value: &Value) -> Result<TypedValue> {
|
||||
let mut f = Fields::new(value, "TypedValue")?;
|
||||
let schema = SchemaRef::from_json(f.value("schema")?)?;
|
||||
let inner = f.value("value")?.clone();
|
||||
f.finish()?;
|
||||
let t = TypedValue {
|
||||
schema,
|
||||
value: inner,
|
||||
};
|
||||
t.validate()?;
|
||||
Ok(t)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("schema", self.schema.to_json()),
|
||||
("value", self.value.clone()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
self.schema.validate()?;
|
||||
if !self.value.is_object() {
|
||||
return err("TypedValue: value must be an object");
|
||||
}
|
||||
let len = self.canonical_len()?;
|
||||
if len > MAX_TYPED_VALUE_BYTES {
|
||||
return err(format!(
|
||||
"TypedValue: {len} bytes of canonical JSON exceeds the {MAX_TYPED_VALUE_BYTES}-byte limit"
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// The four identities
|
||||
|
||||
/// A bus RPC correlation id, `call-<U64>` (bus-v1 section 6). It is not a domain operation id:
|
||||
/// a safe domain retry keeps its [`DomainRequestId`] and gets a new `BusCallId`.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
|
||||
pub struct BusCallId(String);
|
||||
|
||||
/// A domain operation id, `req-` plus a canonical `U64` serial (ipc-v1 section 5).
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
|
||||
pub struct DomainRequestId(String);
|
||||
|
||||
/// The identity of an immutable artifact: store incarnation, artifact id and generation.
|
||||
/// Not an address, not authority to read, and not an [`crate::workers::AssetRef`].
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
|
||||
pub struct ArtifactIdentity {
|
||||
pub store_id: String,
|
||||
pub artifact_id: String,
|
||||
pub generation: u64,
|
||||
}
|
||||
|
||||
/// Which kind of ownership root a token names.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
|
||||
pub enum OwnerKind {
|
||||
/// One recipient's delivery, `dlv-<U64>`.
|
||||
Delivery,
|
||||
/// An explicit artifact hold, `own-<U64>`.
|
||||
Hold,
|
||||
}
|
||||
|
||||
/// A delivery or explicit-hold owner token. Connection-private: it never appears in a domain
|
||||
/// payload or a canonical body digest.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
|
||||
pub struct OwnerToken {
|
||||
token: String,
|
||||
kind: OwnerKind,
|
||||
}
|
||||
|
||||
macro_rules! serial_identity {
|
||||
($type:ty, $prefix:literal, $what:literal) => {
|
||||
impl $type {
|
||||
/// Parses the canonical `prefix-<U64>` form; any other prefix is refused, which is
|
||||
/// what keeps the four identities from being swapped for one another.
|
||||
pub fn parse(s: &str) -> Result<Self> {
|
||||
match wire::parse_serial_id($prefix, s) {
|
||||
Some(_) => Ok(Self(s.to_owned())),
|
||||
None => err(concat!($what, " must be canonical ", $prefix, "-<U64>")),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn from_serial(serial: u64) -> Self {
|
||||
Self(wire::serial_id($prefix, serial))
|
||||
}
|
||||
|
||||
pub fn serial(&self) -> u64 {
|
||||
wire::parse_serial_id($prefix, &self.0).expect("validated on construction")
|
||||
}
|
||||
|
||||
pub fn as_str(&self) -> &str {
|
||||
&self.0
|
||||
}
|
||||
|
||||
pub fn read(f: &mut Fields<'_>, key: &'static str) -> Result<Self> {
|
||||
let s = f.string(key)?;
|
||||
Self::parse(s).map_err(|e| wire_err(format!("{key}: {e}")))
|
||||
}
|
||||
|
||||
pub fn read_nullable(f: &mut Fields<'_>, key: &'static str) -> Result<Option<Self>> {
|
||||
match f.value(key)? {
|
||||
Value::Null => Ok(None),
|
||||
_ => Self::read(f, key).map(Some),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn to_json(&self) -> Value {
|
||||
Value::String(self.0.clone())
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
serial_identity!(BusCallId, "call", "a bus callId");
|
||||
serial_identity!(DomainRequestId, "req", "a domain requestId");
|
||||
|
||||
impl OwnerToken {
|
||||
pub fn parse(s: &str) -> Result<OwnerToken> {
|
||||
if wire::parse_serial_id("dlv", s).is_some() {
|
||||
return Ok(OwnerToken {
|
||||
token: s.to_owned(),
|
||||
kind: OwnerKind::Delivery,
|
||||
});
|
||||
}
|
||||
if wire::parse_serial_id("own", s).is_some() {
|
||||
return Ok(OwnerToken {
|
||||
token: s.to_owned(),
|
||||
kind: OwnerKind::Hold,
|
||||
});
|
||||
}
|
||||
err("an owner token must be canonical dlv-<U64> or own-<U64>")
|
||||
}
|
||||
|
||||
pub fn delivery(serial: u64) -> OwnerToken {
|
||||
OwnerToken {
|
||||
token: wire::serial_id("dlv", serial),
|
||||
kind: OwnerKind::Delivery,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn hold(serial: u64) -> OwnerToken {
|
||||
OwnerToken {
|
||||
token: wire::serial_id("own", serial),
|
||||
kind: OwnerKind::Hold,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn kind(&self) -> OwnerKind {
|
||||
self.kind
|
||||
}
|
||||
|
||||
pub fn as_str(&self) -> &str {
|
||||
&self.token
|
||||
}
|
||||
}
|
||||
|
||||
impl ArtifactIdentity {
|
||||
/// The identity half of a bus `ArtifactRef`: the parts that name the bytes, without the
|
||||
/// byte length, content type or optional digest.
|
||||
pub fn of(reference: &flybus::wire::ArtifactRef) -> ArtifactIdentity {
|
||||
ArtifactIdentity {
|
||||
store_id: reference.store_id.clone(),
|
||||
artifact_id: reference.artifact_id.clone(),
|
||||
generation: reference.generation,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn validate(&self) -> Result<()> {
|
||||
if !is_id(&self.store_id) {
|
||||
return err("ArtifactIdentity: storeId is not a valid id");
|
||||
}
|
||||
if !is_id(&self.artifact_id) {
|
||||
return err("ArtifactIdentity: artifactId is not a valid id");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
1109
services/flysim/crates/fly-session-types/src/schema.rs
Normal file
1109
services/flysim/crates/fly-session-types/src/schema.rs
Normal file
File diff suppressed because it is too large
Load diff
70
services/flysim/crates/fly-session-types/src/seed.rs
Normal file
70
services/flysim/crates/fly-session-types/src/seed.rs
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
//! `seed-derivation-v1`: independent per-agent seeds from one recorded master seed.
|
||||
//!
|
||||
//! The specification is `docs/design/session-framework/seed-derivation-v1.md`; this is its
|
||||
//! reference implementation, and `fixtures/seed-vectors.json` its test vectors, which the
|
||||
//! TypeScript package reproduces.
|
||||
|
||||
use sha2::{Digest as _, Sha256};
|
||||
|
||||
use crate::canonical;
|
||||
use crate::scalar::{Result, err, is_id};
|
||||
|
||||
/// The algorithm identity. It is part of composition identity: changing any byte of the
|
||||
/// derivation requires a new id.
|
||||
pub const ALGORITHM: &str = "seed-derivation-v1";
|
||||
|
||||
/// The domain separation prefix hashed before the inputs.
|
||||
pub const PREFIX: &str = "flybrain/seed-derivation-v1";
|
||||
|
||||
/// The SHA-256 of the derivation material for one agent, lowercase hex.
|
||||
pub fn material_digest(master_seed: u64, agent_id: &str) -> Result<String> {
|
||||
Ok(canonical::sha256_hex(&material(master_seed, agent_id)?))
|
||||
}
|
||||
|
||||
/// The exact bytes hashed: the prefix, the master seed as a canonical `U64` decimal string and
|
||||
/// the agent id, each followed by one `\n`.
|
||||
pub fn material(master_seed: u64, agent_id: &str) -> Result<Vec<u8>> {
|
||||
if !is_id(agent_id) {
|
||||
return err("seed derivation: agentId is not a valid id");
|
||||
}
|
||||
Ok(format!("{PREFIX}\n{master_seed}\n{agent_id}\n").into_bytes())
|
||||
}
|
||||
|
||||
/// The signed 32-bit seed `Agent.Initialize` takes for `agent_id`.
|
||||
///
|
||||
/// The digest is read as eight big-endian `u32` lanes; the first nonzero lane becomes the
|
||||
/// seed, reinterpreted as two's-complement `i32`. Skipping zero lanes keeps the seed usable
|
||||
/// by an xorshift generator, whose state must not be zero. If every lane were zero the
|
||||
/// material is rehashed with a counter suffix, which no observed input has needed.
|
||||
pub fn agent_seed(master_seed: u64, agent_id: &str) -> Result<i32> {
|
||||
let mut material = material(master_seed, agent_id)?;
|
||||
for round in 0u32..4 {
|
||||
if round > 0 {
|
||||
material.extend_from_slice(format!("{round}\n").as_bytes());
|
||||
}
|
||||
let digest = Sha256::digest(&material);
|
||||
for lane in digest.chunks_exact(4) {
|
||||
let word = u32::from_be_bytes([lane[0], lane[1], lane[2], lane[3]]);
|
||||
if word != 0 {
|
||||
return Ok(word as i32);
|
||||
}
|
||||
}
|
||||
}
|
||||
err("seed derivation: every lane of four digests was zero")
|
||||
}
|
||||
|
||||
/// The seeds of a whole composition, in the order the agent ids are given.
|
||||
///
|
||||
/// Equal ids deliberately derive equal seeds: "Identical explicit seeds are allowed only when
|
||||
/// the experiment intentionally declares them" (workers-v1 section 2), so a composition with a
|
||||
/// repeated agent id is refused here rather than silently sharing a seed.
|
||||
pub fn composition_seeds(master_seed: u64, agent_ids: &[String]) -> Result<Vec<i32>> {
|
||||
crate::scalar::require_unique(
|
||||
agent_ids.iter().map(String::as_str),
|
||||
"seed derivation: agentIds",
|
||||
)?;
|
||||
agent_ids
|
||||
.iter()
|
||||
.map(|id| agent_seed(master_seed, id))
|
||||
.collect()
|
||||
}
|
||||
486
services/flysim/crates/fly-session-types/src/trace.rs
Normal file
486
services/flysim/crates/fly-session-types/src/trace.rs
Normal file
|
|
@ -0,0 +1,486 @@
|
|||
//! The trace format of step-v1 section 8, split into behaviour and operational metadata.
|
||||
//!
|
||||
//! Section 8 requires a record, for every transition, of the scope, the Prepare request ids,
|
||||
//! the agent/profile ids, tick counts and remainders, decision digests, the complete batch id
|
||||
//! and control digest, the acknowledged world boundary, observation producing boundaries, task
|
||||
//! event/outcome ids in order, every Commit acknowledgment and the published boundary. It then
|
||||
//! requires that sequential, concurrent and reversed runs "match, excluding wall time, request
|
||||
//! ids and other explicitly operational metadata".
|
||||
//!
|
||||
//! So this record has two halves. [`TraceBehaviour`] is what must match: it is ordered by
|
||||
//! agent id rather than by completion order, so a reversed dispatch produces an identical
|
||||
//! value. [`TraceOperational`] is what section 8 requires recording but excludes from the
|
||||
//! comparison: wall time, the domain request ids, the bus callIds and the delivery ids.
|
||||
//! [`TransitionTrace::behaviour_equals`] compares only the first half, and
|
||||
//! [`TransitionTrace::behaviour_diff`] names the fields that differ.
|
||||
|
||||
use flybus::wire::Fields;
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::canonical;
|
||||
use crate::scalar::{
|
||||
BusCallId, DomainRequestId, DomainType, OwnerToken, RationalNs, Result, Scope, err, is_digest,
|
||||
is_id, list, obj, require_unique, u64_json,
|
||||
};
|
||||
use crate::workers::{MAX_AGENTS, MAX_RATE_ROLES};
|
||||
|
||||
/// One agent's behaviour in one transition.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct TraceAgent {
|
||||
pub agent_id: String,
|
||||
pub profile_digest: String,
|
||||
pub ticks_advanced: u64,
|
||||
pub brain_ticks: u64,
|
||||
pub remainder: RationalNs,
|
||||
pub decision_digest: String,
|
||||
/// The boundary this agent acknowledged in its Commit reply.
|
||||
pub committed_step: u64,
|
||||
}
|
||||
|
||||
/// One view's producing boundary, as observed in this transition.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct TraceObservation {
|
||||
pub view_id: String,
|
||||
pub produced_step: u64,
|
||||
}
|
||||
|
||||
/// The fields two runs of the same transition must agree on.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct TraceBehaviour {
|
||||
pub scope: Scope,
|
||||
/// Sorted by agent id, never by completion order.
|
||||
pub agents: Vec<TraceAgent>,
|
||||
pub batch_id: String,
|
||||
pub control_digest: String,
|
||||
pub acknowledged_boundary: u64,
|
||||
/// Sorted by view id.
|
||||
pub observation_boundaries: Vec<TraceObservation>,
|
||||
/// Task outcome ids in task order.
|
||||
pub outcome_ids: Vec<String>,
|
||||
/// Task event ids in task order.
|
||||
pub event_ids: Vec<String>,
|
||||
pub published_boundary: u64,
|
||||
}
|
||||
|
||||
impl TraceBehaviour {
|
||||
/// Sorts the order-free collections, so a trace recorded in completion order compares
|
||||
/// equal to one recorded in dispatch order.
|
||||
pub fn normalized(&self) -> TraceBehaviour {
|
||||
let mut out = self.clone();
|
||||
out.agents.sort_by(|a, b| a.agent_id.cmp(&b.agent_id));
|
||||
out.observation_boundaries
|
||||
.sort_by(|a, b| a.view_id.cmp(&b.view_id));
|
||||
out
|
||||
}
|
||||
|
||||
pub fn digest(&self) -> Result<String> {
|
||||
canonical::digest_of(&self.normalized().to_json())
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for TraceBehaviour {
|
||||
const TYPE_NAME: &'static str = "TraceBehaviour";
|
||||
|
||||
fn from_json(value: &Value) -> Result<TraceBehaviour> {
|
||||
let mut f = Fields::new(value, "TraceBehaviour")?;
|
||||
let scope = Scope::from_json(f.value("scope")?)?;
|
||||
let agents = list(&mut f, "agents", 1, MAX_AGENTS, |v| {
|
||||
let mut a = Fields::new(v, "TraceBehaviour.agents")?;
|
||||
let agent_id = a.id("agentId")?;
|
||||
let profile_digest = a.string("profileDigest")?.to_owned();
|
||||
let ticks_advanced = a.u64_string("ticksAdvanced")?;
|
||||
let brain_ticks = a.u64_string("brainTicks")?;
|
||||
let remainder = RationalNs::from_json(a.value("remainder")?)?;
|
||||
let decision_digest = a.string("decisionDigest")?.to_owned();
|
||||
let committed_step = a.u64_string("committedStep")?;
|
||||
a.finish()?;
|
||||
Ok(TraceAgent {
|
||||
agent_id,
|
||||
profile_digest,
|
||||
ticks_advanced,
|
||||
brain_ticks,
|
||||
remainder,
|
||||
decision_digest,
|
||||
committed_step,
|
||||
})
|
||||
})?;
|
||||
let batch_id = f.id("batchId")?;
|
||||
let control_digest = f.string("controlDigest")?.to_owned();
|
||||
let acknowledged_boundary = f.u64_string("acknowledgedBoundary")?;
|
||||
let observation_boundaries = list(
|
||||
&mut f,
|
||||
"observationBoundaries",
|
||||
0,
|
||||
crate::media::MAX_VIEWS * 2,
|
||||
|v| {
|
||||
let mut o = Fields::new(v, "TraceBehaviour.observationBoundaries")?;
|
||||
let view_id = o.id("viewId")?;
|
||||
let produced_step = o.u64_string("producedStep")?;
|
||||
o.finish()?;
|
||||
Ok(TraceObservation {
|
||||
view_id,
|
||||
produced_step,
|
||||
})
|
||||
},
|
||||
)?;
|
||||
let outcome_ids = crate::scalar::id_list(&mut f, "outcomeIds", 0, MAX_RATE_ROLES)?;
|
||||
let event_ids = crate::scalar::id_list(&mut f, "eventIds", 0, MAX_RATE_ROLES)?;
|
||||
let published_boundary = f.u64_string("publishedBoundary")?;
|
||||
f.finish()?;
|
||||
let b = TraceBehaviour {
|
||||
scope,
|
||||
agents,
|
||||
batch_id,
|
||||
control_digest,
|
||||
acknowledged_boundary,
|
||||
observation_boundaries,
|
||||
outcome_ids,
|
||||
event_ids,
|
||||
published_boundary,
|
||||
};
|
||||
b.validate()?;
|
||||
Ok(b)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("scope", self.scope.to_json()),
|
||||
(
|
||||
"agents",
|
||||
Value::Array(
|
||||
self.agents
|
||||
.iter()
|
||||
.map(|a| {
|
||||
obj(vec![
|
||||
("agentId", a.agent_id.clone().into()),
|
||||
("profileDigest", a.profile_digest.clone().into()),
|
||||
("ticksAdvanced", u64_json(a.ticks_advanced)),
|
||||
("brainTicks", u64_json(a.brain_ticks)),
|
||||
("remainder", a.remainder.to_json()),
|
||||
("decisionDigest", a.decision_digest.clone().into()),
|
||||
("committedStep", u64_json(a.committed_step)),
|
||||
])
|
||||
})
|
||||
.collect(),
|
||||
),
|
||||
),
|
||||
("batchId", self.batch_id.clone().into()),
|
||||
("controlDigest", self.control_digest.clone().into()),
|
||||
("acknowledgedBoundary", u64_json(self.acknowledged_boundary)),
|
||||
(
|
||||
"observationBoundaries",
|
||||
Value::Array(
|
||||
self.observation_boundaries
|
||||
.iter()
|
||||
.map(|o| {
|
||||
obj(vec![
|
||||
("viewId", o.view_id.clone().into()),
|
||||
("producedStep", u64_json(o.produced_step)),
|
||||
])
|
||||
})
|
||||
.collect(),
|
||||
),
|
||||
),
|
||||
(
|
||||
"outcomeIds",
|
||||
Value::Array(self.outcome_ids.iter().map(|i| i.clone().into()).collect()),
|
||||
),
|
||||
(
|
||||
"eventIds",
|
||||
Value::Array(self.event_ids.iter().map(|i| i.clone().into()).collect()),
|
||||
),
|
||||
("publishedBoundary", u64_json(self.published_boundary)),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
self.scope.validate()?;
|
||||
if self.agents.is_empty() || self.agents.len() > MAX_AGENTS {
|
||||
return err("TraceBehaviour: 1..=4 agents");
|
||||
}
|
||||
require_unique(
|
||||
self.agents.iter().map(|a| a.agent_id.as_str()),
|
||||
"TraceBehaviour.agents",
|
||||
)?;
|
||||
for agent in &self.agents {
|
||||
if !is_id(&agent.agent_id) {
|
||||
return err("TraceBehaviour: agentId is not a valid id");
|
||||
}
|
||||
if !is_digest(&agent.profile_digest) || !is_digest(&agent.decision_digest) {
|
||||
return err("TraceBehaviour: agent digests must be 64 lowercase hex digits");
|
||||
}
|
||||
agent.remainder.validate()?;
|
||||
if agent.committed_step != self.scope.step + 1 {
|
||||
return err(
|
||||
"TraceBehaviour: every commit acknowledgment is the transition's next boundary",
|
||||
);
|
||||
}
|
||||
}
|
||||
if !is_id(&self.batch_id) {
|
||||
return err("TraceBehaviour: batchId is not a valid id");
|
||||
}
|
||||
if !is_digest(&self.control_digest) {
|
||||
return err("TraceBehaviour: controlDigest must be 64 lowercase hex digits");
|
||||
}
|
||||
if self.acknowledged_boundary != self.scope.step + 1 {
|
||||
return err("TraceBehaviour: the acknowledged boundary is scope.step + 1");
|
||||
}
|
||||
if self.published_boundary != self.acknowledged_boundary {
|
||||
return err(
|
||||
"TraceBehaviour: the published boundary is the boundary every agent committed",
|
||||
);
|
||||
}
|
||||
require_unique(
|
||||
self.observation_boundaries
|
||||
.iter()
|
||||
.map(|o| o.view_id.as_str()),
|
||||
"TraceBehaviour.observationBoundaries",
|
||||
)?;
|
||||
require_unique(
|
||||
self.event_ids.iter().map(String::as_str),
|
||||
"TraceBehaviour.eventIds",
|
||||
)?;
|
||||
require_unique(
|
||||
self.outcome_ids.iter().map(String::as_str),
|
||||
"TraceBehaviour.outcomeIds",
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// One agent's domain request id for one phase.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct TraceRequest {
|
||||
pub agent_id: String,
|
||||
pub request_id: DomainRequestId,
|
||||
}
|
||||
|
||||
/// What step-v1 section 8 records but excludes from the comparison.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct TraceOperational {
|
||||
/// Wall time is for pacing, health and presentation only (step-v1 section 5).
|
||||
pub wall_time_ns: u64,
|
||||
pub prepare_request_ids: Vec<TraceRequest>,
|
||||
pub advance_request_id: DomainRequestId,
|
||||
pub commit_request_ids: Vec<TraceRequest>,
|
||||
/// The transport correlation ids this transition happened to use. A safe retry changes
|
||||
/// these and nothing in [`TraceBehaviour`].
|
||||
pub bus_call_ids: Vec<BusCallId>,
|
||||
pub delivery_ids: Vec<OwnerToken>,
|
||||
}
|
||||
|
||||
impl DomainType for TraceOperational {
|
||||
const TYPE_NAME: &'static str = "TraceOperational";
|
||||
|
||||
fn from_json(value: &Value) -> Result<TraceOperational> {
|
||||
let mut f = Fields::new(value, "TraceOperational")?;
|
||||
let wall_time_ns = f.u64_string("wallTimeNs")?;
|
||||
let read_requests = |v: &Value| -> Result<TraceRequest> {
|
||||
let mut r = Fields::new(v, "TraceOperational request")?;
|
||||
let agent_id = r.id("agentId")?;
|
||||
let request_id = DomainRequestId::read(&mut r, "requestId")?;
|
||||
r.finish()?;
|
||||
Ok(TraceRequest {
|
||||
agent_id,
|
||||
request_id,
|
||||
})
|
||||
};
|
||||
let prepare_request_ids = list(&mut f, "prepareRequestIds", 1, MAX_AGENTS, read_requests)?;
|
||||
let advance_request_id = DomainRequestId::read(&mut f, "advanceRequestId")?;
|
||||
let commit_request_ids = list(&mut f, "commitRequestIds", 1, MAX_AGENTS, read_requests)?;
|
||||
let bus_call_ids = list(&mut f, "busCallIds", 0, 64, |v| match v.as_str() {
|
||||
Some(s) => BusCallId::parse(s),
|
||||
None => err("every busCallId must be a string"),
|
||||
})?;
|
||||
let delivery_ids = list(&mut f, "deliveryIds", 0, 64, |v| match v.as_str() {
|
||||
Some(s) => OwnerToken::parse(s),
|
||||
None => err("every deliveryId must be a string"),
|
||||
})?;
|
||||
f.finish()?;
|
||||
let o = TraceOperational {
|
||||
wall_time_ns,
|
||||
prepare_request_ids,
|
||||
advance_request_id,
|
||||
commit_request_ids,
|
||||
bus_call_ids,
|
||||
delivery_ids,
|
||||
};
|
||||
o.validate()?;
|
||||
Ok(o)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
let requests = |items: &[TraceRequest]| {
|
||||
Value::Array(
|
||||
items
|
||||
.iter()
|
||||
.map(|r| {
|
||||
obj(vec![
|
||||
("agentId", r.agent_id.clone().into()),
|
||||
("requestId", r.request_id.to_json()),
|
||||
])
|
||||
})
|
||||
.collect(),
|
||||
)
|
||||
};
|
||||
obj(vec![
|
||||
("wallTimeNs", u64_json(self.wall_time_ns)),
|
||||
("prepareRequestIds", requests(&self.prepare_request_ids)),
|
||||
("advanceRequestId", self.advance_request_id.to_json()),
|
||||
("commitRequestIds", requests(&self.commit_request_ids)),
|
||||
(
|
||||
"busCallIds",
|
||||
Value::Array(self.bus_call_ids.iter().map(BusCallId::to_json).collect()),
|
||||
),
|
||||
(
|
||||
"deliveryIds",
|
||||
Value::Array(
|
||||
self.delivery_ids
|
||||
.iter()
|
||||
.map(|t| Value::String(t.as_str().to_owned()))
|
||||
.collect(),
|
||||
),
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
require_unique(
|
||||
self.prepare_request_ids.iter().map(|r| r.agent_id.as_str()),
|
||||
"TraceOperational.prepareRequestIds",
|
||||
)?;
|
||||
require_unique(
|
||||
self.commit_request_ids.iter().map(|r| r.agent_id.as_str()),
|
||||
"TraceOperational.commitRequestIds",
|
||||
)?;
|
||||
require_unique(
|
||||
self.bus_call_ids.iter().map(BusCallId::as_str),
|
||||
"TraceOperational.busCallIds",
|
||||
)?;
|
||||
require_unique(
|
||||
self.delivery_ids.iter().map(OwnerToken::as_str),
|
||||
"TraceOperational.deliveryIds",
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// One transition's trace: behaviour plus operational metadata.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct TransitionTrace {
|
||||
pub behaviour: TraceBehaviour,
|
||||
pub operational: TraceOperational,
|
||||
}
|
||||
|
||||
impl TransitionTrace {
|
||||
/// Behaviour equality: the comparison step-v1 section 8 asks for.
|
||||
pub fn behaviour_equals(&self, other: &TransitionTrace) -> bool {
|
||||
self.behaviour.normalized() == other.behaviour.normalized()
|
||||
}
|
||||
|
||||
/// The behaviour fields that differ, named. Empty when [`Self::behaviour_equals`] holds.
|
||||
pub fn behaviour_diff(&self, other: &TransitionTrace) -> Vec<String> {
|
||||
let (a, b) = (self.behaviour.normalized(), other.behaviour.normalized());
|
||||
let mut out = Vec::new();
|
||||
if a.scope != b.scope {
|
||||
out.push(format!("scope: {:?} vs {:?}", a.scope, b.scope));
|
||||
}
|
||||
if a.batch_id != b.batch_id {
|
||||
out.push(format!("batchId: {} vs {}", a.batch_id, b.batch_id));
|
||||
}
|
||||
if a.control_digest != b.control_digest {
|
||||
out.push("controlDigest differs".to_owned());
|
||||
}
|
||||
if a.acknowledged_boundary != b.acknowledged_boundary {
|
||||
out.push(format!(
|
||||
"acknowledgedBoundary: {} vs {}",
|
||||
a.acknowledged_boundary, b.acknowledged_boundary
|
||||
));
|
||||
}
|
||||
if a.published_boundary != b.published_boundary {
|
||||
out.push(format!(
|
||||
"publishedBoundary: {} vs {}",
|
||||
a.published_boundary, b.published_boundary
|
||||
));
|
||||
}
|
||||
if a.observation_boundaries != b.observation_boundaries {
|
||||
out.push("observationBoundaries differ".to_owned());
|
||||
}
|
||||
if a.outcome_ids != b.outcome_ids {
|
||||
out.push("outcomeIds differ".to_owned());
|
||||
}
|
||||
if a.event_ids != b.event_ids {
|
||||
out.push("eventIds differ".to_owned());
|
||||
}
|
||||
let ids_a: Vec<&str> = a.agents.iter().map(|x| x.agent_id.as_str()).collect();
|
||||
let ids_b: Vec<&str> = b.agents.iter().map(|x| x.agent_id.as_str()).collect();
|
||||
if ids_a != ids_b {
|
||||
out.push(format!(
|
||||
"agents: [{}] vs [{}]",
|
||||
ids_a.join(", "),
|
||||
ids_b.join(", ")
|
||||
));
|
||||
} else {
|
||||
for (left, right) in a.agents.iter().zip(&b.agents) {
|
||||
if left != right {
|
||||
out.push(format!("agent {}: behaviour differs", left.agent_id));
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Two whole runs agree on behaviour, transition by transition.
|
||||
pub fn runs_equal(left: &[TransitionTrace], right: &[TransitionTrace]) -> bool {
|
||||
left.len() == right.len() && left.iter().zip(right).all(|(a, b)| a.behaviour_equals(b))
|
||||
}
|
||||
}
|
||||
|
||||
impl DomainType for TransitionTrace {
|
||||
const TYPE_NAME: &'static str = "TransitionTrace";
|
||||
|
||||
fn from_json(value: &Value) -> Result<TransitionTrace> {
|
||||
let mut f = Fields::new(value, "TransitionTrace")?;
|
||||
let behaviour = TraceBehaviour::from_json(f.value("behaviour")?)?;
|
||||
let operational = TraceOperational::from_json(f.value("operational")?)?;
|
||||
f.finish()?;
|
||||
let t = TransitionTrace {
|
||||
behaviour,
|
||||
operational,
|
||||
};
|
||||
t.validate()?;
|
||||
Ok(t)
|
||||
}
|
||||
|
||||
fn to_json(&self) -> Value {
|
||||
obj(vec![
|
||||
("behaviour", self.behaviour.to_json()),
|
||||
("operational", self.operational.to_json()),
|
||||
])
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
self.behaviour.validate()?;
|
||||
self.operational.validate()?;
|
||||
let behaviour_agents: Vec<&str> = self
|
||||
.behaviour
|
||||
.agents
|
||||
.iter()
|
||||
.map(|a| a.agent_id.as_str())
|
||||
.collect();
|
||||
for phase in [
|
||||
&self.operational.prepare_request_ids,
|
||||
&self.operational.commit_request_ids,
|
||||
] {
|
||||
for request in phase {
|
||||
if !behaviour_agents.contains(&request.agent_id.as_str()) {
|
||||
return err(format!(
|
||||
"TransitionTrace: request recorded for {:?}, which is not in the transition",
|
||||
request.agent_id
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
2405
services/flysim/crates/fly-session-types/src/workers.rs
Normal file
2405
services/flysim/crates/fly-session-types/src/workers.rs
Normal file
File diff suppressed because it is too large
Load diff
192
services/flysim/crates/fly-session-types/tests/canonical_json.rs
Normal file
192
services/flysim/crates/fly-session-types/tests/canonical_json.rs
Normal file
|
|
@ -0,0 +1,192 @@
|
|||
//! Canonical JSON (RFC 8785) and the digest rules of ipc-v1 section 5.
|
||||
|
||||
use fly_session_types::scalar::{DomainType, Scope};
|
||||
use fly_session_types::{canonical, fixtures};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
#[test]
|
||||
fn object_keys_are_sorted_by_utf16_code_unit() {
|
||||
let value = json!({"b": 1, "a": 2, "A": 3, "\u{00e9}": 4, "\u{10400}": 5, "\u{ff21}": 6});
|
||||
assert_eq!(
|
||||
canonical::canonicalize(&value).expect("canonicalizable"),
|
||||
"{\"A\":3,\"a\":2,\"b\":1,\"\u{00e9}\":4,\"\u{10400}\":5,\"\u{ff21}\":6}",
|
||||
"keys sort by UTF-16 code unit, so an astral key (leading surrogate D801) sorts \
|
||||
before U+FF21, which is where a JavaScript string sort puts it too and where a sort \
|
||||
by Unicode code point would not"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn numbers_print_the_way_ecmascript_prints_them() {
|
||||
let file = fixtures::load("boundaries.json").expect("boundaries.json");
|
||||
for case in file.get("doubles").and_then(Value::as_array).expect("doubles") {
|
||||
let value = case.get("value").expect("value");
|
||||
let accept = case.get("accept").and_then(Value::as_bool).expect("accept");
|
||||
let outcome = canonical::canonicalize(value);
|
||||
assert_eq!(
|
||||
outcome.is_ok(),
|
||||
accept,
|
||||
"{value}: {}",
|
||||
fixtures::field(case, "reason").unwrap_or("")
|
||||
);
|
||||
if let (Ok(text), Some(expected)) = (outcome, case.get("canonical").and_then(Value::as_str))
|
||||
{
|
||||
assert_eq!(text, expected, "{value} must print as {expected}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn strings_are_escaped_the_way_json_stringify_escapes_them() {
|
||||
let value = json!({"s": "quote \" backslash \\ tab \t newline \n bell \u{7} del \u{7f} e\u{301}"});
|
||||
assert_eq!(
|
||||
canonical::canonicalize(&value).expect("canonicalizable"),
|
||||
"{\"s\":\"quote \\\" backslash \\\\ tab \\t newline \\n bell \\u0007 del \u{7f} e\u{301}\"}",
|
||||
"only the escapes JSON.stringify emits, with lowercase hex"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn canonical_form_does_not_depend_on_the_input_formatting() {
|
||||
let compact = br#"{"b":[1,2,{"y":true,"x":null}],"a":"z"}"#;
|
||||
let pretty = br#"{
|
||||
"a" : "z",
|
||||
"b": [ 1, 2, { "x": null, "y": true } ]
|
||||
}"#;
|
||||
let left = canonical::parse_strict(compact).expect("parses");
|
||||
let right = canonical::parse_strict(pretty).expect("parses");
|
||||
assert_eq!(
|
||||
canonical::canonicalize(&left).expect("canonicalizable"),
|
||||
canonical::canonicalize(&right).expect("canonicalizable")
|
||||
);
|
||||
assert_eq!(
|
||||
canonical::digest_of(&left).expect("digest"),
|
||||
canonical::digest_of(&right).expect("digest"),
|
||||
"whitespace and key order are not content"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn duplicate_keys_and_invalid_utf8_never_parse() {
|
||||
assert!(canonical::parse_strict(br#"{"a":1,"a":2}"#).is_err());
|
||||
assert!(canonical::parse_strict(b"{\"a\":\"\xff\"}").is_err());
|
||||
assert!(canonical::parse_strict(br#"{"a":1} {"b":2}"#).is_err());
|
||||
assert!(canonical::parse_strict(br#"{"a":NaN}"#).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_envelope_over_64_kib_is_refused() {
|
||||
let big = json!({"pad": "a".repeat(canonical::MAX_ENVELOPE_BYTES)});
|
||||
assert!(canonical::require_envelope_fit(&big, 0).is_err());
|
||||
let small = json!({"pad": "a"});
|
||||
let length = canonical::canonicalize(&small).expect("canonicalizable").len();
|
||||
assert_eq!(
|
||||
canonical::require_envelope_fit(&small, canonical::MAX_ENVELOPE_BYTES - length)
|
||||
.expect("fits exactly"),
|
||||
canonical::MAX_ENVELOPE_BYTES
|
||||
);
|
||||
assert!(
|
||||
canonical::require_envelope_fit(&small, canonical::MAX_ENVELOPE_BYTES - length + 1)
|
||||
.is_err(),
|
||||
"one byte past the ceiling is refused"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn operation_keys_match_the_fixture_and_separate_the_operations_they_should() {
|
||||
let file = fixtures::load("operations.json").expect("operations.json");
|
||||
let mut digests: Vec<(String, String)> = Vec::new();
|
||||
for case in file.get("keys").and_then(Value::as_array).expect("keys") {
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let scope = Scope::from_json(case.get("scope").expect("scope")).expect("scope");
|
||||
let method = fixtures::field(case, "method").expect("method");
|
||||
let worker = fixtures::field(case, "workerId").expect("workerId");
|
||||
let key = canonical::OperationKey::new(scope, method, worker).expect("a key");
|
||||
let digest = key.digest().expect("digest");
|
||||
assert_eq!(
|
||||
digest,
|
||||
fixtures::field(case, "digest").expect("digest"),
|
||||
"{name}: operation key digest must match the fixture"
|
||||
);
|
||||
digests.push((name.to_owned(), digest));
|
||||
}
|
||||
for (index, (name, digest)) in digests.iter().enumerate() {
|
||||
for (other_name, other) in &digests[index + 1..] {
|
||||
assert_ne!(
|
||||
digest, other,
|
||||
"{name} and {other_name} are different operations"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn canonical_bodies_match_the_fixture() {
|
||||
let file = fixtures::load("operations.json").expect("operations.json");
|
||||
for case in file.get("bodies").and_then(Value::as_array).expect("bodies") {
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let method = fixtures::field(case, "method").expect("method");
|
||||
let scope = match case.get("scope") {
|
||||
Some(Value::Null) | None => None,
|
||||
Some(v) => Some(Scope::from_json(v).expect("scope")),
|
||||
};
|
||||
let params = case.get("params").expect("params");
|
||||
assert_eq!(
|
||||
canonical::body_digest(method, scope.as_ref(), params).expect("digest"),
|
||||
fixtures::field(case, "digest").expect("digest"),
|
||||
"{name}: canonical body digest must match the fixture"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The pairs that decide whether a duplicate is a safe replay or a CONFLICT.
|
||||
#[test]
|
||||
fn operation_pairs_agree_with_the_fixture_about_sameness() {
|
||||
let file = fixtures::load("operations.json").expect("operations.json");
|
||||
for case in file.get("pairs").and_then(Value::as_array).expect("pairs") {
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let reason = fixtures::field(case, "reason").expect("reason");
|
||||
let worker = fixtures::field(case, "workerId").expect("workerId");
|
||||
let right_worker = fixtures::field(case, "rightWorkerId").unwrap_or(worker);
|
||||
let side = |key: &str, worker: &str| {
|
||||
let value = case.get(key).expect("side");
|
||||
let method = fixtures::field(value, "method").expect("method");
|
||||
let scope = Scope::from_json(value.get("scope").expect("scope")).expect("scope");
|
||||
let params = value.get("params").expect("params");
|
||||
let key_digest = canonical::OperationKey::new(scope.clone(), method, worker)
|
||||
.expect("a key")
|
||||
.digest()
|
||||
.expect("digest");
|
||||
let body = canonical::body_digest(method, Some(&scope), params).expect("digest");
|
||||
(key_digest, body)
|
||||
};
|
||||
let (left_key, left_body) = side("left", worker);
|
||||
let (right_key, right_body) = side("right", right_worker);
|
||||
assert_eq!(
|
||||
left_key == right_key,
|
||||
case.get("sameKey").and_then(Value::as_bool).expect("sameKey"),
|
||||
"{name}: operation key sameness. {reason}"
|
||||
);
|
||||
assert_eq!(
|
||||
left_body == right_body,
|
||||
case.get("sameBody").and_then(Value::as_bool).expect("sameBody"),
|
||||
"{name}: canonical body sameness. {reason}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_domain_body_can_never_carry_a_bus_identity() {
|
||||
let file = fixtures::load("operations.json").expect("operations.json");
|
||||
for case in file.get("rejected").and_then(Value::as_array).expect("rejected") {
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let method = fixtures::field(case, "method").expect("method");
|
||||
let scope = Scope::from_json(case.get("scope").expect("scope")).expect("scope");
|
||||
let params = case.get("params").expect("params");
|
||||
assert!(
|
||||
canonical::body_digest(method, Some(&scope), params).is_err(),
|
||||
"{name} must be refused: {}",
|
||||
fixtures::field(case, "reason").unwrap_or("")
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,165 @@
|
|||
//! The `FLYSESS1` envelope layout: its fixture, its offsets and the corruptions it refuses.
|
||||
|
||||
use fly_session_types::{canonical, checkpoint, fixtures};
|
||||
use serde_json::Value;
|
||||
|
||||
fn envelope_bytes(file: &Value) -> Vec<u8> {
|
||||
fixtures::decode_base64(
|
||||
file.get("envelope")
|
||||
.and_then(|e| e.get("base64"))
|
||||
.and_then(Value::as_str)
|
||||
.expect("base64"),
|
||||
)
|
||||
.expect("base64")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_fixture_envelope_decodes_to_its_recorded_layout() {
|
||||
let file = fixtures::load("checkpoint-envelope.json").expect("checkpoint-envelope.json");
|
||||
let bytes = envelope_bytes(&file);
|
||||
let envelope = checkpoint::decode(&bytes).expect("a valid envelope");
|
||||
checkpoint::validate_manifest(&envelope).expect("a complete manifest");
|
||||
|
||||
let layout = file["envelope"]["layout"].clone();
|
||||
assert_eq!(&bytes[0..8], checkpoint::MAGIC);
|
||||
assert_eq!(
|
||||
bytes.len().to_string(),
|
||||
layout["totalBytes"].as_str().expect("totalBytes")
|
||||
);
|
||||
assert_eq!(
|
||||
envelope.layout.table_offset.to_string(),
|
||||
layout["tableOffset"].as_str().expect("tableOffset")
|
||||
);
|
||||
assert_eq!(
|
||||
envelope.layout.manifest_bytes,
|
||||
layout["manifestBytes"].as_u64().expect("manifestBytes") as u32
|
||||
);
|
||||
let entries = layout["entries"].as_array().expect("entries");
|
||||
assert_eq!(envelope.layout.entries.len(), entries.len());
|
||||
for (entry, recorded) in envelope.layout.entries.iter().zip(entries) {
|
||||
assert_eq!(entry.name, recorded["name"].as_str().expect("name"));
|
||||
assert_eq!(
|
||||
entry.offset.to_string(),
|
||||
recorded["offset"].as_str().expect("offset")
|
||||
);
|
||||
assert_eq!(
|
||||
entry.byte_length.to_string(),
|
||||
recorded["byteLength"].as_str().expect("byteLength")
|
||||
);
|
||||
assert_eq!(
|
||||
checkpoint::hex(&entry.digest),
|
||||
recorded["digest"].as_str().expect("digest")
|
||||
);
|
||||
assert_eq!(entry.offset % 8, 0, "payloads start on an eight-byte boundary");
|
||||
}
|
||||
|
||||
for payload in file["payloads"].as_array().expect("payloads") {
|
||||
let name = payload["name"].as_str().expect("name");
|
||||
let expected = fixtures::decode_base64(payload["base64"].as_str().expect("base64"))
|
||||
.expect("base64");
|
||||
assert_eq!(
|
||||
envelope.payload(name).expect("a payload"),
|
||||
expected.as_slice(),
|
||||
"payload {name} must come back byte for byte"
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
envelope.manifest,
|
||||
file["manifest"],
|
||||
"the manifest round trips as canonical JSON"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_recorded_corruption_is_refused() {
|
||||
let file = fixtures::load("checkpoint-envelope.json").expect("checkpoint-envelope.json");
|
||||
let bytes = envelope_bytes(&file);
|
||||
for case in file["corruption"].as_array().expect("corruption") {
|
||||
let name = case["name"].as_str().expect("name");
|
||||
let offset = case["offset"].as_u64().expect("offset") as usize;
|
||||
let mut corrupted = bytes.clone();
|
||||
corrupted[offset] ^= 0x01;
|
||||
assert!(
|
||||
checkpoint::decode(&corrupted).is_err(),
|
||||
"{name} must be refused: {}",
|
||||
case["reason"].as_str().unwrap_or("")
|
||||
);
|
||||
}
|
||||
let truncated = &bytes[..bytes.len() - 1];
|
||||
assert!(
|
||||
checkpoint::decode(truncated).is_err(),
|
||||
"a truncated envelope must be refused"
|
||||
);
|
||||
assert!(
|
||||
checkpoint::decode(&bytes[..8]).is_err(),
|
||||
"a header alone is not an envelope"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_flysim01_envelope_is_not_read_as_a_session_checkpoint() {
|
||||
// The historical envelope: magic, u32 manifest length, manifest, chunks, CRC32.
|
||||
let mut legacy = Vec::new();
|
||||
legacy.extend_from_slice(b"FLYSIM01");
|
||||
let manifest = br#"{"schemaVersion":2,"chunks":[]}"#;
|
||||
legacy.extend_from_slice(&(manifest.len() as u32).to_le_bytes());
|
||||
legacy.extend_from_slice(manifest);
|
||||
legacy.extend_from_slice(&0u32.to_le_bytes());
|
||||
assert!(
|
||||
checkpoint::decode(&legacy).is_err(),
|
||||
"FLYSESS1 is a new format; the old reader stays separate"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_layout_is_deterministic_and_the_manifest_is_canonical() {
|
||||
let manifest = canonical::parse_strict(br#"{"b":2,"a":1}"#).expect("parses");
|
||||
let payloads = vec![
|
||||
("one".to_owned(), b"first".to_vec()),
|
||||
("two".to_owned(), vec![0u8; 9]),
|
||||
];
|
||||
let bytes = checkpoint::encode(&manifest, &payloads).expect("encode");
|
||||
let again = checkpoint::encode(&manifest, &payloads).expect("encode");
|
||||
assert_eq!(bytes, again, "the same inputs produce the same bytes");
|
||||
let envelope = checkpoint::decode(&bytes).expect("decode");
|
||||
let start = checkpoint::HEADER_BYTES;
|
||||
let end = start + envelope.layout.manifest_bytes as usize;
|
||||
assert_eq!(
|
||||
std::str::from_utf8(&bytes[start..end]).expect("utf-8"),
|
||||
"{\"a\":1,\"b\":2}",
|
||||
"the manifest is stored as canonical JSON"
|
||||
);
|
||||
assert_eq!(envelope.layout.entries[1].offset % 8, 0);
|
||||
assert!(
|
||||
checkpoint::encode(
|
||||
&manifest,
|
||||
&[("one".to_owned(), vec![]), ("one".to_owned(), vec![])]
|
||||
)
|
||||
.is_err(),
|
||||
"payload names are unique"
|
||||
);
|
||||
assert!(
|
||||
checkpoint::encode(&manifest, &[("One".to_owned(), vec![])]).is_err(),
|
||||
"payload names are Ids, and the widening from letters-only is deliberate, not arbitrary"
|
||||
);
|
||||
assert!(
|
||||
checkpoint::encode(&manifest, &[("a".to_owned(), Vec::new())]).is_ok(),
|
||||
"an empty payload is still a payload"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_manifest_missing_a_required_field_is_not_a_complete_checkpoint() {
|
||||
let file = fixtures::load("checkpoint-envelope.json").expect("checkpoint-envelope.json");
|
||||
let full = file["manifest"].clone();
|
||||
for field in checkpoint::REQUIRED_MANIFEST_FIELDS {
|
||||
let mut manifest = full.clone();
|
||||
manifest.as_object_mut().expect("object").remove(*field);
|
||||
let bytes = checkpoint::encode(&manifest, &[]).expect("encode");
|
||||
let envelope = checkpoint::decode(&bytes).expect("decode");
|
||||
assert!(
|
||||
checkpoint::validate_manifest(&envelope).is_err(),
|
||||
"a manifest without {field:?} must be refused"
|
||||
);
|
||||
}
|
||||
}
|
||||
133
services/flysim/crates/fly-session-types/tests/common/mod.rs
Normal file
133
services/flysim/crates/fly-session-types/tests/common/mod.rs
Normal file
|
|
@ -0,0 +1,133 @@
|
|||
//! One place that knows how to read every type named by a fixture.
|
||||
|
||||
use flybus::wire::WireError;
|
||||
use fly_session_types::media::*;
|
||||
use fly_session_types::publishing::*;
|
||||
use fly_session_types::rpc::*;
|
||||
use fly_session_types::scalar::*;
|
||||
use fly_session_types::trace::*;
|
||||
use fly_session_types::workers::*;
|
||||
use serde_json::Value;
|
||||
|
||||
/// Reads the value as `type_name`, re-runs its validate step and writes it back out.
|
||||
///
|
||||
/// Every fixture assertion goes through this, so a type that reads a field but forgets to
|
||||
/// write it back fails the round trip.
|
||||
pub fn round_trip(type_name: &str, value: &Value) -> std::result::Result<Value, WireError> {
|
||||
macro_rules! arm {
|
||||
($t:ty) => {
|
||||
if type_name == <$t as DomainType>::TYPE_NAME {
|
||||
let parsed = <$t as DomainType>::from_json(value)?;
|
||||
parsed.validate()?;
|
||||
return Ok(parsed.to_json());
|
||||
}
|
||||
};
|
||||
}
|
||||
arm!(Scope);
|
||||
arm!(RationalNs);
|
||||
arm!(SchemaRef);
|
||||
arm!(TypedValue);
|
||||
arm!(SessionRpcRequest);
|
||||
arm!(SessionRpcSuccess);
|
||||
arm!(SessionRpcFailure);
|
||||
arm!(AssetRef);
|
||||
arm!(SensoryInput);
|
||||
arm!(Stimulus);
|
||||
arm!(Reward);
|
||||
arm!(AgentTelemetry);
|
||||
arm!(AgentInitializeParams);
|
||||
arm!(AgentInitializeResult);
|
||||
arm!(PrepareParams);
|
||||
arm!(PreparedDecision);
|
||||
arm!(CommitParams);
|
||||
arm!(AgentCommitResult);
|
||||
arm!(ControllerSchema);
|
||||
arm!(PortControl);
|
||||
arm!(EnvironmentDescriptor);
|
||||
arm!(EnvironmentInitializeParams);
|
||||
arm!(EnvironmentInitializeResult);
|
||||
arm!(WorldObservation);
|
||||
arm!(AdvanceParams);
|
||||
arm!(StepResult);
|
||||
arm!(HelloParams);
|
||||
arm!(HelloResult);
|
||||
arm!(StatusResult);
|
||||
arm!(AcknowledgeParams);
|
||||
arm!(AcknowledgeResult);
|
||||
arm!(ShutdownParams);
|
||||
arm!(ShutdownResult);
|
||||
arm!(TaskEvent);
|
||||
arm!(EpisodeRequest);
|
||||
arm!(ViewDescriptor);
|
||||
arm!(ViewRef);
|
||||
arm!(AudioDescriptor);
|
||||
arm!(AudioRef);
|
||||
arm!(CaptureParams);
|
||||
arm!(CaptureResult);
|
||||
arm!(StageRestoreParams);
|
||||
arm!(StageRestoreResult);
|
||||
arm!(ActivateRestoreParams);
|
||||
arm!(ActivateRestoreResult);
|
||||
arm!(SessionDescriptor);
|
||||
arm!(CommittedSnapshot);
|
||||
arm!(TraceBehaviour);
|
||||
arm!(TraceOperational);
|
||||
arm!(TransitionTrace);
|
||||
Err(WireError(format!(
|
||||
"no fixture reader for type {type_name:?}"
|
||||
)))
|
||||
}
|
||||
|
||||
/// The type names `round_trip` knows.
|
||||
pub const READABLE_TYPES: &[&str] = &[
|
||||
"Scope",
|
||||
"RationalNs",
|
||||
"SchemaRef",
|
||||
"TypedValue",
|
||||
"SessionRpcRequest",
|
||||
"SessionRpcSuccess",
|
||||
"SessionRpcFailure",
|
||||
"AssetRef",
|
||||
"SensoryInput",
|
||||
"Stimulus",
|
||||
"Reward",
|
||||
"AgentTelemetry",
|
||||
"AgentInitializeParams",
|
||||
"AgentInitializeResult",
|
||||
"PrepareParams",
|
||||
"PreparedDecision",
|
||||
"CommitParams",
|
||||
"AgentCommitResult",
|
||||
"ControllerSchema",
|
||||
"PortControl",
|
||||
"EnvironmentDescriptor",
|
||||
"EnvironmentInitializeParams",
|
||||
"EnvironmentInitializeResult",
|
||||
"WorldObservation",
|
||||
"AdvanceParams",
|
||||
"StepResult",
|
||||
"HelloParams",
|
||||
"HelloResult",
|
||||
"StatusResult",
|
||||
"AcknowledgeParams",
|
||||
"AcknowledgeResult",
|
||||
"ShutdownParams",
|
||||
"ShutdownResult",
|
||||
"TaskEvent",
|
||||
"EpisodeRequest",
|
||||
"ViewDescriptor",
|
||||
"ViewRef",
|
||||
"AudioDescriptor",
|
||||
"AudioRef",
|
||||
"CaptureParams",
|
||||
"CaptureResult",
|
||||
"StageRestoreParams",
|
||||
"StageRestoreResult",
|
||||
"ActivateRestoreParams",
|
||||
"ActivateRestoreResult",
|
||||
"SessionDescriptor",
|
||||
"CommittedSnapshot",
|
||||
"TraceBehaviour",
|
||||
"TraceOperational",
|
||||
"TransitionTrace",
|
||||
];
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
//! Rules that need a descriptor in hand: complete batches, the observation delay rule, byte
|
||||
//! shapes and descriptor agreement.
|
||||
|
||||
use fly_session_types::fixtures;
|
||||
use fly_session_types::publishing::{CommittedSnapshot, SessionDescriptor};
|
||||
use fly_session_types::scalar::{DomainType, Result};
|
||||
use fly_session_types::workers::{
|
||||
EnvironmentDescriptor, PortControl, SensoryInput, StepResult, WorldObservation,
|
||||
};
|
||||
|
||||
#[test]
|
||||
fn every_descriptor_check_lands_the_way_the_fixture_says() {
|
||||
let file = fixtures::load("descriptor-checks.json").expect("descriptor-checks.json");
|
||||
let descriptor =
|
||||
EnvironmentDescriptor::from_json(file.get("descriptor").expect("descriptor")).expect("descriptor");
|
||||
let delayed = EnvironmentDescriptor::from_json(file.get("delayedDescriptor").expect("delayed"))
|
||||
.expect("delayed descriptor");
|
||||
let session = SessionDescriptor::from_json(file.get("sessionDescriptor").expect("session"))
|
||||
.expect("session descriptor");
|
||||
let previous = WorldObservation::from_json(file.get("stepResultPrevious").expect("previous"))
|
||||
.expect("previous observation");
|
||||
|
||||
for case in fixtures::cases(&file).expect("cases") {
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let kind = fixtures::field(case, "kind").expect("kind");
|
||||
let reason = fixtures::field(case, "reason").unwrap_or("");
|
||||
let expect_accept = fixtures::field(case, "expect").expect("expect") == "accept";
|
||||
let value = case.get("value").expect("value");
|
||||
let outcome: Result<()> = match kind {
|
||||
"portControl" => PortControl::from_json(value).and_then(|control| {
|
||||
match descriptor.port(&control.port_id) {
|
||||
Some(port) => control.validate_against(&port.controls),
|
||||
None => fly_session_types::scalar::err("no such port"),
|
||||
}
|
||||
}),
|
||||
"advanceControls" => value
|
||||
.as_array()
|
||||
.expect("an array of controls")
|
||||
.iter()
|
||||
.map(PortControl::from_json)
|
||||
.collect::<Result<Vec<_>>>()
|
||||
.and_then(|controls| descriptor.validate_batch(&controls)),
|
||||
"sensoryInput" => SensoryInput::from_json(value)
|
||||
.and_then(|input| input.validate_against(&descriptor.views)),
|
||||
"sensoryInputDelayed" => {
|
||||
SensoryInput::from_json(value).and_then(|input| input.validate_against(&delayed.views))
|
||||
}
|
||||
"worldObservation" => WorldObservation::from_json(value)
|
||||
.and_then(|observation| observation.validate_against(&descriptor)),
|
||||
"stepResult" => StepResult::from_json(value)
|
||||
.and_then(|result| result.validate_against(&descriptor, &previous)),
|
||||
"snapshot" => CommittedSnapshot::from_json(value)
|
||||
.and_then(|snapshot| snapshot.validate_against(&session)),
|
||||
other => panic!("unknown descriptor check kind {other:?}"),
|
||||
};
|
||||
assert_eq!(
|
||||
outcome.is_ok(),
|
||||
expect_accept,
|
||||
"{name}: expected {}. {reason}. outcome: {outcome:?}",
|
||||
if expect_accept { "accept" } else { "reject" }
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The delay rule itself, stated once: `max(0, boundary - observationDelaySteps)`.
|
||||
#[test]
|
||||
fn the_required_producing_boundary_saturates_at_zero() {
|
||||
let file = fixtures::load("descriptor-checks.json").expect("descriptor-checks.json");
|
||||
let delayed = EnvironmentDescriptor::from_json(file.get("delayedDescriptor").expect("delayed"))
|
||||
.expect("delayed descriptor");
|
||||
let view = &delayed.views[0];
|
||||
assert_eq!(view.observation_delay_steps, 2);
|
||||
assert_eq!(view.required_produced_step(0), 0);
|
||||
assert_eq!(view.required_produced_step(1), 0);
|
||||
assert_eq!(view.required_produced_step(2), 0);
|
||||
assert_eq!(view.required_produced_step(3), 1);
|
||||
assert_eq!(view.frame_bytes(), u64::from(160u32 * 4 * 144));
|
||||
}
|
||||
169
services/flysim/crates/fly-session-types/tests/encodings.rs
Normal file
169
services/flysim/crates/fly-session-types/tests/encodings.rs
Normal file
|
|
@ -0,0 +1,169 @@
|
|||
//! The scalar encodings agree with the bus's, and the four identities cannot be confused.
|
||||
|
||||
use fly_session_types::fixtures;
|
||||
use fly_session_types::scalar::{
|
||||
ArtifactIdentity, BusCallId, DomainRequestId, OwnerKind, OwnerToken, is_digest, is_id, parse_u64,
|
||||
};
|
||||
use serde_json::Value;
|
||||
|
||||
/// The domain `Id`, `U64` and `Digest` are the bus encodings, not a second opinion about them.
|
||||
#[test]
|
||||
fn domain_scalars_are_the_bus_scalars() {
|
||||
let ids = [
|
||||
"a",
|
||||
"fly-a",
|
||||
"0",
|
||||
"a.b_c-d",
|
||||
"",
|
||||
"A",
|
||||
"-a",
|
||||
".a",
|
||||
"a b",
|
||||
"fly/a",
|
||||
&"a".repeat(64),
|
||||
&"a".repeat(65),
|
||||
];
|
||||
for id in ids {
|
||||
assert_eq!(
|
||||
is_id(id),
|
||||
flybus::wire::is_id(id),
|
||||
"Id encoding must agree with the bus for {id:?}"
|
||||
);
|
||||
}
|
||||
let numbers = [
|
||||
"0",
|
||||
"1",
|
||||
"18446744073709551615",
|
||||
"18446744073709551616",
|
||||
"01",
|
||||
"",
|
||||
"-1",
|
||||
"1.0",
|
||||
" 1",
|
||||
];
|
||||
for text in numbers {
|
||||
assert_eq!(
|
||||
parse_u64(text),
|
||||
flybus::wire::parse_u64(text),
|
||||
"U64 encoding must agree with the bus for {text:?}"
|
||||
);
|
||||
}
|
||||
let digests = [
|
||||
&"a".repeat(64),
|
||||
&"0".repeat(64),
|
||||
&"A".repeat(64),
|
||||
&"g".repeat(64),
|
||||
&"a".repeat(63),
|
||||
];
|
||||
for digest in digests {
|
||||
assert_eq!(
|
||||
is_digest(digest),
|
||||
flybus::wire::is_digest(digest),
|
||||
"Digest encoding must agree with the bus"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn u64_boundaries_reject_from_the_fixture() {
|
||||
let file = fixtures::load("boundaries.json").expect("boundaries.json");
|
||||
let cases = file.get("u64").and_then(Value::as_array).expect("u64");
|
||||
for case in cases {
|
||||
let text = fixtures::field(case, "text").expect("text");
|
||||
let accept = case.get("accept").and_then(Value::as_bool).expect("accept");
|
||||
assert_eq!(
|
||||
parse_u64(text).is_some(),
|
||||
accept,
|
||||
"{text:?}: {}",
|
||||
fixtures::field(case, "reason").unwrap_or("")
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// bus callId, domain requestId and delivery/hold owner tokens are four types. The fixture
|
||||
/// says, for every spelling, which of them accept it: no string is accepted by two.
|
||||
#[test]
|
||||
fn the_four_identities_never_accept_each_others_spellings() {
|
||||
let file = fixtures::load("identities.json").expect("identities.json");
|
||||
for case in fixtures::cases(&file).expect("cases") {
|
||||
let text = fixtures::field(case, "text").expect("text");
|
||||
let call = case.get("busCallId").and_then(Value::as_bool).expect("busCallId");
|
||||
let request = case
|
||||
.get("domainRequestId")
|
||||
.and_then(Value::as_bool)
|
||||
.expect("domainRequestId");
|
||||
let owner = case.get("ownerToken").expect("ownerToken");
|
||||
assert_eq!(BusCallId::parse(text).is_ok(), call, "busCallId {text:?}");
|
||||
assert_eq!(
|
||||
DomainRequestId::parse(text).is_ok(),
|
||||
request,
|
||||
"domainRequestId {text:?}"
|
||||
);
|
||||
match owner {
|
||||
Value::Null => assert!(
|
||||
OwnerToken::parse(text).is_err(),
|
||||
"owner token {text:?} must be refused"
|
||||
),
|
||||
Value::String(kind) => {
|
||||
let parsed = OwnerToken::parse(text).expect("an owner token");
|
||||
let expected = match kind.as_str() {
|
||||
"delivery" => OwnerKind::Delivery,
|
||||
"hold" => OwnerKind::Hold,
|
||||
other => panic!("unknown owner kind {other:?}"),
|
||||
};
|
||||
assert_eq!(parsed.kind(), expected, "owner kind of {text:?}");
|
||||
}
|
||||
other => panic!("unexpected ownerToken field {other:?}"),
|
||||
}
|
||||
let accepted = [call, request, OwnerToken::parse(text).is_ok()]
|
||||
.iter()
|
||||
.filter(|a| **a)
|
||||
.count();
|
||||
assert!(
|
||||
accepted <= 1,
|
||||
"{text:?} is accepted by more than one identity type"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// An artifact identity is the naming half of a bus ArtifactRef, and nothing else in these
|
||||
/// contracts is one: an AssetRef is persistent installed content, not live bytes.
|
||||
#[test]
|
||||
fn artifact_identity_is_the_naming_half_of_an_artifact_ref() {
|
||||
let file = fixtures::load("identities.json").expect("identities.json");
|
||||
let section = file.get("artifact").expect("artifact");
|
||||
let reference =
|
||||
flybus::wire::ArtifactRef::from_json(section.get("ref").expect("ref")).expect("a ref");
|
||||
let identity = ArtifactIdentity::of(&reference);
|
||||
identity.validate().expect("a valid identity");
|
||||
let expected = section.get("identity").expect("identity");
|
||||
assert_eq!(identity.store_id, expected["storeId"].as_str().unwrap());
|
||||
assert_eq!(identity.artifact_id, expected["artifactId"].as_str().unwrap());
|
||||
assert_eq!(identity.generation.to_string(), expected["generation"].as_str().unwrap());
|
||||
|
||||
let asset = fly_session_types::workers::AssetRef::from_json(section_asset(&file)).expect("asset");
|
||||
assert_ne!(
|
||||
asset.id, identity.artifact_id,
|
||||
"the fixture's asset and artifact are deliberately different things"
|
||||
);
|
||||
}
|
||||
|
||||
fn section_asset(file: &Value) -> &Value {
|
||||
file.get("asset").expect("asset")
|
||||
}
|
||||
|
||||
/// A delivery id or hold token is connection-private. The domain reader has no field that
|
||||
/// takes one, which is what `canonical::reject_bus_identities` enforces; here we only pin
|
||||
/// that the two prefixes the bus issues are the two kinds this type knows.
|
||||
#[test]
|
||||
fn owner_tokens_come_in_exactly_two_kinds() {
|
||||
assert_eq!(OwnerToken::delivery(7).as_str(), "dlv-7");
|
||||
assert_eq!(OwnerToken::hold(9).as_str(), "own-9");
|
||||
assert_eq!(OwnerToken::delivery(7).kind(), OwnerKind::Delivery);
|
||||
assert_eq!(OwnerToken::hold(9).kind(), OwnerKind::Hold);
|
||||
assert_eq!(BusCallId::from_serial(12).as_str(), "call-12");
|
||||
assert_eq!(DomainRequestId::from_serial(41).as_str(), "req-41");
|
||||
assert_eq!(DomainRequestId::from_serial(41).serial(), 41);
|
||||
}
|
||||
|
||||
use fly_session_types::scalar::DomainType as _;
|
||||
177
services/flysim/crates/fly-session-types/tests/payloads.rs
Normal file
177
services/flysim/crates/fly-session-types/tests/payloads.rs
Normal file
|
|
@ -0,0 +1,177 @@
|
|||
//! The `valid.json`, `invalid.json`, `raw.json` and `generated.json` fixtures.
|
||||
|
||||
mod common;
|
||||
|
||||
use fly_session_types::scalar::{DomainType, MAX_TYPED_VALUE_BYTES, SchemaRef, TypedValue};
|
||||
use fly_session_types::{canonical, fixtures};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use common::round_trip;
|
||||
|
||||
#[test]
|
||||
fn every_valid_case_round_trips_and_canonicalizes_to_its_recorded_bytes() {
|
||||
let file = fixtures::load("valid.json").expect("valid.json");
|
||||
let cases = fixtures::cases(&file).expect("cases");
|
||||
for case in cases {
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let type_name = fixtures::field(case, "type").expect("type");
|
||||
let value = case.get("value").expect("value");
|
||||
let written = round_trip(type_name, value)
|
||||
.unwrap_or_else(|e| panic!("{name} ({type_name}) must be accepted: {e}"));
|
||||
let canonical_in = canonical::canonicalize(value).expect("canonicalizable");
|
||||
let canonical_out = canonical::canonicalize(&written).expect("canonicalizable");
|
||||
assert_eq!(
|
||||
canonical_in, canonical_out,
|
||||
"{name}: reading and writing must preserve every field"
|
||||
);
|
||||
assert_eq!(
|
||||
canonical_in,
|
||||
fixtures::field(case, "canonical").expect("canonical"),
|
||||
"{name}: canonical JSON must match the fixture"
|
||||
);
|
||||
assert_eq!(
|
||||
canonical::sha256_hex(canonical_in.as_bytes()),
|
||||
fixtures::field(case, "digest").expect("digest"),
|
||||
"{name}: digest must match the fixture"
|
||||
);
|
||||
}
|
||||
assert!(cases.len() >= 70, "the valid fixture should stay broad");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_type_the_readers_know_appears_in_the_valid_fixture() {
|
||||
let file = fixtures::load("valid.json").expect("valid.json");
|
||||
let cases = fixtures::cases(&file).expect("cases");
|
||||
let covered: Vec<&str> = cases
|
||||
.iter()
|
||||
.map(|case| fixtures::field(case, "type").expect("type"))
|
||||
.collect();
|
||||
let missing: Vec<&&str> = common::READABLE_TYPES
|
||||
.iter()
|
||||
.filter(|t| !covered.contains(*t))
|
||||
.collect();
|
||||
assert!(
|
||||
missing.is_empty(),
|
||||
"every readable type needs at least one accepted fixture; missing {missing:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_invalid_case_is_refused() {
|
||||
let file = fixtures::load("invalid.json").expect("invalid.json");
|
||||
let cases = fixtures::cases(&file).expect("cases");
|
||||
for case in cases {
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let type_name = fixtures::field(case, "type").expect("type");
|
||||
let reason = fixtures::field(case, "reason").expect("reason");
|
||||
let value = case.get("value").expect("value");
|
||||
let outcome = round_trip(type_name, value);
|
||||
assert!(
|
||||
outcome.is_err(),
|
||||
"{name} ({type_name}) must be refused: {reason}"
|
||||
);
|
||||
}
|
||||
assert!(cases.len() >= 80, "the invalid fixture should stay broad");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_raw_byte_case_is_refused_before_or_during_validation() {
|
||||
let file = fixtures::load("raw.json").expect("raw.json");
|
||||
for case in fixtures::cases(&file).expect("cases") {
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let type_name = fixtures::field(case, "type").expect("type");
|
||||
let reason = fixtures::field(case, "reason").expect("reason");
|
||||
let bytes = fixtures::base64(case, "base64").unwrap_or_default();
|
||||
let outcome = canonical::parse_strict(&bytes).and_then(|value| {
|
||||
round_trip(type_name, &value).map_err(|e| fly_session_types::scalar::wire_err(e.0))
|
||||
});
|
||||
assert!(outcome.is_err(), "{name} must be refused: {reason}");
|
||||
}
|
||||
}
|
||||
|
||||
/// The recipes in `generated.json`: payloads too large to store as fixtures.
|
||||
#[test]
|
||||
fn generated_boundary_cases_land_on_the_right_side_of_every_limit() {
|
||||
let file = fixtures::load("generated.json").expect("generated.json");
|
||||
let pad_schema = SchemaRef::from_json(file.get("padSchema").expect("padSchema")).expect("schema");
|
||||
for case in fixtures::cases(&file).expect("cases") {
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let kind = fixtures::field(case, "kind").expect("kind");
|
||||
let expect_accept = fixtures::field(case, "expect").expect("expect") == "accept";
|
||||
let outcome: Result<(), String> = match kind {
|
||||
"padded-typed-value" => {
|
||||
let pad = case.get("padCharacters").and_then(Value::as_u64).expect("pad") as usize;
|
||||
TypedValue::new(pad_schema.clone(), json!({"pad": "a".repeat(pad)}))
|
||||
.map(|_| ())
|
||||
.map_err(|e| e.0)
|
||||
}
|
||||
"padded-request" => {
|
||||
let pad = case.get("padCharacters").and_then(Value::as_u64).expect("pad") as usize;
|
||||
let total = case
|
||||
.get("envelopeTotal")
|
||||
.and_then(Value::as_u64)
|
||||
.expect("envelopeTotal") as usize;
|
||||
let request = json!({
|
||||
"requestId": "req-1",
|
||||
"scope": Value::Null,
|
||||
"params": {"pad": "a".repeat(pad)},
|
||||
});
|
||||
let body = round_trip("SessionRpcRequest", &request).expect("a request");
|
||||
let length = canonical::canonicalize(&body).expect("canonicalizable").len();
|
||||
canonical::require_envelope_fit(&body, total - length)
|
||||
.map(|_| ())
|
||||
.map_err(|e| e.0)
|
||||
}
|
||||
"error-message" | "error-message-astral" => {
|
||||
let points = case.get("codePoints").and_then(Value::as_u64).expect("codePoints")
|
||||
as usize;
|
||||
let character = if kind == "error-message" { 'x' } else { '\u{10400}' };
|
||||
let message: String = std::iter::repeat_n(character, points).collect();
|
||||
let failure = json!({
|
||||
"type": "error",
|
||||
"requestId": "req-41",
|
||||
"workerId": "fly-a",
|
||||
"incarnationId": "inc-1",
|
||||
"scope": Value::Null,
|
||||
"error": {"code": "INTERNAL", "message": message, "mutation": "unknown"},
|
||||
});
|
||||
round_trip("SessionRpcFailure", &failure)
|
||||
.map(|_| ())
|
||||
.map_err(|e| e.0)
|
||||
}
|
||||
other => panic!("unknown generated case kind {other:?}"),
|
||||
};
|
||||
assert_eq!(
|
||||
outcome.is_ok(),
|
||||
expect_accept,
|
||||
"{name}: expected {}, got {outcome:?}",
|
||||
if expect_accept { "accept" } else { "reject" }
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_typed_value_at_the_cap_is_accepted_and_one_byte_more_is_not() {
|
||||
let schema = SchemaRef::new("pad.v1", 1, &canonical::sha256_hex(b"pad.v1")).expect("schema");
|
||||
let overhead = canonical::canonicalize(
|
||||
&TypedValue::new(schema.clone(), json!({"pad": ""}))
|
||||
.expect("empty")
|
||||
.to_json(),
|
||||
)
|
||||
.expect("canonicalizable")
|
||||
.len();
|
||||
let at_cap = TypedValue::new(
|
||||
schema.clone(),
|
||||
json!({"pad": "a".repeat(MAX_TYPED_VALUE_BYTES - overhead)}),
|
||||
)
|
||||
.expect("exactly at the cap");
|
||||
assert_eq!(at_cap.canonical_len().expect("length"), MAX_TYPED_VALUE_BYTES);
|
||||
assert!(
|
||||
TypedValue::new(
|
||||
schema,
|
||||
json!({"pad": "a".repeat(MAX_TYPED_VALUE_BYTES - overhead + 1)})
|
||||
)
|
||||
.is_err(),
|
||||
"one byte over the cap must fail"
|
||||
);
|
||||
}
|
||||
159
services/flysim/crates/fly-session-types/tests/rational.rs
Normal file
159
services/flysim/crates/fly-session-types/tests/rational.rs
Normal file
|
|
@ -0,0 +1,159 @@
|
|||
//! Checked rational arithmetic and the step-v1 section 5 tick accumulator.
|
||||
|
||||
use fly_session_types::scalar::{DomainType, RationalNs};
|
||||
use fly_session_types::fixtures;
|
||||
use serde_json::Value;
|
||||
|
||||
fn rational(value: &Value) -> RationalNs {
|
||||
RationalNs::from_json(value).expect("a valid rational")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_accumulator_produces_the_fixture_tick_counts_and_remainders() {
|
||||
let file = fixtures::load("rational.json").expect("rational.json");
|
||||
for case in file
|
||||
.get("accumulator")
|
||||
.and_then(Value::as_array)
|
||||
.expect("accumulator")
|
||||
{
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let step = rational(case.get("stepDuration").expect("stepDuration"));
|
||||
let tick = rational(case.get("tickDuration").expect("tickDuration"));
|
||||
let mut accumulator = RationalNs::ZERO;
|
||||
let mut total = 0u64;
|
||||
for (index, expected) in case
|
||||
.get("steps")
|
||||
.and_then(Value::as_array)
|
||||
.expect("steps")
|
||||
.iter()
|
||||
.enumerate()
|
||||
{
|
||||
accumulator = accumulator.checked_add(&step).expect("checked add");
|
||||
let (ticks, remainder) = accumulator.divide_floor(&tick).expect("checked divide");
|
||||
accumulator = remainder;
|
||||
total += ticks;
|
||||
assert_eq!(
|
||||
ticks.to_string(),
|
||||
fixtures::field(expected, "ticks").expect("ticks"),
|
||||
"{name}: tick count at step {index}"
|
||||
);
|
||||
assert_eq!(
|
||||
remainder,
|
||||
rational(expected.get("remainder").expect("remainder")),
|
||||
"{name}: remainder at step {index}"
|
||||
);
|
||||
assert!(
|
||||
remainder < tick,
|
||||
"{name}: the remainder is always less than one model tick"
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
total.to_string(),
|
||||
fixtures::field(case, "totalTicks").expect("totalTicks"),
|
||||
"{name}: total ticks"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn checked_arithmetic_reduces_or_refuses() {
|
||||
let file = fixtures::load("rational.json").expect("rational.json");
|
||||
for case in file.get("add").and_then(Value::as_array).expect("add") {
|
||||
let outcome = rational(case.get("a").expect("a")).checked_add(&rational(case.get("b").expect("b")));
|
||||
match case.get("sum") {
|
||||
Some(sum) => assert_eq!(outcome.expect("a sum"), rational(sum)),
|
||||
None => assert!(outcome.is_err(), "the sum must overflow: {case}"),
|
||||
}
|
||||
}
|
||||
for case in file
|
||||
.get("subtract")
|
||||
.and_then(Value::as_array)
|
||||
.expect("subtract")
|
||||
{
|
||||
let outcome =
|
||||
rational(case.get("a").expect("a")).checked_sub(&rational(case.get("b").expect("b")));
|
||||
match case.get("difference") {
|
||||
Some(difference) => assert_eq!(outcome.expect("a difference"), rational(difference)),
|
||||
None => assert!(outcome.is_err(), "the subtraction must fail: {case}"),
|
||||
}
|
||||
}
|
||||
for case in file
|
||||
.get("multiply")
|
||||
.and_then(Value::as_array)
|
||||
.expect("multiply")
|
||||
{
|
||||
let k: u64 = fixtures::field(case, "k")
|
||||
.expect("k")
|
||||
.parse()
|
||||
.expect("a u64");
|
||||
let outcome = rational(case.get("a").expect("a")).checked_mul_u64(k);
|
||||
match case.get("product") {
|
||||
Some(product) => assert_eq!(outcome.expect("a product"), rational(product)),
|
||||
None => assert!(outcome.is_err(), "the product must overflow: {case}"),
|
||||
}
|
||||
}
|
||||
for case in file
|
||||
.get("compare")
|
||||
.and_then(Value::as_array)
|
||||
.expect("compare")
|
||||
{
|
||||
let left = rational(case.get("a").expect("a"));
|
||||
let right = rational(case.get("b").expect("b"));
|
||||
let ordering = match fixtures::field(case, "ordering").expect("ordering") {
|
||||
"less" => std::cmp::Ordering::Less,
|
||||
"equal" => std::cmp::Ordering::Equal,
|
||||
"greater" => std::cmp::Ordering::Greater,
|
||||
other => panic!("unknown ordering {other:?}"),
|
||||
};
|
||||
assert_eq!(left.cmp(&right), ordering, "{case}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn zero_has_exactly_one_encoding_and_durations_must_be_positive() {
|
||||
assert_eq!(RationalNs::ZERO, RationalNs::new(0, 1).expect("0/1"));
|
||||
assert!(RationalNs::new(0, 2).is_err(), "zero is encoded 0/1");
|
||||
assert!(RationalNs::new(1, 0).is_err(), "denominators are positive");
|
||||
assert!(RationalNs::new(2, 4).is_err(), "fractions are reduced");
|
||||
assert!(RationalNs::ZERO.require_positive("worldTime").is_err());
|
||||
assert!(
|
||||
RationalNs::new(1, 3)
|
||||
.expect("1/3")
|
||||
.require_positive("tickDuration")
|
||||
.is_ok()
|
||||
);
|
||||
assert!(
|
||||
RationalNs::ZERO.divide_floor(&RationalNs::ZERO).is_err(),
|
||||
"dividing by a zero tick is refused, not infinite"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reduction_refuses_a_result_that_does_not_fit_u64() {
|
||||
let big = RationalNs::new(u64::MAX, 1).expect("a whole number");
|
||||
assert!(big.checked_mul_u64(2).is_err());
|
||||
assert!(big.checked_add(&big).is_err());
|
||||
assert_eq!(
|
||||
RationalNs::reduced(u128::from(u64::MAX) * 2, 2).expect("reduces back into range"),
|
||||
big
|
||||
);
|
||||
}
|
||||
|
||||
/// The review case: cross-multiplying two reduced fractions near the `U64` maximum fits
|
||||
/// `u128`, but adding the two products does not. Unchecked, that panics in debug and wraps in
|
||||
/// release, after which the reduction returns a confidently wrong rational.
|
||||
#[test]
|
||||
fn adding_two_fractions_near_the_u64_maximum_is_refused_not_wrapped() {
|
||||
let left = RationalNs::new(u64::MAX, u64::MAX - 1).expect("consecutive integers are coprime");
|
||||
let right = RationalNs::new(u64::MAX - 2, u64::MAX - 4).expect("two odd numbers differing by 2");
|
||||
let outcome = left.checked_add(&right);
|
||||
let message = outcome.expect_err("the sum reaches about 2^129").0;
|
||||
assert!(
|
||||
message.contains("overflow"),
|
||||
"the failure names the overflow rather than the reduction: {message}"
|
||||
);
|
||||
assert!(
|
||||
left.checked_add(&RationalNs::ZERO).is_ok(),
|
||||
"the checked path still adds ordinary operands"
|
||||
);
|
||||
}
|
||||
182
services/flysim/crates/fly-session-types/tests/schema_set.rs
Normal file
182
services/flysim/crates/fly-session-types/tests/schema_set.rs
Normal file
|
|
@ -0,0 +1,182 @@
|
|||
//! The canonical schema set, `contractDigest` and the freshness of every derived fixture.
|
||||
|
||||
use fly_session_types::{canonical, fixtures, schema};
|
||||
use serde_json::Value;
|
||||
|
||||
#[allow(dead_code, reason = "this file uses the readers' type list, not the readers")]
|
||||
mod common;
|
||||
|
||||
#[path = "../examples/update_fixtures.rs"]
|
||||
#[allow(dead_code, reason = "the example's main is not used by the test that reuses its writers")]
|
||||
mod updater;
|
||||
|
||||
/// Every derived fixture is exactly what the updater writes today. If this fails, run
|
||||
/// `cargo run -p fly-session-types --example update_fixtures` and review the diff.
|
||||
#[test]
|
||||
fn derived_fixtures_are_current() {
|
||||
for (name, expected) in updater::derived() {
|
||||
let path = fixtures::dir().join(&name);
|
||||
let found = std::fs::read_to_string(&path).expect("a checked-in fixture");
|
||||
assert_eq!(
|
||||
found, expected,
|
||||
"{name} is stale; regenerate it with the update_fixtures example"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_contract_digest_is_the_digest_of_the_checked_in_schema_set() {
|
||||
let text = fixtures::load_bytes("schema-set.json").expect("schema-set.json");
|
||||
let recorded = fixtures::load("contract-digest.json").expect("contract-digest.json");
|
||||
let expected = recorded
|
||||
.get("contractDigest")
|
||||
.and_then(Value::as_str)
|
||||
.expect("contractDigest");
|
||||
assert_eq!(schema::contract_digest(), expected);
|
||||
// The file is the canonical schema set plus one trailing newline.
|
||||
assert_eq!(
|
||||
canonical::sha256_hex(text.strip_suffix(b"\n").expect("trailing newline")),
|
||||
expected,
|
||||
"the digest is over the canonical schema set, byte for byte"
|
||||
);
|
||||
}
|
||||
|
||||
/// The digest comes from the schema declaration, not from the source text: reparsing the
|
||||
/// checked-in file with different whitespace and key order gives the same digest.
|
||||
#[test]
|
||||
fn the_contract_digest_survives_reformatting() {
|
||||
let bytes = fixtures::load_bytes("schema-set.json").expect("schema-set.json");
|
||||
let parsed = canonical::parse_strict(bytes.strip_suffix(b"\n").expect("newline")).expect("parses");
|
||||
let pretty = serde_json::to_vec_pretty(&parsed).expect("serializable");
|
||||
let reparsed = canonical::parse_strict(&pretty).expect("parses");
|
||||
assert_eq!(
|
||||
canonical::digest_of(&reparsed).expect("digest"),
|
||||
schema::contract_digest(),
|
||||
"pretty printing the schema set does not change its digest"
|
||||
);
|
||||
let shuffled = canonical::parse_strict(
|
||||
br#"{"version":1,"contract":"fly-session-types-other"}"#,
|
||||
)
|
||||
.expect("parses");
|
||||
assert_ne!(
|
||||
canonical::digest_of(&shuffled).expect("digest"),
|
||||
schema::contract_digest()
|
||||
);
|
||||
}
|
||||
|
||||
/// ... and it changes when a schema changes: a renamed field, a widened bound, one more enum
|
||||
/// member or one fewer type all move the digest.
|
||||
#[test]
|
||||
fn the_contract_digest_changes_when_a_schema_changes() {
|
||||
let baseline = schema::contract_digest();
|
||||
let mutate = |mutation: fn(&mut Value)| {
|
||||
let mut set = schema::schema_set();
|
||||
mutation(&mut set);
|
||||
canonical::digest_of(&set).expect("digest")
|
||||
};
|
||||
let renamed_field = mutate(|set| {
|
||||
set["types"][0]["fields"][0]["name"] = Value::String("sessionIdentifier".to_owned());
|
||||
});
|
||||
let widened_bound = mutate(|set| {
|
||||
for limit in set["limits"].as_array_mut().expect("limits") {
|
||||
if limit["name"] == Value::String("maxAgents".to_owned()) {
|
||||
limit["value"] = Value::from(8u64);
|
||||
}
|
||||
}
|
||||
});
|
||||
let extra_enum_member = mutate(|set| {
|
||||
set["enums"][0]["members"]
|
||||
.as_array_mut()
|
||||
.expect("members")
|
||||
.push(Value::String("s16le-interleaved".to_owned()));
|
||||
});
|
||||
let dropped_type = mutate(|set| {
|
||||
set["types"].as_array_mut().expect("types").pop();
|
||||
});
|
||||
let relaxed_constraint = mutate(|set| {
|
||||
set["types"][0]["fields"][0]["constraint"] = Value::String("anything".to_owned());
|
||||
});
|
||||
for (what, digest) in [
|
||||
("a renamed field", renamed_field),
|
||||
("a widened bound", widened_bound),
|
||||
("an extra enum member", extra_enum_member),
|
||||
("a dropped type", dropped_type),
|
||||
("a relaxed constraint", relaxed_constraint),
|
||||
] {
|
||||
assert_ne!(digest, baseline, "{what} must change contractDigest");
|
||||
}
|
||||
}
|
||||
|
||||
/// Every type the crate can read is declared in the schema set, so a new payload type cannot
|
||||
/// ship outside `contractDigest`. The list is the readers' own, not a copy of it.
|
||||
#[test]
|
||||
fn the_schema_set_names_every_type_the_crate_reads() {
|
||||
let set = schema::schema_set();
|
||||
let names: Vec<&str> = set["types"]
|
||||
.as_array()
|
||||
.expect("types")
|
||||
.iter()
|
||||
.map(|t| t["name"].as_str().expect("name"))
|
||||
.collect();
|
||||
let missing: Vec<&&str> = common::READABLE_TYPES
|
||||
.iter()
|
||||
.filter(|expected| !names.contains(*expected))
|
||||
.collect();
|
||||
assert!(
|
||||
missing.is_empty(),
|
||||
"every readable type must be in the schema set; missing {missing:?}"
|
||||
);
|
||||
let mut sorted = names.clone();
|
||||
sorted.sort_unstable();
|
||||
assert_eq!(names, sorted, "the rendered set is sorted by type name");
|
||||
let mut unique = sorted.clone();
|
||||
unique.dedup();
|
||||
assert_eq!(unique.len(), names.len(), "no type is declared twice");
|
||||
}
|
||||
|
||||
/// Every bound the schema set publishes is the constant the code enforces, and every bound
|
||||
/// this crate chose rather than read from a document says so.
|
||||
#[test]
|
||||
fn published_limits_match_the_constants_and_name_their_source() {
|
||||
let set = schema::schema_set();
|
||||
let limits = set["limits"].as_array().expect("limits");
|
||||
let find = |name: &str| -> u64 {
|
||||
limits
|
||||
.iter()
|
||||
.find(|l| l["name"] == Value::String(name.to_owned()))
|
||||
.and_then(|l| l["value"].as_u64())
|
||||
.unwrap_or_else(|| panic!("the schema set must publish {name}"))
|
||||
};
|
||||
assert_eq!(find("maxAgents"), fly_session_types::workers::MAX_AGENTS as u64);
|
||||
assert_eq!(find("maxPorts"), fly_session_types::workers::MAX_PORTS as u64);
|
||||
assert_eq!(
|
||||
find("maxRateRoles"),
|
||||
fly_session_types::workers::MAX_RATE_ROLES as u64
|
||||
);
|
||||
assert_eq!(find("maxViews"), fly_session_types::media::MAX_VIEWS as u64);
|
||||
assert_eq!(
|
||||
find("maxTypedValueBytes"),
|
||||
fly_session_types::scalar::MAX_TYPED_VALUE_BYTES as u64
|
||||
);
|
||||
assert_eq!(
|
||||
find("maxEnvelopeBytes"),
|
||||
canonical::MAX_ENVELOPE_BYTES as u64
|
||||
);
|
||||
let crate_chosen: Vec<&str> = limits
|
||||
.iter()
|
||||
.filter(|l| l["source"] == Value::String("crate".to_owned()))
|
||||
.map(|l| l["name"].as_str().expect("name"))
|
||||
.collect();
|
||||
assert_eq!(
|
||||
crate_chosen,
|
||||
[
|
||||
"maxAssets",
|
||||
"maxAudioStreams",
|
||||
"maxCapabilities",
|
||||
"maxSnapshotEvents",
|
||||
"maxSupportedMajors",
|
||||
"maxSupportedStimuli",
|
||||
],
|
||||
"a bound with no stated source must be declared as this crate's choice"
|
||||
);
|
||||
}
|
||||
120
services/flysim/crates/fly-session-types/tests/seeds.rs
Normal file
120
services/flysim/crates/fly-session-types/tests/seeds.rs
Normal file
|
|
@ -0,0 +1,120 @@
|
|||
//! `seed-derivation-v1` against its test vectors.
|
||||
|
||||
use fly_session_types::{fixtures, seed};
|
||||
use serde_json::Value;
|
||||
|
||||
#[test]
|
||||
fn every_vector_derives_its_recorded_seed() {
|
||||
let file = fixtures::load("seed-vectors.json").expect("seed-vectors.json");
|
||||
assert_eq!(
|
||||
file.get("algorithm").and_then(Value::as_str),
|
||||
Some(seed::ALGORITHM)
|
||||
);
|
||||
let vectors = file.get("vectors").and_then(Value::as_array).expect("vectors");
|
||||
for case in vectors {
|
||||
let master: u64 = fixtures::field(case, "masterSeed")
|
||||
.expect("masterSeed")
|
||||
.parse()
|
||||
.expect("a U64");
|
||||
let agent = fixtures::field(case, "agentId").expect("agentId");
|
||||
assert_eq!(
|
||||
String::from_utf8(seed::material(master, agent).expect("material")).expect("utf-8"),
|
||||
fixtures::field(case, "material").expect("material"),
|
||||
"the hashed material is part of the specification"
|
||||
);
|
||||
assert_eq!(
|
||||
seed::material_digest(master, agent).expect("digest"),
|
||||
fixtures::field(case, "materialDigest").expect("materialDigest")
|
||||
);
|
||||
assert_eq!(
|
||||
i64::from(seed::agent_seed(master, agent).expect("seed")),
|
||||
case.get("seed").and_then(Value::as_i64).expect("seed"),
|
||||
"seed for {agent} under master {master}"
|
||||
);
|
||||
}
|
||||
assert!(vectors.len() >= 20, "keep the vector table broad");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn one_composition_gets_independent_seeds() {
|
||||
let file = fixtures::load("seed-vectors.json").expect("seed-vectors.json");
|
||||
let composition = file.get("composition").expect("composition");
|
||||
let master: u64 = fixtures::field(composition, "masterSeed")
|
||||
.expect("masterSeed")
|
||||
.parse()
|
||||
.expect("a U64");
|
||||
let ids: Vec<String> = composition
|
||||
.get("agentIds")
|
||||
.and_then(Value::as_array)
|
||||
.expect("agentIds")
|
||||
.iter()
|
||||
.map(|v| v.as_str().expect("an id").to_owned())
|
||||
.collect();
|
||||
let seeds = seed::composition_seeds(master, &ids).expect("seeds");
|
||||
let recorded: Vec<i64> = composition
|
||||
.get("seeds")
|
||||
.and_then(Value::as_array)
|
||||
.expect("seeds")
|
||||
.iter()
|
||||
.map(|v| v.as_i64().expect("a seed"))
|
||||
.collect();
|
||||
assert_eq!(
|
||||
seeds.iter().map(|s| i64::from(*s)).collect::<Vec<_>>(),
|
||||
recorded
|
||||
);
|
||||
let mut unique = seeds.clone();
|
||||
unique.sort_unstable();
|
||||
unique.dedup();
|
||||
assert_eq!(unique.len(), seeds.len(), "per-agent seeds are independent");
|
||||
assert!(
|
||||
seeds.iter().all(|s| *s != 0),
|
||||
"a zero seed would stall an xorshift generator"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_different_master_seed_or_agent_id_derives_a_different_seed() {
|
||||
assert_ne!(
|
||||
seed::agent_seed(0, "fly-a").expect("seed"),
|
||||
seed::agent_seed(1, "fly-a").expect("seed")
|
||||
);
|
||||
assert_ne!(
|
||||
seed::agent_seed(0, "fly-a").expect("seed"),
|
||||
seed::agent_seed(0, "fly-b").expect("seed")
|
||||
);
|
||||
assert_eq!(
|
||||
seed::agent_seed(7, "fly-a").expect("seed"),
|
||||
seed::agent_seed(7, "fly-a").expect("seed"),
|
||||
"the derivation is a function of its recorded inputs"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalid_inputs_are_refused_rather_than_normalized() {
|
||||
let file = fixtures::load("seed-vectors.json").expect("seed-vectors.json");
|
||||
for case in file.get("invalid").and_then(Value::as_array).expect("invalid") {
|
||||
let master: u64 = fixtures::field(case, "masterSeed")
|
||||
.expect("masterSeed")
|
||||
.parse()
|
||||
.expect("a U64");
|
||||
if let Ok(agent) = fixtures::field(case, "agentId") {
|
||||
assert!(
|
||||
seed::agent_seed(master, agent).is_err(),
|
||||
"{agent:?} must be refused: {}",
|
||||
fixtures::field(case, "reason").unwrap_or("")
|
||||
);
|
||||
} else {
|
||||
let ids: Vec<String> = case
|
||||
.get("agentIds")
|
||||
.and_then(Value::as_array)
|
||||
.expect("agentIds")
|
||||
.iter()
|
||||
.map(|v| v.as_str().expect("an id").to_owned())
|
||||
.collect();
|
||||
assert!(
|
||||
seed::composition_seeds(master, &ids).is_err(),
|
||||
"a repeated agent id must be refused"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
113
services/flysim/crates/fly-session-types/tests/traces.rs
Normal file
113
services/flysim/crates/fly-session-types/tests/traces.rs
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
//! The step-v1 section 8 trace comparator: behaviour only.
|
||||
|
||||
use fly_session_types::fixtures;
|
||||
use fly_session_types::scalar::DomainType;
|
||||
use fly_session_types::trace::TransitionTrace;
|
||||
use serde_json::Value;
|
||||
|
||||
fn trace(value: &Value) -> TransitionTrace {
|
||||
TransitionTrace::from_json(value).expect("a valid trace")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_variant_compares_the_way_the_fixture_says() {
|
||||
let file = fixtures::load("traces.json").expect("traces.json");
|
||||
let baseline = trace(file.get("baseline").expect("baseline"));
|
||||
for case in file
|
||||
.get("variants")
|
||||
.and_then(Value::as_array)
|
||||
.expect("variants")
|
||||
{
|
||||
let name = fixtures::field(case, "name").expect("name");
|
||||
let variant = trace(case.get("trace").expect("trace"));
|
||||
let expected = case
|
||||
.get("behaviourEquals")
|
||||
.and_then(Value::as_bool)
|
||||
.expect("behaviourEquals");
|
||||
let equal = baseline.behaviour_equals(&variant);
|
||||
let diff = baseline.behaviour_diff(&variant);
|
||||
assert_eq!(
|
||||
equal, expected,
|
||||
"{name}: behaviour equality. differences: {diff:?}"
|
||||
);
|
||||
assert_eq!(
|
||||
diff.is_empty(),
|
||||
expected,
|
||||
"{name}: the diff must be empty exactly when the behaviour matches"
|
||||
);
|
||||
if let Ok(needle) = fixtures::field(case, "diffContains") {
|
||||
assert!(
|
||||
diff.iter().any(|line| line.contains(needle)),
|
||||
"{name}: the diff should name {needle:?}, got {diff:?}"
|
||||
);
|
||||
}
|
||||
if expected {
|
||||
assert_eq!(
|
||||
baseline.behaviour.digest().expect("digest"),
|
||||
variant.behaviour.digest().expect("digest"),
|
||||
"{name}: equal behaviour has one digest"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_whole_run_compares_transition_by_transition() {
|
||||
let file = fixtures::load("traces.json").expect("traces.json");
|
||||
let baseline = trace(file.get("baseline").expect("baseline"));
|
||||
let variants = file
|
||||
.get("variants")
|
||||
.and_then(Value::as_array)
|
||||
.expect("variants");
|
||||
let reversed = trace(variants[0].get("trace").expect("trace"));
|
||||
let changed = trace(
|
||||
variants
|
||||
.iter()
|
||||
.find(|case| fixtures::field(case, "name").unwrap_or("") == "one extra neural tick")
|
||||
.expect("the extra tick variant")
|
||||
.get("trace")
|
||||
.expect("trace"),
|
||||
);
|
||||
assert!(TransitionTrace::runs_equal(
|
||||
&[baseline.clone(), baseline.clone()],
|
||||
&[reversed, baseline.clone()]
|
||||
));
|
||||
assert!(!TransitionTrace::runs_equal(
|
||||
std::slice::from_ref(&baseline),
|
||||
&[changed]
|
||||
));
|
||||
let longer = [baseline.clone(), baseline.clone()];
|
||||
assert!(
|
||||
!TransitionTrace::runs_equal(std::slice::from_ref(&baseline), &longer),
|
||||
"a run with more transitions is not the same run"
|
||||
);
|
||||
}
|
||||
|
||||
/// The operational half is recorded, and never part of the comparison.
|
||||
#[test]
|
||||
fn operational_metadata_is_recorded_and_excluded() {
|
||||
let file = fixtures::load("traces.json").expect("traces.json");
|
||||
let baseline = trace(file.get("baseline").expect("baseline"));
|
||||
assert_eq!(baseline.operational.bus_call_ids.len(), 3);
|
||||
assert_eq!(baseline.operational.prepare_request_ids.len(), 2);
|
||||
assert_eq!(baseline.operational.delivery_ids.len(), 2);
|
||||
assert!(baseline.operational.wall_time_ns > 0);
|
||||
let retried = trace(
|
||||
file.get("variants")
|
||||
.and_then(Value::as_array)
|
||||
.expect("variants")
|
||||
.iter()
|
||||
.find(|case| {
|
||||
fixtures::field(case, "name").unwrap_or("")
|
||||
== "a safe retry with fresh bus callIds, delivery ids and wall time"
|
||||
})
|
||||
.expect("the retry variant")
|
||||
.get("trace")
|
||||
.expect("trace"),
|
||||
);
|
||||
assert_ne!(
|
||||
baseline.operational, retried.operational,
|
||||
"the retry really did change the operational half"
|
||||
);
|
||||
assert!(baseline.behaviour_equals(&retried));
|
||||
}
|
||||
25
services/flysim/crates/fly-session/Cargo.toml
Normal file
25
services/flysim/crates/fly-session/Cargo.toml
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
[package]
|
||||
name = "fly-session"
|
||||
version.workspace = true
|
||||
edition = "2024"
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
publish = false
|
||||
description = "The lockstep session coordinator, its phase machine and a synthetic composition over flybus."
|
||||
|
||||
[lib]
|
||||
name = "fly_session"
|
||||
path = "src/lib.rs"
|
||||
|
||||
[dependencies]
|
||||
# The domain contract (scalars, payloads, canonical digests, the trace format) and the bus.
|
||||
# Everything else this crate needs is std or Tokio.
|
||||
fly-session-types = { path = "../fly-session-types" }
|
||||
flybus = { path = "../flybus" }
|
||||
|
||||
serde_json = { workspace = true }
|
||||
tokio = { version = "1", features = ["rt", "sync", "time", "macros"] }
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = "3"
|
||||
tokio = { version = "1", features = ["rt-multi-thread", "macros", "time"] }
|
||||
157
services/flysim/crates/fly-session/README.md
Normal file
157
services/flysim/crates/fly-session/README.md
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
# fly-session
|
||||
|
||||
The lockstep session coordinator, its phase machine and a synthetic composition over
|
||||
[`flybus`](../flybus).
|
||||
|
||||
This crate is the SESSION-01 slice of the session-framework implementation guide: the
|
||||
sequential transaction of `step-v1`, driven over the Flybus router, with small fake workers
|
||||
standing in for a brain and an emulator. It contains no public controller API, no implicit
|
||||
best-effort retry, no real emulator and no real brain.
|
||||
|
||||
The domain scalars, method payloads, their validation, the canonical digests and the trace
|
||||
format all come from [`fly-session-types`](../fly-session-types), the CONTRACT-01 crate. This
|
||||
crate adds only what is not part of the type contract: a session-side error value, the
|
||||
synthetic composition's schema and event-id derivations, and the coordinator-local
|
||||
`ControllerIntent`, `PortBinding` and `AgentOutcome` that never cross the bus.
|
||||
|
||||
```text
|
||||
Ready(k) ─ Prepare all agents concurrently ────────────> every agent Prepared(k)
|
||||
─ one executor per agent, sorted agent-id order
|
||||
─ one complete port batch, descriptor port order
|
||||
─ exactly one Environment.Advance(k, batch) ──> boundary k+1
|
||||
─ task.evaluate_transition, once
|
||||
─ Commit all agents concurrently ─────────────> every agent Ready(k+1)
|
||||
─ committed boundary k+1, publish, next Prepare allowed
|
||||
```
|
||||
|
||||
## Layout
|
||||
|
||||
| Module | Contents |
|
||||
| --- | --- |
|
||||
| `types` | A facade over the [`fly-session-types`](../fly-session-types) crate, plus the session-side additions a coordinator needs |
|
||||
| `clock` | The `step-v1` section 5 rational tick accumulator and the coordinator's pacing |
|
||||
| `phase` | The `step-v1` section 2 state machine as an explicit edge table |
|
||||
| `dedup` | The `ipc-v1` section 5 operation keys, result caches and retention |
|
||||
| `worker` | The worker dispatch shell: one service, the common `Worker.*` methods, admission |
|
||||
| `agent` | A fake agent worker: seeded model, mutation counter, fixed readout stub |
|
||||
| `environment` | The counter arena: one complete batch per advance, one native frame |
|
||||
| `task` | The task and executor traits, the deterministic counter task, the identity executor |
|
||||
| `rpc` | Domain calls: `req-<U64>` serials, incarnation pinning, the retry rule |
|
||||
| `coordinator` | The transaction, the trace, the failure rules and the publication boundary |
|
||||
| `harness` | The runnable composition: router, two agents, one arena, one coordinator |
|
||||
|
||||
## What it implements
|
||||
|
||||
- **The transaction, in order.** Prepare all agents concurrently; run each task-local executor
|
||||
once in sorted agent-id order; assemble all configured port controls in descriptor port
|
||||
order; send exactly one `Environment.Advance`; evaluate the task once; commit all agents
|
||||
concurrently. The committed boundary moves only when every commit has succeeded.
|
||||
- **The state machine**, including `Paused` and `Failed`, with every transition recorded. A
|
||||
transition the `step-v1` section 2 table does not list returns `INVALID_PHASE`.
|
||||
- **The committed boundary rule.** Only `Ready(k)` or `Paused(k)` is a committed boundary; a
|
||||
snapshot publishes one of those and never an in-progress mix of new agent state and an old
|
||||
world.
|
||||
- **Time and pacing** with checked rational accumulation. A 60 Hz world with a 1 ms model tick
|
||||
produces 16, 17, 17 ticks over three steps, totalling 50, with a remainder of exactly zero.
|
||||
Wall time is only pacing: when behind, the coordinator omits the sleep and reports the lag.
|
||||
- **Initialization, pause and episodes.** The environment initializes first, while stopped;
|
||||
the task bootstraps; then the agents warm up with learning disabled. Nothing in bootstrap
|
||||
advances the world or produces a gameplay reward. A pause arriving mid-step completes the
|
||||
transition and pauses at its committed boundary. A terminal task event commits its final
|
||||
rewards, then the session pauses; no worker resets itself.
|
||||
- **The failure rules.** A partial commit fails the epoch; an uncertain Advance is resolved
|
||||
against its original domain request id and never becomes a second batch; a worker
|
||||
incarnation change invalidates the epoch.
|
||||
- **Domain deduplication over bus calls.** Same key, request and body replays its cached
|
||||
reply with fresh delivery ownership over retained artifacts; a changed body is `CONFLICT`; a
|
||||
duplicate of a running operation is `IN_PROGRESS` for that bus call while the original
|
||||
completes; an evicted record is `RESULT_EXPIRED`; a newly issued request naming an old step
|
||||
is `STALE_STEP`. `Worker.Acknowledge` releases a domain result cache, which is not a bus
|
||||
`delivery.consumed`.
|
||||
|
||||
## API
|
||||
|
||||
```rust
|
||||
let harness = SessionHarness::start(Via::Unix, dir.path(), HarnessConfig::default()).await?;
|
||||
harness.coordinator.bootstrap().await?; // Ready(0), world stopped at boundary 0
|
||||
let reports = harness.coordinator.run(3).await?; // three transitions
|
||||
harness.coordinator.pause_handle().request(); // finish this transition, then pause
|
||||
harness.coordinator.trace.behavior(); // the step-v1 section 8 behaviour trace
|
||||
harness.shutdown().await;
|
||||
```
|
||||
|
||||
- `Coordinator::dispatch` selects `Sequential`, `Concurrent` or `Reversed` per-agent dispatch.
|
||||
All three must produce the same behaviour trace; that is a test.
|
||||
- `Coordinator::injections` asks for one deliberate message fault at one step: a duplicate
|
||||
Prepare or Commit, an abandoned Advance result, an altered control batch, or a consumed
|
||||
result artifact followed by a replay. `injection_log` reports what came back.
|
||||
- `Coordinator::probe_raw` sends one domain request as it stands and returns the worker's own
|
||||
terminal outcome, without letting the answer change session state.
|
||||
- `AgentFaults` and `EnvironmentFaults` ask a worker for a deliberate delay or failure.
|
||||
|
||||
## The synthetic composition
|
||||
|
||||
- **Agents.** A fake model is an LCG with an explicit seed and one counter of everything that
|
||||
mutated it: ticks, stimulations, reinforcements and input installs. The worker reports that
|
||||
counter as its `progressCounter`, which is how a test proves a duplicate repeated nothing.
|
||||
The readout is a fixed stub: it reads bits of the current state, masked by the declared
|
||||
available actions, and never changes its own weights or invents a default winner.
|
||||
- **Environment.** A signed counter. `inc` adds one, `dec` subtracts one, and one bipolar
|
||||
`bias` axis is carried and validated but does not move the world. Each observation seals one
|
||||
immutable 4x4 RGBA frame whose bytes carry the counter, so an agent reading its sensory view
|
||||
reads the world rather than a constant.
|
||||
- **Task.** Rewards are the counter delta of each agent's own port control, with deterministic
|
||||
event ids derived from epoch, source step, rule and ordinal.
|
||||
- **Executors.** The stateless identity executor only, as v1 specifies.
|
||||
|
||||
## Where this crate narrows or adds to the contract crate
|
||||
|
||||
- **Required views.** `WorldObservation::validate_against` checks the views a result carries
|
||||
against their descriptors. Requiring every *declared* view to be there at all is the
|
||||
coordinator's Phase C check, so `verify_step_result` makes it: a missing required sensory
|
||||
view fails the transition with `BUFFER_INVALID` rather than being replaced by an older frame.
|
||||
- **`ControllerIntent`.** `workers-v1` section 4 calls the task and executor interfaces local
|
||||
libraries, so their types live here rather than in the payload contract. An intent is a
|
||||
`PortControl` without its port, and only the coordinator adds the port.
|
||||
- **The phase machine.** `step-v1` section 2 is this crate's, not the contract crate's; the
|
||||
trace's phase path is recorded beside the contract's `TransitionTrace`. The mid-step pause
|
||||
it takes -- the transition finishes, then the session pauses at the boundary it just
|
||||
committed -- is now written into the section 2 machine as a dated amendment.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Fake workers.** There is no neural model and no emulator. What is modelled exactly is the
|
||||
ordering, the identity rules and the retry rules, not any numerical behaviour.
|
||||
- **No state methods.** `State.Capture`, `State.StageRestore` and `State.ActivateRestore` are
|
||||
STATE-01. The phase machine has their edges (`Capturing`, `Restoring`) and the workers do not
|
||||
advertise them as implemented methods.
|
||||
- **One process.** SESSION-01 runs every participant in one process over the same router.
|
||||
SESSION-02 is the per-fly process split.
|
||||
- **No audience input.** The admitted pre-step stimulation list exists and is always empty.
|
||||
- **Pacing is coarse.** The pacing deadline rounds one step to whole nanoseconds for sleeping
|
||||
only; simulation time stays rational and that rounding never re-enters the accumulator.
|
||||
|
||||
## Tests
|
||||
|
||||
```text
|
||||
cargo test -p fly-session # unit + both integration suites
|
||||
cargo run -p fly-session --example session # the runnable synthetic session
|
||||
```
|
||||
|
||||
Every integration test runs over both transports, through the same router code: all but one
|
||||
are generated twice by `both_transports!`, and
|
||||
`sequential_concurrent_and_reversed_orders_agree` walks both transports inside one test
|
||||
because it compares their behaviour traces against each other.
|
||||
|
||||
- `tests/session.rs`: one world advance per complete batch; every agent Prepared before the
|
||||
advance; one task evaluation per transition; every agent committed before the next Prepare or
|
||||
any committed publication; the 16/17/17 tick profile with a zero remainder; a mid-step pause
|
||||
completing its transition; bootstrap advancing nothing; the committed snapshot naming the
|
||||
transition that just ended; a terminal episode pausing at its own boundary; `Worker.Status`
|
||||
during a session; and sequential, concurrent and reversed dispatch producing one behaviour
|
||||
trace.
|
||||
- `tests/failures.rs`: a duplicate Prepare after a lost reply; a duplicate Commit; the same
|
||||
batch with altered controls; a lost Advance result; a cached artifact consumed by its first
|
||||
caller; one Commit failing after another succeeded; a replaced registration; a reply from
|
||||
another incarnation; a world that advanced without sensory data; an exact duplicate of a
|
||||
running operation; and an old-epoch operation.
|
||||
55
services/flysim/crates/fly-session/examples/session.rs
Normal file
55
services/flysim/crates/fly-session/examples/session.rs
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
//! The synthetic sequential transaction, run over both transports and printed.
|
||||
//!
|
||||
//! ```text
|
||||
//! cargo run -p fly-session --example session
|
||||
//! ```
|
||||
//!
|
||||
//! Two fake agents, one counter arena, one coordinator, one router. Nothing here needs a ROM,
|
||||
//! a dataset, a GPU or a network.
|
||||
|
||||
use fly_session::types::*;
|
||||
use fly_session::harness::{HarnessConfig, SessionHarness, Via};
|
||||
|
||||
#[tokio::main(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn main() {
|
||||
for via in [Via::Memory, Via::Unix] {
|
||||
let dir = tempfile::tempdir().expect("a temporary directory");
|
||||
let mut harness = SessionHarness::start(via, dir.path(), HarnessConfig::default())
|
||||
.await
|
||||
.expect("the session starts");
|
||||
harness.coordinator.bootstrap().await.expect("bootstrap");
|
||||
println!("--- {via:?}: Ready(0) with the world stopped at boundary 0");
|
||||
let reports = harness.coordinator.run(3).await.expect("three transitions");
|
||||
|
||||
for (k, transition) in harness.coordinator.trace.transitions.iter().enumerate() {
|
||||
let ticks: Vec<String> = transition
|
||||
.behaviour
|
||||
.agents
|
||||
.iter()
|
||||
.map(|a| format!("{}={} ticks", a.agent_id, a.ticks_advanced))
|
||||
.collect();
|
||||
println!(
|
||||
"step {k}: {} batch={} boundary={} events={}",
|
||||
ticks.join(" "),
|
||||
transition.behaviour.batch_id,
|
||||
transition.behaviour.acknowledged_boundary,
|
||||
transition.behaviour.event_ids.len()
|
||||
);
|
||||
}
|
||||
let progress = harness.coordinator.task_progress();
|
||||
println!(
|
||||
"{via:?}: {} advances, counter {}, reward {}, {} publications, last boundary {}",
|
||||
harness.coordinator.stats().advances,
|
||||
progress.integer("counter").unwrap_or_default(),
|
||||
progress.number("totalReward").unwrap_or_default(),
|
||||
harness.coordinator.stats().publications,
|
||||
reports.last().map(|r| r.boundary).unwrap_or_default(),
|
||||
);
|
||||
// The behaviour trace is what two runs in different dispatch orders must agree on.
|
||||
for line in harness.coordinator.trace.behavior() {
|
||||
println!(" behaviour: {line}");
|
||||
}
|
||||
harness.shutdown().await;
|
||||
drop(dir);
|
||||
}
|
||||
}
|
||||
681
services/flysim/crates/fly-session/src/agent.rs
Normal file
681
services/flysim/crates/fly-session/src/agent.rs
Normal file
|
|
@ -0,0 +1,681 @@
|
|||
//! A small fake agent worker: an explicit seed, a mutation counter the tests read, and a fixed
|
||||
//! readout stub.
|
||||
//!
|
||||
//! There is no neural model here and no attempt to imitate one. What it does model exactly is
|
||||
//! the *ordering* `workers-v1` section 2 requires: Prepare applies pre-step stimulation, then
|
||||
//! advances whole ticks, then decodes; Commit installs the next input, then applies task
|
||||
//! stimulation, then reinforces once, and executes no tick at all. Every mutating step bumps
|
||||
//! one counter, which is how a test proves a duplicate request changed nothing.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::clock::TickAccumulator;
|
||||
use crate::dedup::OpClass;
|
||||
use crate::task::{context_schema, decision_schema};
|
||||
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
|
||||
// glob keeps the contract's own names in sight instead of restating them.
|
||||
use crate::types::*;
|
||||
use crate::worker::{BoxFuture, HandlerCtx, HandlerReply, StatusCell, WorkerEndpoint};
|
||||
|
||||
/// The fake numerical model: a seeded stream and a count of everything that mutated it.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct FakeModel {
|
||||
seed: i32,
|
||||
state: u64,
|
||||
mutations: u64,
|
||||
ticks: u64,
|
||||
stimulations: u64,
|
||||
reinforcements: u64,
|
||||
learning_enabled: bool,
|
||||
learning_updates: u64,
|
||||
learning_changed: u64,
|
||||
last_signal: f64,
|
||||
input_value: i64,
|
||||
input_installs: u64,
|
||||
}
|
||||
|
||||
impl FakeModel {
|
||||
/// A model at its initial state for `seed`. The seed is run configuration and state, and
|
||||
/// the same seed always produces the same stream.
|
||||
pub fn new(seed: i32) -> FakeModel {
|
||||
FakeModel {
|
||||
seed,
|
||||
// Sign-extend so a negative seed is a distinct stream rather than a truncation.
|
||||
state: (seed as i64 as u64) ^ 0x9e37_79b9_7f4a_7c15,
|
||||
mutations: 0,
|
||||
ticks: 0,
|
||||
stimulations: 0,
|
||||
reinforcements: 0,
|
||||
learning_enabled: false,
|
||||
learning_updates: 0,
|
||||
learning_changed: 0,
|
||||
last_signal: 0.0,
|
||||
input_value: 0,
|
||||
input_installs: 0,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn seed(&self) -> i32 {
|
||||
self.seed
|
||||
}
|
||||
|
||||
/// Every mutation this model has taken: ticks, stimulations, reinforcements and installs.
|
||||
///
|
||||
/// Tests read this to prove a duplicate request repeated nothing.
|
||||
pub fn mutations(&self) -> u64 {
|
||||
self.mutations
|
||||
}
|
||||
|
||||
pub fn ticks(&self) -> u64 {
|
||||
self.ticks
|
||||
}
|
||||
|
||||
pub fn reinforcements(&self) -> u64 {
|
||||
self.reinforcements
|
||||
}
|
||||
|
||||
pub fn stimulations(&self) -> u64 {
|
||||
self.stimulations
|
||||
}
|
||||
|
||||
/// How many times the next sensory input was installed: once per Initialize and Commit.
|
||||
pub fn input_installs(&self) -> u64 {
|
||||
self.input_installs
|
||||
}
|
||||
|
||||
/// The scalar the encoder last installed.
|
||||
pub fn input_value(&self) -> i64 {
|
||||
self.input_value
|
||||
}
|
||||
|
||||
fn draw(&mut self) -> u64 {
|
||||
self.state = self
|
||||
.state
|
||||
.wrapping_mul(6_364_136_223_846_793_005)
|
||||
.wrapping_add(1_442_695_040_888_963_407);
|
||||
self.mutations += 1;
|
||||
self.state
|
||||
}
|
||||
|
||||
/// Advances `ticks` whole model ticks. This is the only place ticks are executed.
|
||||
fn advance(&mut self, ticks: u64) {
|
||||
for _ in 0..ticks {
|
||||
self.draw();
|
||||
self.ticks += 1;
|
||||
}
|
||||
}
|
||||
|
||||
/// Applies one declared stimulus. The kind resolves through the profile; a caller never
|
||||
/// names a neuron or a drive value.
|
||||
fn stimulate(&mut self, stimulus: &Stimulus) {
|
||||
let kind = u64::from_le_bytes({
|
||||
let d = digest_of_bytes(stimulus.kind_id.as_bytes());
|
||||
let bytes = d.as_bytes();
|
||||
let mut out = [0u8; 8];
|
||||
out.copy_from_slice(&bytes[..8]);
|
||||
out
|
||||
});
|
||||
self.state ^= kind ^ (stimulus.duration_ms.to_bits());
|
||||
self.mutations += 1;
|
||||
self.stimulations += 1;
|
||||
}
|
||||
|
||||
/// Installs the next encoded sensory input for the following Prepare.
|
||||
fn install_input(&mut self, value: i64) {
|
||||
self.input_value = value;
|
||||
self.state ^= value as u64;
|
||||
self.mutations += 1;
|
||||
self.input_installs += 1;
|
||||
}
|
||||
|
||||
/// Reinforces once at the current brain time. A zero sum still reinforces: the profile's
|
||||
/// legacy-equivalent behaviour is not optimized away without evidence.
|
||||
fn reinforce(&mut self, signal: f64) {
|
||||
self.last_signal = signal;
|
||||
self.reinforcements += 1;
|
||||
self.mutations += 1;
|
||||
if self.learning_enabled {
|
||||
self.learning_updates += 1;
|
||||
if signal != 0.0 {
|
||||
self.learning_changed += 1;
|
||||
self.state ^= signal.to_bits();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The fixed readout: a deterministic decode of the current state, masked by the declared
|
||||
/// available actions. It never invents a default winner or changes its own weights.
|
||||
fn readout(&self, available: &[String]) -> (bool, bool, f64) {
|
||||
// Separate bits of one rotated word, because an LCG's low bits are too regular to
|
||||
// read a decision off directly.
|
||||
let draw = self.state.rotate_right(29);
|
||||
let mut inc = draw & 1 == 1;
|
||||
let mut dec = !inc && (draw >> 7) & 1 == 1;
|
||||
if !available.iter().any(|a| a == "inc") {
|
||||
inc = false;
|
||||
}
|
||||
if !available.iter().any(|a| a == "dec") {
|
||||
dec = false;
|
||||
}
|
||||
// Exactly representable in f64, so a digest over the decision is stable.
|
||||
let bias = (((draw >> 13) % 5) as f64 - 2.0) / 4.0;
|
||||
(inc, dec, bias)
|
||||
}
|
||||
|
||||
fn telemetry(&self) -> AgentTelemetry {
|
||||
let draw = self.state >> 29;
|
||||
AgentTelemetry {
|
||||
brain_ticks: self.ticks,
|
||||
population_rate_hz: (draw % 1000) as f64 / 10.0,
|
||||
rates: vec![
|
||||
RateSample { role_id: id("kc"), hz: (draw % 700) as f64 / 10.0 },
|
||||
RateSample { role_id: id("mbon"), hz: (draw % 310) as f64 / 10.0 },
|
||||
],
|
||||
learning: LearningTelemetry {
|
||||
enabled: self.learning_enabled,
|
||||
updates: self.learning_updates,
|
||||
changed: self.learning_changed,
|
||||
signal: self.last_signal,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Deliberate faults a test can ask this worker to produce.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct AgentFaults {
|
||||
/// Fail `Agent.Commit` at this step, after the next input was installed, so the coordinator
|
||||
/// meets a partially applied mutation rather than a clean refusal.
|
||||
pub fail_commit_at_step: Option<u64>,
|
||||
/// Hold `Agent.Prepare` open for this long, to reorder completions.
|
||||
pub prepare_delay_ms: u64,
|
||||
/// Hold `Agent.Commit` open for this long.
|
||||
pub commit_delay_ms: u64,
|
||||
}
|
||||
|
||||
/// One fake agent worker's configuration.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct AgentConfig {
|
||||
pub session_id: Id,
|
||||
pub agent_id: Id,
|
||||
pub incarnation_id: Id,
|
||||
pub tick_duration: RationalNs,
|
||||
pub warmup_ticks: u64,
|
||||
pub faults: AgentFaults,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
enum AgentPhase {
|
||||
Uninitialized,
|
||||
Ready(u64),
|
||||
Prepared(u64),
|
||||
Failed,
|
||||
}
|
||||
|
||||
/// The agent worker endpoint: model, sensor encoder and readout in one bundle.
|
||||
pub struct FakeAgentWorker {
|
||||
config: AgentConfig,
|
||||
status: StatusCell,
|
||||
phase: AgentPhase,
|
||||
epoch: Option<Id>,
|
||||
profile: Option<AssetRef>,
|
||||
accumulator: Option<TickAccumulator>,
|
||||
model: FakeModel,
|
||||
context: Option<TypedValue>,
|
||||
context_digest: Option<Digest>,
|
||||
prepared: Option<(DomainRequestId, PreparedDecision)>,
|
||||
}
|
||||
|
||||
impl FakeAgentWorker {
|
||||
pub fn new(config: AgentConfig) -> FakeAgentWorker {
|
||||
FakeAgentWorker {
|
||||
status: StatusCell::new(),
|
||||
phase: AgentPhase::Uninitialized,
|
||||
epoch: None,
|
||||
profile: None,
|
||||
accumulator: None,
|
||||
model: FakeModel::new(0),
|
||||
context: None,
|
||||
context_digest: None,
|
||||
prepared: None,
|
||||
config,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn status(&self) -> StatusCell {
|
||||
self.status.clone()
|
||||
}
|
||||
|
||||
fn check_epoch(&self, scope: &Scope) -> DomainResult<()> {
|
||||
if scope.session_id != self.config.session_id {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"this worker belongs to another session",
|
||||
));
|
||||
}
|
||||
match &self.epoch {
|
||||
Some(epoch) if *epoch == scope.epoch => Ok(()),
|
||||
Some(_) => Err(DomainError::before(
|
||||
ErrorCode::StaleEpoch,
|
||||
"this scope names an epoch this worker has left",
|
||||
)),
|
||||
None => Err(DomainError::before(
|
||||
ErrorCode::InvalidPhase,
|
||||
"this worker is uninitialized",
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Encodes a sensory input into the one scalar the fake model consumes.
|
||||
///
|
||||
/// Reading the pixels is what proves the attachment was a live owned handle rather than a
|
||||
/// bare reference; a missing required view is an error, never zero input.
|
||||
async fn encode(&self, ctx: &HandlerCtx<'_>, input: &SensoryInput) -> DomainResult<i64> {
|
||||
input.validate().map_err(DomainError::invalid)?;
|
||||
let mut total: i64 = 0;
|
||||
for view in &input.views {
|
||||
let name = format!("view.{}", view.view_id);
|
||||
let artifact = ctx.artifact(&name)?;
|
||||
if artifact.reference() != &view.pixels {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::BufferInvalid,
|
||||
format!("attachment {name} is not the artifact the payload names"),
|
||||
));
|
||||
}
|
||||
let bytes = artifact.read_all().await.map_err(|e| {
|
||||
DomainError::before(
|
||||
ErrorCode::BufferInvalid,
|
||||
format!("view {} could not be read: {}", view.view_id, e.message),
|
||||
)
|
||||
})?;
|
||||
if bytes.len() as u64 != view.pixels.byte_length {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::BufferInvalid,
|
||||
format!("view {} is the wrong length", view.view_id),
|
||||
));
|
||||
}
|
||||
total += i64::from(bytes.first().copied().unwrap_or_default());
|
||||
}
|
||||
if let Some(structured) = &input.structured {
|
||||
total += structured.integer("counter").map_err(DomainError::invalid)?;
|
||||
}
|
||||
Ok(total)
|
||||
}
|
||||
|
||||
fn available_actions(context: &TypedValue) -> DomainResult<Vec<String>> {
|
||||
if context.schema != context_schema() {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"the decision context does not carry the schema this profile allows",
|
||||
));
|
||||
}
|
||||
let list = context
|
||||
.value
|
||||
.get("available")
|
||||
.and_then(Value::as_array)
|
||||
.ok_or_else(|| DomainError::invalid("the decision context declares no actions"))?;
|
||||
list.iter()
|
||||
.map(|v| {
|
||||
v.as_str()
|
||||
.map(str::to_owned)
|
||||
.ok_or_else(|| DomainError::invalid("an available action is not a string"))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn decision(&self, available: &[String]) -> TypedValue {
|
||||
let (inc, dec, bias) = self.model.readout(available);
|
||||
let intent = ControllerIntent {
|
||||
buttons: vec![
|
||||
ButtonState { id: id("inc"), down: inc },
|
||||
ButtonState { id: id("dec"), down: dec },
|
||||
],
|
||||
axes: vec![AxisValue { id: id("bias"), value: bias }],
|
||||
};
|
||||
TypedValue::new(decision_schema(), intent.to_json())
|
||||
.expect("a direct-control decision fits the contract")
|
||||
}
|
||||
|
||||
async fn initialize(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
|
||||
let scope = ctx.scope()?.clone();
|
||||
if self.phase != AgentPhase::Uninitialized {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::InvalidPhase,
|
||||
"Agent.Initialize is only allowed on an uninitialized agent; restore uses the \
|
||||
state interface",
|
||||
));
|
||||
}
|
||||
if scope.step != 0 {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::FutureStep,
|
||||
"Agent.Initialize uses the new epoch at step 0",
|
||||
));
|
||||
}
|
||||
if scope.session_id != self.config.session_id {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"this worker belongs to another session",
|
||||
));
|
||||
}
|
||||
let params: AgentInitializeParams = ctx.params()?;
|
||||
if params.agent_id != self.config.agent_id {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"Agent.Initialize names another agent",
|
||||
));
|
||||
}
|
||||
if params.worker_threads == 0 {
|
||||
return Err(DomainError::invalid("workerThreads must be >= 1"));
|
||||
}
|
||||
params.initial_decision_context.validate().map_err(DomainError::invalid)?;
|
||||
let available = FakeAgentWorker::available_actions(¶ms.initial_decision_context)?;
|
||||
// Everything is validated before the model is constructed.
|
||||
let encoded = self.encode(ctx, ¶ms.initial_input).await?;
|
||||
if params.initial_input.boundary != 0 {
|
||||
return Err(DomainError::invalid("the initial input must observe boundary 0"));
|
||||
}
|
||||
|
||||
let mut accumulator =
|
||||
TickAccumulator::new(self.config.tick_duration).map_err(DomainError::invalid)?;
|
||||
let mut model = FakeModel::new(params.seed);
|
||||
model.install_input(encoded);
|
||||
// Warm-up runs with learning disabled and produces no gameplay reward or control.
|
||||
model.advance(self.config.warmup_ticks);
|
||||
accumulator.warm_up(self.config.warmup_ticks).map_err(|e| {
|
||||
DomainError::new(ErrorCode::Internal, e, MutationCertainty::Applied)
|
||||
})?;
|
||||
// Calibration happens on settled rates, after warm-up. The readout is a pure read of
|
||||
// the model, so calibrating it mutates nothing.
|
||||
let _calibration = model.readout(&available);
|
||||
model.learning_enabled = true;
|
||||
|
||||
self.model = model;
|
||||
self.accumulator = Some(accumulator);
|
||||
self.epoch = Some(scope.epoch.clone());
|
||||
self.profile = Some(params.profile.clone());
|
||||
self.context_digest = Some(params.initial_decision_context.digest());
|
||||
self.context = Some(params.initial_decision_context);
|
||||
self.phase = AgentPhase::Ready(0);
|
||||
self.status.set_state(WorkerState::Ready);
|
||||
self.status.set_scope(Some(scope.clone()));
|
||||
self.status.advance_to(self.model.mutations());
|
||||
|
||||
let result = AgentInitializeResult {
|
||||
agent_id: self.config.agent_id.clone(),
|
||||
profile_digest: params.profile.digest.clone(),
|
||||
tick_duration: self.config.tick_duration,
|
||||
warmup_ticks: self.config.warmup_ticks,
|
||||
committed_step: 0,
|
||||
decision_context_digest: self.context_digest.clone().expect("just set"),
|
||||
telemetry: self.model.telemetry(),
|
||||
};
|
||||
Ok(HandlerReply::from(&result))
|
||||
}
|
||||
|
||||
async fn prepare(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
|
||||
let scope = ctx.scope()?.clone();
|
||||
self.check_epoch(&scope)?;
|
||||
let AgentPhase::Ready(k) = self.phase.clone() else {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::InvalidPhase,
|
||||
format!("Agent.Prepare needs Ready(k); this worker is {:?}", self.phase),
|
||||
));
|
||||
};
|
||||
if scope.step < k {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::StaleStep,
|
||||
"Agent.Prepare names a step this worker has left",
|
||||
));
|
||||
}
|
||||
if scope.step > k {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::FutureStep,
|
||||
"Agent.Prepare names a step beyond this worker's committed boundary",
|
||||
));
|
||||
}
|
||||
let params: PrepareParams = ctx.params()?;
|
||||
if params.agent_id != self.config.agent_id {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"Agent.Prepare names another agent",
|
||||
));
|
||||
}
|
||||
let profile = self.profile.as_ref().expect("initialized");
|
||||
if params.profile_digest != profile.digest {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"Agent.Prepare names another profile",
|
||||
));
|
||||
}
|
||||
if Some(¶ms.decision_context_digest) != self.context_digest.as_ref() {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"the cached decision context digest does not match",
|
||||
));
|
||||
}
|
||||
params.interval.validate().map_err(DomainError::invalid)?;
|
||||
if params.pre_step_stimulations.len() > MAX_STIMULI {
|
||||
return Err(DomainError::invalid("at most 64 pre-step stimulations"));
|
||||
}
|
||||
for stimulus in ¶ms.pre_step_stimulations {
|
||||
stimulus.validate().map_err(DomainError::invalid)?;
|
||||
}
|
||||
let available =
|
||||
FakeAgentWorker::available_actions(self.context.as_ref().expect("initialized"))?;
|
||||
|
||||
self.status.set_state(WorkerState::Preparing);
|
||||
if self.config.faults.prepare_delay_ms > 0 {
|
||||
tokio::time::sleep(std::time::Duration::from_millis(
|
||||
self.config.faults.prepare_delay_ms,
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
// 1. admitted pre-step stimulation, in deterministic command sequence order
|
||||
for stimulus in ¶ms.pre_step_stimulations {
|
||||
self.model.stimulate(stimulus);
|
||||
}
|
||||
// 2. advance the numerical model for the environment interval
|
||||
let ticks = {
|
||||
let accumulator = self.accumulator.as_mut().expect("initialized");
|
||||
accumulator.advance(¶ms.interval).map_err(|e| {
|
||||
DomainError::new(ErrorCode::InvalidArgument, e, MutationCertainty::Applied)
|
||||
})?
|
||||
};
|
||||
self.model.advance(ticks);
|
||||
// 3. read rates and perform the fixed readout with the declared decision context
|
||||
let decision = self.decision(&available);
|
||||
let (brain_ticks, remainder) = {
|
||||
let accumulator = self.accumulator.as_ref().expect("initialized");
|
||||
(accumulator.brain_ticks(), accumulator.remainder())
|
||||
};
|
||||
let prepared = PreparedDecision {
|
||||
agent_id: self.config.agent_id.clone(),
|
||||
ticks_advanced: ticks,
|
||||
brain_ticks,
|
||||
remainder,
|
||||
decision,
|
||||
};
|
||||
self.prepared = Some((ctx.request.request_id.clone(), prepared.clone()));
|
||||
self.phase = AgentPhase::Prepared(k);
|
||||
self.status.set_state(WorkerState::Prepared);
|
||||
self.status.advance_to(self.model.mutations());
|
||||
Ok(HandlerReply::from(&prepared))
|
||||
}
|
||||
|
||||
async fn commit(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
|
||||
let scope = ctx.scope()?.clone();
|
||||
self.check_epoch(&scope)?;
|
||||
let AgentPhase::Prepared(k) = self.phase.clone() else {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::InvalidPhase,
|
||||
format!("Agent.Commit needs Prepared(k); this worker is {:?}", self.phase),
|
||||
));
|
||||
};
|
||||
if scope.step != k {
|
||||
return Err(DomainError::before(
|
||||
if scope.step < k { ErrorCode::StaleStep } else { ErrorCode::FutureStep },
|
||||
"Agent.Commit must carry the step of its transition, not the new boundary",
|
||||
));
|
||||
}
|
||||
let params: CommitParams = ctx.params()?;
|
||||
if params.agent_id != self.config.agent_id {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"Agent.Commit names another agent",
|
||||
));
|
||||
}
|
||||
let (prepared_request, _) = self.prepared.as_ref().expect("prepared");
|
||||
if params.prepared_request_id != *prepared_request {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"Agent.Commit does not match this worker's Prepare request",
|
||||
));
|
||||
}
|
||||
if params.next_input.boundary != k + 1 {
|
||||
return Err(DomainError::invalid(
|
||||
"the next sensory input must observe boundary k+1",
|
||||
));
|
||||
}
|
||||
if params.rewards.len() > MAX_REWARDS || params.task_stimulations.len() > MAX_STIMULI
|
||||
{
|
||||
return Err(DomainError::invalid("at most 64 rewards and 64 stimulations"));
|
||||
}
|
||||
for reward in ¶ms.rewards {
|
||||
reward.validate().map_err(DomainError::invalid)?;
|
||||
}
|
||||
for stimulus in ¶ms.task_stimulations {
|
||||
stimulus.validate().map_err(DomainError::invalid)?;
|
||||
}
|
||||
params.next_decision_context.validate().map_err(DomainError::invalid)?;
|
||||
FakeAgentWorker::available_actions(¶ms.next_decision_context)?;
|
||||
// The complete request and every required owned artifact are validated before anything
|
||||
// is applied.
|
||||
let encoded = self.encode(ctx, ¶ms.next_input).await?;
|
||||
|
||||
self.status.set_state(WorkerState::Committing);
|
||||
if self.config.faults.commit_delay_ms > 0 {
|
||||
tokio::time::sleep(std::time::Duration::from_millis(
|
||||
self.config.faults.commit_delay_ms,
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
// 1. encode and install the next sensory input for the following Prepare
|
||||
self.model.install_input(encoded);
|
||||
if self.config.faults.fail_commit_at_step == Some(k) {
|
||||
// A deliberate fault after the input was installed: the brain is already mutated,
|
||||
// so the coordinator has to recover the group rather than retry this agent.
|
||||
self.phase = AgentPhase::Failed;
|
||||
self.status.set_state(WorkerState::Failed);
|
||||
return Err(DomainError::new(
|
||||
ErrorCode::BackendFailure,
|
||||
"injected commit failure after the next input was installed",
|
||||
MutationCertainty::Applied,
|
||||
));
|
||||
}
|
||||
// 2. apply task-derived stimulation in returned event order
|
||||
for stimulus in ¶ms.task_stimulations {
|
||||
self.model.stimulate(stimulus);
|
||||
}
|
||||
// 3. sum this agent's rewards in returned event order and reinforce once
|
||||
let mut signal = 0.0f64;
|
||||
for reward in ¶ms.rewards {
|
||||
signal += reward.value;
|
||||
}
|
||||
self.model.reinforce(signal);
|
||||
// 4. retain the next decision context and acknowledge boundary k+1
|
||||
self.context_digest = Some(params.next_decision_context.digest());
|
||||
self.context = Some(params.next_decision_context);
|
||||
self.prepared = None;
|
||||
self.phase = AgentPhase::Ready(k + 1);
|
||||
self.status.set_state(WorkerState::Ready);
|
||||
self.status.set_scope(Some(scope_at(&scope.session_id, &scope.epoch, k + 1)));
|
||||
self.status.advance_to(self.model.mutations());
|
||||
|
||||
let result = AgentCommitResult {
|
||||
agent_id: self.config.agent_id.clone(),
|
||||
committed_step: k + 1,
|
||||
decision_context_digest: self.context_digest.clone().expect("just set"),
|
||||
telemetry: self.model.telemetry(),
|
||||
};
|
||||
Ok(HandlerReply::from(&result))
|
||||
}
|
||||
|
||||
/// The model, for a test that wants its mutation counter directly.
|
||||
pub fn model(&self) -> &FakeModel {
|
||||
&self.model
|
||||
}
|
||||
}
|
||||
|
||||
impl WorkerEndpoint for FakeAgentWorker {
|
||||
fn worker_id(&self) -> Id {
|
||||
self.config.agent_id.clone()
|
||||
}
|
||||
|
||||
fn incarnation_id(&self) -> Id {
|
||||
self.config.incarnation_id.clone()
|
||||
}
|
||||
|
||||
fn session_id(&self) -> Id {
|
||||
self.config.session_id.clone()
|
||||
}
|
||||
|
||||
fn role(&self) -> Role {
|
||||
Role::Agent
|
||||
}
|
||||
|
||||
fn capabilities(&self) -> Vec<Id> {
|
||||
vec![id("agent-step-v1"), id("pixel-observation-v1")]
|
||||
}
|
||||
|
||||
fn status_cell(&self) -> StatusCell {
|
||||
self.status.clone()
|
||||
}
|
||||
|
||||
fn methods(&self) -> Vec<&'static str> {
|
||||
vec!["Agent.Initialize", "Agent.Prepare", "Agent.Commit"]
|
||||
}
|
||||
|
||||
fn handle<'a>(&'a mut self, ctx: HandlerCtx<'a>) -> BoxFuture<'a, DomainResult<HandlerReply>> {
|
||||
Box::pin(async move {
|
||||
match ctx.method {
|
||||
"Agent.Initialize" => self.initialize(&ctx).await,
|
||||
"Agent.Prepare" => self.prepare(&ctx).await,
|
||||
"Agent.Commit" => self.commit(&ctx).await,
|
||||
other => Err(DomainError::before(
|
||||
ErrorCode::Unsupported,
|
||||
format!("{other} is not an agent method"),
|
||||
)),
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// The retention class table an agent endpoint follows, for a caller that wants it.
|
||||
pub fn agent_op_class(method: &str) -> Option<OpClass> {
|
||||
match method {
|
||||
"Agent.Initialize" => Some(OpClass::Lifecycle),
|
||||
"Agent.Prepare" | "Agent.Commit" => Some(OpClass::StepMutation),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// A synthetic profile asset for one agent. The digest covers its effective identities.
|
||||
pub fn synthetic_profile(agent_id: &Id, tick_duration: &RationalNs, warmup_ticks: u64) -> AssetRef {
|
||||
let text = format!(
|
||||
"arena-direct-v1\nagent={agent_id}\ntick={}/{}\nwarmup={warmup_ticks}\n",
|
||||
tick_duration.numerator, tick_duration.denominator
|
||||
);
|
||||
AssetRef {
|
||||
id: id("arena-direct-v1"),
|
||||
digest: digest_of_bytes(text.as_bytes()),
|
||||
byte_length: text.len() as u64,
|
||||
format: id("fly-profile-v1"),
|
||||
}
|
||||
}
|
||||
|
||||
/// The per-agent contexts a bootstrap produced, keyed by agent id.
|
||||
pub type Contexts = BTreeMap<Id, TypedValue>;
|
||||
223
services/flysim/crates/fly-session/src/clock.rs
Normal file
223
services/flysim/crates/fly-session/src/clock.rs
Normal file
|
|
@ -0,0 +1,223 @@
|
|||
//! Time and pacing: `step-v1` section 5.
|
||||
//!
|
||||
//! The accumulator is the contract crate's exact rational arithmetic, never rounded
|
||||
//! nanoseconds. A 60 Hz world with a 1 ms model tick advances 16, 17, 17 ticks over its first
|
||||
//! three steps and comes back to a remainder of exactly zero; rounding to microseconds does
|
||||
//! not.
|
||||
|
||||
use crate::types::{DomainError, DomainType, ErrorCode, MutationCertainty, RationalNs};
|
||||
|
||||
/// One agent's tick accumulator: its tick duration, its remainder and its executed count.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct TickAccumulator {
|
||||
tick_duration: RationalNs,
|
||||
remainder: RationalNs,
|
||||
executed_ticks: u64,
|
||||
warmup_offset: u64,
|
||||
}
|
||||
|
||||
impl TickAccumulator {
|
||||
/// A fresh accumulator. The tick duration must be positive.
|
||||
pub fn new(tick_duration: RationalNs) -> Result<TickAccumulator, String> {
|
||||
tick_duration.validate().map_err(|e| e.0)?;
|
||||
tick_duration.require_positive("tick duration").map_err(|e| e.0)?;
|
||||
Ok(TickAccumulator {
|
||||
tick_duration,
|
||||
remainder: RationalNs::ZERO,
|
||||
executed_ticks: 0,
|
||||
warmup_offset: 0,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn tick_duration(&self) -> RationalNs {
|
||||
self.tick_duration
|
||||
}
|
||||
|
||||
/// The persisted remainder: always >= 0 and < one model tick.
|
||||
pub fn remainder(&self) -> RationalNs {
|
||||
self.remainder
|
||||
}
|
||||
|
||||
/// Every tick this accumulator has executed, warm-up included.
|
||||
pub fn executed_ticks(&self) -> u64 {
|
||||
self.executed_ticks
|
||||
}
|
||||
|
||||
/// The warm-up ticks executed before the first gameplay transition.
|
||||
pub fn warmup_offset(&self) -> u64 {
|
||||
self.warmup_offset
|
||||
}
|
||||
|
||||
/// Accounts for `ticks` of warm-up. Warm-up does not consume an environment interval, so
|
||||
/// it never touches the remainder.
|
||||
pub fn warm_up(&mut self, ticks: u64) -> Result<(), String> {
|
||||
self.warmup_offset = self
|
||||
.warmup_offset
|
||||
.checked_add(ticks)
|
||||
.ok_or_else(|| "warm-up tick count overflows".to_owned())?;
|
||||
self.executed_ticks = self
|
||||
.executed_ticks
|
||||
.checked_add(ticks)
|
||||
.ok_or_else(|| "executed tick count overflows".to_owned())?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Adds one environment interval and returns the whole ticks it covers.
|
||||
///
|
||||
/// ```text
|
||||
/// accumulator += environment step duration
|
||||
/// ticks = floor(accumulator / model tick duration)
|
||||
/// accumulator -= ticks * model tick duration
|
||||
/// ```
|
||||
pub fn advance(&mut self, interval: &RationalNs) -> Result<u64, String> {
|
||||
interval.validate().map_err(|e| e.0)?;
|
||||
interval.require_positive("environment interval").map_err(|e| e.0)?;
|
||||
let accumulated = self.remainder.checked_add(interval).map_err(|e| e.0)?;
|
||||
let (ticks, remainder) =
|
||||
accumulated.divide_floor(&self.tick_duration).map_err(|e| e.0)?;
|
||||
debug_assert!(
|
||||
remainder < self.tick_duration,
|
||||
"the remainder must stay below one model tick"
|
||||
);
|
||||
self.remainder = remainder;
|
||||
self.executed_ticks = self
|
||||
.executed_ticks
|
||||
.checked_add(ticks)
|
||||
.ok_or_else(|| "executed tick count overflows".to_owned())?;
|
||||
Ok(ticks)
|
||||
}
|
||||
|
||||
pub fn brain_ticks(&self) -> u64 {
|
||||
self.executed_ticks
|
||||
}
|
||||
}
|
||||
|
||||
/// The coordinator's pacing authority: absolute deadlines after committed boundaries.
|
||||
///
|
||||
/// Only one pacing authority may be active, so this is the coordinator's and the backend does
|
||||
/// not throttle as well. When behind, it omits the sleep and reports the lag; it never skips a
|
||||
/// world step or drops a neural tick.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Pacing {
|
||||
step_duration: RationalNs,
|
||||
next_deadline: Option<std::time::Instant>,
|
||||
lag: std::time::Duration,
|
||||
lagged_steps: u64,
|
||||
}
|
||||
|
||||
impl Pacing {
|
||||
pub fn new(step_duration: RationalNs) -> Pacing {
|
||||
Pacing {
|
||||
step_duration,
|
||||
next_deadline: None,
|
||||
lag: std::time::Duration::ZERO,
|
||||
lagged_steps: 0,
|
||||
}
|
||||
}
|
||||
|
||||
/// The wall-clock period of one step, rounded for sleeping only. Simulation time stays
|
||||
/// rational; this value is never fed back into the accumulator.
|
||||
fn period(&self) -> std::time::Duration {
|
||||
let ns = u128::from(self.step_duration.numerator)
|
||||
/ u128::from(self.step_duration.denominator).max(1);
|
||||
std::time::Duration::from_nanos(u64::try_from(ns).unwrap_or(u64::MAX))
|
||||
}
|
||||
|
||||
/// Waits until this step's deadline. Returns the lag if the deadline had already passed.
|
||||
pub async fn wait(&mut self) -> Option<std::time::Duration> {
|
||||
let now = std::time::Instant::now();
|
||||
let deadline = self.next_deadline.unwrap_or(now);
|
||||
let outcome = if deadline > now {
|
||||
tokio::time::sleep_until(tokio::time::Instant::from_std(deadline)).await;
|
||||
None
|
||||
} else {
|
||||
let behind = now.duration_since(deadline);
|
||||
if !behind.is_zero() {
|
||||
self.lag += behind;
|
||||
self.lagged_steps += 1;
|
||||
}
|
||||
Some(behind)
|
||||
};
|
||||
self.next_deadline = Some(deadline.max(now) + self.period());
|
||||
outcome
|
||||
}
|
||||
|
||||
pub fn total_lag(&self) -> std::time::Duration {
|
||||
self.lag
|
||||
}
|
||||
|
||||
pub fn lagged_steps(&self) -> u64 {
|
||||
self.lagged_steps
|
||||
}
|
||||
}
|
||||
|
||||
/// Converts a whole tick count to the legacy f64 millisecond clock, refusing a run that has
|
||||
/// left the exactly representable range.
|
||||
pub fn ticks_to_legacy_millis(ticks: u64, tick_duration: &RationalNs) -> Result<f64, DomainError> {
|
||||
// 2^53 is the last integer f64 represents exactly; beyond it a millisecond clock starts
|
||||
// skipping representable ticks, so the run is refused rather than silently rounded.
|
||||
const EXACT_F64_INTEGERS: u64 = 1 << 53;
|
||||
if ticks >= EXACT_F64_INTEGERS {
|
||||
return Err(DomainError::new(
|
||||
ErrorCode::InvalidArgument,
|
||||
"tick count exceeds the range the legacy millisecond clock represents exactly",
|
||||
MutationCertainty::None,
|
||||
));
|
||||
}
|
||||
let per_tick_ms =
|
||||
tick_duration.numerator as f64 / (tick_duration.denominator as f64 * 1_000_000.0);
|
||||
Ok(ticks as f64 * per_tick_ms)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::types::{hz, millis};
|
||||
|
||||
#[test]
|
||||
fn a_60_hz_world_with_a_1_ms_tick_runs_16_17_17() {
|
||||
let step = hz(60).unwrap();
|
||||
let mut acc = TickAccumulator::new(millis(1).unwrap()).unwrap();
|
||||
let ticks: Vec<u64> = (0..3).map(|_| acc.advance(&step).unwrap()).collect();
|
||||
assert_eq!(ticks, vec![16, 17, 17]);
|
||||
assert_eq!(ticks.iter().sum::<u64>(), 50);
|
||||
assert!(acc.remainder().is_zero());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_remainder_stays_below_one_tick_and_never_goes_negative() {
|
||||
let step = hz(60).unwrap();
|
||||
let tick = millis(1).unwrap();
|
||||
let mut acc = TickAccumulator::new(tick).unwrap();
|
||||
for _ in 0..600 {
|
||||
acc.advance(&step).unwrap();
|
||||
assert!(acc.remainder() < tick);
|
||||
}
|
||||
// 600 steps of 1/60 s is exactly 10 s, which is 10,000 whole milliseconds.
|
||||
assert_eq!(acc.executed_ticks(), 10_000);
|
||||
assert!(acc.remainder().is_zero());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warm_up_ticks_do_not_touch_the_remainder() {
|
||||
let mut acc = TickAccumulator::new(millis(1).unwrap()).unwrap();
|
||||
acc.warm_up(25).unwrap();
|
||||
assert_eq!(acc.executed_ticks(), 25);
|
||||
assert_eq!(acc.warmup_offset(), 25);
|
||||
assert!(acc.remainder().is_zero());
|
||||
assert_eq!(acc.advance(&hz(60).unwrap()).unwrap(), 16);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_zero_interval_is_refused_rather_than_silently_producing_no_ticks() {
|
||||
let mut acc = TickAccumulator::new(millis(1).unwrap()).unwrap();
|
||||
assert!(acc.advance(&RationalNs::ZERO).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_legacy_millisecond_clock_refuses_a_run_beyond_its_exact_range() {
|
||||
let tick = millis(1).unwrap();
|
||||
assert_eq!(ticks_to_legacy_millis(50, &tick).unwrap(), 50.0);
|
||||
assert!(ticks_to_legacy_millis(1 << 53, &tick).is_err());
|
||||
}
|
||||
}
|
||||
2110
services/flysim/crates/fly-session/src/coordinator.rs
Normal file
2110
services/flysim/crates/fly-session/src/coordinator.rs
Normal file
File diff suppressed because it is too large
Load diff
446
services/flysim/crates/fly-session/src/dedup.rs
Normal file
446
services/flysim/crates/fly-session/src/dedup.rs
Normal file
|
|
@ -0,0 +1,446 @@
|
|||
//! Domain deduplication and the result cache of `ipc-v1` section 5.
|
||||
//!
|
||||
//! The operation key for a step mutation is `(sessionId, epoch, step, method, workerId)`, and
|
||||
//! there is at most one Prepare, Commit or Advance for it. The cache decides, *before* any
|
||||
//! phase check or artifact dereference, whether an arriving request is the original, a safe
|
||||
//! replay, a conflicting change, a duplicate of something still running, or a retry whose
|
||||
//! result is gone.
|
||||
//!
|
||||
//! A cached reply owns its artifacts through explicit holds, so a replay is still valid after
|
||||
//! the first caller consumed its delivery. Dropping the record drops those holds.
|
||||
|
||||
use std::collections::{BTreeMap, VecDeque};
|
||||
|
||||
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
|
||||
// glob keeps the contract's own names in sight instead of restating them.
|
||||
use crate::types::*;
|
||||
|
||||
/// `ipc-v1` section 5: unacknowledged lifecycle replies are bounded at 16, then BUSY.
|
||||
pub const MAX_UNACKNOWLEDGED: usize = 16;
|
||||
/// `ipc-v1` section 5: Status and Acknowledge keep a cache of their last 16 replies.
|
||||
pub const MAX_READONLY_REPLIES: usize = 16;
|
||||
/// `ipc-v1` section 5: keep the current and the immediately previous step's records.
|
||||
pub const RETAINED_STEPS: u64 = 2;
|
||||
|
||||
/// Which retention rule an operation follows.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum OpClass {
|
||||
/// Prepare, Commit, Advance: keyed by scope and retained for two steps.
|
||||
StepMutation,
|
||||
/// Initialize and capture: retained until `Worker.Acknowledge`.
|
||||
Lifecycle,
|
||||
/// Status and Acknowledge: a small last-16 reply cache, no mutation key.
|
||||
ReadOnly,
|
||||
}
|
||||
|
||||
/// A step mutation's identity. Not a bus call id and not a batch id.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub struct OperationKey {
|
||||
pub session_id: Id,
|
||||
pub epoch: Id,
|
||||
pub step: u64,
|
||||
pub method: String,
|
||||
pub worker_id: Id,
|
||||
}
|
||||
|
||||
/// A terminal domain reply plus the artifact holds that keep its attachments readable.
|
||||
#[derive(Clone)]
|
||||
pub struct CachedReply {
|
||||
pub outcome: SessionRpcOutcome,
|
||||
pub artifacts: Vec<(String, flybus::Artifact)>,
|
||||
}
|
||||
|
||||
impl CachedReply {
|
||||
pub fn new(outcome: SessionRpcOutcome) -> CachedReply {
|
||||
CachedReply { outcome, artifacts: Vec::new() }
|
||||
}
|
||||
|
||||
pub fn with_artifacts(
|
||||
outcome: SessionRpcOutcome,
|
||||
artifacts: Vec<(String, flybus::Artifact)>,
|
||||
) -> CachedReply {
|
||||
CachedReply { outcome, artifacts }
|
||||
}
|
||||
|
||||
/// The attachment list for a fresh `rpc.reply`, over the same immutable bytes.
|
||||
pub fn attachments(&self) -> Vec<(&str, &flybus::Artifact)> {
|
||||
self.artifacts.iter().map(|(n, a)| (n.as_str(), a)).collect()
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for CachedReply {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("CachedReply")
|
||||
.field("outcome", &self.outcome)
|
||||
.field("artifacts", &self.artifacts.len())
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
/// What the cache decided about an arriving request.
|
||||
#[derive(Debug)]
|
||||
pub enum Admission {
|
||||
/// The original: run it, then `record` its reply.
|
||||
Execute,
|
||||
/// A safe replay: return this reply again, with fresh delivery ownership.
|
||||
Replay(CachedReply),
|
||||
/// A terminal domain error for this bus call, with no second mutation started.
|
||||
Refuse(DomainError),
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct Record {
|
||||
request_id: DomainRequestId,
|
||||
body: Digest,
|
||||
reply: CachedReply,
|
||||
}
|
||||
|
||||
/// One worker's domain request cache.
|
||||
pub struct ResultCache {
|
||||
steps: BTreeMap<OperationKey, Record>,
|
||||
active: BTreeMap<OperationKey, (DomainRequestId, Digest)>,
|
||||
lifecycle: BTreeMap<String, Record>,
|
||||
lifecycle_order: VecDeque<String>,
|
||||
readonly: VecDeque<(String, CachedReply)>,
|
||||
highest_serial: Option<u64>,
|
||||
step_watermark: Option<u64>,
|
||||
/// Serials retired by `Worker.Acknowledge`; reuse below this is refused without keeping a
|
||||
/// tombstone per request.
|
||||
acknowledged_watermark: Option<u64>,
|
||||
}
|
||||
|
||||
impl Default for ResultCache {
|
||||
fn default() -> ResultCache {
|
||||
ResultCache::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl ResultCache {
|
||||
pub fn new() -> ResultCache {
|
||||
ResultCache {
|
||||
steps: BTreeMap::new(),
|
||||
active: BTreeMap::new(),
|
||||
lifecycle: BTreeMap::new(),
|
||||
lifecycle_order: VecDeque::new(),
|
||||
readonly: VecDeque::new(),
|
||||
highest_serial: None,
|
||||
step_watermark: None,
|
||||
acknowledged_watermark: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Decides what to do with an arriving request. Identity is checked before phase.
|
||||
pub fn admit(
|
||||
&mut self,
|
||||
class: OpClass,
|
||||
key: &OperationKey,
|
||||
request: DomainRequestId,
|
||||
body: &Digest,
|
||||
) -> Admission {
|
||||
match class {
|
||||
OpClass::StepMutation => self.admit_step(key, request, body),
|
||||
OpClass::Lifecycle => self.admit_lifecycle(request, body),
|
||||
OpClass::ReadOnly => Admission::Execute,
|
||||
}
|
||||
}
|
||||
|
||||
fn admit_step(&mut self, key: &OperationKey, request: DomainRequestId, body: &Digest) -> Admission {
|
||||
if let Some(record) = self.steps.get(key) {
|
||||
if record.request_id == request && record.body == *body {
|
||||
return Admission::Replay(record.reply.clone());
|
||||
}
|
||||
return Admission::Refuse(DomainError::new(
|
||||
ErrorCode::Conflict,
|
||||
"this operation key already holds a different request or body",
|
||||
// The earlier result stands; refusing the duplicate undoes nothing.
|
||||
MutationCertainty::None,
|
||||
));
|
||||
}
|
||||
if !self.active.is_empty() && !self.active.contains_key(key) {
|
||||
// One mutation executes at a time. A second, different one is refused before
|
||||
// admission rather than queued behind the first.
|
||||
return Admission::Refuse(DomainError::before(
|
||||
ErrorCode::Busy,
|
||||
"another step mutation is already executing on this worker",
|
||||
));
|
||||
}
|
||||
if let Some((active_request, active_body)) = self.active.get(key) {
|
||||
if *active_request == request && *active_body == *body {
|
||||
// The duplicate bus call started no work at all, so its certainty is none;
|
||||
// it must not be mistaken for the original operation's failure.
|
||||
return Admission::Refuse(DomainError::before(
|
||||
ErrorCode::InProgress,
|
||||
"the original operation is still executing; this duplicate started no work",
|
||||
));
|
||||
}
|
||||
return Admission::Refuse(DomainError::new(
|
||||
ErrorCode::Conflict,
|
||||
"an operation with a different body is active for this key",
|
||||
MutationCertainty::None,
|
||||
));
|
||||
}
|
||||
// No record and nothing active. Eviction must never re-enable execution, so a step
|
||||
// below the retention window is refused on the serial and step watermarks.
|
||||
if let Some(watermark) = self.step_watermark
|
||||
&& key.step + RETAINED_STEPS <= watermark
|
||||
{
|
||||
let issued = self.highest_serial.is_some_and(|h| request.serial() <= h);
|
||||
return Admission::Refuse(if issued {
|
||||
DomainError::new(
|
||||
ErrorCode::ResultExpired,
|
||||
"the retained result for this request is gone; it is never recomputed",
|
||||
MutationCertainty::Unknown,
|
||||
)
|
||||
} else {
|
||||
DomainError::before(
|
||||
ErrorCode::StaleStep,
|
||||
"a newly issued request cannot name a step the worker has left behind",
|
||||
)
|
||||
});
|
||||
}
|
||||
if let Some(watermark) = self.acknowledged_watermark
|
||||
&& request.serial() <= watermark
|
||||
{
|
||||
return Admission::Refuse(DomainError::new(
|
||||
ErrorCode::ResultExpired,
|
||||
"this request serial was acknowledged and cannot be reused",
|
||||
MutationCertainty::Unknown,
|
||||
));
|
||||
}
|
||||
self.begin(key.clone(), request, body.clone());
|
||||
Admission::Execute
|
||||
}
|
||||
|
||||
fn admit_lifecycle(&mut self, request: DomainRequestId, body: &Digest) -> Admission {
|
||||
let id = request.as_str().to_owned();
|
||||
if let Some(record) = self.lifecycle.get(&id) {
|
||||
if record.body == *body {
|
||||
return Admission::Replay(record.reply.clone());
|
||||
}
|
||||
return Admission::Refuse(DomainError::before(
|
||||
ErrorCode::Conflict,
|
||||
"this lifecycle request id already holds a different body",
|
||||
));
|
||||
}
|
||||
if let Some(watermark) = self.acknowledged_watermark
|
||||
&& request.serial() <= watermark
|
||||
{
|
||||
return Admission::Refuse(DomainError::new(
|
||||
ErrorCode::ResultExpired,
|
||||
"this request serial was acknowledged and cannot be reused",
|
||||
MutationCertainty::Unknown,
|
||||
));
|
||||
}
|
||||
if self.lifecycle.len() >= MAX_UNACKNOWLEDGED {
|
||||
return Admission::Refuse(DomainError::before(
|
||||
ErrorCode::Busy,
|
||||
"16 lifecycle replies are unacknowledged; acknowledge some before sending more",
|
||||
));
|
||||
}
|
||||
Admission::Execute
|
||||
}
|
||||
|
||||
fn begin(&mut self, key: OperationKey, request: DomainRequestId, body: Digest) {
|
||||
self.highest_serial = Some(self.highest_serial.map_or(request.serial(), |h| h.max(request.serial())));
|
||||
self.step_watermark = Some(self.step_watermark.map_or(key.step, |w| w.max(key.step)));
|
||||
self.active.insert(key, (request, body));
|
||||
}
|
||||
|
||||
/// Stores a terminal reply for a step mutation and prunes what has aged out.
|
||||
pub fn record(
|
||||
&mut self,
|
||||
key: OperationKey,
|
||||
request: DomainRequestId,
|
||||
body: Digest,
|
||||
reply: CachedReply,
|
||||
) {
|
||||
self.active.remove(&key);
|
||||
let step = key.step;
|
||||
self.steps.insert(key, Record { request_id: request, body, reply });
|
||||
self.step_watermark = Some(self.step_watermark.map_or(step, |w| w.max(step)));
|
||||
self.prune();
|
||||
}
|
||||
|
||||
/// Stores a lifecycle reply, retained until `Worker.Acknowledge`.
|
||||
pub fn record_lifecycle(&mut self, request: DomainRequestId, body: Digest, reply: CachedReply) {
|
||||
let id = request.as_str().to_owned();
|
||||
self.highest_serial = Some(self.highest_serial.map_or(request.serial(), |h| h.max(request.serial())));
|
||||
if self.lifecycle.insert(id.clone(), Record { request_id: request, body, reply }).is_none()
|
||||
{
|
||||
self.lifecycle_order.push_back(id);
|
||||
}
|
||||
}
|
||||
|
||||
/// Stores a read-only reply in the last-16 cache.
|
||||
pub fn record_readonly(&mut self, request: DomainRequestId, reply: CachedReply) {
|
||||
let id = request.as_str().to_owned();
|
||||
self.readonly.retain(|(existing, _)| *existing != id);
|
||||
self.readonly.push_back((id, reply));
|
||||
while self.readonly.len() > MAX_READONLY_REPLIES {
|
||||
self.readonly.pop_front();
|
||||
}
|
||||
}
|
||||
|
||||
/// Releases an operation that did not mutate anything, so the key stays free.
|
||||
pub fn abandon(&mut self, key: &OperationKey) {
|
||||
self.active.remove(key);
|
||||
}
|
||||
|
||||
/// `Worker.Acknowledge`: drops those lifecycle records, ignoring unknown ids, and raises
|
||||
/// the serial watermark so an acknowledged id cannot be reused.
|
||||
pub fn acknowledge(&mut self, ids: &[DomainRequestId]) -> Vec<DomainRequestId> {
|
||||
let mut out = Vec::new();
|
||||
for id in ids {
|
||||
if let Some(record) = self.lifecycle.remove(id.as_str()) {
|
||||
self.lifecycle_order.retain(|existing| existing != id.as_str());
|
||||
self.acknowledged_watermark = Some(
|
||||
self.acknowledged_watermark
|
||||
.map_or(record.request_id.serial(), |w| w.max(record.request_id.serial())),
|
||||
);
|
||||
out.push(id.clone());
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
pub fn unacknowledged(&self) -> usize {
|
||||
self.lifecycle.len()
|
||||
}
|
||||
|
||||
pub fn retained_steps(&self) -> usize {
|
||||
self.steps.len()
|
||||
}
|
||||
|
||||
pub fn has_active(&self) -> bool {
|
||||
!self.active.is_empty()
|
||||
}
|
||||
|
||||
/// Deliberately drops a step record, so a retry meets RESULT_EXPIRED instead of a replay.
|
||||
pub fn expire_step(&mut self, key: &OperationKey) -> bool {
|
||||
self.steps.remove(key).is_some()
|
||||
}
|
||||
|
||||
fn prune(&mut self) {
|
||||
let Some(watermark) = self.step_watermark else { return };
|
||||
let floor = watermark.saturating_sub(RETAINED_STEPS - 1);
|
||||
self.steps.retain(|key, _| key.step >= floor);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use serde_json::Value;
|
||||
|
||||
use super::*;
|
||||
|
||||
fn key(step: u64, method: &str) -> OperationKey {
|
||||
OperationKey {
|
||||
session_id: id("demo"),
|
||||
epoch: id("e1"),
|
||||
step,
|
||||
method: method.to_owned(),
|
||||
worker_id: id("fly-a"),
|
||||
}
|
||||
}
|
||||
|
||||
fn reply(tag: &str) -> CachedReply {
|
||||
let mut result = serde_json::Map::new();
|
||||
result.insert("tag".into(), tag.into());
|
||||
CachedReply::new(SessionRpcOutcome::Success(SessionRpcSuccess {
|
||||
request_id: DomainRequestId::from_serial(1),
|
||||
worker_id: id("fly-a"),
|
||||
incarnation_id: id("inc-1"),
|
||||
scope: Some(scope_at("demo", "e1", 0)),
|
||||
result: Value::Object(result),
|
||||
}))
|
||||
}
|
||||
|
||||
fn body(s: &str) -> Digest {
|
||||
digest_of_bytes(s.as_bytes())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_same_key_request_and_body_replays() {
|
||||
let mut c = ResultCache::new();
|
||||
let k = key(0, "Agent.Prepare");
|
||||
assert!(matches!(c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a")), Admission::Execute));
|
||||
c.record(k.clone(), DomainRequestId::from_serial(1), body("a"), reply("first"));
|
||||
match c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a")) {
|
||||
Admission::Replay(r) => {
|
||||
assert_eq!(r.outcome.result().unwrap()["tag"], "first");
|
||||
}
|
||||
other => panic!("wanted a replay, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_changed_body_for_a_recorded_key_is_a_conflict() {
|
||||
let mut c = ResultCache::new();
|
||||
let k = key(0, "Environment.Advance");
|
||||
c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a"));
|
||||
c.record(k.clone(), DomainRequestId::from_serial(1), body("a"), reply("first"));
|
||||
match c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("b")) {
|
||||
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::Conflict),
|
||||
other => panic!("wanted CONFLICT, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_duplicate_while_the_original_runs_is_in_progress() {
|
||||
let mut c = ResultCache::new();
|
||||
let k = key(0, "Agent.Prepare");
|
||||
c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a"));
|
||||
match c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a")) {
|
||||
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::InProgress),
|
||||
other => panic!("wanted IN_PROGRESS, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_evicted_step_gives_result_expired_and_a_fresh_serial_gives_stale_step() {
|
||||
let mut c = ResultCache::new();
|
||||
for step in 0..4u64 {
|
||||
let k = key(step, "Agent.Prepare");
|
||||
c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(step + 1), &body("a"));
|
||||
c.record(k, DomainRequestId::from_serial(step + 1), body("a"), reply("x"));
|
||||
}
|
||||
assert_eq!(c.retained_steps(), 2);
|
||||
// req-1 named step 0, whose record is long gone.
|
||||
match c.admit(OpClass::StepMutation, &key(0, "Agent.Prepare"), DomainRequestId::from_serial(1), &body("a")) {
|
||||
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::ResultExpired),
|
||||
other => panic!("wanted RESULT_EXPIRED, got {other:?}"),
|
||||
}
|
||||
// A serial above the highest issued is a new operation naming an old step.
|
||||
match c.admit(OpClass::StepMutation, &key(0, "Agent.Prepare"), DomainRequestId::from_serial(99), &body("a")) {
|
||||
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::StaleStep),
|
||||
other => panic!("wanted STALE_STEP, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lifecycle_replies_are_bounded_and_released_by_acknowledge() {
|
||||
let mut c = ResultCache::new();
|
||||
for serial in 1..=MAX_UNACKNOWLEDGED as u64 {
|
||||
assert!(matches!(
|
||||
c.admit(OpClass::Lifecycle, &key(0, "Agent.Initialize"), DomainRequestId::from_serial(serial), &body("a")),
|
||||
Admission::Execute
|
||||
));
|
||||
c.record_lifecycle(DomainRequestId::from_serial(serial), body("a"), reply("init"));
|
||||
}
|
||||
match c.admit(OpClass::Lifecycle, &key(0, "Agent.Initialize"), DomainRequestId::from_serial(99), &body("a")) {
|
||||
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::Busy),
|
||||
other => panic!("wanted BUSY, got {other:?}"),
|
||||
}
|
||||
let dropped = c.acknowledge(&[
|
||||
DomainRequestId::from_serial(1),
|
||||
DomainRequestId::from_serial(404),
|
||||
]);
|
||||
assert_eq!(dropped, vec![DomainRequestId::from_serial(1)]);
|
||||
assert_eq!(c.unacknowledged(), MAX_UNACKNOWLEDGED - 1);
|
||||
// The acknowledged serial cannot come back.
|
||||
match c.admit(OpClass::Lifecycle, &key(0, "Agent.Initialize"), DomainRequestId::from_serial(1), &body("a")) {
|
||||
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::ResultExpired),
|
||||
other => panic!("wanted RESULT_EXPIRED, got {other:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
454
services/flysim/crates/fly-session/src/environment.rs
Normal file
454
services/flysim/crates/fly-session/src/environment.rs
Normal file
|
|
@ -0,0 +1,454 @@
|
|||
//! The counter arena: one environment worker with no emulator behind it.
|
||||
//!
|
||||
//! It holds a signed counter, applies one complete control batch per Advance, advances exactly
|
||||
//! one interval, and returns boundary `k+1` with its world time advanced by `stepDuration`. It
|
||||
//! never advances while waiting for the next request, and it does not free-run during agent
|
||||
//! initialization.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
use std::io::Write;
|
||||
|
||||
use crate::task::{controller_schema_ref, inspection, inspection_schema};
|
||||
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
|
||||
// glob keeps the contract's own names in sight instead of restating them.
|
||||
use crate::types::*;
|
||||
use crate::worker::{BoxFuture, HandlerCtx, HandlerReply, StatusCell, WorkerEndpoint};
|
||||
|
||||
/// The arena's view: a 4x4 RGBA8 tile whose bytes carry the counter.
|
||||
pub const VIEW_WIDTH: u64 = 4;
|
||||
pub const VIEW_HEIGHT: u64 = 4;
|
||||
|
||||
/// Deliberate faults a test can ask the environment for.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct EnvironmentFaults {
|
||||
/// Hold `Environment.Advance` open for this long, after the world already moved.
|
||||
pub advance_delay_ms: u64,
|
||||
/// Drop the required sensory view from the result at this boundary, so the coordinator
|
||||
/// meets a world that advanced with no usable sensory data.
|
||||
pub omit_view_at_boundary: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct EnvironmentConfig {
|
||||
pub session_id: Id,
|
||||
pub worker_id: Id,
|
||||
pub incarnation_id: Id,
|
||||
/// The world's fixed reduced step duration. 60 Hz is `1/60` s.
|
||||
pub step_duration: RationalNs,
|
||||
pub ports: Vec<Id>,
|
||||
pub faults: EnvironmentFaults,
|
||||
}
|
||||
|
||||
/// The counter environment endpoint.
|
||||
pub struct CounterEnvironment {
|
||||
config: EnvironmentConfig,
|
||||
status: StatusCell,
|
||||
epoch: Option<Id>,
|
||||
episode_id: Option<Id>,
|
||||
descriptor: Option<EnvironmentDescriptor>,
|
||||
bindings: Vec<PortBinding>,
|
||||
boundary: u64,
|
||||
counter: i64,
|
||||
world_time: RationalNs,
|
||||
advances: u64,
|
||||
batches: BTreeSet<Id>,
|
||||
}
|
||||
|
||||
impl CounterEnvironment {
|
||||
pub fn new(config: EnvironmentConfig) -> CounterEnvironment {
|
||||
CounterEnvironment {
|
||||
status: StatusCell::new(),
|
||||
epoch: None,
|
||||
episode_id: None,
|
||||
descriptor: None,
|
||||
bindings: Vec::new(),
|
||||
boundary: 0,
|
||||
counter: 0,
|
||||
world_time: RationalNs::ZERO,
|
||||
advances: 0,
|
||||
batches: BTreeSet::new(),
|
||||
config,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn status(&self) -> StatusCell {
|
||||
self.status.clone()
|
||||
}
|
||||
|
||||
/// How many intervals this world has advanced. One complete batch advances it once.
|
||||
pub fn advances(&self) -> u64 {
|
||||
self.advances
|
||||
}
|
||||
|
||||
pub fn counter(&self) -> i64 {
|
||||
self.counter
|
||||
}
|
||||
|
||||
pub fn boundary(&self) -> u64 {
|
||||
self.boundary
|
||||
}
|
||||
|
||||
/// The controller every port of this arena declares: two buttons and one bipolar axis.
|
||||
pub fn controller_schema() -> ControllerSchema {
|
||||
ControllerSchema {
|
||||
schema: controller_schema_ref(),
|
||||
buttons: vec![id("inc"), id("dec")],
|
||||
axes: vec![AxisSchema {
|
||||
id: id("bias"),
|
||||
range: AxisRange::Bipolar,
|
||||
neutral: 0.0,
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
pub fn view_descriptor() -> ViewDescriptor {
|
||||
ViewDescriptor {
|
||||
view_id: id("arena"),
|
||||
width: VIEW_WIDTH,
|
||||
height: VIEW_HEIGHT,
|
||||
row_stride: VIEW_WIDTH * 4,
|
||||
pixel_aspect_numerator: 1,
|
||||
pixel_aspect_denominator: 1,
|
||||
observation_delay_steps: 0,
|
||||
}
|
||||
}
|
||||
|
||||
fn build_descriptor(&self) -> DomainResult<EnvironmentDescriptor> {
|
||||
let descriptor = EnvironmentDescriptor {
|
||||
backend_digest: digest_of_bytes(b"counter-arena-backend-v1"),
|
||||
content_digest: digest_of_bytes(b"counter-arena-content-v1"),
|
||||
configuration_digest: digest_of_bytes(
|
||||
format!(
|
||||
"counter-arena-config-v1\nstep={}/{}\nports={}\n",
|
||||
self.config.step_duration.numerator,
|
||||
self.config.step_duration.denominator,
|
||||
self.config.ports.len()
|
||||
)
|
||||
.as_bytes(),
|
||||
),
|
||||
step_duration: self.config.step_duration,
|
||||
ports: self
|
||||
.config
|
||||
.ports
|
||||
.iter()
|
||||
.map(|port_id| PortDescriptor {
|
||||
port_id: port_id.clone(),
|
||||
controls: CounterEnvironment::controller_schema(),
|
||||
})
|
||||
.collect(),
|
||||
inspection_schema: inspection_schema(),
|
||||
views: vec![CounterEnvironment::view_descriptor()],
|
||||
audio: Vec::new(),
|
||||
recovery: Recovery::ExactCheckpoint,
|
||||
determinism: Determinism::FixedBuild,
|
||||
};
|
||||
descriptor.validate().map_err(DomainError::invalid)?;
|
||||
Ok(descriptor)
|
||||
}
|
||||
|
||||
/// Seals one immutable native frame for the current counter and returns the handle.
|
||||
async fn render(
|
||||
&self,
|
||||
ctx: &HandlerCtx<'_>,
|
||||
) -> DomainResult<(ViewRef, flybus::Artifact)> {
|
||||
let descriptor = CounterEnvironment::view_descriptor();
|
||||
let len = descriptor.byte_length();
|
||||
let mut writer = ctx
|
||||
.client
|
||||
.artifacts()
|
||||
.allocate(len, "image/x-rgba")
|
||||
.await
|
||||
.map_err(|e| {
|
||||
DomainError::new(
|
||||
ErrorCode::BackendFailure,
|
||||
format!("frame allocation failed: {}", e.message),
|
||||
MutationCertainty::Applied,
|
||||
)
|
||||
})?;
|
||||
// Every pixel carries the counter's low byte, so an agent reading the frame reads the
|
||||
// world rather than a constant.
|
||||
let byte = (self.counter & 0xff) as u8;
|
||||
writer
|
||||
.write_all(&vec![byte; len as usize])
|
||||
.map_err(|e| {
|
||||
DomainError::new(
|
||||
ErrorCode::BackendFailure,
|
||||
format!("frame write failed: {e}"),
|
||||
MutationCertainty::Applied,
|
||||
)
|
||||
})?;
|
||||
let artifact = writer.seal().await.map_err(|e| {
|
||||
DomainError::new(
|
||||
ErrorCode::BackendFailure,
|
||||
format!("frame seal failed: {}", e.message),
|
||||
MutationCertainty::Applied,
|
||||
)
|
||||
})?;
|
||||
let view = ViewRef {
|
||||
view_id: descriptor.view_id.clone(),
|
||||
produced_step: descriptor.required_produced_step(self.boundary),
|
||||
pixels: artifact.reference().clone(),
|
||||
};
|
||||
Ok((view, artifact))
|
||||
}
|
||||
|
||||
async fn observation(
|
||||
&self,
|
||||
ctx: &HandlerCtx<'_>,
|
||||
) -> DomainResult<(WorldObservation, Vec<(String, flybus::Artifact)>)> {
|
||||
let omit = self.config.faults.omit_view_at_boundary == Some(self.boundary);
|
||||
let (views, attachments) = if omit {
|
||||
(Vec::new(), Vec::new())
|
||||
} else {
|
||||
let (view, artifact) = self.render(ctx).await?;
|
||||
let name = format!("view.{}", view.view_id);
|
||||
(vec![view], vec![(name, artifact)])
|
||||
};
|
||||
let observation = WorldObservation {
|
||||
boundary: self.boundary,
|
||||
world_time: self.world_time,
|
||||
engine_frame: Some(self.boundary.to_string()),
|
||||
sensory_views: views.clone(),
|
||||
inspection: inspection(self.counter, self.boundary),
|
||||
// The same immutable object serves the broadcast view; nothing is rendered twice.
|
||||
broadcast_views: views,
|
||||
audio: Vec::new(),
|
||||
};
|
||||
Ok((observation, attachments))
|
||||
}
|
||||
|
||||
async fn initialize(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
|
||||
let scope = ctx.scope()?.clone();
|
||||
if self.descriptor.is_some() {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::InvalidPhase,
|
||||
"this environment is already initialized",
|
||||
));
|
||||
}
|
||||
if scope.session_id != self.config.session_id {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"this environment belongs to another session",
|
||||
));
|
||||
}
|
||||
if scope.step != 0 {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::FutureStep,
|
||||
"Environment.Initialize uses the new epoch at step 0",
|
||||
));
|
||||
}
|
||||
let params: EnvironmentInitializeParams = ctx.params()?;
|
||||
if params.port_bindings.len() > MAX_PORTS {
|
||||
return Err(DomainError::invalid("at most 4 ports in the first composition"));
|
||||
}
|
||||
let mut seen = BTreeSet::new();
|
||||
for (port_id, _agent_id) in ¶ms.port_bindings {
|
||||
if !self.config.ports.contains(port_id) {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
format!("port {port_id} is not a port of this arena"),
|
||||
));
|
||||
}
|
||||
if !seen.insert(port_id.clone()) {
|
||||
return Err(DomainError::invalid(format!("port {port_id} is bound twice")));
|
||||
}
|
||||
}
|
||||
let descriptor = self.build_descriptor()?;
|
||||
self.epoch = Some(scope.epoch.clone());
|
||||
self.episode_id = Some(params.episode_id.clone());
|
||||
self.bindings = params
|
||||
.port_bindings
|
||||
.iter()
|
||||
.map(|(port_id, agent_id)| PortBinding {
|
||||
port_id: port_id.clone(),
|
||||
agent_id: agent_id.clone(),
|
||||
})
|
||||
.collect();
|
||||
self.boundary = 0;
|
||||
self.counter = 0;
|
||||
self.world_time = RationalNs::ZERO;
|
||||
self.batches.clear();
|
||||
self.descriptor = Some(descriptor.clone());
|
||||
// The world is stopped when O[0] goes out and cannot free-run while the brains boot.
|
||||
self.status.set_state(WorkerState::Ready);
|
||||
self.status.set_scope(Some(scope.clone()));
|
||||
self.status.progress(1);
|
||||
|
||||
let (observation, attachments) = self.observation(ctx).await?;
|
||||
let result = EnvironmentInitializeResult { descriptor, observation };
|
||||
let mut reply = HandlerReply::from(&result);
|
||||
reply.artifacts = attachments;
|
||||
Ok(reply)
|
||||
}
|
||||
|
||||
async fn advance(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
|
||||
let scope = ctx.scope()?.clone();
|
||||
let Some(descriptor) = self.descriptor.clone() else {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::InvalidPhase,
|
||||
"this environment is uninitialized",
|
||||
));
|
||||
};
|
||||
if scope.session_id != self.config.session_id {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"this environment belongs to another session",
|
||||
));
|
||||
}
|
||||
match &self.epoch {
|
||||
Some(epoch) if *epoch == scope.epoch => {}
|
||||
_ => {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::StaleEpoch,
|
||||
"this scope names an epoch this environment has left",
|
||||
));
|
||||
}
|
||||
}
|
||||
if scope.step != self.boundary {
|
||||
return Err(DomainError::before(
|
||||
if scope.step < self.boundary {
|
||||
ErrorCode::StaleStep
|
||||
} else {
|
||||
ErrorCode::FutureStep
|
||||
},
|
||||
"Environment.Advance must name the boundary the world is at",
|
||||
));
|
||||
}
|
||||
let params: AdvanceParams = ctx.params()?;
|
||||
// Batch ids are unique within an epoch; reusing one for a different request or step is
|
||||
// a conflict, not a second world mutation.
|
||||
if self.batches.contains(¶ms.batch_id) {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::Conflict,
|
||||
format!("batch {} was already applied in this epoch", params.batch_id),
|
||||
));
|
||||
}
|
||||
self.check_batch(&descriptor, ¶ms.controls)?;
|
||||
let digest = controls_digest(¶ms.controls);
|
||||
|
||||
let applied_from = self.boundary;
|
||||
// Apply the complete batch to its interval and advance exactly one framework step.
|
||||
let mut delta = 0i64;
|
||||
for control in ¶ms.controls {
|
||||
delta += crate::task::CounterTask::delta_of(control);
|
||||
}
|
||||
self.counter += delta;
|
||||
self.boundary += 1;
|
||||
self.world_time = self
|
||||
.world_time
|
||||
.checked_add(&descriptor.step_duration)
|
||||
.map_err(|e| DomainError::new(ErrorCode::Internal, e, MutationCertainty::Applied))?;
|
||||
self.advances += 1;
|
||||
self.batches.insert(params.batch_id.clone());
|
||||
self.status.set_batch(params.batch_id.clone());
|
||||
self.status.set_scope(Some(scope_at(
|
||||
&scope.session_id,
|
||||
&scope.epoch,
|
||||
self.boundary,
|
||||
)));
|
||||
self.status.progress(1);
|
||||
|
||||
if self.config.faults.advance_delay_ms > 0 {
|
||||
tokio::time::sleep(std::time::Duration::from_millis(
|
||||
self.config.faults.advance_delay_ms,
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
let (observation, attachments) = self.observation(ctx).await?;
|
||||
// The record of batch id, result and next boundary exists before the acknowledgment.
|
||||
let result = StepResult {
|
||||
batch_id: params.batch_id,
|
||||
applied_from_step: applied_from,
|
||||
next_step: self.boundary,
|
||||
applied_controls_digest: digest,
|
||||
observation,
|
||||
};
|
||||
let mut reply = HandlerReply::from(&result);
|
||||
reply.artifacts = attachments;
|
||||
Ok(reply)
|
||||
}
|
||||
|
||||
/// Every active port must appear exactly once, with every declared button and axis in
|
||||
/// descriptor order. Uncontrolled ports were configured neutral before the epoch.
|
||||
fn check_batch(
|
||||
&self,
|
||||
descriptor: &EnvironmentDescriptor,
|
||||
controls: &[PortControl],
|
||||
) -> DomainResult<()> {
|
||||
if controls.len() != descriptor.ports.len() {
|
||||
return Err(DomainError::invalid(format!(
|
||||
"the batch has {} port controls; the descriptor declares {}",
|
||||
controls.len(),
|
||||
descriptor.ports.len()
|
||||
)));
|
||||
}
|
||||
for (declared, given) in descriptor.ports.iter().zip(controls) {
|
||||
if declared.port_id != given.port_id {
|
||||
return Err(DomainError::invalid(format!(
|
||||
"port {} is out of descriptor order; expected {}",
|
||||
given.port_id, declared.port_id
|
||||
)));
|
||||
}
|
||||
declared.controls.check(given).map_err(DomainError::invalid)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl WorkerEndpoint for CounterEnvironment {
|
||||
fn worker_id(&self) -> Id {
|
||||
self.config.worker_id.clone()
|
||||
}
|
||||
|
||||
fn incarnation_id(&self) -> Id {
|
||||
self.config.incarnation_id.clone()
|
||||
}
|
||||
|
||||
fn session_id(&self) -> Id {
|
||||
self.config.session_id.clone()
|
||||
}
|
||||
|
||||
fn role(&self) -> Role {
|
||||
Role::Environment
|
||||
}
|
||||
|
||||
fn capabilities(&self) -> Vec<Id> {
|
||||
vec![
|
||||
id("world-step-v1"),
|
||||
id("pixel-observation-v1"),
|
||||
id("checkpoint-v1"),
|
||||
]
|
||||
}
|
||||
|
||||
fn status_cell(&self) -> StatusCell {
|
||||
self.status.clone()
|
||||
}
|
||||
|
||||
fn methods(&self) -> Vec<&'static str> {
|
||||
vec!["Environment.Initialize", "Environment.Advance"]
|
||||
}
|
||||
|
||||
fn handle<'a>(&'a mut self, ctx: HandlerCtx<'a>) -> BoxFuture<'a, DomainResult<HandlerReply>> {
|
||||
Box::pin(async move {
|
||||
match ctx.method {
|
||||
"Environment.Initialize" => self.initialize(&ctx).await,
|
||||
"Environment.Advance" => self.advance(&ctx).await,
|
||||
other => Err(DomainError::before(
|
||||
ErrorCode::Unsupported,
|
||||
format!("{other} is not an environment method"),
|
||||
)),
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// A synthetic backend/task configuration asset for the arena.
|
||||
pub fn synthetic_asset(asset_id: &str, body: &str) -> AssetRef {
|
||||
AssetRef {
|
||||
id: id(asset_id),
|
||||
digest: digest_of_bytes(body.as_bytes()),
|
||||
byte_length: body.len() as u64,
|
||||
format: id("fly-config-v1"),
|
||||
}
|
||||
}
|
||||
381
services/flysim/crates/fly-session/src/harness.rs
Normal file
381
services/flysim/crates/fly-session/src/harness.rs
Normal file
|
|
@ -0,0 +1,381 @@
|
|||
//! The runnable synthetic composition: one router, two fake agents, one counter arena and one
|
||||
//! coordinator, over either transport.
|
||||
//!
|
||||
//! All participants use router semantics even when colocated, so the in-memory and
|
||||
//! Unix-socket runs exercise the same code. The caller owns the store root directory, which
|
||||
//! keeps this module free of a temporary-directory dependency.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Mutex;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
|
||||
use flybus::{
|
||||
Client, ClientConfig, Grants, Pattern, Policy, Router, RouterConfig, ServiceConfig, Transport,
|
||||
UnixListenerHandle,
|
||||
};
|
||||
|
||||
use crate::agent::{AgentConfig, AgentFaults, FakeAgentWorker, synthetic_profile};
|
||||
use crate::coordinator::{AgentSlot, Coordinator};
|
||||
use crate::environment::{CounterEnvironment, EnvironmentConfig, EnvironmentFaults};
|
||||
use crate::rpc::WorkerRef;
|
||||
use crate::task::{ActionExecutor, CounterTask, IdentityExecutor, Terminal};
|
||||
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
|
||||
// glob keeps the contract's own names in sight instead of restating them.
|
||||
use crate::types::*;
|
||||
use crate::worker::{StatusCell, WorkerHandle, serve};
|
||||
|
||||
/// Which transport the session runs over. Both must produce the same behaviour.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum Via {
|
||||
Memory,
|
||||
Unix,
|
||||
}
|
||||
|
||||
/// One agent in the composition.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct AgentSpec {
|
||||
pub agent_id: Id,
|
||||
pub port_id: Id,
|
||||
/// An explicit seed. The first synthetic composition supports hand-selected seeds; the
|
||||
/// derivation algorithm is specified before the real agent slice.
|
||||
pub seed: i32,
|
||||
pub faults: AgentFaults,
|
||||
}
|
||||
|
||||
/// The composition the harness builds.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct HarnessConfig {
|
||||
pub session_id: Id,
|
||||
pub epoch: Id,
|
||||
pub episode_id: Id,
|
||||
pub agents: Vec<AgentSpec>,
|
||||
/// The world's cadence. 60 Hz with a 1 ms tick is the `step-v1` section 5 example.
|
||||
pub step_hz: u64,
|
||||
pub tick_ms: u64,
|
||||
pub warmup_ticks: u64,
|
||||
pub terminal: Terminal,
|
||||
pub environment_faults: EnvironmentFaults,
|
||||
}
|
||||
|
||||
impl Default for HarnessConfig {
|
||||
fn default() -> HarnessConfig {
|
||||
HarnessConfig {
|
||||
session_id: id("demo"),
|
||||
epoch: id("e1"),
|
||||
episode_id: id("ep1"),
|
||||
agents: vec![
|
||||
AgentSpec {
|
||||
agent_id: id("fly-a"),
|
||||
port_id: id("p1"),
|
||||
seed: 7,
|
||||
faults: AgentFaults::default(),
|
||||
},
|
||||
AgentSpec {
|
||||
agent_id: id("fly-b"),
|
||||
port_id: id("p2"),
|
||||
seed: 11,
|
||||
faults: AgentFaults::default(),
|
||||
},
|
||||
],
|
||||
step_hz: 60,
|
||||
tick_ms: 1,
|
||||
warmup_ticks: 10,
|
||||
terminal: Terminal::Never,
|
||||
environment_faults: EnvironmentFaults::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const ENV_SERVICE: &str = "env.arena";
|
||||
const ENV_CLIENT: &str = "environment";
|
||||
const ENV_WORKER: &str = "arena";
|
||||
|
||||
fn agent_service(agent_id: &Id) -> String {
|
||||
format!("agent.{agent_id}")
|
||||
}
|
||||
|
||||
fn agent_client(agent_id: &Id) -> String {
|
||||
format!("worker-{agent_id}")
|
||||
}
|
||||
|
||||
fn grants(f: impl FnOnce(&mut Grants)) -> Grants {
|
||||
let mut g = Grants::default();
|
||||
f(&mut g);
|
||||
g
|
||||
}
|
||||
|
||||
/// Makes a connection for one launcher-bound participant, over the chosen transport.
|
||||
struct Connector {
|
||||
router: Router,
|
||||
via: Via,
|
||||
store_root: PathBuf,
|
||||
sockets: PathBuf,
|
||||
next_socket: AtomicU64,
|
||||
listeners: Mutex<Vec<UnixListenerHandle>>,
|
||||
}
|
||||
|
||||
impl Connector {
|
||||
async fn client(&self, id: &str) -> Result<Client, flybus::BusError> {
|
||||
let transport = match self.via {
|
||||
Via::Memory => self.router.connect_in_memory_as(id),
|
||||
Via::Unix => {
|
||||
let n = self.next_socket.fetch_add(1, Ordering::Relaxed);
|
||||
let path = self.sockets.join(format!("{id}-{n}.sock"));
|
||||
let listener = self.router.listen_unix_as(&path, id).await.map_err(|e| {
|
||||
flybus::BusError::new(flybus::ErrorCode::RouterLost, format!("listen: {e}"))
|
||||
})?;
|
||||
let transport = Transport::unix(&path).await.map_err(|e| {
|
||||
flybus::BusError::new(flybus::ErrorCode::RouterLost, format!("connect: {e}"))
|
||||
})?;
|
||||
self.listeners.lock().expect("not poisoned").push(listener);
|
||||
transport
|
||||
}
|
||||
};
|
||||
Client::connect(transport, ClientConfig::new(id, &self.store_root)).await
|
||||
}
|
||||
}
|
||||
|
||||
/// What a restarted worker looks like from the outside: a new registration and a new
|
||||
/// incarnation, both different from the ones the coordinator pinned.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Restarted {
|
||||
pub service: String,
|
||||
pub service_incarnation: String,
|
||||
pub incarnation_id: Id,
|
||||
}
|
||||
|
||||
/// A running synthetic session.
|
||||
pub struct SessionHarness {
|
||||
pub coordinator: Coordinator,
|
||||
pub environment: WorkerHandle,
|
||||
pub agents: BTreeMap<Id, WorkerHandle>,
|
||||
pub config: HarnessConfig,
|
||||
pub via: Via,
|
||||
connector: Connector,
|
||||
observers: Mutex<Vec<Client>>,
|
||||
}
|
||||
|
||||
impl SessionHarness {
|
||||
/// Builds the router, the workers and the coordinator. Nothing has stepped yet.
|
||||
pub async fn start(
|
||||
via: Via,
|
||||
root: &Path,
|
||||
config: HarnessConfig,
|
||||
) -> Result<SessionHarness, flybus::BusError> {
|
||||
let store_root = root.join("store");
|
||||
let sockets = root.join("sockets");
|
||||
std::fs::create_dir_all(&sockets).expect("the caller owns a writable directory");
|
||||
|
||||
let mut policy = Policy::closed()
|
||||
.client(
|
||||
"coordinator",
|
||||
grants(|g| {
|
||||
g.call = vec![Pattern::prefix("agent."), Pattern::prefix("env.")];
|
||||
g.publish = vec![Pattern::prefix("session.")];
|
||||
g.manage_topics = vec![Pattern::prefix("session.")];
|
||||
}),
|
||||
)
|
||||
.client(ENV_CLIENT, grants(|g| g.register = vec![Pattern::exact(ENV_SERVICE)]))
|
||||
.client("observer", grants(|g| g.subscribe = vec![Pattern::prefix("session.")]));
|
||||
for spec in &config.agents {
|
||||
let service = agent_service(&spec.agent_id);
|
||||
policy = policy.client(
|
||||
&agent_client(&spec.agent_id),
|
||||
grants(|g| g.register = vec![Pattern::exact(&service)]),
|
||||
);
|
||||
// A replacement worker connects under its own client id, so a restart is visibly a
|
||||
// new participant rather than a silent reattachment to the active epoch.
|
||||
policy = policy.client(
|
||||
&format!("{}-r2", agent_client(&spec.agent_id)),
|
||||
grants(|g| g.register = vec![Pattern::exact(&service)]),
|
||||
);
|
||||
}
|
||||
let mut router_config = RouterConfig::new(&store_root);
|
||||
router_config.policy = policy;
|
||||
let router = Router::new(router_config).map_err(|e| {
|
||||
flybus::BusError::new(flybus::ErrorCode::StoreFailure, format!("router: {e}"))
|
||||
})?;
|
||||
let connector = Connector {
|
||||
router,
|
||||
via,
|
||||
store_root,
|
||||
sockets,
|
||||
next_socket: AtomicU64::new(0),
|
||||
listeners: Mutex::new(Vec::new()),
|
||||
};
|
||||
|
||||
let step_duration = hz(config.step_hz).expect("a positive cadence");
|
||||
let tick_duration = millis(config.tick_ms).expect("a positive tick");
|
||||
|
||||
// The environment first: it owns the world and the descriptor.
|
||||
let env_client = connector.client(ENV_CLIENT).await?;
|
||||
let env_service = env_client.register(ENV_SERVICE, ServiceConfig::default()).await?;
|
||||
let env_incarnation = env_service.incarnation().to_owned();
|
||||
let environment = serve(
|
||||
env_client,
|
||||
env_service,
|
||||
CounterEnvironment::new(EnvironmentConfig {
|
||||
session_id: config.session_id.clone(),
|
||||
worker_id: id(ENV_WORKER),
|
||||
incarnation_id: id("arena-inc-1"),
|
||||
step_duration,
|
||||
ports: config.agents.iter().map(|a| a.port_id.clone()).collect(),
|
||||
faults: config.environment_faults.clone(),
|
||||
}),
|
||||
);
|
||||
|
||||
let mut slots = Vec::new();
|
||||
let mut agents = BTreeMap::new();
|
||||
for spec in &config.agents {
|
||||
let service_name = agent_service(&spec.agent_id);
|
||||
let client = connector.client(&agent_client(&spec.agent_id)).await?;
|
||||
let service = client.register(&service_name, ServiceConfig::default()).await?;
|
||||
let incarnation = service.incarnation().to_owned();
|
||||
let handle = serve(
|
||||
client,
|
||||
service,
|
||||
FakeAgentWorker::new(AgentConfig {
|
||||
session_id: config.session_id.clone(),
|
||||
agent_id: spec.agent_id.clone(),
|
||||
incarnation_id: parse_id(&format!("{}-inc-1", spec.agent_id))
|
||||
.expect("an agent id plus a suffix is an Id"),
|
||||
tick_duration,
|
||||
warmup_ticks: config.warmup_ticks,
|
||||
faults: spec.faults.clone(),
|
||||
}),
|
||||
);
|
||||
slots.push(AgentSlot::new(
|
||||
WorkerRef::new(&service_name, &incarnation, &spec.agent_id),
|
||||
spec.agent_id.clone(),
|
||||
spec.port_id.clone(),
|
||||
synthetic_profile(&spec.agent_id, &tick_duration, config.warmup_ticks),
|
||||
spec.seed,
|
||||
));
|
||||
agents.insert(spec.agent_id.clone(), handle);
|
||||
}
|
||||
|
||||
let coordinator_client = connector.client("coordinator").await?;
|
||||
let executors: BTreeMap<Id, Box<dyn ActionExecutor>> = config
|
||||
.agents
|
||||
.iter()
|
||||
.map(|spec| {
|
||||
(spec.agent_id.clone(), Box::new(IdentityExecutor) as Box<dyn ActionExecutor>)
|
||||
})
|
||||
.collect();
|
||||
let coordinator = Coordinator::new(
|
||||
coordinator_client,
|
||||
config.session_id.clone(),
|
||||
config.epoch.clone(),
|
||||
config.episode_id.clone(),
|
||||
WorkerRef::new(ENV_SERVICE, &env_incarnation, &id(ENV_WORKER)),
|
||||
slots,
|
||||
Box::new(CounterTask::new(&config.epoch, config.terminal)),
|
||||
executors,
|
||||
);
|
||||
|
||||
Ok(SessionHarness {
|
||||
coordinator,
|
||||
environment,
|
||||
agents,
|
||||
config,
|
||||
via,
|
||||
connector,
|
||||
observers: Mutex::new(Vec::new()),
|
||||
})
|
||||
}
|
||||
|
||||
pub fn router(&self) -> &Router {
|
||||
&self.connector.router
|
||||
}
|
||||
|
||||
/// A client for `id`, connected the same way every participant is.
|
||||
pub async fn client(&self, id: &str) -> Result<Client, flybus::BusError> {
|
||||
self.connector.client(id).await
|
||||
}
|
||||
|
||||
/// An extra subscriber, for a test that watches the published boundaries.
|
||||
pub async fn observer(&self) -> Result<Client, flybus::BusError> {
|
||||
let client = self.connector.client("observer").await?;
|
||||
self.observers.lock().expect("not poisoned").push(client.clone());
|
||||
Ok(client)
|
||||
}
|
||||
|
||||
/// Replaces one agent's worker with a fresh incarnation, as a restore would.
|
||||
///
|
||||
/// The coordinator still pins the old registration, so its next call to that agent fails
|
||||
/// rather than silently reaching another brain.
|
||||
pub async fn restart_agent(&mut self, agent_id: &Id) -> Result<Restarted, flybus::BusError> {
|
||||
let tick_duration = millis(self.config.tick_ms).expect("a positive tick");
|
||||
if let Some(old) = self.agents.remove(agent_id) {
|
||||
old.stop().await;
|
||||
}
|
||||
let service_name = agent_service(agent_id);
|
||||
let client = self.connector.client(&format!("{}-r2", agent_client(agent_id))).await?;
|
||||
let service = loop {
|
||||
match client.register(&service_name, ServiceConfig::default()).await {
|
||||
Ok(service) => break service,
|
||||
Err(e) if e.code == flybus::ErrorCode::Conflict => {
|
||||
// The old registration is released when its connection finishes closing.
|
||||
tokio::time::sleep(std::time::Duration::from_millis(5)).await;
|
||||
}
|
||||
Err(e) => return Err(e),
|
||||
}
|
||||
};
|
||||
let spec = self
|
||||
.config
|
||||
.agents
|
||||
.iter()
|
||||
.find(|spec| spec.agent_id == *agent_id)
|
||||
.expect("a configured agent")
|
||||
.clone();
|
||||
let incarnation_id =
|
||||
parse_id(&format!("{agent_id}-inc-2")).expect("an agent id plus a suffix is an Id");
|
||||
let restarted = Restarted {
|
||||
service: service_name,
|
||||
service_incarnation: service.incarnation().to_owned(),
|
||||
incarnation_id: incarnation_id.clone(),
|
||||
};
|
||||
let handle = serve(
|
||||
client,
|
||||
service,
|
||||
FakeAgentWorker::new(AgentConfig {
|
||||
session_id: self.config.session_id.clone(),
|
||||
agent_id: agent_id.clone(),
|
||||
incarnation_id,
|
||||
tick_duration,
|
||||
warmup_ticks: self.config.warmup_ticks,
|
||||
faults: spec.faults,
|
||||
}),
|
||||
);
|
||||
self.agents.insert(agent_id.clone(), handle);
|
||||
Ok(restarted)
|
||||
}
|
||||
|
||||
/// The agent worker's progress counter, which is its fake model's mutation count.
|
||||
pub fn agent_mutations(&self, agent_id: &Id) -> u64 {
|
||||
self.agents.get(agent_id).map(WorkerHandle::progress_counter).unwrap_or_default()
|
||||
}
|
||||
|
||||
pub fn environment_mutations(&self) -> u64 {
|
||||
self.environment.progress_counter()
|
||||
}
|
||||
|
||||
pub fn agent_status(&self, agent_id: &Id) -> Option<StatusCell> {
|
||||
self.agents.get(agent_id).map(|handle| handle.status.clone())
|
||||
}
|
||||
|
||||
/// Stops every worker and closes the router.
|
||||
pub async fn shutdown(self) {
|
||||
let SessionHarness { coordinator, environment, agents, connector, observers, .. } = self;
|
||||
drop(coordinator);
|
||||
environment.stop().await;
|
||||
for (_, handle) in agents {
|
||||
handle.stop().await;
|
||||
}
|
||||
for observer in observers.into_inner().expect("not poisoned") {
|
||||
observer.close().await;
|
||||
}
|
||||
connector.router.shutdown();
|
||||
}
|
||||
}
|
||||
41
services/flysim/crates/fly-session/src/lib.rs
Normal file
41
services/flysim/crates/fly-session/src/lib.rs
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
//! `fly-session`: the lockstep session coordinator and a synthetic composition over Flybus.
|
||||
//!
|
||||
//! This crate is the SESSION-01 slice of the session-framework implementation guide: the
|
||||
//! sequential transaction of [`step-v1`], driven over the [`flybus`] router, with small fake
|
||||
//! workers standing in for a brain and an emulator.
|
||||
//!
|
||||
//! ```text
|
||||
//! Coordinator ── Agent.Prepare ──> agent workers (fake model + fixed readout stub)
|
||||
//! ── Environment.Advance ──> environment (a counter arena, no emulator)
|
||||
//! ── task.evaluate_transition (once)
|
||||
//! ── Agent.Commit ──> agent workers
|
||||
//! ── committed snapshot ──> session.<id>.snapshots
|
||||
//! ```
|
||||
//!
|
||||
//! Every arrow is a Flybus RPC to an incarnation-pinned service, with domain request ids and
|
||||
//! the result caches of `ipc-v1` section 5 in front of every mutation. Nothing here contains a
|
||||
//! public controller API, an implicit best-effort retry, a real emulator or a real brain.
|
||||
//!
|
||||
//! The domain types come from the CONTRACT-01 crate [`fly_session_types`]; [`types`] is a
|
||||
//! facade over it plus the few session-side additions a coordinator needs.
|
||||
//!
|
||||
//! [`step-v1`]: https://example.invalid/step-v1
|
||||
|
||||
pub mod agent;
|
||||
pub mod clock;
|
||||
pub mod coordinator;
|
||||
pub mod dedup;
|
||||
pub mod environment;
|
||||
pub mod harness;
|
||||
pub mod phase;
|
||||
pub mod rpc;
|
||||
pub mod task;
|
||||
pub mod worker;
|
||||
|
||||
// CONTRACT-01 owns the domain types; `types` is a facade over its crate plus the few
|
||||
// session-side additions a coordinator needs.
|
||||
pub mod types;
|
||||
pub use fly_session_types;
|
||||
|
||||
pub use coordinator::{Coordinator, DispatchOrder, Injections, SessionFailure, StepReport};
|
||||
pub use phase::{Phase, PhaseMachine};
|
||||
223
services/flysim/crates/fly-session/src/phase.rs
Normal file
223
services/flysim/crates/fly-session/src/phase.rs
Normal file
|
|
@ -0,0 +1,223 @@
|
|||
//! The session state machine of `step-v1` section 2, as an explicit edge table.
|
||||
//!
|
||||
//! ```text
|
||||
//! Starting -> Ready(k) -> Preparing(k) -> Applying(k) -> Observing(k+1)
|
||||
//! ^ |
|
||||
//! +---------------- Ready(k+1) <- Committing(k)
|
||||
//!
|
||||
//! Ready(k) -> Paused(k) -> Ready(k)
|
||||
//! Ready(k) / Paused(k) -> Capturing(k) -> same boundary
|
||||
//! any unresolved partial failure -> Failed -> Restoring(new epoch) -> Paused(k)
|
||||
//! terminal episode -> Paused(k) -> Resetting(new epoch) -> Ready(0)
|
||||
//! ```
|
||||
//!
|
||||
//! A transition the table does not list is a bug, not a recoverable condition, so it returns
|
||||
//! INVALID_PHASE rather than being silently applied.
|
||||
|
||||
use crate::types::{DomainError, ErrorCode};
|
||||
|
||||
/// Where the session is. The number is the committed boundary the phase belongs to.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum Phase {
|
||||
Starting,
|
||||
Ready(u64),
|
||||
Preparing(u64),
|
||||
Applying(u64),
|
||||
/// `Observing(k+1)`: the world has reached `k+1` but nothing is committed yet.
|
||||
Observing(u64),
|
||||
/// `Committing(k)`: completing the transition `k -> k+1`.
|
||||
Committing(u64),
|
||||
Paused(u64),
|
||||
Capturing(u64),
|
||||
Failed,
|
||||
/// `Restoring(k)`: installing a coherent checkpoint of boundary `k` under a new epoch.
|
||||
Restoring(u64),
|
||||
/// `Resetting(k)`: leaving boundary `k` for a new epoch and episode at step 0.
|
||||
Resetting(u64),
|
||||
}
|
||||
|
||||
impl Phase {
|
||||
pub fn label(&self) -> String {
|
||||
match self {
|
||||
Phase::Starting => "Starting".to_owned(),
|
||||
Phase::Ready(k) => format!("Ready({k})"),
|
||||
Phase::Preparing(k) => format!("Preparing({k})"),
|
||||
Phase::Applying(k) => format!("Applying({k})"),
|
||||
Phase::Observing(k) => format!("Observing({k})"),
|
||||
Phase::Committing(k) => format!("Committing({k})"),
|
||||
Phase::Paused(k) => format!("Paused({k})"),
|
||||
Phase::Capturing(k) => format!("Capturing({k})"),
|
||||
Phase::Failed => "Failed".to_owned(),
|
||||
Phase::Restoring(k) => format!("Restoring({k})"),
|
||||
Phase::Resetting(k) => format!("Resetting({k})"),
|
||||
}
|
||||
}
|
||||
|
||||
/// The committed boundary, where the phase has one. `Observing(k+1)` does not: the world
|
||||
/// has moved but nothing is committed, so the committed boundary is still `k`.
|
||||
pub fn committed_boundary(&self) -> Option<u64> {
|
||||
match self {
|
||||
Phase::Ready(k) | Phase::Paused(k) | Phase::Capturing(k) => Some(*k),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Only a committed boundary is eligible for a checkpoint or a normal pause.
|
||||
pub fn is_committed_boundary(&self) -> bool {
|
||||
matches!(self, Phase::Ready(_) | Phase::Paused(_))
|
||||
}
|
||||
}
|
||||
|
||||
/// The phase, plus the edge check that guards every change to it.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct PhaseMachine {
|
||||
phase: Phase,
|
||||
/// Where a `Capturing(k)` must return to.
|
||||
capture_origin: Option<Phase>,
|
||||
}
|
||||
|
||||
impl Default for PhaseMachine {
|
||||
fn default() -> PhaseMachine {
|
||||
PhaseMachine::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl PhaseMachine {
|
||||
pub fn new() -> PhaseMachine {
|
||||
PhaseMachine { phase: Phase::Starting, capture_origin: None }
|
||||
}
|
||||
|
||||
pub fn phase(&self) -> Phase {
|
||||
self.phase
|
||||
}
|
||||
|
||||
/// True when `next` is an edge of the section 2 machine.
|
||||
pub fn allows(&self, next: Phase) -> bool {
|
||||
use Phase::*;
|
||||
// Any unresolved partial failure fails the epoch, from wherever the session was.
|
||||
if next == Failed {
|
||||
return self.phase != Failed;
|
||||
}
|
||||
match (self.phase, next) {
|
||||
(Starting, Ready(0)) => true,
|
||||
(Ready(k), Preparing(j)) => k == j,
|
||||
(Preparing(k), Applying(j)) => k == j,
|
||||
(Applying(k), Observing(j)) => j == k + 1,
|
||||
(Observing(j), Committing(k)) => j == k + 1,
|
||||
(Committing(k), Ready(j)) => j == k + 1,
|
||||
(Ready(k), Paused(j)) => k == j,
|
||||
(Paused(k), Ready(j)) => k == j,
|
||||
(Ready(k), Capturing(j)) | (Paused(k), Capturing(j)) => k == j,
|
||||
// Capturing returns to the boundary it came from, and only to that one.
|
||||
(Capturing(k), Ready(j)) => k == j && self.capture_origin == Some(Ready(k)),
|
||||
(Capturing(k), Paused(j)) => k == j && self.capture_origin == Some(Paused(k)),
|
||||
(Failed, Restoring(_)) => true,
|
||||
(Restoring(k), Paused(j)) => k == j,
|
||||
(Paused(k), Resetting(j)) => k == j,
|
||||
(Resetting(_), Ready(0)) => true,
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Applies an edge, reporting the old and new labels for the trace.
|
||||
pub fn to(&mut self, next: Phase) -> Result<(String, String), DomainError> {
|
||||
if !self.allows(next) {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::InvalidPhase,
|
||||
format!("{} cannot move to {}", self.phase.label(), next.label()),
|
||||
));
|
||||
}
|
||||
let from = self.phase.label();
|
||||
if matches!(next, Phase::Capturing(_)) {
|
||||
self.capture_origin = Some(self.phase);
|
||||
} else if !matches!(self.phase, Phase::Capturing(_)) {
|
||||
self.capture_origin = None;
|
||||
}
|
||||
self.phase = next;
|
||||
Ok((from, next.label()))
|
||||
}
|
||||
|
||||
/// Fails the epoch from wherever the session was.
|
||||
pub fn fail(&mut self) -> (String, String) {
|
||||
let from = self.phase.label();
|
||||
self.phase = Phase::Failed;
|
||||
self.capture_origin = None;
|
||||
(from, self.phase.label())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn the_happy_path_walks_the_section_2_diagram() {
|
||||
let mut m = PhaseMachine::new();
|
||||
m.to(Phase::Ready(0)).unwrap();
|
||||
m.to(Phase::Preparing(0)).unwrap();
|
||||
m.to(Phase::Applying(0)).unwrap();
|
||||
m.to(Phase::Observing(1)).unwrap();
|
||||
m.to(Phase::Committing(0)).unwrap();
|
||||
m.to(Phase::Ready(1)).unwrap();
|
||||
assert_eq!(m.phase(), Phase::Ready(1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_step_cannot_be_skipped_or_rewound() {
|
||||
let mut m = PhaseMachine::new();
|
||||
m.to(Phase::Ready(0)).unwrap();
|
||||
assert!(m.to(Phase::Preparing(1)).is_err());
|
||||
m.to(Phase::Preparing(0)).unwrap();
|
||||
assert!(m.to(Phase::Observing(1)).is_err());
|
||||
m.to(Phase::Applying(0)).unwrap();
|
||||
assert!(m.to(Phase::Observing(0)).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn only_a_committed_boundary_pauses_or_captures() {
|
||||
let mut m = PhaseMachine::new();
|
||||
m.to(Phase::Ready(0)).unwrap();
|
||||
m.to(Phase::Preparing(0)).unwrap();
|
||||
assert!(m.to(Phase::Paused(0)).is_err());
|
||||
assert!(m.to(Phase::Capturing(0)).is_err());
|
||||
assert!(!Phase::Preparing(0).is_committed_boundary());
|
||||
assert!(Phase::Ready(0).is_committed_boundary());
|
||||
assert!(Phase::Paused(3).is_committed_boundary());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_capture_returns_to_the_boundary_it_came_from() {
|
||||
let mut m = PhaseMachine::new();
|
||||
m.to(Phase::Ready(0)).unwrap();
|
||||
m.to(Phase::Paused(0)).unwrap();
|
||||
m.to(Phase::Capturing(0)).unwrap();
|
||||
assert!(m.to(Phase::Ready(0)).is_err());
|
||||
m.to(Phase::Paused(0)).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn failure_leads_to_restore_and_then_to_a_paused_boundary() {
|
||||
let mut m = PhaseMachine::new();
|
||||
m.to(Phase::Ready(0)).unwrap();
|
||||
m.to(Phase::Preparing(0)).unwrap();
|
||||
m.fail();
|
||||
assert_eq!(m.phase(), Phase::Failed);
|
||||
assert!(m.to(Phase::Ready(0)).is_err());
|
||||
m.to(Phase::Restoring(0)).unwrap();
|
||||
m.to(Phase::Paused(0)).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_episode_reset_leaves_a_pause_and_lands_on_step_zero() {
|
||||
let mut m = PhaseMachine::new();
|
||||
m.to(Phase::Ready(0)).unwrap();
|
||||
m.to(Phase::Preparing(0)).unwrap();
|
||||
m.to(Phase::Applying(0)).unwrap();
|
||||
m.to(Phase::Observing(1)).unwrap();
|
||||
m.to(Phase::Committing(0)).unwrap();
|
||||
m.to(Phase::Ready(1)).unwrap();
|
||||
m.to(Phase::Paused(1)).unwrap();
|
||||
m.to(Phase::Resetting(1)).unwrap();
|
||||
m.to(Phase::Ready(0)).unwrap();
|
||||
}
|
||||
}
|
||||
139
services/flysim/crates/fly-session/src/rpc.rs
Normal file
139
services/flysim/crates/fly-session/src/rpc.rs
Normal file
|
|
@ -0,0 +1,139 @@
|
|||
//! Domain RPC over Flybus: `req-<u64>` serials, incarnation pinning and the retry rule.
|
||||
//!
|
||||
//! A domain retry keeps its `requestId` and body and takes a fresh bus `callId`. Nothing here
|
||||
//! retries on its own: an uncertain call is resolved by the caller, which is the coordinator.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use serde_json::{Map, Value};
|
||||
|
||||
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
|
||||
// glob keeps the contract's own names in sight instead of restating them.
|
||||
use crate::types::*;
|
||||
|
||||
/// A named worker endpoint, pinned to one bus registration and one domain incarnation.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct WorkerRef {
|
||||
pub service: String,
|
||||
/// The bus `serviceIncarnation` every call pins. Not a worker process id.
|
||||
pub bus_incarnation: String,
|
||||
pub worker_id: Id,
|
||||
/// The domain `incarnationId` Hello negotiated, once it has.
|
||||
pub domain_incarnation: Option<Id>,
|
||||
}
|
||||
|
||||
impl WorkerRef {
|
||||
pub fn new(service: &str, bus_incarnation: &str, worker_id: &Id) -> WorkerRef {
|
||||
WorkerRef {
|
||||
service: service.to_owned(),
|
||||
bus_incarnation: bus_incarnation.to_owned(),
|
||||
worker_id: worker_id.clone(),
|
||||
domain_incarnation: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One terminal domain reply and the artifacts it brought.
|
||||
pub struct DomainReply {
|
||||
pub outcome: SessionRpcOutcome,
|
||||
pub request_id: DomainRequestId,
|
||||
pub artifacts: BTreeMap<String, flybus::Artifact>,
|
||||
}
|
||||
|
||||
impl DomainReply {
|
||||
/// The success `result`, or the domain error.
|
||||
pub fn result(&self) -> Result<&Value, DomainError> {
|
||||
outcome_result(&self.outcome)
|
||||
}
|
||||
|
||||
/// Reads and validates the success `result` as a method payload.
|
||||
pub fn parse<T: DomainType>(&self) -> Result<T, DomainError> {
|
||||
let result = outcome_result(&self.outcome)?;
|
||||
T::from_json(result).map_err(|e| DomainError::invalid(format!("unreadable result: {e}")))
|
||||
}
|
||||
}
|
||||
|
||||
/// Issues one domain call and waits for its terminal reply.
|
||||
///
|
||||
/// `want_artifacts` names the attachments to extract before the result delivery is dropped.
|
||||
/// A bus-level failure is not a domain failure: it is reported with the dispatch certainty the
|
||||
/// bus gave, because a caller-side timeout must not imply that nothing was mutated.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub async fn call(
|
||||
bus: &flybus::Client,
|
||||
target: &WorkerRef,
|
||||
method: &str,
|
||||
scope: Option<Scope>,
|
||||
params: Map<String, Value>,
|
||||
attachments: &[(&str, &flybus::Artifact)],
|
||||
request_id: DomainRequestId,
|
||||
want_artifacts: &[String],
|
||||
) -> Result<DomainReply, DomainError> {
|
||||
let request = SessionRpcRequest { request_id: request_id.clone(), scope, params: Value::Object(params) };
|
||||
let mut pending = bus
|
||||
.call(
|
||||
&target.service,
|
||||
Some(&target.bus_incarnation),
|
||||
method,
|
||||
object(request.to_json()),
|
||||
attachments,
|
||||
)
|
||||
.await
|
||||
.map_err(|e| bus_error(method, &e))?;
|
||||
let result = pending.result().await.map_err(|e| bus_error(method, &e))?;
|
||||
let outcome = SessionRpcOutcome::from_json(&Value::Object(result.outcome().clone()))
|
||||
.map_err(|e| DomainError::invalid(format!("{method}: {e}")))?;
|
||||
let mut artifacts = BTreeMap::new();
|
||||
for name in want_artifacts {
|
||||
if let Ok(artifact) = result.artifact(name) {
|
||||
// An independent explicit hold, so the handle outlives this delivery and can be
|
||||
// forwarded to several Commit calls and to publication.
|
||||
match artifact.retain().await {
|
||||
Ok(hold) => {
|
||||
artifacts.insert(name.clone(), hold);
|
||||
}
|
||||
Err(_) => {
|
||||
artifacts.insert(name.clone(), artifact);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
drop(result);
|
||||
Ok(DomainReply { outcome, request_id, artifacts })
|
||||
}
|
||||
|
||||
/// Maps a bus failure onto a domain error, preserving how certain the mutation is.
|
||||
fn bus_error(method: &str, e: &flybus::BusError) -> DomainError {
|
||||
let mutation = match e.dispatch {
|
||||
flybus::Dispatch::NotDispatched => MutationCertainty::None,
|
||||
flybus::Dispatch::Dispatched | flybus::Dispatch::Unknown => MutationCertainty::Unknown,
|
||||
};
|
||||
let code = match e.code {
|
||||
flybus::ErrorCode::TargetChanged | flybus::ErrorCode::NoService => {
|
||||
ErrorCode::IdentityMismatch
|
||||
}
|
||||
flybus::ErrorCode::Backpressure | flybus::ErrorCode::QuotaExceeded => ErrorCode::Busy,
|
||||
flybus::ErrorCode::ArtifactGone
|
||||
| flybus::ErrorCode::ArtifactUnsealed
|
||||
| flybus::ErrorCode::OwnerInvalid
|
||||
| flybus::ErrorCode::ArtifactMismatch => ErrorCode::BufferInvalid,
|
||||
_ => ErrorCode::BackendFailure,
|
||||
};
|
||||
DomainError::new(code, format!("{method}: bus {:?}: {}", e.code, e.message), mutation)
|
||||
}
|
||||
|
||||
/// Per-worker request serials. A newly issued operation takes the next one; a retry does not.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct Serials(BTreeMap<String, u64>);
|
||||
|
||||
impl Serials {
|
||||
pub fn next(&mut self, service: &str) -> DomainRequestId {
|
||||
let slot = self.0.entry(service.to_owned()).or_insert(0);
|
||||
*slot += 1;
|
||||
DomainRequestId::from_serial(*slot)
|
||||
}
|
||||
|
||||
pub fn highest(&self, service: &str) -> u64 {
|
||||
self.0.get(service).copied().unwrap_or_default()
|
||||
}
|
||||
}
|
||||
358
services/flysim/crates/fly-session/src/task.rs
Normal file
358
services/flysim/crates/fly-session/src/task.rs
Normal file
|
|
@ -0,0 +1,358 @@
|
|||
//! Coordinator-local task and executor interfaces (`workers-v1` section 4), plus the
|
||||
//! deterministic counter task and the identity executor the synthetic composition uses.
|
||||
//!
|
||||
//! These are library interfaces, not extra bus services. The task interprets inspection data
|
||||
//! and asks for outcomes; the executor translates a selected decision using read-only current
|
||||
//! game state and task progress; the coordinator orders and applies the results. Nothing here
|
||||
//! writes a controller or neural state directly.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use serde_json::{Map, Value, json};
|
||||
|
||||
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
|
||||
// glob keeps the contract's own names in sight instead of restating them.
|
||||
use crate::types::*;
|
||||
|
||||
/// The schemas the synthetic arena composition registers.
|
||||
pub fn inspection_schema() -> SchemaRef {
|
||||
synthetic_schema("arena.inspection.v1", 1)
|
||||
}
|
||||
|
||||
pub fn decision_schema() -> SchemaRef {
|
||||
synthetic_schema("arena.decision.v1", 1)
|
||||
}
|
||||
|
||||
pub fn context_schema() -> SchemaRef {
|
||||
synthetic_schema("arena.context.v1", 1)
|
||||
}
|
||||
|
||||
pub fn progress_schema() -> SchemaRef {
|
||||
synthetic_schema("arena.progress.v1", 1)
|
||||
}
|
||||
|
||||
pub fn event_schema() -> SchemaRef {
|
||||
synthetic_schema("arena.event.v1", 1)
|
||||
}
|
||||
|
||||
pub fn episode_schema() -> SchemaRef {
|
||||
synthetic_schema("arena.episode.v1", 1)
|
||||
}
|
||||
|
||||
pub fn controller_schema_ref() -> SchemaRef {
|
||||
synthetic_schema("arena.controller.v1", 1)
|
||||
}
|
||||
|
||||
/// What `Task.bootstrap` produced.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Bootstrap {
|
||||
pub contexts: BTreeMap<Id, TypedValue>,
|
||||
pub progress: TypedValue,
|
||||
pub events: Vec<TaskEvent>,
|
||||
}
|
||||
|
||||
/// What `Task.evaluate_transition` produced, for exactly one transition.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Evaluation {
|
||||
/// Every configured agent has an entry, including an empty one.
|
||||
pub outcomes: BTreeMap<Id, AgentOutcome>,
|
||||
pub next_contexts: BTreeMap<Id, TypedValue>,
|
||||
pub progress: TypedValue,
|
||||
pub events: Vec<TaskEvent>,
|
||||
pub episode: Option<EpisodeRequest>,
|
||||
}
|
||||
|
||||
/// A checkpointable task ledger and the two evaluation entry points.
|
||||
pub trait Task: Send {
|
||||
fn schema(&self) -> SchemaRef;
|
||||
|
||||
/// Called once, at boundary 0, before any agent is initialized.
|
||||
fn bootstrap(
|
||||
&mut self,
|
||||
initial_inspection: &TypedValue,
|
||||
bindings: &[PortBinding],
|
||||
) -> DomainResult<Bootstrap>;
|
||||
|
||||
/// Called exactly once per acknowledged world step, never against a later observation.
|
||||
fn evaluate_transition(
|
||||
&mut self,
|
||||
scope: &Scope,
|
||||
old_inspection: &TypedValue,
|
||||
new_inspection: &TypedValue,
|
||||
applied_controls: &[PortControl],
|
||||
) -> DomainResult<Evaluation>;
|
||||
|
||||
fn progress(&self) -> TypedValue;
|
||||
|
||||
/// How many times `evaluate_transition` has run. A transition must evaluate once.
|
||||
fn evaluations(&self) -> u64;
|
||||
}
|
||||
|
||||
/// Translates one selected decision into a controller intent, with no port assignment.
|
||||
pub trait ActionExecutor: Send {
|
||||
fn apply(
|
||||
&mut self,
|
||||
scope: &Scope,
|
||||
decision: &TypedValue,
|
||||
current_game_state: &TypedValue,
|
||||
progress: &TypedValue,
|
||||
clock: &RationalNs,
|
||||
) -> DomainResult<(ControllerIntent, Vec<TaskEvent>)>;
|
||||
}
|
||||
|
||||
/// The only executor v1 supports: it passes a direct-control decision through unchanged.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct IdentityExecutor;
|
||||
|
||||
impl ActionExecutor for IdentityExecutor {
|
||||
fn apply(
|
||||
&mut self,
|
||||
_scope: &Scope,
|
||||
decision: &TypedValue,
|
||||
_current_game_state: &TypedValue,
|
||||
_progress: &TypedValue,
|
||||
_clock: &RationalNs,
|
||||
) -> DomainResult<(ControllerIntent, Vec<TaskEvent>)> {
|
||||
if decision.schema != decision_schema() {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"the decision does not carry the profile's registered intent schema",
|
||||
));
|
||||
}
|
||||
let intent = ControllerIntent::from_json(&decision.value)
|
||||
.map_err(|e| DomainError::invalid(format!("decision: {e}")))?;
|
||||
Ok((intent, Vec::new()))
|
||||
}
|
||||
}
|
||||
|
||||
/// When the counter task asks for a terminal episode transition.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
|
||||
pub enum Terminal {
|
||||
/// The episode runs until the application stops it.
|
||||
#[default]
|
||||
Never,
|
||||
/// The counter reached this value or higher.
|
||||
Counter(i64),
|
||||
/// This many transitions were evaluated.
|
||||
AfterTransitions(u64),
|
||||
}
|
||||
|
||||
/// The deterministic counter task: rewards come from the arena counter each agent moved.
|
||||
pub struct CounterTask {
|
||||
epoch: Id,
|
||||
agents: Vec<Id>,
|
||||
bindings: Vec<PortBinding>,
|
||||
transitions: u64,
|
||||
evaluations: u64,
|
||||
total_reward: f64,
|
||||
counter: i64,
|
||||
terminal: Terminal,
|
||||
}
|
||||
|
||||
impl CounterTask {
|
||||
pub fn new(epoch: &Id, terminal: Terminal) -> CounterTask {
|
||||
CounterTask {
|
||||
epoch: epoch.clone(),
|
||||
agents: Vec::new(),
|
||||
bindings: Vec::new(),
|
||||
transitions: 0,
|
||||
evaluations: 0,
|
||||
total_reward: 0.0,
|
||||
counter: 0,
|
||||
terminal,
|
||||
}
|
||||
}
|
||||
|
||||
fn context(&self, step: u64, boot: bool) -> TypedValue {
|
||||
TypedValue::new(context_schema(), json!({
|
||||
"available": ["inc", "dec"],
|
||||
"boot": boot,
|
||||
"step": step,
|
||||
}))
|
||||
.expect("a synthetic typed value fits the contract")
|
||||
}
|
||||
|
||||
fn progress_value(&self) -> TypedValue {
|
||||
TypedValue::new(progress_schema(), json!({
|
||||
"counter": self.counter,
|
||||
"transitions": self.transitions,
|
||||
"totalReward": self.total_reward,
|
||||
}))
|
||||
.expect("a synthetic typed value fits the contract")
|
||||
}
|
||||
|
||||
/// The counter delta one port control asks for: `inc` adds one, `dec` subtracts one.
|
||||
///
|
||||
/// This is the task's reading of a control, kept identical to the environment's rule so a
|
||||
/// reward describes the transition the world actually took.
|
||||
pub fn delta_of(control: &PortControl) -> i64 {
|
||||
let mut delta = 0;
|
||||
for button in &control.buttons {
|
||||
if button.down {
|
||||
match button.id.as_str() {
|
||||
"inc" => delta += 1,
|
||||
"dec" => delta -= 1,
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
delta
|
||||
}
|
||||
|
||||
fn agent_of_port(&self, port_id: &Id) -> Option<&Id> {
|
||||
self.bindings.iter().find(|b| b.port_id == *port_id).map(|b| &b.agent_id)
|
||||
}
|
||||
}
|
||||
|
||||
impl Task for CounterTask {
|
||||
fn schema(&self) -> SchemaRef {
|
||||
progress_schema()
|
||||
}
|
||||
|
||||
fn bootstrap(
|
||||
&mut self,
|
||||
initial_inspection: &TypedValue,
|
||||
bindings: &[PortBinding],
|
||||
) -> DomainResult<Bootstrap> {
|
||||
if initial_inspection.schema != inspection_schema() {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"the environment's inspection schema is not the one this task reads",
|
||||
));
|
||||
}
|
||||
self.counter = initial_inspection.integer("counter").map_err(DomainError::invalid)?;
|
||||
self.bindings = bindings.to_vec();
|
||||
self.agents = bindings.iter().map(|b| b.agent_id.clone()).collect();
|
||||
self.agents.sort();
|
||||
let contexts = self
|
||||
.agents
|
||||
.iter()
|
||||
.map(|agent| (agent.clone(), self.context(0, true)))
|
||||
.collect();
|
||||
// A bootstrap event has sourceStep 0 and carries no reward: warm-up produces no
|
||||
// gameplay outcome at all.
|
||||
let events = vec![TaskEvent {
|
||||
id: event_id(&self.epoch, 0, "bootstrap", 0),
|
||||
kind_id: id("arena.bootstrap"),
|
||||
source_step: 0,
|
||||
agent_id: None,
|
||||
payload: TypedValue::new(event_schema(), json!({"counter": self.counter}))
|
||||
.expect("a synthetic typed value fits the contract"),
|
||||
}];
|
||||
Ok(Bootstrap { contexts, progress: self.progress_value(), events })
|
||||
}
|
||||
|
||||
fn evaluate_transition(
|
||||
&mut self,
|
||||
scope: &Scope,
|
||||
old_inspection: &TypedValue,
|
||||
new_inspection: &TypedValue,
|
||||
applied_controls: &[PortControl],
|
||||
) -> DomainResult<Evaluation> {
|
||||
let old = old_inspection.integer("counter").map_err(DomainError::invalid)?;
|
||||
let new = new_inspection.integer("counter").map_err(DomainError::invalid)?;
|
||||
let source_step = scope.step + 1;
|
||||
self.evaluations += 1;
|
||||
self.transitions += 1;
|
||||
self.counter = new;
|
||||
|
||||
let mut outcomes: BTreeMap<Id, AgentOutcome> = self
|
||||
.agents
|
||||
.iter()
|
||||
.map(|agent| (agent.clone(), AgentOutcome::default()))
|
||||
.collect();
|
||||
let mut events = Vec::new();
|
||||
let mut ordinal = 0u32;
|
||||
// Controls arrive in descriptor port order, so the reward order is deterministic.
|
||||
for control in applied_controls {
|
||||
let Some(agent) = self.agent_of_port(&control.port_id).cloned() else {
|
||||
return Err(DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
format!("port {} is bound to no agent", control.port_id),
|
||||
));
|
||||
};
|
||||
let delta = CounterTask::delta_of(control);
|
||||
let event = event_id(&self.epoch, source_step, "counter-delta", ordinal);
|
||||
ordinal += 1;
|
||||
events.push(TaskEvent {
|
||||
id: event.clone(),
|
||||
kind_id: id("arena.counter-delta"),
|
||||
source_step,
|
||||
agent_id: Some(agent.clone()),
|
||||
payload: TypedValue::new(event_schema(), json!({"delta": delta, "counter": new}))
|
||||
.expect("a synthetic typed value fits the contract"),
|
||||
});
|
||||
let outcome = outcomes.get_mut(&agent).ok_or_else(|| {
|
||||
DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
format!("port {} names agent {agent}, which is not configured", control.port_id),
|
||||
)
|
||||
})?;
|
||||
// A shipped positive-only profile would reject a negative value; this task is
|
||||
// signed on purpose, so the profile that consumes it declares signed rewards.
|
||||
outcome.rewards.push(Reward {
|
||||
event_id: event,
|
||||
rule_id: id("counter-delta"),
|
||||
value: delta as f64,
|
||||
});
|
||||
self.total_reward += delta as f64;
|
||||
// One declared stimulus when the counter moved past a multiple of five, so the
|
||||
// stimulation path is exercised without depending on reward.
|
||||
if delta != 0 && new.rem_euclid(5) == 0 {
|
||||
outcome.stimulations.push(Stimulus {
|
||||
id: event_id(&self.epoch, source_step, "milestone", ordinal),
|
||||
kind_id: id("arena.milestone"),
|
||||
duration_ms: 4.0,
|
||||
});
|
||||
}
|
||||
}
|
||||
if new != old + applied_controls.iter().map(CounterTask::delta_of).sum::<i64>() {
|
||||
return Err(DomainError::new(
|
||||
ErrorCode::BackendFailure,
|
||||
"the world did not move by the batch the task read",
|
||||
MutationCertainty::Unknown,
|
||||
));
|
||||
}
|
||||
|
||||
let next_contexts = self
|
||||
.agents
|
||||
.iter()
|
||||
.map(|agent| (agent.clone(), self.context(source_step, false)))
|
||||
.collect();
|
||||
let terminal = match self.terminal {
|
||||
Terminal::Never => false,
|
||||
Terminal::Counter(target) => new >= target,
|
||||
Terminal::AfterTransitions(n) => self.transitions >= n,
|
||||
};
|
||||
// The contract's `EpisodeRequest` is terminal by construction: `kind` is a constant.
|
||||
let episode = terminal.then(|| EpisodeRequest {
|
||||
reason: id("counter-target"),
|
||||
outcome: TypedValue::new(episode_schema(), json!({"counter": new, "transitions": self.transitions}))
|
||||
.expect("a synthetic typed value fits the contract"),
|
||||
});
|
||||
Ok(Evaluation {
|
||||
outcomes,
|
||||
next_contexts,
|
||||
progress: self.progress_value(),
|
||||
events,
|
||||
episode,
|
||||
})
|
||||
}
|
||||
|
||||
fn progress(&self) -> TypedValue {
|
||||
self.progress_value()
|
||||
}
|
||||
|
||||
fn evaluations(&self) -> u64 {
|
||||
self.evaluations
|
||||
}
|
||||
}
|
||||
|
||||
/// The inspection value the counter environment publishes.
|
||||
pub fn inspection(counter: i64, boundary: u64) -> TypedValue {
|
||||
let mut value = Map::new();
|
||||
value.insert("counter".into(), counter.into());
|
||||
value.insert("boundary".into(), boundary.into());
|
||||
TypedValue::new(inspection_schema(), Value::Object(value))
|
||||
.expect("the arena inspection fits the contract")
|
||||
}
|
||||
518
services/flysim/crates/fly-session/src/types.rs
Normal file
518
services/flysim/crates/fly-session/src/types.rs
Normal file
|
|
@ -0,0 +1,518 @@
|
|||
//! The domain types this crate uses, all from the shared `fly-session-types` crate.
|
||||
//!
|
||||
//! CONTRACT-01 owns the scalars, the method payloads, their validation, the canonical digests
|
||||
//! and the trace format. This module is only a facade over that crate plus the few things a
|
||||
//! coordinator needs that are not part of the contract: a session-side error value, the
|
||||
//! synthetic composition's schema and event-id derivations, and a log of recorded traces.
|
||||
//!
|
||||
//! `Id` and `Digest` are type aliases, because the shared crate carries both as validated
|
||||
//! `String`s from `flybus::wire` rather than forking the encodings into newtypes.
|
||||
|
||||
use serde_json::{Map, Value};
|
||||
|
||||
pub use fly_session_types::ArtifactRef;
|
||||
pub use fly_session_types::canonical::{
|
||||
self, OperationKey, body_digest, canonicalize, digest_of, sha256_hex,
|
||||
};
|
||||
pub use fly_session_types::media::{AudioDescriptor, AudioRef, ViewDescriptor, ViewRef};
|
||||
pub use fly_session_types::rpc::{
|
||||
ErrorCode, MutationCertainty, SessionRpcFailure, SessionRpcOutcome, SessionRpcRequest,
|
||||
SessionRpcSuccess,
|
||||
};
|
||||
pub use fly_session_types::scalar::{
|
||||
ArtifactIdentity, BusCallId, DomainRequestId, DomainType, MAX_TYPED_VALUE_BYTES, RationalNs,
|
||||
SchemaRef, Scope, TypedValue, is_digest, is_id,
|
||||
};
|
||||
pub use fly_session_types::schema::contract_digest;
|
||||
pub use fly_session_types::trace::{
|
||||
TraceAgent, TraceBehaviour, TraceObservation, TraceOperational, TraceRequest, TransitionTrace,
|
||||
};
|
||||
pub use fly_session_types::workers::{
|
||||
AcknowledgeParams, AcknowledgeResult, AdvanceParams, AgentCommitResult, AgentInitializeParams,
|
||||
AgentInitializeResult, AgentTelemetry, AssetRef, AxisRange, AxisSchema, AxisValue, ButtonState,
|
||||
CommitParams, ControllerSchema, Determinism, EnvironmentDescriptor,
|
||||
EnvironmentInitializeParams, EnvironmentInitializeResult, EpisodeRequest, HelloParams,
|
||||
HelloResult, LearningTelemetry, MAX_ACKNOWLEDGE, MAX_AGENTS, MAX_PORTS, MAX_RATE_ROLES,
|
||||
MAX_REWARDS, MAX_STIMULI, PortControl, PortDescriptor, PrepareParams, PreparedDecision,
|
||||
RateSample, Recovery, Reward, Role, SensoryInput, ShutdownParams, ShutdownResult, StatusResult,
|
||||
StepResult, Stimulus, TaskEvent, WorkerState, WorldObservation,
|
||||
};
|
||||
|
||||
/// A validated `Id`: `^[a-z0-9][a-z0-9._-]{0,63}$`, the bus encoding the contracts reuse.
|
||||
pub type Id = String;
|
||||
|
||||
/// 64 lowercase hexadecimal digits: a SHA-256.
|
||||
pub type Digest = String;
|
||||
|
||||
/// An `Id` from a trusted literal. It panics on a malformed one, which is a bug in the
|
||||
/// composition rather than a runtime condition.
|
||||
pub fn id(s: &str) -> Id {
|
||||
assert!(is_id(s), "{s:?} is not a valid Id");
|
||||
s.to_owned()
|
||||
}
|
||||
|
||||
/// An `Id` from an untrusted string.
|
||||
pub fn parse_id(s: &str) -> Result<Id, String> {
|
||||
if is_id(s) {
|
||||
Ok(s.to_owned())
|
||||
} else {
|
||||
Err(format!("{s:?} is not a valid Id"))
|
||||
}
|
||||
}
|
||||
|
||||
/// The SHA-256 of some bytes, hex-encoded.
|
||||
pub fn digest_of_bytes(bytes: &[u8]) -> Digest {
|
||||
sha256_hex(bytes)
|
||||
}
|
||||
|
||||
/// A `Scope` from trusted composition values.
|
||||
pub fn scope_at(session_id: &str, epoch: &str, step: u64) -> Scope {
|
||||
Scope::new(session_id, epoch, step).expect("a composition scope is valid")
|
||||
}
|
||||
|
||||
/// `hz` steps per second as an exact nanosecond duration.
|
||||
pub fn hz(hz: u64) -> Result<RationalNs, String> {
|
||||
RationalNs::reduced(1_000_000_000, u128::from(hz)).map_err(|e| e.0)
|
||||
}
|
||||
|
||||
/// A whole number of milliseconds as an exact nanosecond duration.
|
||||
pub fn millis(ms: u64) -> Result<RationalNs, String> {
|
||||
RationalNs::reduced(u128::from(ms) * 1_000_000, 1).map_err(|e| e.0)
|
||||
}
|
||||
|
||||
/// A schema reference whose digest is derived from its own name and version, so the synthetic
|
||||
/// composition has stable identities without a schema registry file.
|
||||
pub fn synthetic_schema(name: &str, version: u16) -> SchemaRef {
|
||||
let digest = digest_of_bytes(format!("fly-session-schema-v1\n{name}\n{version}\n").as_bytes());
|
||||
SchemaRef::new(name, version, &digest).expect("a synthetic schema reference is valid")
|
||||
}
|
||||
|
||||
/// A deterministic task event id from epoch, source step, rule and ordinal.
|
||||
pub fn event_id(epoch: &str, source_step: u64, rule: &str, ordinal: u32) -> Id {
|
||||
let digest = digest_of_bytes(format!("{epoch}\n{source_step}\n{rule}\n{ordinal}\n").as_bytes());
|
||||
id(&format!("ev-{}", &digest[..16]))
|
||||
}
|
||||
|
||||
/// The digest of a validated, canonical control batch, in descriptor port order.
|
||||
pub fn controls_digest(controls: &[PortControl]) -> Digest {
|
||||
let array = Value::Array(controls.iter().map(DomainType::to_json).collect());
|
||||
digest_of(&array).expect("a validated control batch canonicalizes")
|
||||
}
|
||||
|
||||
/// The canonical digest of a typed value, schema identity included.
|
||||
pub fn typed_digest(value: &TypedValue) -> Digest {
|
||||
digest_of(&value.to_json()).expect("a validated typed value canonicalizes")
|
||||
}
|
||||
|
||||
/// The byte length one view's artifact must have.
|
||||
pub fn view_byte_length(descriptor: &ViewDescriptor) -> u64 {
|
||||
descriptor.row_stride * descriptor.height
|
||||
}
|
||||
|
||||
/// The boundary a required sensory view must have been produced at.
|
||||
pub fn required_produced_step(descriptor: &ViewDescriptor, boundary: u64) -> u64 {
|
||||
boundary.saturating_sub(descriptor.observation_delay_steps)
|
||||
}
|
||||
|
||||
/// Reads a finite number out of a typed value.
|
||||
pub fn typed_number(value: &TypedValue, key: &str) -> Result<f64, String> {
|
||||
value
|
||||
.value
|
||||
.get(key)
|
||||
.and_then(Value::as_f64)
|
||||
.filter(|v| v.is_finite())
|
||||
.ok_or_else(|| format!("typed value has no finite number {key:?}"))
|
||||
}
|
||||
|
||||
/// Reads an integer out of a typed value.
|
||||
pub fn typed_integer(value: &TypedValue, key: &str) -> Result<i64, String> {
|
||||
value
|
||||
.value
|
||||
.get(key)
|
||||
.and_then(Value::as_i64)
|
||||
.ok_or_else(|| format!("typed value has no integer {key:?}"))
|
||||
}
|
||||
|
||||
/// Builds a typed value from a JSON object, refusing one the contract would reject.
|
||||
pub fn typed(schema: SchemaRef, value: Value) -> Result<TypedValue, String> {
|
||||
TypedValue::new(schema, value).map_err(|e| e.0)
|
||||
}
|
||||
|
||||
/// The object form of a payload, for a bus `payload` or `outcome` field.
|
||||
pub fn object(value: Value) -> Map<String, Value> {
|
||||
match value {
|
||||
Value::Object(m) => m,
|
||||
_ => Map::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// `ipc-v1` section 7: the domain error a worker returns, with its mutation certainty.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct DomainError {
|
||||
pub code: ErrorCode,
|
||||
pub message: String,
|
||||
pub mutation: MutationCertainty,
|
||||
}
|
||||
|
||||
/// `ipc-v1` section 7: messages are at most 512 code points and carry no raw memory.
|
||||
pub const MAX_ERROR_MESSAGE_CHARS: usize = 512;
|
||||
|
||||
impl DomainError {
|
||||
pub fn new(
|
||||
code: ErrorCode,
|
||||
message: impl std::fmt::Display,
|
||||
mutation: MutationCertainty,
|
||||
) -> DomainError {
|
||||
let message: String = message.to_string();
|
||||
let message = if message.chars().count() > MAX_ERROR_MESSAGE_CHARS {
|
||||
message.chars().take(MAX_ERROR_MESSAGE_CHARS).collect()
|
||||
} else {
|
||||
message
|
||||
};
|
||||
DomainError { code, message, mutation }
|
||||
}
|
||||
|
||||
/// An error raised before anything was mutated.
|
||||
pub fn before(code: ErrorCode, message: impl std::fmt::Display) -> DomainError {
|
||||
DomainError::new(code, message, MutationCertainty::None)
|
||||
}
|
||||
|
||||
pub fn invalid(message: impl std::fmt::Display) -> DomainError {
|
||||
DomainError::before(ErrorCode::InvalidArgument, message)
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for DomainError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(f, "{} ({}): {}", self.code.as_str(), self.mutation.as_str(), self.message)
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for DomainError {}
|
||||
|
||||
pub type DomainResult<T> = Result<T, DomainError>;
|
||||
|
||||
/// The success `result` of a terminal outcome, or its domain error.
|
||||
pub fn outcome_result(outcome: &SessionRpcOutcome) -> DomainResult<&Value> {
|
||||
match outcome {
|
||||
SessionRpcOutcome::Success(s) => Ok(&s.result),
|
||||
SessionRpcOutcome::Failure(f) => Err(DomainError::new(
|
||||
f.code,
|
||||
f.message.clone(),
|
||||
f.mutation,
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
/// The worker id and incarnation a terminal outcome came from, and the scope it echoes.
|
||||
pub fn outcome_identity(
|
||||
outcome: &SessionRpcOutcome,
|
||||
) -> (&Id, &Id, &Option<Scope>) {
|
||||
match outcome {
|
||||
SessionRpcOutcome::Success(s) => (&s.worker_id, &s.incarnation_id, &s.scope),
|
||||
SessionRpcOutcome::Failure(f) => (&f.worker_id, &f.incarnation_id, &f.scope),
|
||||
}
|
||||
}
|
||||
|
||||
/// One session phase transition, recorded whether or not it ends a step.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct PhaseTransition {
|
||||
pub from: String,
|
||||
pub to: String,
|
||||
}
|
||||
|
||||
/// Everything a run recorded: its phase transitions and its completed transitions.
|
||||
///
|
||||
/// The transitions are the contract's [`TransitionTrace`]; the phase path is this crate's own,
|
||||
/// because the state machine lives here and not in the type contract.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct TraceLog {
|
||||
pub phases: Vec<PhaseTransition>,
|
||||
pub transitions: Vec<TransitionTrace>,
|
||||
}
|
||||
|
||||
impl TraceLog {
|
||||
pub fn phase(&mut self, from: String, to: String) {
|
||||
self.phases.push(PhaseTransition { from, to });
|
||||
}
|
||||
|
||||
pub fn transition(&mut self, trace: TransitionTrace) {
|
||||
self.transitions.push(trace);
|
||||
}
|
||||
|
||||
/// The behaviour of every transition, in order, with operational metadata excluded.
|
||||
///
|
||||
/// This is what `step-v1` section 8 compares across dispatch and completion orders.
|
||||
pub fn behavior(&self) -> Vec<String> {
|
||||
self.transitions
|
||||
.iter()
|
||||
.map(|t| {
|
||||
canonicalize(&t.behaviour.to_json())
|
||||
.expect("a recorded behaviour canonicalizes")
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The phase path, as `from -> to` strings.
|
||||
pub fn phase_path(&self) -> Vec<String> {
|
||||
self.phases.iter().map(|p| format!("{} -> {}", p.from, p.to)).collect()
|
||||
}
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------------------------
|
||||
// Coordinator-local types
|
||||
//
|
||||
// `workers-v1` section 4 calls these library interfaces rather than payloads, so the contract
|
||||
// crate does not carry them: they never cross the bus.
|
||||
|
||||
/// What an executor returns: buttons and axes, with no port assignment.
|
||||
///
|
||||
/// The coordinator supplies the port, which is why this is not a [`PortControl`].
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct ControllerIntent {
|
||||
pub buttons: Vec<ButtonState>,
|
||||
pub axes: Vec<AxisValue>,
|
||||
}
|
||||
|
||||
impl ControllerIntent {
|
||||
/// The canonical JSON of the intent: exactly a `PortControl` without its port.
|
||||
pub fn to_json(&self) -> Value {
|
||||
let buttons = self
|
||||
.buttons
|
||||
.iter()
|
||||
.map(|b| {
|
||||
let mut m = Map::new();
|
||||
m.insert("id".into(), b.id.clone().into());
|
||||
m.insert("down".into(), Value::Bool(b.down));
|
||||
Value::Object(m)
|
||||
})
|
||||
.collect();
|
||||
let axes = self
|
||||
.axes
|
||||
.iter()
|
||||
.map(|a| {
|
||||
let mut m = Map::new();
|
||||
m.insert("id".into(), a.id.clone().into());
|
||||
m.insert("value".into(), Value::from(a.value));
|
||||
Value::Object(m)
|
||||
})
|
||||
.collect();
|
||||
let mut m = Map::new();
|
||||
m.insert("buttons".into(), Value::Array(buttons));
|
||||
m.insert("axes".into(), Value::Array(axes));
|
||||
Value::Object(m)
|
||||
}
|
||||
|
||||
/// Reads an intent out of a decision payload.
|
||||
pub fn from_json(value: &Value) -> Result<ControllerIntent, String> {
|
||||
let control = Value::Object({
|
||||
let mut m = object(value.clone());
|
||||
m.insert("portId".into(), "p0".into());
|
||||
m
|
||||
});
|
||||
let control = PortControl::from_json(&control).map_err(|e| e.0)?;
|
||||
Ok(ControllerIntent { buttons: control.buttons, axes: control.axes })
|
||||
}
|
||||
|
||||
/// Binds this intent to a port, which only the coordinator may do.
|
||||
pub fn at_port(self, port_id: &str) -> PortControl {
|
||||
PortControl {
|
||||
port_id: port_id.to_owned(),
|
||||
buttons: self.buttons,
|
||||
axes: self.axes,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One port-to-agent assignment. It crosses the bus as a pair inside
|
||||
/// [`EnvironmentInitializeParams`]; inside the session it is named.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct PortBinding {
|
||||
pub port_id: Id,
|
||||
pub agent_id: Id,
|
||||
}
|
||||
|
||||
impl PortBinding {
|
||||
pub fn pair(&self) -> (Id, Id) {
|
||||
(self.port_id.clone(), self.agent_id.clone())
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything the task routes to one agent for this transition. Always explicit, even empty.
|
||||
#[derive(Clone, Debug, Default, PartialEq)]
|
||||
pub struct AgentOutcome {
|
||||
pub rewards: Vec<Reward>,
|
||||
pub stimulations: Vec<Stimulus>,
|
||||
}
|
||||
|
||||
/// Every button up and every axis at its declared neutral.
|
||||
///
|
||||
/// Uncontrolled ports are configured neutral before the epoch, not supplied ad hoc.
|
||||
pub fn neutral_control(schema: &ControllerSchema, port_id: &str) -> PortControl {
|
||||
PortControl {
|
||||
port_id: port_id.to_owned(),
|
||||
buttons: schema
|
||||
.buttons
|
||||
.iter()
|
||||
.map(|id| ButtonState { id: id.clone(), down: false })
|
||||
.collect(),
|
||||
axes: schema
|
||||
.axes
|
||||
.iter()
|
||||
.map(|a| AxisValue { id: a.id.clone(), value: a.neutral })
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------------------------
|
||||
// Session-side conveniences over the contract types
|
||||
//
|
||||
// These are extension traits rather than forks: the data and the rules stay in the contract
|
||||
// crate, and these only spell out the readings a coordinator and its workers keep needing.
|
||||
|
||||
/// Reading a typed value the way the synthetic composition writes it.
|
||||
pub trait TypedValueExt {
|
||||
/// The canonical digest of the whole typed value, schema identity included.
|
||||
fn digest(&self) -> Digest;
|
||||
fn number(&self, key: &str) -> Result<f64, String>;
|
||||
fn integer(&self, key: &str) -> Result<i64, String>;
|
||||
}
|
||||
|
||||
impl TypedValueExt for TypedValue {
|
||||
fn digest(&self) -> Digest {
|
||||
typed_digest(self)
|
||||
}
|
||||
|
||||
fn number(&self, key: &str) -> Result<f64, String> {
|
||||
typed_number(self, key)
|
||||
}
|
||||
|
||||
fn integer(&self, key: &str) -> Result<i64, String> {
|
||||
typed_integer(self, key)
|
||||
}
|
||||
}
|
||||
|
||||
/// Checking and building one port's complete controller state.
|
||||
pub trait ControllerSchemaExt {
|
||||
/// Checks that `control` names every declared button and axis, in descriptor order, with
|
||||
/// every value inside its range.
|
||||
fn check(&self, control: &PortControl) -> Result<(), String>;
|
||||
/// Every button up and every axis at its declared neutral.
|
||||
fn neutral(&self, port_id: &str) -> PortControl;
|
||||
}
|
||||
|
||||
impl ControllerSchemaExt for ControllerSchema {
|
||||
fn check(&self, control: &PortControl) -> Result<(), String> {
|
||||
control.validate_against(self).map_err(|e| e.0)
|
||||
}
|
||||
|
||||
fn neutral(&self, port_id: &str) -> PortControl {
|
||||
neutral_control(self, port_id)
|
||||
}
|
||||
}
|
||||
|
||||
/// The byte shape and producing boundary one declared view requires.
|
||||
pub trait ViewDescriptorExt {
|
||||
fn byte_length(&self) -> u64;
|
||||
fn required_produced_step(&self, boundary: u64) -> u64;
|
||||
}
|
||||
|
||||
impl ViewDescriptorExt for ViewDescriptor {
|
||||
fn byte_length(&self) -> u64 {
|
||||
view_byte_length(self)
|
||||
}
|
||||
|
||||
fn required_produced_step(&self, boundary: u64) -> u64 {
|
||||
required_produced_step(self, boundary)
|
||||
}
|
||||
}
|
||||
|
||||
/// `RationalNs` readings the session uses.
|
||||
pub trait RationalExt {
|
||||
fn is_positive(&self) -> bool;
|
||||
}
|
||||
|
||||
impl RationalExt for RationalNs {
|
||||
fn is_positive(&self) -> bool {
|
||||
!self.is_zero()
|
||||
}
|
||||
}
|
||||
|
||||
/// The next step of a scope, which is the only arithmetic a coordinator does on one.
|
||||
pub trait ScopeExt {
|
||||
fn next(&self) -> Scope;
|
||||
}
|
||||
|
||||
impl ScopeExt for Scope {
|
||||
fn next(&self) -> Scope {
|
||||
scope_at(&self.session_id, &self.epoch, self.step + 1)
|
||||
}
|
||||
}
|
||||
|
||||
/// The `Id` form of a domain request id, for a payload field that carries it as a string.
|
||||
pub trait DomainRequestIdExt {
|
||||
fn id(&self) -> Id;
|
||||
}
|
||||
|
||||
impl DomainRequestIdExt for DomainRequestId {
|
||||
fn id(&self) -> Id {
|
||||
self.as_str().to_owned()
|
||||
}
|
||||
}
|
||||
|
||||
/// Carrying a terminal domain outcome in a bus `outcome` object.
|
||||
pub trait SessionRpcOutcomeExt: Sized {
|
||||
fn to_outcome(&self) -> Map<String, Value>;
|
||||
fn from_outcome(outcome: &Map<String, Value>) -> Result<Self, String>;
|
||||
fn result(&self) -> DomainResult<&Value>;
|
||||
}
|
||||
|
||||
impl SessionRpcOutcomeExt for SessionRpcOutcome {
|
||||
fn to_outcome(&self) -> Map<String, Value> {
|
||||
object(self.to_json())
|
||||
}
|
||||
|
||||
fn from_outcome(outcome: &Map<String, Value>) -> Result<SessionRpcOutcome, String> {
|
||||
SessionRpcOutcome::from_json(&Value::Object(outcome.clone())).map_err(|e| e.0)
|
||||
}
|
||||
|
||||
fn result(&self) -> DomainResult<&Value> {
|
||||
outcome_result(self)
|
||||
}
|
||||
}
|
||||
|
||||
/// One domain failure, ready to send.
|
||||
pub fn failure_outcome(
|
||||
request_id: &DomainRequestId,
|
||||
worker_id: &str,
|
||||
incarnation_id: &str,
|
||||
scope: Option<Scope>,
|
||||
error: DomainError,
|
||||
) -> SessionRpcOutcome {
|
||||
SessionRpcOutcome::Failure(SessionRpcFailure {
|
||||
request_id: request_id.clone(),
|
||||
worker_id: worker_id.to_owned(),
|
||||
incarnation_id: incarnation_id.to_owned(),
|
||||
scope,
|
||||
code: error.code,
|
||||
message: error.message,
|
||||
mutation: error.mutation,
|
||||
})
|
||||
}
|
||||
|
||||
/// One domain success, ready to send.
|
||||
pub fn success_outcome(
|
||||
request_id: &DomainRequestId,
|
||||
worker_id: &str,
|
||||
incarnation_id: &str,
|
||||
scope: Option<Scope>,
|
||||
result: Map<String, Value>,
|
||||
) -> SessionRpcOutcome {
|
||||
SessionRpcOutcome::Success(SessionRpcSuccess {
|
||||
request_id: request_id.clone(),
|
||||
worker_id: worker_id.to_owned(),
|
||||
incarnation_id: incarnation_id.to_owned(),
|
||||
scope,
|
||||
result: Value::Object(result),
|
||||
})
|
||||
}
|
||||
695
services/flysim/crates/fly-session/src/worker.rs
Normal file
695
services/flysim/crates/fly-session/src/worker.rs
Normal file
|
|
@ -0,0 +1,695 @@
|
|||
//! The worker dispatch shell: one Flybus service, the common `Worker.*` methods, and the
|
||||
//! domain deduplication of `ipc-v1` section 5 in front of every mutation.
|
||||
//!
|
||||
//! The shell owns request admission order and the result cache. An endpoint owns the
|
||||
//! mutation. Exactly one mutation runs at a time -- the endpoint sits behind its own mutex --
|
||||
//! while `Worker.Status` is answered from a small shared cell, so a status query never waits
|
||||
//! for a numerical operation and never advances the progress counter itself.
|
||||
|
||||
use std::future::Future;
|
||||
use std::pin::Pin;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use serde_json::{Map, Value};
|
||||
|
||||
use crate::dedup::{Admission, CachedReply, OpClass, OperationKey, ResultCache};
|
||||
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
|
||||
// glob keeps the contract's own names in sight instead of restating them.
|
||||
use crate::types::*;
|
||||
|
||||
/// The build identity a worker reports in Hello. It is not a profile digest.
|
||||
pub fn build_digest() -> Digest {
|
||||
digest_of_bytes(b"fly-session/synthetic-workers-v1")
|
||||
}
|
||||
|
||||
/// The status a worker reports, kept outside the endpoint mutex so `Worker.Status` stays
|
||||
/// responsive while a mutation runs.
|
||||
#[derive(Clone)]
|
||||
pub struct StatusCell(Arc<Mutex<StatusInner>>);
|
||||
|
||||
struct StatusInner {
|
||||
state: WorkerState,
|
||||
current_scope: Option<Scope>,
|
||||
active_request_id: Option<DomainRequestId>,
|
||||
last_completed_request_id: Option<DomainRequestId>,
|
||||
last_batch_id: Option<Id>,
|
||||
progress_counter: u64,
|
||||
}
|
||||
|
||||
impl Default for StatusCell {
|
||||
fn default() -> StatusCell {
|
||||
StatusCell::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl StatusCell {
|
||||
pub fn new() -> StatusCell {
|
||||
StatusCell(Arc::new(Mutex::new(StatusInner {
|
||||
state: WorkerState::Uninitialized,
|
||||
current_scope: None,
|
||||
active_request_id: None,
|
||||
last_completed_request_id: None,
|
||||
last_batch_id: None,
|
||||
progress_counter: 0,
|
||||
})))
|
||||
}
|
||||
|
||||
fn with<T>(&self, f: impl FnOnce(&mut StatusInner) -> T) -> T {
|
||||
let mut inner = self.0.lock().expect("the status cell is never poisoned");
|
||||
f(&mut inner)
|
||||
}
|
||||
|
||||
pub fn set_state(&self, state: WorkerState) {
|
||||
self.with(|s| s.state = state);
|
||||
}
|
||||
|
||||
pub fn state(&self) -> WorkerState {
|
||||
self.with(|s| s.state)
|
||||
}
|
||||
|
||||
pub fn set_scope(&self, scope: Option<Scope>) {
|
||||
self.with(|s| s.current_scope = scope);
|
||||
}
|
||||
|
||||
pub fn set_active(&self, request_id: Option<DomainRequestId>) {
|
||||
self.with(|s| s.active_request_id = request_id);
|
||||
}
|
||||
|
||||
pub fn set_completed(&self, request_id: DomainRequestId) {
|
||||
self.with(|s| {
|
||||
s.active_request_id = None;
|
||||
s.last_completed_request_id = Some(request_id);
|
||||
});
|
||||
}
|
||||
|
||||
pub fn set_batch(&self, batch_id: Id) {
|
||||
self.with(|s| s.last_batch_id = Some(batch_id));
|
||||
}
|
||||
|
||||
/// Records computational or phase progress. A status query never calls this.
|
||||
pub fn progress(&self, by: u64) {
|
||||
self.with(|s| s.progress_counter = s.progress_counter.saturating_add(by));
|
||||
}
|
||||
|
||||
/// Raises the progress counter to `value`, which lets a worker report its model's own
|
||||
/// mutation count as its progress. It never moves backwards.
|
||||
pub fn advance_to(&self, value: u64) {
|
||||
self.with(|s| s.progress_counter = s.progress_counter.max(value));
|
||||
}
|
||||
|
||||
pub fn progress_counter(&self) -> u64 {
|
||||
self.with(|s| s.progress_counter)
|
||||
}
|
||||
|
||||
pub fn snapshot(&self) -> StatusResult {
|
||||
self.with(|s| StatusResult {
|
||||
state: s.state,
|
||||
current_scope: s.current_scope.clone(),
|
||||
active_request_id: s.active_request_id.clone(),
|
||||
last_completed_request_id: s.last_completed_request_id.clone(),
|
||||
last_batch_id: s.last_batch_id.clone(),
|
||||
progress_counter: s.progress_counter,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// What a handler produced: a domain `result` object and the artifacts it attaches.
|
||||
pub struct HandlerReply {
|
||||
pub result: Map<String, Value>,
|
||||
pub artifacts: Vec<(String, flybus::Artifact)>,
|
||||
/// True when a failure left the endpoint mutated; the shell reports it as such.
|
||||
pub mutated: bool,
|
||||
}
|
||||
|
||||
impl HandlerReply {
|
||||
pub fn new(result: Map<String, Value>) -> HandlerReply {
|
||||
HandlerReply { result, artifacts: Vec::new(), mutated: true }
|
||||
}
|
||||
|
||||
pub fn with_artifacts(
|
||||
result: Map<String, Value>,
|
||||
artifacts: Vec<(String, flybus::Artifact)>,
|
||||
) -> HandlerReply {
|
||||
HandlerReply { result, artifacts, mutated: true }
|
||||
}
|
||||
|
||||
/// The canonical JSON of one method result.
|
||||
pub fn from<T: DomainType>(value: &T) -> HandlerReply {
|
||||
HandlerReply::new(object(value.to_json()))
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything a handler is given: the parsed domain request and the bus request behind it.
|
||||
pub struct HandlerCtx<'a> {
|
||||
pub method: &'a str,
|
||||
pub request: &'a SessionRpcRequest,
|
||||
pub client: &'a flybus::Client,
|
||||
pub incoming: &'a flybus::Request,
|
||||
}
|
||||
|
||||
impl HandlerCtx<'_> {
|
||||
/// Reads and validates `params` as a method payload, reporting INVALID_ARGUMENT.
|
||||
///
|
||||
/// The contract crate does the reading, so a misspelled required field fails here rather
|
||||
/// than silently defaulting.
|
||||
pub fn params<T: DomainType>(&self) -> DomainResult<T> {
|
||||
T::from_json(&self.request.params)
|
||||
.map_err(|e| DomainError::invalid(format!("{}: {e}", self.method)))
|
||||
}
|
||||
|
||||
/// The scope the request must carry.
|
||||
pub fn scope(&self) -> DomainResult<&Scope> {
|
||||
self.request
|
||||
.scope
|
||||
.as_ref()
|
||||
.ok_or_else(|| DomainError::invalid(format!("{} requires a scope", self.method)))
|
||||
}
|
||||
|
||||
/// An owned handle on one of the request's declared attachments.
|
||||
pub fn artifact(&self, name: &str) -> DomainResult<flybus::Artifact> {
|
||||
self.incoming.artifact(name).map_err(|e| {
|
||||
DomainError::before(
|
||||
ErrorCode::BufferInvalid,
|
||||
format!("attachment {name:?} is missing or unowned: {}", e.message),
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// A handler's boxed future, so the endpoint trait stays dyn-compatible.
|
||||
pub type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;
|
||||
|
||||
/// One worker's domain behaviour. The shell owns everything else.
|
||||
pub trait WorkerEndpoint: Send + 'static {
|
||||
fn worker_id(&self) -> Id;
|
||||
fn incarnation_id(&self) -> Id;
|
||||
fn session_id(&self) -> Id;
|
||||
fn role(&self) -> Role;
|
||||
fn capabilities(&self) -> Vec<Id>;
|
||||
fn status_cell(&self) -> StatusCell;
|
||||
|
||||
/// The domain methods this endpoint implements, beyond the common `Worker.*` set.
|
||||
/// Anything else returns UNSUPPORTED without entering the endpoint.
|
||||
fn methods(&self) -> Vec<&'static str>;
|
||||
|
||||
fn handle<'a>(&'a mut self, ctx: HandlerCtx<'a>) -> BoxFuture<'a, DomainResult<HandlerReply>>;
|
||||
}
|
||||
|
||||
/// A running worker: its bus client, its service and the task serving it.
|
||||
pub struct WorkerHandle {
|
||||
pub worker_id: Id,
|
||||
pub incarnation_id: Id,
|
||||
pub service_name: String,
|
||||
pub service_incarnation: String,
|
||||
pub status: StatusCell,
|
||||
pub cache: Arc<tokio::sync::Mutex<ResultCache>>,
|
||||
task: tokio::task::JoinHandle<()>,
|
||||
client: flybus::Client,
|
||||
}
|
||||
|
||||
impl WorkerHandle {
|
||||
/// The worker's own progress counter, which is the fake model's mutation count.
|
||||
pub fn progress_counter(&self) -> u64 {
|
||||
self.status.progress_counter()
|
||||
}
|
||||
|
||||
pub fn state(&self) -> WorkerState {
|
||||
self.status.state()
|
||||
}
|
||||
|
||||
/// Stops serving and closes the worker's bus connection, which drops its registration and
|
||||
/// every owner it held. A later reply from it can attach to nothing.
|
||||
pub async fn stop(self) {
|
||||
self.task.abort();
|
||||
let _ = self.task.await;
|
||||
self.client.close().await;
|
||||
}
|
||||
}
|
||||
|
||||
/// Registers `service_name` and serves `endpoint` on it until the service ends or Shutdown.
|
||||
pub fn serve<E: WorkerEndpoint>(
|
||||
client: flybus::Client,
|
||||
service: flybus::Service,
|
||||
endpoint: E,
|
||||
) -> WorkerHandle {
|
||||
let worker_id = endpoint.worker_id();
|
||||
let incarnation_id = endpoint.incarnation_id();
|
||||
let status = endpoint.status_cell();
|
||||
let service_name = service.name().to_owned();
|
||||
let service_incarnation = service.incarnation().to_owned();
|
||||
let cache = Arc::new(tokio::sync::Mutex::new(ResultCache::new()));
|
||||
let task = tokio::spawn(run(client.clone(), service, Arc::new(tokio::sync::Mutex::new(endpoint)), cache.clone()));
|
||||
WorkerHandle {
|
||||
worker_id,
|
||||
incarnation_id,
|
||||
service_name,
|
||||
service_incarnation,
|
||||
status,
|
||||
cache,
|
||||
task,
|
||||
client,
|
||||
}
|
||||
}
|
||||
|
||||
async fn run<E: WorkerEndpoint>(
|
||||
client: flybus::Client,
|
||||
mut service: flybus::Service,
|
||||
endpoint: Arc<tokio::sync::Mutex<E>>,
|
||||
cache: Arc<tokio::sync::Mutex<ResultCache>>,
|
||||
) {
|
||||
// Identity and capabilities are fixed for the endpoint's lifetime, so the shell reads them
|
||||
// once and never takes the endpoint mutex to answer Hello or Status.
|
||||
let (worker_id, incarnation_id, session_id, role, capabilities, status, methods) = {
|
||||
let e = endpoint.lock().await;
|
||||
(
|
||||
e.worker_id(),
|
||||
e.incarnation_id(),
|
||||
e.session_id(),
|
||||
e.role(),
|
||||
e.capabilities(),
|
||||
e.status_cell(),
|
||||
e.methods(),
|
||||
)
|
||||
};
|
||||
let mut running: Vec<tokio::task::JoinHandle<()>> = Vec::new();
|
||||
while let Some(incoming) = service.next().await {
|
||||
let method = incoming.method().to_owned();
|
||||
let responder = incoming.responder();
|
||||
let request = match SessionRpcRequest::from_json(&Value::Object(
|
||||
incoming.payload().clone(),
|
||||
)) {
|
||||
Ok(request) => request,
|
||||
Err(e) => {
|
||||
// A malformed envelope has no usable requestId, so the reply names req-0.
|
||||
let failure = failure(
|
||||
&DomainRequestId::from_serial(0),
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
None,
|
||||
DomainError::invalid(format!("{method}: {e}")),
|
||||
);
|
||||
let _ = responder.reply(failure.to_outcome(), &[]).await;
|
||||
continue;
|
||||
}
|
||||
};
|
||||
// The contract type already validated the `req-<U64>` form when it read the envelope.
|
||||
let serial = request.request_id.clone();
|
||||
|
||||
// The common methods never enter the endpoint mutex, so they answer during a mutation.
|
||||
match method.as_str() {
|
||||
"Worker.Hello" => {
|
||||
let outcome = hello(
|
||||
&request,
|
||||
&session_id,
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
role,
|
||||
&capabilities,
|
||||
);
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
continue;
|
||||
}
|
||||
"Worker.Status" => {
|
||||
let result = status.snapshot().to_json();
|
||||
let outcome = success(
|
||||
&request,
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
object(result),
|
||||
);
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
continue;
|
||||
}
|
||||
"Worker.Acknowledge" => {
|
||||
let outcome = match acknowledge(&request, &cache).await {
|
||||
Ok(result) => success(&request, &worker_id, &incarnation_id, result),
|
||||
Err(e) => {
|
||||
failure(&request.request_id, &worker_id, &incarnation_id, request.scope.clone(), e)
|
||||
}
|
||||
};
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
continue;
|
||||
}
|
||||
"Worker.Shutdown" => {
|
||||
let outcome = match request.params.get("reason") {
|
||||
Some(_) => {
|
||||
match ShutdownParams::from_json(&request.params) {
|
||||
Ok(_) => {
|
||||
status.set_state(WorkerState::Stopping);
|
||||
Ok(ShutdownResult)
|
||||
}
|
||||
Err(e) => Err(DomainError::invalid(format!("Worker.Shutdown: {e}"))),
|
||||
}
|
||||
}
|
||||
None => Err(DomainError::invalid("Worker.Shutdown requires a reason")),
|
||||
};
|
||||
match outcome {
|
||||
Ok(result) => {
|
||||
let value = result.to_json();
|
||||
let outcome =
|
||||
success(&request, &worker_id, &incarnation_id, object(value));
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
let outcome = failure(
|
||||
&request.request_id,
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
request.scope.clone(),
|
||||
e,
|
||||
);
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
let class = if methods.contains(&method.as_str()) {
|
||||
classify_default(&method)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let Some(class) = class else {
|
||||
let outcome = failure(
|
||||
&request.request_id,
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
request.scope.clone(),
|
||||
DomainError::before(ErrorCode::Unsupported, format!("{method} is not supported")),
|
||||
);
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
continue;
|
||||
};
|
||||
|
||||
let body = match request.body_digest(&method) {
|
||||
Ok(body) => body,
|
||||
Err(e) => {
|
||||
let outcome = failure(
|
||||
&request.request_id,
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
request.scope.clone(),
|
||||
DomainError::invalid(format!("{method}: {e}")),
|
||||
);
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
continue;
|
||||
}
|
||||
};
|
||||
let key = match (class, &request.scope) {
|
||||
(OpClass::StepMutation, Some(scope)) => OperationKey {
|
||||
session_id: scope.session_id.clone(),
|
||||
epoch: scope.epoch.clone(),
|
||||
step: scope.step,
|
||||
method: method.clone(),
|
||||
worker_id: worker_id.clone(),
|
||||
},
|
||||
(OpClass::StepMutation, None) => {
|
||||
let outcome = failure(
|
||||
&request.request_id,
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
None,
|
||||
DomainError::invalid(format!("{method} requires a scope")),
|
||||
);
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
continue;
|
||||
}
|
||||
_ => OperationKey {
|
||||
session_id: session_id.clone(),
|
||||
epoch: id("lifecycle"),
|
||||
step: 0,
|
||||
method: method.clone(),
|
||||
worker_id: worker_id.clone(),
|
||||
},
|
||||
};
|
||||
|
||||
// Retained request identity is checked before phase checks and before any attachment
|
||||
// is dereferenced, because a duplicate may arrive after its input delivery was
|
||||
// consumed and needs only the cached result.
|
||||
let admission = {
|
||||
let mut c = cache.lock().await;
|
||||
c.admit(class, &key, serial.clone(), &body)
|
||||
};
|
||||
match admission {
|
||||
Admission::Replay(reply) => {
|
||||
let _ = responder.reply(reply.outcome.to_outcome(), &reply.attachments()).await;
|
||||
continue;
|
||||
}
|
||||
Admission::Refuse(e) => {
|
||||
let outcome = failure(
|
||||
&request.request_id,
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
request.scope.clone(),
|
||||
e,
|
||||
);
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
continue;
|
||||
}
|
||||
Admission::Execute => {}
|
||||
}
|
||||
|
||||
status.set_active(Some(request.request_id.clone()));
|
||||
// The mutation runs in its own task so the shell keeps reading. That is what lets an
|
||||
// exact duplicate arriving mid-execution be refused with IN_PROGRESS while the
|
||||
// original bus call still completes normally. The endpoint mutex, not this loop,
|
||||
// enforces one mutation at a time.
|
||||
running.retain(|task| !task.is_finished());
|
||||
running.push(tokio::spawn(execute(
|
||||
client.clone(),
|
||||
endpoint.clone(),
|
||||
cache.clone(),
|
||||
status.clone(),
|
||||
worker_id.clone(),
|
||||
incarnation_id.clone(),
|
||||
class,
|
||||
key,
|
||||
serial,
|
||||
body,
|
||||
method,
|
||||
request,
|
||||
incoming,
|
||||
)));
|
||||
}
|
||||
for task in running {
|
||||
let _ = task.await;
|
||||
}
|
||||
}
|
||||
|
||||
/// Runs one admitted mutation, records its reply and answers the bus call.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn execute<E: WorkerEndpoint>(
|
||||
client: flybus::Client,
|
||||
endpoint: Arc<tokio::sync::Mutex<E>>,
|
||||
cache: Arc<tokio::sync::Mutex<ResultCache>>,
|
||||
status: StatusCell,
|
||||
worker_id: Id,
|
||||
incarnation_id: Id,
|
||||
class: OpClass,
|
||||
key: OperationKey,
|
||||
serial: DomainRequestId,
|
||||
body: Digest,
|
||||
method: String,
|
||||
request: SessionRpcRequest,
|
||||
incoming: flybus::Request,
|
||||
) {
|
||||
let responder = incoming.responder();
|
||||
let outcome = {
|
||||
// One mutation at a time: the endpoint mutex is the worker's simulation lock, and it is
|
||||
// never held across a bus round trip taken by anything else.
|
||||
let mut e = endpoint.lock().await;
|
||||
let ctx = HandlerCtx {
|
||||
method: &method,
|
||||
request: &request,
|
||||
client: &client,
|
||||
incoming: &incoming,
|
||||
};
|
||||
e.handle(ctx).await
|
||||
};
|
||||
match outcome {
|
||||
Ok(reply) => {
|
||||
let outcome = success(&request, &worker_id, &incarnation_id, reply.result.clone());
|
||||
let mut holds = Vec::with_capacity(reply.artifacts.len());
|
||||
for (name, artifact) in &reply.artifacts {
|
||||
// The cache owns its own hold, so a replay survives the first caller
|
||||
// consuming its delivery.
|
||||
match artifact.retain().await {
|
||||
Ok(hold) => holds.push((name.clone(), hold)),
|
||||
Err(_) => holds.push((name.clone(), artifact.clone())),
|
||||
}
|
||||
}
|
||||
let cached = CachedReply::with_artifacts(outcome.clone(), holds);
|
||||
{
|
||||
let mut c = cache.lock().await;
|
||||
match class {
|
||||
OpClass::StepMutation => c.record(key, serial, body, cached),
|
||||
OpClass::Lifecycle => c.record_lifecycle(serial, body, cached),
|
||||
OpClass::ReadOnly => c.record_readonly(serial, cached),
|
||||
}
|
||||
}
|
||||
status.set_completed(request.request_id.clone());
|
||||
let attachments: Vec<(&str, &flybus::Artifact)> =
|
||||
reply.artifacts.iter().map(|(n, a)| (n.as_str(), a)).collect();
|
||||
let _ = responder.reply(outcome.to_outcome(), &attachments).await;
|
||||
}
|
||||
Err(e) => {
|
||||
if e.mutation == MutationCertainty::None {
|
||||
// Nothing happened, so the key stays free for the corrected request.
|
||||
let mut c = cache.lock().await;
|
||||
c.abandon(&key);
|
||||
} else {
|
||||
let cached = CachedReply::new(failure_outcome(
|
||||
&request.request_id,
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
request.scope.clone(),
|
||||
e.clone(),
|
||||
));
|
||||
let mut c = cache.lock().await;
|
||||
if class == OpClass::StepMutation {
|
||||
c.record(key, serial, body, cached);
|
||||
} else {
|
||||
c.abandon(&key);
|
||||
}
|
||||
status.set_state(WorkerState::Failed);
|
||||
}
|
||||
status.set_active(None);
|
||||
let outcome = failure(
|
||||
&request.request_id,
|
||||
&worker_id,
|
||||
&incarnation_id,
|
||||
request.scope.clone(),
|
||||
e,
|
||||
);
|
||||
let _ = responder.reply(outcome.to_outcome(), &[]).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The retention class of every method name the contracts define.
|
||||
fn classify_default(method: &str) -> Option<OpClass> {
|
||||
match method {
|
||||
"Agent.Prepare" | "Agent.Commit" | "Environment.Advance" => Some(OpClass::StepMutation),
|
||||
"Agent.Initialize" | "Environment.Initialize" => Some(OpClass::Lifecycle),
|
||||
"Worker.Hello" | "Worker.Status" | "Worker.Acknowledge" | "Worker.Shutdown" => {
|
||||
Some(OpClass::ReadOnly)
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn success(
|
||||
request: &SessionRpcRequest,
|
||||
worker_id: &Id,
|
||||
incarnation_id: &Id,
|
||||
result: Map<String, Value>,
|
||||
) -> SessionRpcOutcome {
|
||||
success_outcome(
|
||||
&request.request_id,
|
||||
worker_id,
|
||||
incarnation_id,
|
||||
request.scope.clone(),
|
||||
result,
|
||||
)
|
||||
}
|
||||
|
||||
fn failure(
|
||||
request_id: &DomainRequestId,
|
||||
worker_id: &Id,
|
||||
incarnation_id: &Id,
|
||||
scope: Option<Scope>,
|
||||
error: DomainError,
|
||||
) -> SessionRpcOutcome {
|
||||
failure_outcome(request_id, worker_id, incarnation_id, scope, error)
|
||||
}
|
||||
|
||||
fn hello(
|
||||
request: &SessionRpcRequest,
|
||||
session_id: &Id,
|
||||
worker_id: &Id,
|
||||
incarnation_id: &Id,
|
||||
role: Role,
|
||||
capabilities: &[Id],
|
||||
) -> SessionRpcOutcome {
|
||||
let params: HelloParams = match HelloParams::from_json(&request.params) {
|
||||
Ok(params) => params,
|
||||
Err(e) => {
|
||||
return failure(
|
||||
&request.request_id,
|
||||
worker_id,
|
||||
incarnation_id,
|
||||
None,
|
||||
DomainError::invalid(format!("Worker.Hello: {e}")),
|
||||
);
|
||||
}
|
||||
};
|
||||
if request.scope.is_some() {
|
||||
return failure(
|
||||
&request.request_id,
|
||||
worker_id,
|
||||
incarnation_id,
|
||||
None,
|
||||
DomainError::invalid("Worker.Hello has no scope"),
|
||||
);
|
||||
}
|
||||
if params.session_id != *session_id
|
||||
|| params.expected_worker_id != *worker_id
|
||||
|| params.role != role
|
||||
{
|
||||
return failure(
|
||||
&request.request_id,
|
||||
worker_id,
|
||||
incarnation_id,
|
||||
None,
|
||||
DomainError::before(
|
||||
ErrorCode::IdentityMismatch,
|
||||
"this worker is not the session, worker or role the caller expected",
|
||||
),
|
||||
);
|
||||
}
|
||||
if !params.supported_majors.contains(&1) {
|
||||
return failure(
|
||||
&request.request_id,
|
||||
worker_id,
|
||||
incarnation_id,
|
||||
None,
|
||||
DomainError::before(ErrorCode::Unsupported, "no common major version"),
|
||||
);
|
||||
}
|
||||
let result = HelloResult {
|
||||
worker_id: worker_id.clone(),
|
||||
incarnation_id: incarnation_id.clone(),
|
||||
role,
|
||||
build_digest: build_digest(),
|
||||
contract_digest: contract_digest(),
|
||||
capabilities: capabilities.to_vec(),
|
||||
max_agents: MAX_AGENTS as u64,
|
||||
max_ports: MAX_PORTS as u64,
|
||||
};
|
||||
success(
|
||||
request,
|
||||
worker_id,
|
||||
incarnation_id,
|
||||
object(result.to_json()),
|
||||
)
|
||||
}
|
||||
|
||||
async fn acknowledge(
|
||||
request: &SessionRpcRequest,
|
||||
cache: &Arc<tokio::sync::Mutex<ResultCache>>,
|
||||
) -> DomainResult<Map<String, Value>> {
|
||||
let params: AcknowledgeParams = AcknowledgeParams::from_json(&request.params)
|
||||
.map_err(|e| DomainError::invalid(format!("Worker.Acknowledge: {e}")))?;
|
||||
if params.request_ids.is_empty() || params.request_ids.len() > MAX_ACKNOWLEDGE {
|
||||
return Err(DomainError::invalid("Worker.Acknowledge takes 1..=16 request ids"));
|
||||
}
|
||||
let acknowledged = {
|
||||
let mut c = cache.lock().await;
|
||||
c.acknowledge(¶ms.request_ids)
|
||||
};
|
||||
let result = AcknowledgeResult { acknowledged };
|
||||
Ok(object(result.to_json()))
|
||||
}
|
||||
96
services/flysim/crates/fly-session/tests/common/mod.rs
Normal file
96
services/flysim/crates/fly-session/tests/common/mod.rs
Normal file
|
|
@ -0,0 +1,96 @@
|
|||
//! Shared test fixture: the synthetic session on a temporary store, over either transport.
|
||||
|
||||
#![allow(dead_code)]
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use fly_session::harness::{HarnessConfig, SessionHarness, Via};
|
||||
use fly_session::types::*;
|
||||
|
||||
pub const WAIT: Duration = Duration::from_secs(20);
|
||||
|
||||
/// Generates one test per transport from an `async fn name(via: Via)`.
|
||||
#[macro_export]
|
||||
macro_rules! both_transports {
|
||||
($($name:ident),* $(,)?) => {
|
||||
mod in_memory {
|
||||
$(
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn $name() {
|
||||
super::$name($crate::common::via_memory()).await
|
||||
}
|
||||
)*
|
||||
}
|
||||
mod unix_socket {
|
||||
$(
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn $name() {
|
||||
super::$name($crate::common::via_unix()).await
|
||||
}
|
||||
)*
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
pub fn via_memory() -> Via {
|
||||
Via::Memory
|
||||
}
|
||||
|
||||
pub fn via_unix() -> Via {
|
||||
Via::Unix
|
||||
}
|
||||
|
||||
/// A started session plus the temporary directory its store lives in.
|
||||
pub struct Fixture {
|
||||
pub dir: tempfile::TempDir,
|
||||
pub harness: SessionHarness,
|
||||
}
|
||||
|
||||
impl Fixture {
|
||||
pub async fn shutdown(self) {
|
||||
let Fixture { dir, harness } = self;
|
||||
harness.shutdown().await;
|
||||
drop(dir);
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn fixture(via: Via, config: HarnessConfig) -> Fixture {
|
||||
let dir = tempfile::tempdir().expect("a temporary directory");
|
||||
let harness = SessionHarness::start(via, dir.path(), config)
|
||||
.await
|
||||
.expect("the synthetic session starts");
|
||||
Fixture { dir, harness }
|
||||
}
|
||||
|
||||
/// The default two-agent composition: 60 Hz world, 1 ms model tick.
|
||||
pub async fn default_fixture(via: Via) -> Fixture {
|
||||
fixture(via, HarnessConfig::default()).await
|
||||
}
|
||||
|
||||
pub fn fly_a() -> Id {
|
||||
id("fly-a")
|
||||
}
|
||||
|
||||
pub fn fly_b() -> Id {
|
||||
id("fly-b")
|
||||
}
|
||||
|
||||
/// The index of an audit entry, or a panic naming what was missing.
|
||||
pub fn at(audit: &[String], what: &str) -> usize {
|
||||
audit
|
||||
.iter()
|
||||
.position(|entry| entry == what)
|
||||
.unwrap_or_else(|| panic!("the audit has no {what:?}: {audit:?}"))
|
||||
}
|
||||
|
||||
pub fn count(audit: &[String], what: &str) -> usize {
|
||||
audit.iter().filter(|entry| *entry == what).count()
|
||||
}
|
||||
|
||||
/// Fails the test rather than hanging, so a missed reply is a failure and not a stuck job.
|
||||
pub async fn within<T>(what: &str, f: impl std::future::Future<Output = T>) -> T {
|
||||
match tokio::time::timeout(WAIT, f).await {
|
||||
Ok(v) => v,
|
||||
Err(_) => panic!("{what}: timed out"),
|
||||
}
|
||||
}
|
||||
373
services/flysim/crates/fly-session/tests/failures.rs
Normal file
373
services/flysim/crates/fly-session/tests/failures.rs
Normal file
|
|
@ -0,0 +1,373 @@
|
|||
//! The failure-injection rows of the implementation guide's section 4 that apply to
|
||||
//! SESSION-01, plus the `step-v1` section 7 rules they enforce.
|
||||
//!
|
||||
//! Most rows are proved by comparing an injected run with a clean run of the same
|
||||
//! composition: same seeds, same cadence, same number of steps. If the injected run's
|
||||
//! behaviour trace, model mutation counts and world counter are identical, then the injected
|
||||
//! message added no tick, no RNG draw, no stimulation, no reward and no world step.
|
||||
|
||||
mod common;
|
||||
|
||||
use common::{Fixture, at, fly_a, fly_b, within};
|
||||
use fly_session::agent::AgentFaults;
|
||||
use fly_session::coordinator::Injections;
|
||||
use fly_session::environment::EnvironmentFaults;
|
||||
use fly_session::harness::{HarnessConfig, Via};
|
||||
use fly_session::phase::Phase;
|
||||
use fly_session::types::*;
|
||||
|
||||
both_transports!(
|
||||
a_duplicate_prepare_after_a_lost_reply_repeats_nothing,
|
||||
a_duplicate_commit_replays_without_a_second_reinforcement,
|
||||
the_same_batch_with_altered_controls_conflicts,
|
||||
a_lost_advance_result_resolves_the_same_operation,
|
||||
a_cached_artifact_consumed_by_its_first_caller_survives_a_retry,
|
||||
one_commit_failing_after_another_succeeds_fails_the_epoch,
|
||||
a_replaced_registration_is_not_silently_reached,
|
||||
a_reply_from_another_incarnation_is_rejected,
|
||||
a_world_that_advanced_without_sensory_data_fails_the_transition,
|
||||
an_exact_duplicate_of_a_running_operation_is_in_progress,
|
||||
an_old_epoch_operation_is_refused_with_stale_epoch,
|
||||
);
|
||||
|
||||
const STEPS: u64 = 4;
|
||||
const INJECT_AT: u64 = 2;
|
||||
|
||||
/// What a run of the standard composition produced.
|
||||
struct Run {
|
||||
behaviour: Vec<String>,
|
||||
mutations: Vec<(Id, u64)>,
|
||||
counter: i64,
|
||||
advances: u64,
|
||||
injections: Vec<fly_session::coordinator::InjectionOutcome>,
|
||||
in_progress: u64,
|
||||
}
|
||||
|
||||
async fn run_with(via: Via, injections: Injections) -> Run {
|
||||
let mut f = clean_fixture(via).await;
|
||||
f.harness.coordinator.injections = injections;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("run", f.harness.coordinator.run(STEPS)).await.unwrap();
|
||||
let run = Run {
|
||||
behaviour: f.harness.coordinator.trace.behavior(),
|
||||
mutations: vec![
|
||||
(fly_a(), f.harness.agent_mutations(&fly_a())),
|
||||
(fly_b(), f.harness.agent_mutations(&fly_b())),
|
||||
],
|
||||
counter: f
|
||||
.harness
|
||||
.coordinator
|
||||
.task_progress()
|
||||
.integer("counter")
|
||||
.unwrap(),
|
||||
advances: f.harness.coordinator.stats().advances,
|
||||
injections: f.harness.coordinator.injection_log.clone(),
|
||||
in_progress: f.harness.coordinator.in_progress_replies,
|
||||
};
|
||||
f.shutdown().await;
|
||||
run
|
||||
}
|
||||
|
||||
async fn clean_fixture(via: Via) -> Fixture {
|
||||
common::fixture(via, HarnessConfig::default()).await
|
||||
}
|
||||
|
||||
fn assert_same(injected: &Run, clean: &Run, what: &str) {
|
||||
assert_eq!(injected.behaviour, clean.behaviour, "{what}: behaviour trace");
|
||||
assert_eq!(injected.mutations, clean.mutations, "{what}: model mutations");
|
||||
assert_eq!(injected.counter, clean.counter, "{what}: world counter");
|
||||
assert_eq!(injected.advances, clean.advances, "{what}: world advances");
|
||||
assert_eq!(injected.advances, STEPS, "{what}: one advance per batch");
|
||||
assert_eq!(clean.in_progress, 0, "{what}: a clean run never meets a duplicate");
|
||||
}
|
||||
|
||||
/// Row: duplicate Prepare after a lost reply. No extra ticks, RNG draws, stimulation or
|
||||
/// decode, and the cached decision comes back unchanged.
|
||||
async fn a_duplicate_prepare_after_a_lost_reply_repeats_nothing(via: Via) {
|
||||
let clean = run_with(via, Injections::default()).await;
|
||||
let injected = run_with(
|
||||
via,
|
||||
Injections {
|
||||
at_step: INJECT_AT,
|
||||
duplicate_prepare: Some(fly_a()),
|
||||
..Injections::default()
|
||||
},
|
||||
)
|
||||
.await;
|
||||
let probe = injected
|
||||
.injections
|
||||
.iter()
|
||||
.find(|o| o.what == "duplicate-prepare")
|
||||
.expect("the duplicate was sent");
|
||||
assert_eq!(probe.code, None, "a safe replay is a success, not an error");
|
||||
assert!(probe.identical, "the replay returned the same decision, ticks and remainder");
|
||||
assert_same(&injected, &clean, "duplicate prepare");
|
||||
}
|
||||
|
||||
/// Row: the same Commit again. It replays its cached reply and reinforces nothing twice.
|
||||
async fn a_duplicate_commit_replays_without_a_second_reinforcement(via: Via) {
|
||||
let clean = run_with(via, Injections::default()).await;
|
||||
let injected = run_with(
|
||||
via,
|
||||
Injections {
|
||||
at_step: INJECT_AT,
|
||||
duplicate_commit: Some(fly_b()),
|
||||
..Injections::default()
|
||||
},
|
||||
)
|
||||
.await;
|
||||
let probe = injected
|
||||
.injections
|
||||
.iter()
|
||||
.find(|o| o.what == "duplicate-commit")
|
||||
.expect("the duplicate was sent");
|
||||
assert_eq!(probe.code, None);
|
||||
assert!(probe.identical, "the replay returned the same committed step and telemetry");
|
||||
assert_same(&injected, &clean, "duplicate commit");
|
||||
}
|
||||
|
||||
/// Row: the same batch with altered controls. A conflict, never a second world mutation.
|
||||
async fn the_same_batch_with_altered_controls_conflicts(via: Via) {
|
||||
let clean = run_with(via, Injections::default()).await;
|
||||
let injected = run_with(
|
||||
via,
|
||||
Injections {
|
||||
at_step: INJECT_AT,
|
||||
altered_advance_controls: true,
|
||||
..Injections::default()
|
||||
},
|
||||
)
|
||||
.await;
|
||||
let probe = injected
|
||||
.injections
|
||||
.iter()
|
||||
.find(|o| o.what == "altered-advance-controls")
|
||||
.expect("the altered batch was sent");
|
||||
assert_eq!(probe.code, Some(ErrorCode::Conflict));
|
||||
assert_same(&injected, &clean, "altered controls");
|
||||
}
|
||||
|
||||
/// Row: the Advance result is lost after the world stepped. The same operation is resolved
|
||||
/// against its original request id; no new batch is ever sent.
|
||||
async fn a_lost_advance_result_resolves_the_same_operation(via: Via) {
|
||||
let clean = run_with(via, Injections::default()).await;
|
||||
let injected = run_with(
|
||||
via,
|
||||
Injections { at_step: INJECT_AT, lose_advance_result: true, ..Injections::default() },
|
||||
)
|
||||
.await;
|
||||
assert!(
|
||||
injected
|
||||
.injections
|
||||
.iter()
|
||||
.any(|o| o.what == "lost-advance-result" && o.identical),
|
||||
"the call was abandoned after dispatch, with an uncertain outcome"
|
||||
);
|
||||
assert!(
|
||||
injected.injections.iter().any(|o| o.what == "status-after-loss" && o.identical),
|
||||
"the uncertain call was probed with Status before being resolved"
|
||||
);
|
||||
assert_same(&injected, &clean, "lost advance result");
|
||||
}
|
||||
|
||||
/// Row: the cached RPC artifact is consumed by its first caller. The endpoint's domain cache
|
||||
/// still owns it, so the retry gets valid bytes.
|
||||
async fn a_cached_artifact_consumed_by_its_first_caller_survives_a_retry(via: Via) {
|
||||
let clean = run_with(via, Injections::default()).await;
|
||||
let injected = run_with(
|
||||
via,
|
||||
Injections {
|
||||
at_step: INJECT_AT,
|
||||
consume_advance_artifact_then_retry: true,
|
||||
..Injections::default()
|
||||
},
|
||||
)
|
||||
.await;
|
||||
let probe = injected
|
||||
.injections
|
||||
.iter()
|
||||
.find(|o| o.what == "cached-artifact-after-consumption")
|
||||
.expect("the frame was read, released and replayed");
|
||||
assert!(probe.identical, "the replayed frame has the same bytes as the consumed one");
|
||||
assert_same(&injected, &clean, "cached artifact");
|
||||
}
|
||||
|
||||
/// Row: one Commit fails after another succeeds. No next world step, and the epoch fails
|
||||
/// rather than continuing with a partial match.
|
||||
async fn one_commit_failing_after_another_succeeds_fails_the_epoch(via: Via) {
|
||||
let mut config = HarnessConfig::default();
|
||||
// fly-a commits quickly and succeeds; fly-b fails after its next input was installed.
|
||||
config.agents[1].faults =
|
||||
AgentFaults { fail_commit_at_step: Some(1), commit_delay_ms: 15, ..AgentFaults::default() };
|
||||
let mut f = common::fixture(via, config).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("step", f.harness.coordinator.step()).await.unwrap();
|
||||
let err = within("failing step", f.harness.coordinator.step())
|
||||
.await
|
||||
.expect_err("the epoch fails when one commit fails");
|
||||
assert_eq!(err.error.code, ErrorCode::BackendFailure);
|
||||
assert_eq!(err.error.mutation, MutationCertainty::Applied);
|
||||
assert_eq!(f.harness.coordinator.phase(), Phase::Failed);
|
||||
|
||||
// The world took the step whose commits failed, and it takes no further step.
|
||||
let advances = f.harness.coordinator.stats().advances;
|
||||
assert_eq!(advances, 1, "the failed transition never reached a committed boundary");
|
||||
let env = f.harness.coordinator.environment_ref().clone();
|
||||
let status = within("status", f.harness.coordinator.status(&env)).await.unwrap();
|
||||
assert_eq!(status.current_scope.as_ref().unwrap().step, 2);
|
||||
let again = f.harness.coordinator.step().await.expect_err("no step from Failed");
|
||||
assert_eq!(again.error.code, ErrorCode::InvalidPhase);
|
||||
let status = within("status", f.harness.coordinator.status(&env)).await.unwrap();
|
||||
assert_eq!(
|
||||
status.current_scope.unwrap().step,
|
||||
2,
|
||||
"no world step follows a partial commit"
|
||||
);
|
||||
|
||||
// The agent that succeeded is at the new boundary; the one that failed reports Failed.
|
||||
let a = f.harness.coordinator.agent_ref(&fly_a()).cloned().unwrap();
|
||||
let status = within("status", f.harness.coordinator.status(&a)).await.unwrap();
|
||||
assert_eq!(status.current_scope.unwrap().step, 2);
|
||||
assert_eq!(
|
||||
f.harness.agent_status(&fly_b()).unwrap().state(),
|
||||
WorkerState::Failed
|
||||
);
|
||||
// Nothing was published for the boundary that failed to commit.
|
||||
let audit = f.harness.coordinator.audit.clone();
|
||||
assert!(!audit.iter().any(|entry| entry == "publish:2"));
|
||||
assert!(at(&audit, "publish:1") < at(&audit, "advance:1"));
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// Row: an old worker is replaced. The coordinator pinned the old registration, so its next
|
||||
/// call fails rather than silently reaching another brain.
|
||||
async fn a_replaced_registration_is_not_silently_reached(via: Via) {
|
||||
let mut f = clean_fixture(via).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("step", f.harness.coordinator.step()).await.unwrap();
|
||||
let restarted = f.harness.restart_agent(&fly_b()).await.unwrap();
|
||||
let pinned = f.harness.coordinator.agent_ref(&fly_b()).cloned().unwrap();
|
||||
assert_ne!(
|
||||
restarted.service_incarnation, pinned.bus_incarnation,
|
||||
"a replacement registration is a new incarnation"
|
||||
);
|
||||
let err = within("step after restart", f.harness.coordinator.step())
|
||||
.await
|
||||
.expect_err("the pinned incarnation is gone");
|
||||
assert_eq!(err.error.code, ErrorCode::IdentityMismatch);
|
||||
assert_eq!(f.harness.coordinator.phase(), Phase::Failed);
|
||||
assert_eq!(f.harness.coordinator.stats().advances, 1, "no world step under a lost pin");
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// Row: a reply carrying another domain incarnation is rejected, even when the bus route is
|
||||
/// live and answering.
|
||||
async fn a_reply_from_another_incarnation_is_rejected(via: Via) {
|
||||
let mut f = clean_fixture(via).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
let old = f.harness.coordinator.agent_ref(&fly_b()).cloned().unwrap();
|
||||
let restarted = f.harness.restart_agent(&fly_b()).await.unwrap();
|
||||
// Follow the new registration, but keep pinning the incarnation the old worker negotiated.
|
||||
let stale = fly_session::rpc::WorkerRef {
|
||||
service: restarted.service.clone(),
|
||||
bus_incarnation: restarted.service_incarnation.clone(),
|
||||
worker_id: fly_b(),
|
||||
domain_incarnation: old.domain_incarnation.clone(),
|
||||
};
|
||||
assert_ne!(old.domain_incarnation, Some(restarted.incarnation_id.clone()));
|
||||
let err = within("status", f.harness.coordinator.status(&stale))
|
||||
.await
|
||||
.expect_err("the replacement is not the negotiated incarnation");
|
||||
assert_eq!(err.error.code, ErrorCode::IdentityMismatch);
|
||||
assert_eq!(f.harness.coordinator.phase(), Phase::Failed);
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// `step-v1` section 7: the world advanced but the required sensory data is unavailable. The
|
||||
/// transition fails; nothing is rewarded or continued on guessed input.
|
||||
async fn a_world_that_advanced_without_sensory_data_fails_the_transition(via: Via) {
|
||||
let config = HarnessConfig {
|
||||
environment_faults: EnvironmentFaults {
|
||||
omit_view_at_boundary: Some(1),
|
||||
..EnvironmentFaults::default()
|
||||
},
|
||||
..HarnessConfig::default()
|
||||
};
|
||||
let mut f = common::fixture(via, config).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
let err = within("step", f.harness.coordinator.step())
|
||||
.await
|
||||
.expect_err("a missing required view is not silently replaced");
|
||||
assert_eq!(err.error.code, ErrorCode::BufferInvalid);
|
||||
assert_eq!(f.harness.coordinator.phase(), Phase::Failed);
|
||||
// The task never interpreted the transition, so nothing was rewarded.
|
||||
assert_eq!(f.harness.coordinator.evaluations(), 0);
|
||||
assert_eq!(
|
||||
f.harness.coordinator.task_progress().number("totalReward").unwrap(),
|
||||
0.0
|
||||
);
|
||||
let audit = f.harness.coordinator.audit.clone();
|
||||
assert!(!audit.iter().any(|entry| entry.starts_with("committed:")));
|
||||
assert!(!audit.iter().any(|entry| entry == "publish:1"));
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// `ipc-v1` section 5: an exact duplicate arriving while the original is still executing gets
|
||||
/// IN_PROGRESS for that bus call, and the original completes normally.
|
||||
async fn an_exact_duplicate_of_a_running_operation_is_in_progress(via: Via) {
|
||||
let config = HarnessConfig {
|
||||
environment_faults: EnvironmentFaults {
|
||||
// The world moves, then the reply is held, so the resolution attempt lands while
|
||||
// the original operation is still active.
|
||||
advance_delay_ms: 120,
|
||||
..EnvironmentFaults::default()
|
||||
},
|
||||
..HarnessConfig::default()
|
||||
};
|
||||
let mut f = common::fixture(via, config).await;
|
||||
f.harness.coordinator.injections =
|
||||
Injections { at_step: 0, lose_advance_result: true, ..Injections::default() };
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("step", f.harness.coordinator.step()).await.unwrap();
|
||||
assert!(
|
||||
f.harness.coordinator.in_progress_replies > 0,
|
||||
"the duplicate met the original still running"
|
||||
);
|
||||
// And the original still completed: exactly one world step, at one boundary.
|
||||
assert_eq!(f.harness.coordinator.stats().advances, 1);
|
||||
assert_eq!(f.harness.coordinator.observation().unwrap().boundary, 1);
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// A live worker under this epoch refuses an operation naming another one.
|
||||
async fn an_old_epoch_operation_is_refused_with_stale_epoch(via: Via) {
|
||||
let mut f = clean_fixture(via).await;
|
||||
let harness = &mut f.harness;
|
||||
within("bootstrap", harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("step", harness.coordinator.step()).await.unwrap();
|
||||
|
||||
// The agent is live and initialized under epoch e1. An operation naming another epoch is
|
||||
// refused as stale rather than applied to this brain.
|
||||
let worker = harness.coordinator.agent_ref(&fly_a()).cloned().unwrap();
|
||||
let scope = scope_at("demo", "e0", 1);
|
||||
let params = serde_json::json!({
|
||||
"agentId": "fly-a",
|
||||
"profileDigest": digest_of_bytes(b"whatever").to_string(),
|
||||
"interval": {"numerator": "16666667", "denominator": "1"},
|
||||
"decisionContextDigest": digest_of_bytes(b"whatever").to_string(),
|
||||
"preStepStimulations": [],
|
||||
});
|
||||
let bus = harness.client("coordinator2").await;
|
||||
// The launcher grants no second coordinator, so the old-epoch probe goes through the
|
||||
// session's own client instead.
|
||||
assert!(bus.is_err(), "an unconfigured client id is refused before it can route");
|
||||
let err = within(
|
||||
"stale epoch",
|
||||
harness.coordinator.probe_raw(&worker, "Agent.Prepare", Some(scope), params),
|
||||
)
|
||||
.await
|
||||
.expect_err("an old epoch cannot mutate this worker");
|
||||
assert_eq!(err.code, ErrorCode::StaleEpoch);
|
||||
assert_eq!(harness.coordinator.stats().advances, 1);
|
||||
f.shutdown().await;
|
||||
}
|
||||
404
services/flysim/crates/fly-session/tests/session.rs
Normal file
404
services/flysim/crates/fly-session/tests/session.rs
Normal file
|
|
@ -0,0 +1,404 @@
|
|||
//! SESSION-01 acceptance: the synthetic sequential transaction, over both transports.
|
||||
//!
|
||||
//! Every test here is one of the acceptance bullets of the implementation guide's SESSION-01
|
||||
//! slice, or one of the initialization, pause and episode rules of `step-v1` section 6.
|
||||
|
||||
mod common;
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use common::{Fixture, at, count, default_fixture, fixture, fly_a, fly_b, within};
|
||||
use fly_session::coordinator::DispatchOrder;
|
||||
use fly_session::harness::{AgentSpec, HarnessConfig, Via};
|
||||
use fly_session::phase::Phase;
|
||||
use fly_session::types::*;
|
||||
use fly_session::agent::AgentFaults;
|
||||
|
||||
both_transports!(
|
||||
one_world_advance_per_complete_batch,
|
||||
every_agent_is_prepared_before_the_world_advances,
|
||||
the_task_evaluates_each_transition_once,
|
||||
every_agent_commits_before_the_next_prepare_or_publication,
|
||||
a_60_hz_world_with_a_1_ms_tick_runs_16_17_17,
|
||||
a_pause_mid_step_completes_the_step_and_pauses_at_the_boundary,
|
||||
bootstrap_cannot_advance_the_world_or_produce_a_reward,
|
||||
the_committed_snapshot_names_the_boundary_that_just_ended,
|
||||
a_terminal_episode_pauses_at_its_own_boundary,
|
||||
status_answers_with_the_committed_boundary,
|
||||
a_worker_refuses_a_second_initialize,
|
||||
a_single_agent_composition_runs_the_same_transaction,
|
||||
);
|
||||
|
||||
const STEPS: u64 = 3;
|
||||
|
||||
/// One `Environment.Advance` per complete batch, and one boundary per advance.
|
||||
async fn one_world_advance_per_complete_batch(via: Via) {
|
||||
let mut f = default_fixture(via).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
let before = f.harness.environment_mutations();
|
||||
let reports = within("run", f.harness.coordinator.run(STEPS)).await.unwrap();
|
||||
assert_eq!(reports.len() as u64, STEPS);
|
||||
assert_eq!(f.harness.coordinator.stats().advances, STEPS);
|
||||
assert_eq!(f.harness.coordinator.phase(), Phase::Ready(STEPS));
|
||||
assert_eq!(
|
||||
f.harness.coordinator.observation().unwrap().boundary,
|
||||
STEPS,
|
||||
"the world is at exactly one boundary per batch"
|
||||
);
|
||||
// The environment's progress counter moves once per advance and not otherwise.
|
||||
assert_eq!(f.harness.environment_mutations() - before, STEPS);
|
||||
assert_eq!(count(&f.harness.coordinator.audit, "advance:0"), 1);
|
||||
assert_eq!(f.harness.coordinator.trace.transitions.len() as u64, STEPS);
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// Every agent reaches Prepared before the batch is built and the world advances.
|
||||
async fn every_agent_is_prepared_before_the_world_advances(via: Via) {
|
||||
// Different completion delays, so "all prepared" cannot be an accident of timing.
|
||||
let mut config = HarnessConfig::default();
|
||||
config.agents[0].faults = AgentFaults { prepare_delay_ms: 15, ..AgentFaults::default() };
|
||||
config.agents[1].faults = AgentFaults { prepare_delay_ms: 1, ..AgentFaults::default() };
|
||||
let mut f = fixture(via, config).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("run", f.harness.coordinator.run(STEPS)).await.unwrap();
|
||||
let audit = f.harness.coordinator.audit.clone();
|
||||
for k in 0..STEPS {
|
||||
let advance = at(&audit, &format!("advance:{k}"));
|
||||
for agent in [fly_a(), fly_b()] {
|
||||
let prepared = at(&audit, &format!("prepared:{agent}@{k}"));
|
||||
assert!(
|
||||
prepared < advance,
|
||||
"{agent} must be Prepared({k}) before the world advances: {audit:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// The task's transition evaluation runs exactly once per acknowledged world step.
|
||||
async fn the_task_evaluates_each_transition_once(via: Via) {
|
||||
let mut f = default_fixture(via).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
assert_eq!(f.harness.coordinator.evaluations(), 0, "bootstrap evaluates no transition");
|
||||
within("run", f.harness.coordinator.run(STEPS)).await.unwrap();
|
||||
assert_eq!(f.harness.coordinator.evaluations(), STEPS);
|
||||
let audit = f.harness.coordinator.audit.clone();
|
||||
for k in 0..STEPS {
|
||||
assert_eq!(count(&audit, &format!("evaluate:{k}")), 1);
|
||||
}
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// No agent starts the next Prepare, and nothing is published as committed, until every agent
|
||||
/// has committed this transition.
|
||||
async fn every_agent_commits_before_the_next_prepare_or_publication(via: Via) {
|
||||
let mut config = HarnessConfig::default();
|
||||
config.agents[0].faults = AgentFaults { commit_delay_ms: 12, ..AgentFaults::default() };
|
||||
let mut f = fixture(via, config).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("run", f.harness.coordinator.run(STEPS)).await.unwrap();
|
||||
let audit = f.harness.coordinator.audit.clone();
|
||||
for k in 0..STEPS {
|
||||
let publish = at(&audit, &format!("publish:{}", k + 1));
|
||||
for agent in [fly_a(), fly_b()] {
|
||||
let committed = at(&audit, &format!("committed:{agent}@{k}"));
|
||||
assert!(
|
||||
committed < publish,
|
||||
"{agent} must commit before boundary {} is published: {audit:?}",
|
||||
k + 1
|
||||
);
|
||||
if k + 1 < STEPS {
|
||||
let next = at(&audit, &format!("prepared:{agent}@{}", k + 1));
|
||||
for other in [fly_a(), fly_b()] {
|
||||
let other_commit = at(&audit, &format!("committed:{other}@{k}"));
|
||||
assert!(
|
||||
other_commit < next,
|
||||
"{other} must commit step {k} before {agent} prepares {}: {audit:?}",
|
||||
k + 1
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(f.harness.coordinator.stats().publications, STEPS + 1, "one per boundary, plus 0");
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// `step-v1` section 5: a 60 Hz world with a 1 ms model tick runs 16, 17, 17 ticks over three
|
||||
/// steps, totalling 50, with a remainder of exactly zero.
|
||||
async fn a_60_hz_world_with_a_1_ms_tick_runs_16_17_17(via: Via) {
|
||||
let mut f = default_fixture(via).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("run", f.harness.coordinator.run(3)).await.unwrap();
|
||||
let transitions = &f.harness.coordinator.trace.transitions;
|
||||
assert_eq!(transitions.len(), 3);
|
||||
for agent in [fly_a(), fly_b()] {
|
||||
let ticks: Vec<u64> = transitions
|
||||
.iter()
|
||||
.map(|t| {
|
||||
t.behaviour
|
||||
.agents
|
||||
.iter()
|
||||
.find(|a| a.agent_id == agent)
|
||||
.expect("the agent is in every transition")
|
||||
.ticks_advanced
|
||||
})
|
||||
.collect();
|
||||
assert_eq!(ticks, vec![16, 17, 17], "{agent} tick profile");
|
||||
assert_eq!(ticks.iter().sum::<u64>(), 50);
|
||||
let last = transitions
|
||||
.last()
|
||||
.unwrap()
|
||||
.behaviour
|
||||
.agents
|
||||
.iter()
|
||||
.find(|a| a.agent_id == agent)
|
||||
.unwrap();
|
||||
assert!(last.remainder.is_zero(), "{agent} remainder after three steps");
|
||||
// Warm-up ticks are counted too, so brainTicks is warm-up plus the 50 gameplay ticks.
|
||||
assert_eq!(last.brain_ticks, 50 + f.harness.config.warmup_ticks);
|
||||
}
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// A pause arriving mid-step means "finish this transition, then pause", and it pauses at the
|
||||
/// committed boundary rather than truncating anything.
|
||||
async fn a_pause_mid_step_completes_the_step_and_pauses_at_the_boundary(via: Via) {
|
||||
let mut config = HarnessConfig::default();
|
||||
// Both agents hold their Commit open, so the pause request lands inside the transition.
|
||||
config.agents[0].faults = AgentFaults { commit_delay_ms: 40, ..AgentFaults::default() };
|
||||
config.agents[1].faults = AgentFaults { commit_delay_ms: 60, ..AgentFaults::default() };
|
||||
let mut f = fixture(via, config).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("step", f.harness.coordinator.step()).await.unwrap();
|
||||
|
||||
// A supervisor asks for a pause while the second transition is still running.
|
||||
let handle = f.harness.coordinator.pause_handle();
|
||||
let asked = tokio::spawn(async move {
|
||||
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
|
||||
handle.request();
|
||||
});
|
||||
let report = within("paused step", f.harness.coordinator.step()).await.unwrap();
|
||||
asked.await.unwrap();
|
||||
|
||||
// The transition completed and the session paused at its committed boundary.
|
||||
assert_eq!(report.boundary, 2);
|
||||
assert!(report.paused);
|
||||
assert_eq!(f.harness.coordinator.phase(), Phase::Paused(2));
|
||||
assert!(f.harness.coordinator.phase().is_committed_boundary());
|
||||
assert_eq!(f.harness.coordinator.stats().advances, 2, "the pause truncated no transition");
|
||||
let audit = f.harness.coordinator.audit.clone();
|
||||
let pause = at(&audit, "pause:2");
|
||||
for agent in [fly_a(), fly_b()] {
|
||||
assert!(at(&audit, &format!("committed:{agent}@1")) < pause);
|
||||
}
|
||||
assert!(at(&audit, "publish:2") < pause);
|
||||
|
||||
// A paused worker retains its state and answers Status; the world does not advance.
|
||||
let env = f.harness.coordinator.environment_ref().clone();
|
||||
let status = within("status", f.harness.coordinator.status(&env)).await.unwrap();
|
||||
assert_eq!(status.current_scope.unwrap().step, 2);
|
||||
assert_eq!(f.harness.coordinator.stats().advances, 2);
|
||||
|
||||
f.harness.coordinator.resume().unwrap();
|
||||
assert_eq!(f.harness.coordinator.phase(), Phase::Ready(2));
|
||||
let report = within("resumed step", f.harness.coordinator.step()).await.unwrap();
|
||||
assert_eq!(report.boundary, 3);
|
||||
assert!(!report.paused, "the pause request was consumed by the pause it caused");
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// Bootstrap and warm-up mutate the fake brains but cannot advance the environment or produce
|
||||
/// a gameplay reward.
|
||||
async fn bootstrap_cannot_advance_the_world_or_produce_a_reward(via: Via) {
|
||||
let mut f = default_fixture(via).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
assert_eq!(f.harness.coordinator.phase(), Phase::Ready(0));
|
||||
let observation = f.harness.coordinator.observation().unwrap();
|
||||
assert_eq!(observation.boundary, 0);
|
||||
assert!(observation.world_time.is_zero());
|
||||
assert_eq!(f.harness.coordinator.stats().advances, 0);
|
||||
assert_eq!(f.harness.coordinator.evaluations(), 0);
|
||||
// The environment advanced nothing, so its status is still at boundary 0 with no batch.
|
||||
let env = f.harness.coordinator.environment_ref().clone();
|
||||
let status = within("status", f.harness.coordinator.status(&env)).await.unwrap();
|
||||
assert_eq!(status.current_scope.as_ref().unwrap().step, 0);
|
||||
assert!(status.last_batch_id.is_none(), "no batch was ever applied");
|
||||
// Warm-up did run, with learning disabled, so the models did mutate.
|
||||
for agent in [fly_a(), fly_b()] {
|
||||
assert!(
|
||||
f.harness.agent_mutations(&agent) >= f.harness.config.warmup_ticks,
|
||||
"warm-up ticks are real mutations"
|
||||
);
|
||||
}
|
||||
// And the task ledger has no reward yet.
|
||||
let progress = f.harness.coordinator.task_progress();
|
||||
assert_eq!(progress.number("totalReward").unwrap(), 0.0);
|
||||
assert_eq!(progress.integer("transitions").unwrap(), 0);
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// The published snapshot represents the committed boundary and labels the transition that
|
||||
/// just ended.
|
||||
async fn the_committed_snapshot_names_the_boundary_that_just_ended(via: Via) {
|
||||
let mut f = default_fixture(via).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
// The snapshots topic retains its latest value, so a subscriber joining after boundary 0
|
||||
// still replays it before the boundaries that follow.
|
||||
let observer = f.harness.observer().await.unwrap();
|
||||
let topic = f.harness.coordinator.topics().snapshots.clone();
|
||||
let mut subscription = observer
|
||||
.subscribe(
|
||||
&topic,
|
||||
flybus::SubscriptionConfig::bounded().in_flight(8).replay(true),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
within("run", f.harness.coordinator.run(2)).await.unwrap();
|
||||
|
||||
let mut boundaries = Vec::new();
|
||||
for _ in 0..3 {
|
||||
let message = within("snapshot", subscription.next()).await.expect("a snapshot");
|
||||
let payload = message.payload().clone();
|
||||
let step: u64 = payload["scope"]["step"].as_str().unwrap().parse().unwrap();
|
||||
let decisions_present = payload["agents"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.all(|a| !a["selectedDecision"].is_null());
|
||||
boundaries.push((step, decisions_present));
|
||||
// The frame the snapshot names travels as an owned attachment.
|
||||
if step > 0 {
|
||||
let frame = message.artifact("view.arena").expect("the published frame");
|
||||
assert_eq!(frame.reference().byte_length, 4 * 4 * 4);
|
||||
}
|
||||
}
|
||||
assert_eq!(boundaries[0], (0, false), "boundary 0 has no decision or control");
|
||||
assert_eq!(boundaries[1], (1, true));
|
||||
assert_eq!(boundaries[2], (2, true));
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// A terminal task event is evaluated, its rewards committed once, and the session pauses at
|
||||
/// that boundary before any further gameplay transition.
|
||||
async fn a_terminal_episode_pauses_at_its_own_boundary(via: Via) {
|
||||
let config = HarnessConfig {
|
||||
terminal: fly_session::task::Terminal::AfterTransitions(3),
|
||||
..HarnessConfig::default()
|
||||
};
|
||||
let mut f = fixture(via, config).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
let reports = within("run", f.harness.coordinator.run(5)).await.unwrap();
|
||||
let last = reports.last().unwrap();
|
||||
assert!(last.terminal, "the counter task asked for a terminal transition");
|
||||
assert!(last.paused);
|
||||
assert_eq!(f.harness.coordinator.phase(), Phase::Paused(last.boundary));
|
||||
assert!(f.harness.coordinator.episode_request().is_some());
|
||||
// No worker resets itself, and no further gameplay transition is allowed.
|
||||
let err = f.harness.coordinator.step().await.expect_err("no transition after terminal");
|
||||
assert_eq!(err.error.code, ErrorCode::InvalidPhase);
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// `Worker.Status` answers with the worker's own committed boundary and progress.
|
||||
async fn status_answers_with_the_committed_boundary(via: Via) {
|
||||
let mut f = default_fixture(via).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("run", f.harness.coordinator.run(2)).await.unwrap();
|
||||
for agent in [fly_a(), fly_b()] {
|
||||
let worker = f.harness.coordinator.agent_ref(&agent).cloned().unwrap();
|
||||
let status = within("status", f.harness.coordinator.status(&worker)).await.unwrap();
|
||||
assert_eq!(status.state, WorkerState::Ready);
|
||||
assert_eq!(status.current_scope.unwrap().step, 2);
|
||||
let before = status.progress_counter;
|
||||
// A status query is not progress.
|
||||
let again = within("status", f.harness.coordinator.status(&worker)).await.unwrap();
|
||||
assert_eq!(again.progress_counter, before);
|
||||
}
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
/// `Agent.Initialize` is allowed only on an uninitialized agent.
|
||||
async fn a_worker_refuses_a_second_initialize(via: Via) {
|
||||
let mut f = default_fixture(via).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
let err = within("second bootstrap", f.harness.coordinator.bootstrap())
|
||||
.await
|
||||
.expect_err("the environment is already initialized");
|
||||
assert_eq!(err.error.code, ErrorCode::InvalidPhase);
|
||||
f.shutdown().await;
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------------------------
|
||||
// Dispatch order equivalence: this one builds its own fixtures per order.
|
||||
|
||||
/// `step-v1` section 8: sequential, concurrent and reversed dispatch and completion orders all
|
||||
/// produce the same behaviour trace, excluding request ids and other operational metadata.
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn sequential_concurrent_and_reversed_orders_agree() {
|
||||
let mut behaviours: BTreeMap<String, Vec<String>> = BTreeMap::new();
|
||||
for via in [Via::Memory, Via::Unix] {
|
||||
for order in [
|
||||
DispatchOrder::Sequential,
|
||||
DispatchOrder::Concurrent,
|
||||
DispatchOrder::Reversed,
|
||||
] {
|
||||
let mut config = HarnessConfig::default();
|
||||
// Deliberately unequal completion times, so a concurrent run really does finish
|
||||
// out of dispatch order.
|
||||
config.agents[0].faults =
|
||||
AgentFaults { prepare_delay_ms: 12, commit_delay_ms: 0, ..AgentFaults::default() };
|
||||
config.agents[1].faults =
|
||||
AgentFaults { prepare_delay_ms: 0, commit_delay_ms: 9, ..AgentFaults::default() };
|
||||
let mut f = fixture(via, config).await;
|
||||
f.harness.coordinator.dispatch = order;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("run", f.harness.coordinator.run(4)).await.unwrap();
|
||||
let behaviour = f.harness.coordinator.trace.behavior();
|
||||
assert_eq!(behaviour.len(), 4);
|
||||
behaviours.insert(format!("{via:?}/{order:?}"), behaviour);
|
||||
// The operational metadata is recorded but is not part of the comparison.
|
||||
// The operational metadata is recorded beside the behaviour, not inside it.
|
||||
let operational = &f.harness.coordinator.trace.transitions[0].operational;
|
||||
assert_eq!(operational.prepare_request_ids.len(), 2);
|
||||
assert_eq!(operational.commit_request_ids.len(), 2);
|
||||
f.shutdown().await;
|
||||
}
|
||||
}
|
||||
let mut iter = behaviours.iter();
|
||||
let (first_name, first) = iter.next().expect("at least one run");
|
||||
for (name, behaviour) in iter {
|
||||
assert_eq!(
|
||||
behaviour, first,
|
||||
"{name} produced a different behaviour trace from {first_name}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// A one-agent composition still runs the same transaction, so the barrier is not two-agent
|
||||
/// specific.
|
||||
async fn a_single_agent_composition_runs_the_same_transaction(via: Via) {
|
||||
let config = HarnessConfig {
|
||||
agents: vec![AgentSpec {
|
||||
agent_id: id("fly-a"),
|
||||
port_id: id("p1"),
|
||||
seed: 7,
|
||||
faults: AgentFaults::default(),
|
||||
}],
|
||||
..HarnessConfig::default()
|
||||
};
|
||||
let mut f: Fixture = fixture(via, config).await;
|
||||
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
|
||||
within("run", f.harness.coordinator.run(3)).await.unwrap();
|
||||
assert_eq!(f.harness.coordinator.stats().advances, 3);
|
||||
let ticks: Vec<u64> = f
|
||||
.harness
|
||||
.coordinator
|
||||
.trace
|
||||
.transitions
|
||||
.iter()
|
||||
.map(|t| t.behaviour.agents[0].ticks_advanced)
|
||||
.collect();
|
||||
assert_eq!(ticks, vec![16, 17, 17]);
|
||||
f.shutdown().await;
|
||||
}
|
||||
Loading…
Add table
Reference in a new issue