Merge feat/sf-contract-01 at f043cd7: packages/session-types, the canonical number rule and the seed/envelope specifications

This commit is contained in:
acamilo 2026-09-22 13:03:56 +00:00
commit 574e9fec0c
32 changed files with 4792 additions and 24 deletions

View file

@ -34,6 +34,15 @@ retroactively to existing [public feed](../../feed-protocol.md),
6. [Application/presentation boundary](publishing-v1.md) — snapshots, flexible data and effects. 6. [Application/presentation boundary](publishing-v1.md) — snapshots, flexible data and effects.
7. [Implementation guide](implementation.md) — sequenced build tasks and acceptance tests. 7. [Implementation guide](implementation.md) — sequenced build tasks and acceptance tests.
Two derived specifications, written by CONTRACT-01 because the slices that need them cannot
be built without them:
- [Seed derivation v1](seed-derivation-v1.md) — independent per-agent seeds from one recorded
master seed and stable agent ids, with test vectors in both languages.
- [Checkpoint envelope v1](checkpoint-envelope-v1.md) — the exact bytes of the new `FLYSESS1`
envelope and the durable commit sequence. `FLYSIM01` is unchanged and stays separately
readable.
For context: [modular-session analysis](../malecns-modular-sessions.md) and For context: [modular-session analysis](../malecns-modular-sessions.md) and
[Melee audit](../melee-framework-audit.md). Each contract owns its named subject; step ordering [Melee audit](../melee-framework-audit.md). Each contract owns its named subject; step ordering
wins over an informal diagram, and Flybus owns transport/resource rules. Resolve contradictions wins over an informal diagram, and Flybus owns transport/resource rules. Resolve contradictions

View file

@ -0,0 +1,181 @@
# Checkpoint envelope v1: `FLYSESS1`
Status: **draft 1**, 2026-09-22. Specified by CONTRACT-01 of the
[implementation guide](implementation.md); required by
[session artifacts, native media and recovery](state-media-v1.md) section 4, which says to
"use a new envelope version; specify exact byte layout before production files". Reference
implementations of the layout: `services/flysim/crates/fly-session-types/src/checkpoint.rs`
and `packages/session-types/src/checkpoint.ts`; fixture:
`.../fly-session-types/fixtures/checkpoint-envelope.json`.
This is the byte layout and the durable commit sequence. The store itself, generations,
rotation, the writer thread and the capture RPC flow are the STATE-01 slice.
## 1. Why a new format
The historical envelope (`FLYSIM01`, `crates/flybrain-core/src/envelope.rs`) is a magic, a
`u32` manifest length, a JSON manifest, `u32`-prefixed chunks in manifest order and a CRC32
footer, with chunk names restricted to ASCII letters so the TypeScript reader can never name a
prototype key. It stays exactly as it is, and its reader stays separately readable: nothing in
this document changes a byte of it, and a `FLYSIM01` file is refused by a `FLYSESS1` reader at
the magic.
A coherent all-participant session checkpoint needs what that format does not have:
- payload names that are `Id`s (`agent-fly-a`, `executor-fly-a`), so the letters-only
constraint is widened **deliberately, in a new version**, rather than quietly;
- a per-payload content digest, because state-media-v1 section 1 makes digests mandatory on
checkpoint payloads and a group install must be able to fail one participant's bytes;
- a payload table with explicit offsets and lengths, so a reader can map one participant's
payload without walking every preceding chunk;
- SHA-256 over the whole prefix instead of CRC32, matching the `Digest` type these contracts
already use everywhere else.
## 2. Byte layout
All integers are unsigned little-endian. All digests are raw 32-byte SHA-256 (the manifest
records the same digests as lowercase hex `Digest` strings).
### Header, 32 bytes
| Offset | Size | Field |
| ---: | ---: | --- |
| 0 | 8 | Magic, ASCII `FLYSESS1` |
| 8 | 4 | `envelopeVersion`, `1` |
| 12 | 4 | `headerBytes`, `32` |
| 16 | 4 | `manifestBytes` |
| 20 | 4 | `payloadCount`, at most 64 |
| 24 | 4 | `tableOffset` |
| 28 | 4 | Reserved, must be zero |
### Manifest
`manifestBytes` bytes of canonical JSON (RFC 8785) at offset 32, no trailing newline. It is
canonical so the envelope's own digest is stable under reserialization, and a reader rejects a
manifest that is not already canonical rather than silently accepting a second spelling.
### Payload table
At `tableOffset`, which is `32 + manifestBytes` rounded up to a multiple of 8.
`payloadCount` entries of 112 bytes each, in write order:
| Offset in entry | Size | Field |
| ---: | ---: | --- |
| 0 | 64 | Name: an `Id` in ASCII, NUL-padded, no bytes after the terminator |
| 64 | 8 | `offset` |
| 72 | 8 | `byteLength` |
| 80 | 32 | SHA-256 of exactly `byteLength` bytes at `offset` |
### Payloads
Each payload starts at its declared offset. The first starts at the end of the table rounded
up to a multiple of 8; each subsequent one starts at the previous payload's end rounded up the
same way. Padding bytes are zero. Offsets are ascending and non-overlapping, which a reader
checks rather than assumes.
### Footer, 48 bytes
| Offset from end | Size | Field |
| ---: | ---: | --- |
| 48 | 8 | `fileBytes`, the total length including the footer |
| 40 | 32 | SHA-256 of every byte before the footer |
| 8 | 8 | Magic, ASCII `FLYSESSF` |
A truncated file therefore fails at the footer magic or the recorded length, not at an
arbitrary payload.
## 3. Manifest fields
State-media-v1 section 4 lists what the manifest records. The names below are the JSON field
names; a manifest missing any of them is not a complete checkpoint.
| Field | Contents |
| --- | --- |
| `envelopeVersion` | `1` |
| `checkpointId` | `Id`, the identity every participant's capture shares |
| `sourceScope` | `Scope`: session, epoch and the committed step |
| `episodeId` | `Id` |
| `worldTime` | `RationalNs`, the environment's logical time at that boundary |
| `schedulerId` | The coordinator's scheduler identity, `lockstep-v1` in v1 |
| `compositionDigest` | Coordinator scheduler and configuration identity |
| `portMap` | The exact port-to-agent map, `[{portId, agentId}]` |
| `compatibility` | Backend, content, patch, controller, parser and state-format identities |
| `agents` | Per agent: profile, dataset and model identities, resolved seed, tick count, remainder and the payload name holding its state |
| `coordinator` | Task ledger, prior world inspection, per-agent executor state, admission state and event watermarks, each as a payload name or an inline value |
| `helperState` | External-helper state required for exact resume, as payload names |
| `payloads` | `[{name, byteLength, digest}]`, mirroring the payload table |
`payloads` is redundant with the table on purpose: the table is what a reader needs to map
bytes, and the manifest is what a store lists, compares and reports without opening the
payload area. A reader checks that the two agree.
What the manifest must **not** contain (state-media-v1 section 4): a transient bus `storeId`,
artifact ID, owner token, mapping or pointer. Payload bytes and durable content identity are
the only things that survive; on restore the durable store imports fresh bus artifacts, and
`sourceScope` is provenance, not a claim on the current router.
## 4. What a reader enforces
In this order, so a corrupt file fails on its own terms rather than on a derived value:
1. Length at least header plus footer; magic; version; `headerBytes`; reserved word zero.
2. Footer magic, `fileBytes` equal to the actual length, and the prefix digest.
3. Manifest inside the payload area, valid strict JSON (duplicate keys, invalid UTF-8 and
non-finite numbers refused) and already canonical.
4. `tableOffset` exactly at the laid-out position; the table inside the payload area.
5. Per entry: an `Id` name with no bytes after its terminator, names unique, the declared
offset exactly at the aligned end of the previous payload, the payload inside the payload
area, and its digest matching its bytes.
6. No padding between the last payload and the footer.
7. The required manifest field set, `envelopeVersion` of 1, and a `payloads` list that matches
the table name for name, length for length and digest for digest.
Failing any of these is a corrupt or foreign file. The group install rule of state-media-v1
section 5 then applies: corrupt any participant and installation fails as a group.
## 5. Durable commit
State-media-v1 section 6, in the order the writer performs it:
1. Write the envelope to a temporary generation file in the store directory.
2. `fsync` the file.
3. `rename` it to its final generation name.
4. `fsync` the store directory.
5. Write the store manifest to its own temporary file, `fsync`, `rename`, `fsync` the
directory.
**The store manifest rename is the durable commit point.** Before it, the generation file is
an unreferenced temporary that is never a restore candidate. After it, and only after it, the
writer reports a saved acknowledgment and moves the high-water mark.
Consequences the writer must respect rather than reinterpret:
- Bus publications for `captured`, `queued`, `committed`, `failed` and `superseded` are
distinct events; only durable completion produces the saved acknowledgment.
- A lost save reply never advances durable metadata: the coordinator resolves the same
operation or fails the epoch, and an unreferenced generation stays unreferenced.
- A failed write releases its owned ephemeral captures under the configured retry policy and
reports the failure. It never reports false durability.
- The writer owns the bus artifact handles until the bytes are committed or the job fails, and
drops them afterwards; durable files are outside the bus's ephemeral collection.
- No per-payload `fsync` inside one envelope: the single file `fsync` in step 2 covers it.
## 6. Fixture
`fixtures/checkpoint-envelope.json` holds one complete envelope: the manifest, five payloads
(one agent, one executor, the task ledger, the prior inspection and a world payload), the
envelope's base64 bytes, its exact layout (header size, manifest offset and length, table
offset, every entry's offset, length and digest, footer offset, total length) and six
corruptions a reader must refuse, each naming the byte to flip.
The two implementations are held to it from both directions: each parses the fixture and
checks every recorded offset, and the TypeScript side re-encodes the same manifest and
payloads and requires the bytes to be identical to the fixture. A layout change that only one
language makes therefore fails on the next test run.
## 7. Out of scope
Generations, rotation, hot versus durable copies, the capture queue and its bounds, the
`State.Capture` / `State.StageRestore` / `State.ActivateRestore` flow, compatibility
comparison rules and group fencing. Those are STATE-01, over this layout. `FLYSIM01` and the
legacy composition keep their own format and their own reader, unchanged.

View file

@ -0,0 +1,101 @@
# Seed derivation v1
Status: **draft 1**, 2026-09-22. Specified by CONTRACT-01 of the
[implementation guide](implementation.md), required by
[worker interfaces](workers-v1.md) section 2 before the real-agent slice. Reference
implementations: `services/flysim/crates/fly-session-types/src/seed.rs` and
`packages/session-types/src/seed.ts`; test vectors:
`services/flysim/crates/fly-session-types/fixtures/seed-vectors.json`.
## 1. What this is for
`Agent.Initialize` takes `seed`, a signed 32-bit integer, matching the current RNG input.
Workers-v1 section 2 requires that the coordinator derive independent per-agent seeds from
**its recorded master seed and stable agent IDs** under a versioned algorithm, and that the
algorithm be specified and tested before the real agent slice. This is that algorithm.
It is a reproducibility rule, not a secret: a run manifest records the master seed in the
clear, and anyone with the manifest can recompute every agent's seed. It is not a key
derivation function and must not be used as one.
`seed-derivation-v1` is part of composition identity. Changing any byte of it requires a new
identifier (`seed-derivation-v2`), because two runs that agree on every other identity but
disagree here are not the same experiment.
## 2. Inputs
| Input | Type | Source |
| --- | --- | --- |
| `masterSeed` | `U64` decimal string | Recorded once per run by the application/supervisor |
| `agentId` | `Id` | The configured agent identity, stable across restarts and epochs |
Both are the ipc-v1 section 2 scalars. An `agentId` that is not an `Id` is an error, not
something to normalize. The master seed is the whole 64-bit range: a 32-bit master seed would
be no wider than the seed it derives.
## 3. Derivation
```text
material = "flybrain/seed-derivation-v1" LF masterSeed LF agentId LF
digest = SHA-256(material)
lanes = digest read as eight big-endian uint32 values, in order
seed = the first nonzero lane, reinterpreted as a two's-complement int32
```
`LF` is one `0x0a` byte. `masterSeed` is its canonical decimal form: `"0"`, or no leading
zero. The prefix is a domain separator, so a digest from this algorithm can never collide with
one taken over some other pair of strings.
Zero lanes are skipped because the pinned kernel's RNG is an xorshift generator, whose state
must not be zero: a derivation that could hand out `0` would silently produce a stalled
generator. If every one of the eight lanes were zero, the material is rehashed with a counter
suffix (`material || "1" LF`, then `"2" LF`, then `"3" LF`) and the search repeats; no input
has ever needed it, and four rounds exhausted is an error rather than a fallback seed.
The seed is the *negative* number when the lane's high bit is set. That is deliberate: the
existing RNG input is a signed 32-bit integer, and half the range is negative.
## 4. Properties
- **Deterministic.** The seed is a function of the two recorded inputs and nothing else: not
of wall time, agent order, port assignment, worker process or thread count.
- **Independent per agent.** Distinct agent IDs give unrelated seeds; there is no arithmetic
relationship between `fly-a` and `fly-b` for a caller to exploit or accidentally rely on.
- **Stable across recovery.** Restore, episode reset and a new epoch do not re-derive a
different seed for the same agent ID under the same master seed. The seed is persisted as run
configuration and state, and the capture compatibility digest covers the resolved seed
(workers-v1 section 2), so a checkpoint cannot be installed into a differently seeded
instance.
- **Equal IDs give equal seeds.** That is the only way to get identical seeds, and workers-v1
allows identical seeds only when an experiment declares them. A composition therefore
refuses a repeated agent ID rather than quietly sharing a seed between two agents.
Non-properties, stated so nobody assumes them: this is not uniform over the int32 range beyond
what SHA-256 gives, it is not a stream (one seed per agent per run, not per step), and it says
nothing about how a model consumes its seed.
## 5. Test vectors
`fixtures/seed-vectors.json` carries the full table: five master seeds (`0`, `1`, `42`, `2^63`
and the `U64` maximum) across four agent IDs, each with the exact material string, its SHA-256
and the derived seed, plus one four-agent composition and the inputs that must be refused.
Both implementations reproduce every row, and each records the material as well as the seed so
a third implementation can find where it diverges.
The first two rows:
| masterSeed | agentId | material | seed |
| --- | --- | --- | ---: |
| `0` | `fly-a` | `flybrain/seed-derivation-v1\n0\nfly-a\n` | 1828176714 |
| `0` | `fly-b` | `flybrain/seed-derivation-v1\n0\nfly-b\n` | 1218785088 |
Refused: an agent ID that is not an `Id` (uppercase, empty, over 64 characters), a master seed
that is not a canonical `U64`, and a composition with a repeated agent ID.
## 6. Out of scope
Choosing the master seed, recording it in the run manifest, and the hand-selected explicit
seeds that workers-v1 allows for the first synthetic composition. This document defines only
the derivation. A profile that needs several independent streams inside one agent derives them
from the agent's own seed under its own documented rule; that is a profile concern, not a
session one.

18
package-lock.json generated
View file

@ -14,6 +14,7 @@
"apps/stage": { "apps/stage": {
"name": "@flybrain/stage", "name": "@flybrain/stage",
"version": "0.1.1", "version": "0.1.1",
"license": "Apache-2.0",
"dependencies": { "dependencies": {
"@flybrain/brain": "*", "@flybrain/brain": "*",
"@flybrain/feed": "*", "@flybrain/feed": "*",
@ -892,6 +893,10 @@
"resolved": "packages/feed", "resolved": "packages/feed",
"link": true "link": true
}, },
"node_modules/@flybrain/session-types": {
"resolved": "packages/session-types",
"link": true
},
"node_modules/@flybrain/stage": { "node_modules/@flybrain/stage": {
"resolved": "apps/stage", "resolved": "apps/stage",
"link": true "link": true
@ -3666,6 +3671,7 @@
"packages/brain": { "packages/brain": {
"name": "@flybrain/brain", "name": "@flybrain/brain",
"version": "0.1.1", "version": "0.1.1",
"license": "Apache-2.0",
"devDependencies": { "devDependencies": {
"@types/node": "22.17.0", "@types/node": "22.17.0",
"@types/three": "0.178.1", "@types/three": "0.178.1",
@ -3685,6 +3691,7 @@
"packages/feed": { "packages/feed": {
"name": "@flybrain/feed", "name": "@flybrain/feed",
"version": "0.1.0", "version": "0.1.0",
"license": "Apache-2.0",
"dependencies": { "dependencies": {
"ws": "8.21.3" "ws": "8.21.3"
}, },
@ -3696,9 +3703,20 @@
"typescript": "5.9.2" "typescript": "5.9.2"
} }
}, },
"packages/session-types": {
"name": "@flybrain/session-types",
"version": "0.1.0",
"license": "Apache-2.0",
"devDependencies": {
"@types/node": "22.17.0",
"tsx": "4.20.3",
"typescript": "5.9.2"
}
},
"services/bridge": { "services/bridge": {
"name": "@flybrain/bridge", "name": "@flybrain/bridge",
"version": "0.1.0", "version": "0.1.0",
"license": "Apache-2.0",
"dependencies": { "dependencies": {
"@flybrain/feed": "0.1.0", "@flybrain/feed": "0.1.0",
"@twurple/api": "8.1.4", "@twurple/api": "8.1.4",

View file

@ -0,0 +1,76 @@
# @flybrain/session-types
The session framework contracts in TypeScript: types, validation, canonical JSON (RFC 8785)
and canonical digests.
The other half of [`services/flysim/crates/fly-session-types`](../../services/flysim/crates/fly-session-types).
Same rules, same canonical bytes, same digests, and the same fixture corpus: this package
loads the crate's `fixtures/` directory rather than keeping a copy, so a case written once
holds both languages to it. Nothing here opens a socket; it reads, validates and hashes.
This is the internal session path (`docs/design/session-framework/`). The public feed and
control contracts are unchanged and still live in [`@flybrain/feed`](../feed).
## Modules
| Module | Contents |
| --- | --- |
| `canonical` | `canonicalize`, `digestOf`, `parseStrict`, `requireEnvelopeFit`, `rejectBusIdentities` |
| `scalar` | `Id`, `U64`, `Digest`, `Scope`, `RationalNs` with checked arithmetic, and the four identities as branded types |
| `reader` | `Reader`, which reads one object field by field and then refuses any field it did not read |
| `common` | `readScope`, `readSchemaRef`, `readTypedValue`, `operationKeyDigest`, `bodyDigest` |
| `media` | View and audio descriptors and refs, and the `State.*` payloads |
| `workers` | The closed enums and every Agent/Environment/Worker method payload |
| `rpc` | `SessionRpcRequest`, the success and failure replies, `ErrorCode`, `MutationCertainty` |
| `publishing` | `SessionDescriptor`, `CommittedSnapshot` |
| `trace` | The step-v1 section 8 record and the behaviour-only comparator |
| `seed` | `seed-derivation-v1` |
| `checkpoint` | The `FLYSESS1` envelope layout |
| `fixtures` | Loading the shared corpus |
## Reading a payload
Every reader takes `unknown`, validates, and hands back a value whose fields are exactly the
ones it read. A payload with an unknown or misspelled field fails instead of silently
defaulting, and a round trip through a reader is the test that no field is dropped.
```ts
import { canonicalize, digestOf, readScope, readPrepareParams, bodyDigest } from '@flybrain/session-types';
const scope = readScope(payload.scope);
const params = readPrepareParams(payload.params);
const digest = bodyDigest('Agent.Prepare', scope, params); // the ipc-v1 section 5 comparison
```
Rules that need another value in hand are separate functions, because a payload cannot check
them alone: `validatePortControlAgainst`, `validateBatch`, `validateSensoryInputAgainst`,
`validateObservationAgainst`, `validateStepResultAgainst`, `validateSnapshotAgainst`,
`validateTelemetryRoles`, `validateRemainder`, `validateCommitAgainstScope`.
## Canonical JSON
Three rules make the two implementations agree byte for byte:
- object keys sort by UTF-16 code unit, which is what comparing JavaScript strings does;
- numbers print with `String(number)`, the ECMAScript algorithm RFC 8785 requires;
- a number is canonicalizable when it is finite and, if integral, no larger in magnitude than
`Number.MAX_SAFE_INTEGER`. Larger integers are refused rather than rounded: every counter
and clock in these contracts is a `U64` decimal string. The rule is on the value, not on how
it was written, because `JSON.parse` cannot tell `1e21` from the same digits written out.
`parseStrict` is a small recursive-descent parser rather than a wrapper around `JSON.parse`,
which keeps the last of two duplicate keys instead of failing.
Digests use `node:crypto`. This package is contract tooling for services and tests, not
browser code; the presentation layer consumes the public feed package instead.
## Tests
```sh
npm test --workspace @flybrain/session-types
npm run typecheck --workspace @flybrain/session-types
```
Nine files, all fixture-driven. The one that says the most about the two implementations is in
`tests/checkpoint.test.ts`: a `FLYSESS1` envelope written here is byte-identical to the one the
Rust crate wrote into the fixture.

View file

@ -0,0 +1,22 @@
{
"name": "@flybrain/session-types",
"version": "0.1.0",
"description": "TypeScript types, validation, canonical JSON (RFC 8785) and canonical digests for the session framework contracts, sharing the fixture corpus of services/flysim/crates/fly-session-types.",
"license": "Apache-2.0",
"private": true,
"type": "module",
"main": "./src/index.ts",
"types": "./src/index.ts",
"exports": {
".": "./src/index.ts"
},
"scripts": {
"test": "node --import tsx --test tests/**/*.test.ts",
"typecheck": "tsc -p tsconfig.json --pretty false"
},
"devDependencies": {
"@types/node": "22.17.0",
"tsx": "4.20.3",
"typescript": "5.9.2"
}
}

View file

@ -0,0 +1,381 @@
/**
* Canonical JSON (RFC 8785), strict parsing and canonical digests.
*
* The Rust crate `services/flysim/crates/fly-session-types` is the other half of this
* contract; `fixtures/valid.json` records the canonical bytes and digest of every accepted
* payload, and both languages assert against it.
*
* Three rules make the two agree:
*
* - object keys sort by UTF-16 code unit, which is what comparing JavaScript strings does;
* - numbers print with `String(number)`, the ECMAScript algorithm RFC 8785 requires;
* - a number is canonicalizable when it is finite and, if integral, no larger in magnitude
* than `Number.MAX_SAFE_INTEGER`. Larger integers are refused rather than rounded: every
* counter and clock in these contracts is a `U64` decimal string. The rule is on the value,
* not on how it was written, because `JSON.parse` cannot tell `1e21` from the same digits
* written out.
*/
import { createHash } from 'node:crypto';
/** The largest JSON envelope, in bytes (bus-v1 section 4). */
export const MAX_ENVELOPE_BYTES = 65_536;
/** Thrown by everything in this package. One error type, like the bus's `WireError`. */
export class ContractError extends Error {
constructor(message: string) {
super(message);
this.name = 'ContractError';
}
}
export function fail(message: string): never {
throw new ContractError(message);
}
/** A JSON value, as strictly parsed. */
export type Json = null | boolean | number | string | Json[] | { [key: string]: Json };
/** `String(number)` for a canonicalizable number. */
function numberToString(value: number): string {
if (!Number.isFinite(value)) {
fail(`canonical JSON: ${String(value)} is not a finite number`);
}
if (Number.isInteger(value) && Math.abs(value) > Number.MAX_SAFE_INTEGER) {
fail(`canonical JSON: ${String(value)} is an integral value outside the exact double range`);
}
// String(-0) is already "0".
return String(value);
}
function writeString(out: string[], value: string): void {
out.push('"');
for (const character of value) {
switch (character) {
case '"':
out.push('\\"');
break;
case '\\':
out.push('\\\\');
break;
case '\b':
out.push('\\b');
break;
case '\t':
out.push('\\t');
break;
case '\n':
out.push('\\n');
break;
case '\f':
out.push('\\f');
break;
case '\r':
out.push('\\r');
break;
default: {
const point = character.codePointAt(0) ?? 0;
if (point < 0x20) {
out.push(`\\u${point.toString(16).padStart(4, '0')}`);
} else {
out.push(character);
}
}
}
}
out.push('"');
}
function write(out: string[], value: unknown): void {
if (value === null) {
out.push('null');
return;
}
switch (typeof value) {
case 'boolean':
out.push(value ? 'true' : 'false');
return;
case 'number':
out.push(numberToString(value));
return;
case 'string':
writeString(out, value);
return;
case 'object':
break;
default:
fail(`canonical JSON: ${typeof value} is not a JSON value`);
}
if (Array.isArray(value)) {
out.push('[');
value.forEach((item, index) => {
if (index > 0) out.push(',');
write(out, item);
});
out.push(']');
return;
}
const entries = Object.entries(value as Record<string, unknown>);
for (const [key, item] of entries) {
if (item === undefined) fail(`canonical JSON: ${key} is undefined, which is not a JSON value`);
}
// Comparing JavaScript strings compares UTF-16 code units, which is the order RFC 8785
// section 3.2.3 specifies.
entries.sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0));
out.push('{');
entries.forEach(([key, item], index) => {
if (index > 0) out.push(',');
writeString(out, key);
out.push(':');
write(out, item);
});
out.push('}');
}
/** The canonical JSON text of `value`. */
export function canonicalize(value: unknown): string {
const out: string[] = [];
write(out, value);
return out.join('');
}
/** Lowercase hex SHA-256 of `bytes`. */
export function sha256Hex(bytes: Uint8Array | string): string {
return createHash('sha256')
.update(typeof bytes === 'string' ? Buffer.from(bytes, 'utf8') : bytes)
.digest('hex');
}
/** The canonical digest of a JSON value: SHA-256 over its canonical JSON bytes. */
export function digestOf(value: unknown): string {
return sha256Hex(canonicalize(value));
}
/**
* Parses JSON strictly: duplicate keys at any depth, invalid UTF-8, `NaN`/`Infinity`,
* trailing bytes and control characters inside strings are all refused.
*
* `JSON.parse` keeps the last of two duplicate keys instead of failing, so this is a small
* recursive-descent parser rather than a wrapper around it.
*/
export function parseStrict(input: Uint8Array | string): Json {
let text: string;
if (typeof input === 'string') {
text = input;
} else {
try {
text = new TextDecoder('utf-8', { fatal: true }).decode(input);
} catch {
return fail('invalid UTF-8');
}
}
const parser = new Parser(text);
const value = parser.value();
parser.skipWhitespace();
if (!parser.atEnd()) fail('invalid JSON: trailing data');
return value;
}
class Parser {
private index = 0;
constructor(private readonly text: string) {}
atEnd(): boolean {
return this.index >= this.text.length;
}
skipWhitespace(): void {
while (this.index < this.text.length && ' \t\n\r'.includes(this.text[this.index] as string)) {
this.index += 1;
}
}
value(): Json {
this.skipWhitespace();
const character = this.text[this.index];
if (character === undefined) fail('invalid JSON: unexpected end of input');
switch (character) {
case '{':
return this.object();
case '[':
return this.array();
case '"':
return this.string();
case 't':
this.literal('true');
return true;
case 'f':
this.literal('false');
return false;
case 'n':
this.literal('null');
return null;
default:
return this.number();
}
}
private literal(word: string): void {
if (!this.text.startsWith(word, this.index)) fail(`invalid JSON: expected ${word}`);
this.index += word.length;
}
private object(): Json {
this.index += 1;
const out: { [key: string]: Json } = {};
this.skipWhitespace();
if (this.text[this.index] === '}') {
this.index += 1;
return out;
}
for (;;) {
this.skipWhitespace();
if (this.text[this.index] !== '"') fail('invalid JSON: expected a key');
const key = this.string();
if (Object.prototype.hasOwnProperty.call(out, key)) {
fail(`invalid JSON: duplicate key ${JSON.stringify(key)}`);
}
this.skipWhitespace();
if (this.text[this.index] !== ':') fail('invalid JSON: expected :');
this.index += 1;
out[key] = this.value();
this.skipWhitespace();
const next = this.text[this.index];
if (next === ',') {
this.index += 1;
continue;
}
if (next === '}') {
this.index += 1;
return out;
}
fail('invalid JSON: expected , or }');
}
}
private array(): Json {
this.index += 1;
const out: Json[] = [];
this.skipWhitespace();
if (this.text[this.index] === ']') {
this.index += 1;
return out;
}
for (;;) {
out.push(this.value());
this.skipWhitespace();
const next = this.text[this.index];
if (next === ',') {
this.index += 1;
continue;
}
if (next === ']') {
this.index += 1;
return out;
}
fail('invalid JSON: expected , or ]');
}
}
private string(): string {
this.index += 1;
let out = '';
for (;;) {
const character = this.text[this.index];
if (character === undefined) fail('invalid JSON: unterminated string');
this.index += 1;
if (character === '"') return out;
if (character === '\\') {
const escape = this.text[this.index];
this.index += 1;
switch (escape) {
case '"':
case '\\':
case '/':
out += escape;
break;
case 'b':
out += '\b';
break;
case 'f':
out += '\f';
break;
case 'n':
out += '\n';
break;
case 'r':
out += '\r';
break;
case 't':
out += '\t';
break;
case 'u': {
const hex = this.text.slice(this.index, this.index + 4);
if (!/^[0-9a-fA-F]{4}$/.test(hex)) fail('invalid JSON: bad \\u escape');
out += String.fromCharCode(Number.parseInt(hex, 16));
this.index += 4;
break;
}
default:
fail('invalid JSON: bad escape');
}
continue;
}
if (character.charCodeAt(0) < 0x20) fail('invalid JSON: control character in a string');
out += character;
}
}
private number(): number {
const match = /^-?(0|[1-9][0-9]*)(\.[0-9]+)?([eE][-+]?[0-9]+)?/.exec(
this.text.slice(this.index),
);
if (!match) fail('invalid JSON: expected a value');
this.index += match[0].length;
const value = Number(match[0]);
if (!Number.isFinite(value)) fail('invalid JSON: non-finite number');
return value;
}
}
/**
* Refuses a domain payload that does not fit the bus envelope ceiling. `envelopeOverhead` is
* what the surrounding envelope adds, so a payload that only fits without its envelope fails.
*/
export function requireEnvelopeFit(value: unknown, envelopeOverhead: number): number {
const total = canonicalize(value).length + envelopeOverhead;
if (total > MAX_ENVELOPE_BYTES) {
fail(`envelope: ${total} bytes exceeds the ${MAX_ENVELOPE_BYTES}-byte maximum`);
}
return total;
}
/** Keys that belong to the bus and never to a domain body (ipc-v1 section 5). */
export const BUS_ONLY_KEYS = [
'callId',
'deliveryId',
'ownerId',
'ownerIds',
'deliveryIds',
'requestDeliveryId',
'expectedIncarnation',
'serviceIncarnation',
'connectionId',
'topicSequence',
'subscriptionId',
] as const;
/** Fails if any bus-only key appears anywhere in `value`. */
export function rejectBusIdentities(value: unknown): void {
if (Array.isArray(value)) {
for (const item of value) rejectBusIdentities(item);
return;
}
if (value === null || typeof value !== 'object') return;
for (const [key, item] of Object.entries(value as Record<string, unknown>)) {
if ((BUS_ONLY_KEYS as readonly string[]).includes(key)) {
fail(`canonical body: "${key}" is a bus identity and never part of a domain body`);
}
rejectBusIdentities(item);
}
}

View file

@ -0,0 +1,273 @@
/**
* `FLYSESS1`: the envelope layout of
* `docs/design/session-framework/checkpoint-envelope-v1.md`.
*
* The layout half of the specification, not the store: writing generations, fsyncing and
* committing a manifest belong to the STATE-01 slice. `FLYSIM01` is a different format with a
* different magic and is not touched by any of this.
*/
import { createHash } from 'node:crypto';
import { type Json, canonicalize, fail, parseStrict } from './canonical';
import { requireUnique } from './reader';
import { isId } from './scalar';
export const MAGIC = 'FLYSESS1';
export const FOOTER_MAGIC = 'FLYSESSF';
export const VERSION = 1;
export const HEADER_BYTES = 32;
export const TABLE_ENTRY_BYTES = 112;
export const NAME_BYTES = 64;
export const FOOTER_BYTES = 48;
export const ALIGNMENT = 8;
export const MAX_PAYLOADS = 64;
export interface PayloadEntry {
name: string;
offset: number;
byteLength: number;
digest: string;
}
export interface Layout {
manifestOffset: number;
manifestBytes: number;
tableOffset: number;
entries: PayloadEntry[];
footerOffset: number;
totalBytes: number;
}
export interface Envelope {
manifest: Json;
payloads: { name: string; bytes: Uint8Array }[];
layout: Layout;
}
function alignUp(value: number): number {
return Math.ceil(value / ALIGNMENT) * ALIGNMENT;
}
function sha256(bytes: Uint8Array): string {
return createHash('sha256').update(bytes).digest('hex');
}
export function layoutOf(
manifest: unknown,
payloads: readonly { name: string; bytes: Uint8Array }[],
): Layout {
if (payloads.length > MAX_PAYLOADS) fail('checkpoint envelope: at most 64 payloads');
requireUnique(
payloads.map((payload) => payload.name),
'checkpoint envelope: payload names',
);
for (const payload of payloads) {
if (!isId(payload.name)) {
fail(`checkpoint envelope: payload name "${payload.name}" is not an Id`);
}
}
const manifestBytes = new TextEncoder().encode(canonicalize(manifest)).length;
const manifestOffset = HEADER_BYTES;
const tableOffset = alignUp(manifestOffset + manifestBytes);
let offset = alignUp(tableOffset + payloads.length * TABLE_ENTRY_BYTES);
const entries: PayloadEntry[] = [];
for (const payload of payloads) {
entries.push({
name: payload.name,
offset,
byteLength: payload.bytes.length,
digest: sha256(payload.bytes),
});
offset = alignUp(offset + payload.bytes.length);
}
return {
manifestOffset,
manifestBytes,
tableOffset,
entries,
footerOffset: offset,
totalBytes: offset + FOOTER_BYTES,
};
}
export function encode(
manifest: unknown,
payloads: readonly { name: string; bytes: Uint8Array }[],
): Uint8Array {
const layout = layoutOf(manifest, payloads);
const manifestText = new TextEncoder().encode(canonicalize(manifest));
const out = Buffer.alloc(layout.footerOffset);
out.write(MAGIC, 0, 'ascii');
out.writeUInt32LE(VERSION, 8);
out.writeUInt32LE(HEADER_BYTES, 12);
out.writeUInt32LE(layout.manifestBytes, 16);
out.writeUInt32LE(payloads.length, 20);
out.writeUInt32LE(layout.tableOffset, 24);
out.writeUInt32LE(0, 28);
Buffer.from(manifestText).copy(out, layout.manifestOffset);
layout.entries.forEach((entry, index) => {
const base = layout.tableOffset + index * TABLE_ENTRY_BYTES;
out.write(entry.name, base, 'ascii');
out.writeBigUInt64LE(BigInt(entry.offset), base + NAME_BYTES);
out.writeBigUInt64LE(BigInt(entry.byteLength), base + NAME_BYTES + 8);
Buffer.from(entry.digest, 'hex').copy(out, base + NAME_BYTES + 16);
});
layout.entries.forEach((entry, index) => {
Buffer.from((payloads[index] as { bytes: Uint8Array }).bytes).copy(out, entry.offset);
});
const footer = Buffer.alloc(FOOTER_BYTES);
footer.writeBigUInt64LE(BigInt(layout.totalBytes), 0);
Buffer.from(sha256(out), 'hex').copy(footer, 8);
footer.write(FOOTER_MAGIC, 40, 'ascii');
return Buffer.concat([out, footer]);
}
/** Reads and fully validates one envelope. */
export function decode(input: Uint8Array): Envelope {
const bytes = Buffer.from(input);
if (bytes.length < HEADER_BYTES + FOOTER_BYTES) {
fail('checkpoint envelope: shorter than a header plus a footer');
}
if (bytes.subarray(0, 8).toString('ascii') !== MAGIC) {
fail('checkpoint envelope: wrong magic (FLYSIM01 is a different format)');
}
if (bytes.readUInt32LE(8) !== VERSION) fail('checkpoint envelope: unsupported version');
if (bytes.readUInt32LE(12) !== HEADER_BYTES) {
fail('checkpoint envelope: headerBytes must be 32');
}
if (bytes.readUInt32LE(28) !== 0) {
fail('checkpoint envelope: reserved header word must be zero');
}
const manifestBytes = bytes.readUInt32LE(16);
const payloadCount = bytes.readUInt32LE(20);
const tableOffset = bytes.readUInt32LE(24);
if (payloadCount > MAX_PAYLOADS) fail('checkpoint envelope: at most 64 payloads');
const footerOffset = bytes.length - FOOTER_BYTES;
if (bytes.subarray(footerOffset + 40).toString('ascii') !== FOOTER_MAGIC) {
fail('checkpoint envelope: missing footer magic');
}
if (bytes.readBigUInt64LE(footerOffset) !== BigInt(bytes.length)) {
fail('checkpoint envelope: footer length does not match the file');
}
const recorded = bytes.subarray(footerOffset + 8, footerOffset + 40).toString('hex');
if (recorded !== sha256(bytes.subarray(0, footerOffset))) {
fail('checkpoint envelope: footer digest does not match the contents');
}
const manifestEnd = HEADER_BYTES + manifestBytes;
if (manifestEnd > footerOffset) {
fail('checkpoint envelope: manifest runs past the payload area');
}
const manifestSlice = bytes.subarray(HEADER_BYTES, manifestEnd);
const manifest = parseStrict(manifestSlice);
if (canonicalize(manifest) !== manifestSlice.toString('utf8')) {
fail('checkpoint envelope: the manifest is not canonical JSON');
}
if (tableOffset !== alignUp(manifestEnd)) {
fail('checkpoint envelope: the payload table is not at its laid-out offset');
}
const tableEnd = tableOffset + payloadCount * TABLE_ENTRY_BYTES;
if (tableEnd > footerOffset) {
fail('checkpoint envelope: the payload table runs past the payload area');
}
const entries: PayloadEntry[] = [];
const payloads: { name: string; bytes: Uint8Array }[] = [];
let previousEnd = alignUp(tableEnd);
for (let index = 0; index < payloadCount; index += 1) {
const base = tableOffset + index * TABLE_ENTRY_BYTES;
const nameField = bytes.subarray(base, base + NAME_BYTES);
const terminator = nameField.indexOf(0);
const length = terminator === -1 ? NAME_BYTES : terminator;
if (nameField.subarray(length).some((byte) => byte !== 0)) {
fail('checkpoint envelope: a payload name has bytes after its terminator');
}
const name = nameField.subarray(0, length).toString('utf8');
if (!isId(name)) fail(`checkpoint envelope: payload name "${name}" is not an Id`);
const offset = Number(bytes.readBigUInt64LE(base + NAME_BYTES));
const byteLength = Number(bytes.readBigUInt64LE(base + NAME_BYTES + 8));
const digest = bytes.subarray(base + NAME_BYTES + 16, base + NAME_BYTES + 48).toString('hex');
if (offset !== previousEnd) {
fail(
`checkpoint envelope: payload "${name}" starts at ${offset}, not at its aligned ${previousEnd}`,
);
}
const end = offset + byteLength;
if (end > footerOffset) {
fail(`checkpoint envelope: payload "${name}" runs past the payload area`);
}
const payload = bytes.subarray(offset, end);
if (sha256(payload) !== digest) {
fail(`checkpoint envelope: payload "${name}" fails its digest`);
}
previousEnd = alignUp(end);
entries.push({ name, offset, byteLength, digest });
payloads.push({ name, bytes: Uint8Array.from(payload) });
}
requireUnique(
entries.map((entry) => entry.name),
'checkpoint envelope: payload names',
);
if (previousEnd !== footerOffset) {
fail('checkpoint envelope: padding between the last payload and the footer');
}
return {
manifest,
payloads,
layout: {
manifestOffset: HEADER_BYTES,
manifestBytes,
tableOffset,
entries,
footerOffset,
totalBytes: bytes.length,
},
};
}
/** The manifest fields state-media-v1 section 4 requires. */
export const REQUIRED_MANIFEST_FIELDS = [
'envelopeVersion',
'checkpointId',
'sourceScope',
'episodeId',
'worldTime',
'schedulerId',
'compositionDigest',
'portMap',
'compatibility',
'agents',
'coordinator',
'payloads',
] as const;
/** Checks the required field set and that the manifest's payload table mirrors the envelope's. */
export function validateManifest(envelope: Envelope): void {
const manifest = envelope.manifest;
if (manifest === null || typeof manifest !== 'object' || Array.isArray(manifest)) {
fail('checkpoint manifest: must be an object');
}
const map = manifest as Record<string, Json>;
for (const field of REQUIRED_MANIFEST_FIELDS) {
if (!Object.prototype.hasOwnProperty.call(map, field)) {
fail(`checkpoint manifest: missing "${field}"`);
}
}
if (map.envelopeVersion !== VERSION) fail('checkpoint manifest: envelopeVersion must be 1');
const listed = map.payloads;
if (!Array.isArray(listed)) fail('checkpoint manifest: payloads must be an array');
if (listed.length !== envelope.layout.entries.length) {
fail('checkpoint manifest: payloads does not match the payload table');
}
listed.forEach((declared, index) => {
const entry = envelope.layout.entries[index] as PayloadEntry;
const record = declared as Record<string, Json>;
if (record.name !== entry.name) {
fail('checkpoint manifest: payload name does not match the table');
}
if (record.byteLength !== String(entry.byteLength)) {
fail(`checkpoint manifest: payload "${entry.name}" byteLength does not match the table`);
}
if (record.digest !== entry.digest) {
fail(`checkpoint manifest: payload "${entry.name}" digest does not match the table`);
}
});
}

View file

@ -0,0 +1,114 @@
/**
* `Scope`, `SchemaRef`, `TypedValue`, the operation key and the canonical body
* (ipc-v1 sections 2 and 5).
*/
import { canonicalize, digestOf, fail, rejectBusIdentities } from './canonical';
import { Reader } from './reader';
import {
MAX_TYPED_VALUE_BYTES,
type RationalNs,
type Scope,
type SchemaRef,
type TypedValue,
isId,
isMethod,
validateRational,
} from './scalar';
export function readScope(value: unknown): Scope {
const reader = new Reader(value, 'Scope');
const scope: Scope = {
sessionId: reader.id('sessionId'),
epoch: reader.id('epoch'),
step: reader.u64('step'),
};
reader.finish();
return scope;
}
export function readNullableScope(value: unknown): Scope | null {
return value === null ? null : readScope(value);
}
export function readSchemaRef(value: unknown): SchemaRef {
const reader = new Reader(value, 'SchemaRef');
const schema: SchemaRef = {
id: reader.id('id'),
version: reader.int('version', 1, 65_535),
digest: reader.digest('digest'),
};
reader.finish();
return schema;
}
export function readTypedValue(value: unknown): TypedValue {
const reader = new Reader(value, 'TypedValue');
const typed: TypedValue = {
schema: readSchemaRef(reader.value('schema')),
value: reader.object('value'),
};
reader.finish();
const length = canonicalize(typed).length;
if (length > MAX_TYPED_VALUE_BYTES) {
fail(
`TypedValue: ${length} bytes of canonical JSON exceeds the ${MAX_TYPED_VALUE_BYTES}-byte limit`,
);
}
return typed;
}
export function readNullableTypedValue(value: unknown): TypedValue | null {
return value === null ? null : readTypedValue(value);
}
export { validateRational };
/** `(sessionId, epoch, step, method, workerId)`: the operation key of a step mutation. */
export interface OperationKey {
scope: Scope;
method: string;
workerId: string;
}
export function operationKeyJson(key: OperationKey): Record<string, unknown> {
if (!isMethod(key.method)) {
fail('OperationKey: method must be 1..=128 printable ASCII characters');
}
if (!isId(key.workerId)) fail('OperationKey: workerId is not a valid id');
return { scope: key.scope, method: key.method, workerId: key.workerId };
}
export function operationKeyDigest(key: OperationKey): string {
return digestOf(operationKeyJson(key));
}
/** The canonical body of a domain operation: method, scope and validated params. */
export function canonicalBody(
method: string,
scope: Scope | null,
params: unknown,
): Record<string, unknown> {
if (!isMethod(method)) {
fail('canonical body: method must be 1..=128 printable ASCII characters');
}
if (params === null || typeof params !== 'object' || Array.isArray(params)) {
fail('canonical body: params must be an object');
}
rejectBusIdentities(params);
return { method, scope, params };
}
export function bodyDigest(method: string, scope: Scope | null, params: unknown): string {
return digestOf(canonicalBody(method, scope, params));
}
export function readRational(value: unknown): RationalNs {
const reader = new Reader(value, 'RationalNs');
const rational: RationalNs = {
numerator: reader.u64('numerator'),
denominator: reader.u64('denominator'),
};
reader.finish();
validateRational(rational);
return rational;
}

View file

@ -0,0 +1,62 @@
/**
* Loading the fixture corpus, which lives with the Rust crate:
* `services/flysim/crates/fly-session-types/fixtures`.
*
* One corpus, two implementations. A case written once holds both languages to it.
*/
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { type Json, fail, parseStrict } from './canonical';
const here = fileURLToPath(new URL('.', import.meta.url));
/** The fixture directory. */
export const FIXTURE_DIR = join(
here,
'../../../services/flysim/crates/fly-session-types/fixtures',
);
export function loadBytes(name: string): Uint8Array {
return new Uint8Array(readFileSync(join(FIXTURE_DIR, name)));
}
/** Reads one fixture file, parsed strictly. */
export function load(name: string): Json {
return parseStrict(loadBytes(name));
}
export function section(file: Json, key: string): Json[] {
const value = (file as Record<string, Json>)[key];
if (!Array.isArray(value) || value.length === 0) {
fail(`fixture: ${key} must be a nonempty array`);
}
return value;
}
export function cases(file: Json): Json[] {
return section(file, 'cases');
}
/** A string field of one case. */
export function field(value: Json, key: string): string {
const found = (value as Record<string, Json>)[key];
if (typeof found !== 'string') fail(`fixture case: missing string field "${key}"`);
return found;
}
export function optionalField(value: Json, key: string): string | undefined {
const found = (value as Record<string, Json>)[key];
return typeof found === 'string' ? found : undefined;
}
export function member(value: Json, key: string): Json {
const found = (value as Record<string, Json>)[key];
if (found === undefined) fail(`fixture case: missing field "${key}"`);
return found;
}
export function decodeBase64(text: string): Uint8Array {
return new Uint8Array(Buffer.from(text, 'base64'));
}

View file

@ -0,0 +1,19 @@
/**
* `@flybrain/session-types`: the session framework contracts in TypeScript.
*
* The other half of `services/flysim/crates/fly-session-types`. Same rules, same canonical
* JSON, same digests, same fixtures. Nothing here opens a socket: it reads, validates and
* hashes payloads.
*/
export * from './canonical';
export * from './scalar';
export * from './reader';
export * from './common';
export * from './media';
export * from './workers';
export * from './rpc';
export * from './publishing';
export * from './trace';
export * as seed from './seed';
export * as checkpoint from './checkpoint';
export * as fixtures from './fixtures';

View file

@ -0,0 +1,264 @@
/** Native observation media (state-media-v1 section 2) and the `State.*` payloads (section 5). */
import { fail } from './canonical';
import { readRational, readScope } from './common';
import { Reader, readArtifactRef, requireUnique, u64 } from './reader';
import { type ArtifactRef, type Digest, type Id, type Scope, type U64, isDigest } from './scalar';
/** Max views per sensory input (workers-v1 section 1), and per descriptor list. */
export const MAX_VIEWS = 8;
export const MAX_VIEW_DIMENSION = 4096;
export const MAX_PIXEL_ASPECT = 65_535;
export const MAX_OBSERVATION_DELAY_STEPS = 8;
export const MAX_SAMPLE_FRAMES = 192_000;
/** Not a stated bound; this crate's choice, published in the schema set. */
export const MAX_AUDIO_STREAMS = 8;
export interface ViewDescriptor {
viewId: Id;
width: number;
height: number;
format: 'rgba8';
rowStride: number;
pixelAspect: { numerator: number; denominator: number };
observationDelaySteps: number;
}
export interface ViewRef {
viewId: Id;
producedStep: U64;
pixels: ArtifactRef;
}
export interface AudioDescriptor {
streamId: Id;
sampleRate: number;
channels: number;
format: 'f32le-interleaved';
}
export interface AudioRef {
streamId: Id;
firstSample: U64;
sampleFrames: number;
samples: ArtifactRef;
discontinuity: boolean;
}
export function readViewDescriptor(value: unknown): ViewDescriptor {
const reader = new Reader(value, 'ViewDescriptor');
const viewId = reader.id('viewId');
const width = reader.int('width', 1, MAX_VIEW_DIMENSION);
const height = reader.int('height', 1, MAX_VIEW_DIMENSION);
const format = reader.constant('format', 'rgba8');
const rowStride = reader.int('rowStride', 1, MAX_VIEW_DIMENSION * 4);
const aspectReader = new Reader(reader.value('pixelAspect'), 'ViewDescriptor.pixelAspect');
const pixelAspect = {
numerator: aspectReader.int('numerator', 1, MAX_PIXEL_ASPECT),
denominator: aspectReader.int('denominator', 1, MAX_PIXEL_ASPECT),
};
aspectReader.finish();
const observationDelaySteps = reader.int(
'observationDelaySteps',
0,
MAX_OBSERVATION_DELAY_STEPS,
);
reader.finish();
if (rowStride !== width * 4) {
fail('ViewDescriptor: rowStride must be exactly 4 x width (no padded rows in v1)');
}
return { viewId, width, height, format, rowStride, pixelAspect, observationDelaySteps };
}
/** The exact byte length of one frame of this view. */
export function frameBytes(descriptor: ViewDescriptor): number {
return descriptor.rowStride * descriptor.height;
}
/** `max(0, boundary - observationDelaySteps)` (state-media-v1 section 2). */
export function requiredProducedStep(descriptor: ViewDescriptor, boundary: bigint): bigint {
const delay = BigInt(descriptor.observationDelaySteps);
return boundary > delay ? boundary - delay : 0n;
}
export function readViewRef(value: unknown): ViewRef {
const reader = new Reader(value, 'ViewRef');
const view: ViewRef = {
viewId: reader.id('viewId'),
producedStep: reader.u64('producedStep'),
pixels: readArtifactRef(reader.value('pixels')),
};
reader.finish();
if (u64(view.pixels.byteLength) === 0n) {
fail('ViewRef: pixels must have a positive byte length');
}
return view;
}
export function readViewList(reader: Reader, key: string): ViewRef[] {
const views = reader.list(key, 0, MAX_VIEWS, readViewRef);
requireUnique(
views.map((view) => view.viewId),
key,
);
return views;
}
export function readAudioDescriptor(value: unknown): AudioDescriptor {
const reader = new Reader(value, 'AudioDescriptor');
const descriptor: AudioDescriptor = {
streamId: reader.id('streamId'),
sampleRate: reader.int('sampleRate', 8_000, 192_000),
channels: reader.int('channels', 1, 8),
format: reader.constant('format', 'f32le-interleaved'),
};
reader.finish();
return descriptor;
}
export function readAudioRef(value: unknown): AudioRef {
const reader = new Reader(value, 'AudioRef');
const chunk: AudioRef = {
streamId: reader.id('streamId'),
firstSample: reader.u64('firstSample'),
sampleFrames: reader.int('sampleFrames', 0, MAX_SAMPLE_FRAMES),
samples: readArtifactRef(reader.value('samples')),
discontinuity: reader.boolean('discontinuity'),
};
reader.finish();
if (u64(chunk.firstSample) + BigInt(chunk.sampleFrames) > 18446744073709551615n) {
fail('AudioRef: firstSample + sampleFrames overflows U64');
}
return chunk;
}
export function readAudioList(reader: Reader, key: string): AudioRef[] {
const audio = reader.list(key, 0, MAX_AUDIO_STREAMS, readAudioRef);
requireUnique(
audio.map((chunk) => chunk.streamId),
key,
);
return audio;
}
/** Byte shape and producing boundary against the descriptor that declared this view. */
export function validateViewAgainst(
view: ViewRef,
descriptor: ViewDescriptor,
boundary: bigint | null,
): void {
if (view.viewId !== descriptor.viewId) {
fail(`ViewRef: viewId "${view.viewId}" does not match descriptor "${descriptor.viewId}"`);
}
if (u64(view.pixels.byteLength) !== BigInt(frameBytes(descriptor))) {
fail(
`ViewRef ${view.viewId}: artifact is ${view.pixels.byteLength} bytes, rowStride x height is ${frameBytes(descriptor)}`,
);
}
if (boundary !== null) {
const expected = requiredProducedStep(descriptor, boundary);
if (u64(view.producedStep) !== expected) {
fail(
`ViewRef ${view.viewId}: producedStep ${view.producedStep} must be max(0, ${boundary} - ${descriptor.observationDelaySteps}) = ${expected}`,
);
}
}
}
export function validateAudioAgainst(chunk: AudioRef, descriptor: AudioDescriptor): void {
if (chunk.streamId !== descriptor.streamId) {
fail(`AudioRef: streamId "${chunk.streamId}" does not match the descriptor`);
}
const expected = BigInt(chunk.sampleFrames) * BigInt(descriptor.channels) * 4n;
if (u64(chunk.samples.byteLength) !== expected) {
fail(
`AudioRef ${chunk.streamId}: artifact is ${chunk.samples.byteLength} bytes, sampleFrames x channels x 4 is ${expected}`,
);
}
}
// ---------------------------------------------------------------------------------- State.*
export interface CaptureParams {
checkpointId: Id;
}
export interface CaptureResult {
checkpointId: Id;
boundary: U64;
compatibilityDigest: Digest;
payload: ArtifactRef;
}
export interface StageRestoreParams {
checkpointId: Id;
sourceScope: Scope;
compatibilityDigest: Digest;
payload: ArtifactRef;
}
export interface StageRestoreResult {
checkpointId: Id;
restoreToken: Id;
}
export interface ActivateRestoreParams {
restoreToken: Id;
}
function checkpointPayload(reader: Reader, key: string): ArtifactRef {
const reference = readArtifactRef(reader.value(key));
if (!isDigest(reference.digest)) {
fail(`${key}: a checkpoint payload must carry a content digest`);
}
return reference;
}
export function readCaptureParams(value: unknown): CaptureParams {
const reader = new Reader(value, 'CaptureParams');
const params = { checkpointId: reader.id('checkpointId') };
reader.finish();
return params;
}
export function readCaptureResult(value: unknown): CaptureResult {
const reader = new Reader(value, 'CaptureResult');
const result: CaptureResult = {
checkpointId: reader.id('checkpointId'),
boundary: reader.u64('boundary'),
compatibilityDigest: reader.digest('compatibilityDigest'),
payload: checkpointPayload(reader, 'payload'),
};
reader.finish();
return result;
}
export function readStageRestoreParams(value: unknown): StageRestoreParams {
const reader = new Reader(value, 'StageRestoreParams');
const params: StageRestoreParams = {
checkpointId: reader.id('checkpointId'),
sourceScope: readScope(reader.value('sourceScope')),
compatibilityDigest: reader.digest('compatibilityDigest'),
payload: checkpointPayload(reader, 'payload'),
};
reader.finish();
return params;
}
export function readStageRestoreResult(value: unknown): StageRestoreResult {
const reader = new Reader(value, 'StageRestoreResult');
const result: StageRestoreResult = {
checkpointId: reader.id('checkpointId'),
restoreToken: reader.id('restoreToken'),
};
reader.finish();
return result;
}
export function readActivateRestoreParams(value: unknown): ActivateRestoreParams {
const reader = new Reader(value, 'ActivateRestoreParams');
const params = { restoreToken: reader.id('restoreToken') };
reader.finish();
return params;
}
export { readRational };

View file

@ -0,0 +1,221 @@
/** The publication types of publishing-v1 section 3. */
import { fail } from './canonical';
import { readRational, readScope, readSchemaRef, readTypedValue, readNullableTypedValue } from './common';
import {
type AudioRef,
MAX_VIEWS,
type ViewRef,
readAudioList,
readViewList,
} from './media';
import { Reader, requireUnique, u64 } from './reader';
import { type Digest, type Id, type RationalNs, type SchemaRef, type Scope, type TypedValue, type U64 } from './scalar';
import {
type AgentTelemetry,
type AssetRef,
type EnvironmentDescriptor,
MAX_AGENTS,
MAX_RATE_ROLES,
type PortControl,
findPort,
readAgentTelemetry,
readAssetRef,
readEnvironmentDescriptor,
readPortControl,
validatePortControlAgainst,
validateTelemetryRoles,
} from './workers';
/** Not stated by a document; this crate's choices, published in the schema set. */
export const MAX_SUPPORTED_STIMULI = 64;
export const MAX_ASSETS = 64;
export const MAX_SNAPSHOT_EVENTS = 64;
export interface AgentDescriptor {
agentId: Id;
portId: Id;
profileDigest: Digest;
datasetDigest: Digest;
indexDigest: Digest;
neuronCount: U64;
rateRoles: Id[];
supportedStimuli: Id[];
}
export interface SessionDescriptor {
sessionId: Id;
revision: U64;
compositionDigest: Digest;
schedulerId: 'lockstep-v1';
environment: EnvironmentDescriptor;
taskSchema: SchemaRef;
agents: AgentDescriptor[];
assets: AssetRef[];
}
export interface SnapshotAgent {
agentId: Id;
telemetry: AgentTelemetry;
selectedDecision: TypedValue | null;
appliedControls: PortControl | null;
}
export interface CommittedSnapshot {
descriptorRevision: U64;
publisherIncarnation: Id;
scope: Scope;
episodeId: Id;
sequence: U64;
worldTime: RationalNs;
agents: SnapshotAgent[];
progress: TypedValue;
media: { views: ViewRef[]; audio: AudioRef[] };
eventIds: Id[];
}
export function readSessionDescriptor(value: unknown): SessionDescriptor {
const reader = new Reader(value, 'SessionDescriptor');
const descriptor: SessionDescriptor = {
sessionId: reader.id('sessionId'),
revision: reader.u64('revision'),
compositionDigest: reader.digest('compositionDigest'),
schedulerId: reader.constant('schedulerId', 'lockstep-v1'),
environment: readEnvironmentDescriptor(reader.value('environment')),
taskSchema: readSchemaRef(reader.value('taskSchema')),
agents: reader.list('agents', 1, MAX_AGENTS, (item) => {
const agent = new Reader(item, 'SessionDescriptor.agents');
const entry: AgentDescriptor = {
agentId: agent.id('agentId'),
portId: agent.id('portId'),
profileDigest: agent.digest('profileDigest'),
datasetDigest: agent.digest('datasetDigest'),
indexDigest: agent.digest('indexDigest'),
neuronCount: agent.u64('neuronCount'),
rateRoles: agent.idList('rateRoles', 0, MAX_RATE_ROLES),
supportedStimuli: agent.idList('supportedStimuli', 0, MAX_SUPPORTED_STIMULI),
};
agent.finish();
requireUnique(entry.rateRoles, 'SessionDescriptor.agents rateRoles');
requireUnique(entry.supportedStimuli, 'SessionDescriptor.agents supportedStimuli');
return entry;
}),
assets: reader.list('assets', 0, MAX_ASSETS, readAssetRef),
};
reader.finish();
requireUnique(
descriptor.agents.map((agent) => agent.agentId),
'SessionDescriptor.agents agentId',
);
requireUnique(
descriptor.agents.map((agent) => agent.portId),
'SessionDescriptor.agents portId',
);
requireUnique(
descriptor.assets.map((asset) => asset.id),
'SessionDescriptor.assets',
);
for (const agent of descriptor.agents) {
if (!findPort(descriptor.environment, agent.portId)) {
fail(
`SessionDescriptor: agent "${agent.agentId}" is bound to port "${agent.portId}", which the environment does not declare`,
);
}
}
return descriptor;
}
export function readCommittedSnapshot(value: unknown): CommittedSnapshot {
const reader = new Reader(value, 'CommittedSnapshot');
const descriptorRevision = reader.u64('descriptorRevision');
const publisherIncarnation = reader.id('publisherIncarnation');
const scope = readScope(reader.value('scope'));
const episodeId = reader.id('episodeId');
const sequence = reader.u64('sequence');
const worldTime = readRational(reader.value('worldTime'));
const agents = reader.list('agents', 1, MAX_AGENTS, (item) => {
const agent = new Reader(item, 'CommittedSnapshot.agents');
const controls = agent.value('appliedControls');
const entry: SnapshotAgent = {
agentId: agent.id('agentId'),
telemetry: readAgentTelemetry(agent.value('telemetry')),
selectedDecision: readNullableTypedValue(agent.value('selectedDecision')),
appliedControls: controls === null ? null : readPortControl(controls),
};
agent.finish();
return entry;
});
const progress = readTypedValue(reader.value('progress'));
const mediaReader = new Reader(reader.value('media'), 'CommittedSnapshot.media');
const media = {
views: readViewList(mediaReader, 'views'),
audio: readAudioList(mediaReader, 'audio'),
};
mediaReader.finish();
const eventIds = reader.idList('eventIds', 0, MAX_SNAPSHOT_EVENTS);
reader.finish();
requireUnique(
agents.map((agent) => agent.agentId),
'CommittedSnapshot.agents',
);
requireUnique(
media.views.map((view) => view.viewId),
'CommittedSnapshot.media.views',
);
requireUnique(eventIds, 'CommittedSnapshot.eventIds');
if (media.views.length > MAX_VIEWS) fail('CommittedSnapshot: at most 8 views');
const atBoundaryZero = u64(scope.step) === 0n;
for (const agent of agents) {
// "Decisions/controls describe the transition ending at that boundary, null at initial
// boundary 0." (publishing-v1 section 3)
if (atBoundaryZero && (agent.selectedDecision !== null || agent.appliedControls !== null)) {
fail('CommittedSnapshot: at boundary 0 selectedDecision and appliedControls are null');
}
if (!atBoundaryZero && (agent.selectedDecision === null || agent.appliedControls === null)) {
fail(
'CommittedSnapshot: past boundary 0 every agent has a decision and applied controls',
);
}
}
return {
descriptorRevision,
publisherIncarnation,
scope,
episodeId,
sequence,
worldTime,
agents,
progress,
media,
eventIds,
};
}
/** Descriptor agreement: revision, session, agent set and each agent's assigned port. */
export function validateSnapshotAgainst(
snapshot: CommittedSnapshot,
descriptor: SessionDescriptor,
): void {
if (snapshot.descriptorRevision !== descriptor.revision) {
fail('CommittedSnapshot: descriptorRevision does not match the descriptor');
}
if (snapshot.scope.sessionId !== descriptor.sessionId) {
fail('CommittedSnapshot: sessionId does not match the descriptor');
}
for (const agent of snapshot.agents) {
const declared = descriptor.agents.find((candidate) => candidate.agentId === agent.agentId);
if (!declared) {
fail(`CommittedSnapshot: agent "${agent.agentId}" is not in the descriptor`);
}
validateTelemetryRoles(agent.telemetry, declared.rateRoles);
if (agent.appliedControls !== null) {
if (agent.appliedControls.portId !== declared.portId) {
fail(
`CommittedSnapshot: agent "${agent.agentId}" controls port "${agent.appliedControls.portId}", not its assigned "${declared.portId}"`,
);
}
const port = findPort(descriptor.environment, declared.portId);
if (!port) fail('CommittedSnapshot: assigned port is not declared');
validatePortControlAgainst(agent.appliedControls, port.controls);
}
}
}

View file

@ -0,0 +1,231 @@
/**
* Reading one JSON object field by field, then refusing any field that was not read.
*
* The Rust crate's `flybus::wire::Fields` does the same job; keeping the two shaped alike is
* what lets the fixture corpus hold both languages to the same rules.
*/
import { ContractError, canonicalize, fail } from './canonical';
import {
type Digest,
type Id,
type U64,
type ArtifactRef,
isDigest,
isId,
parseU64,
requireU64,
} from './scalar';
export class Reader {
private readonly map: Record<string, unknown>;
private readonly seen = new Set<string>();
constructor(
value: unknown,
private readonly what: string,
) {
if (value === null || typeof value !== 'object' || Array.isArray(value)) {
fail(`${what} must be an object`);
}
this.map = value as Record<string, unknown>;
}
value(key: string): unknown {
this.seen.add(key);
if (!Object.prototype.hasOwnProperty.call(this.map, key)) {
fail(`${this.what}: missing field "${key}"`);
}
return this.map[key];
}
string(key: string): string {
const value = this.value(key);
if (typeof value !== 'string') fail(`${this.what}: ${key} must be a string`);
return value;
}
id(key: string): Id {
const value = this.string(key);
if (!isId(value)) fail(`${this.what}: ${key} is not a valid id`);
return value;
}
nullableId(key: string): Id | null {
const value = this.value(key);
if (value === null) return null;
return this.id(key);
}
digest(key: string): Digest {
const value = this.string(key);
if (!isDigest(value)) fail(`${this.what}: ${key} must be 64 lowercase hex digits`);
return value;
}
u64(key: string): U64 {
return requireU64(this.value(key), `${this.what}: ${key}`);
}
int(key: string, low: number, high: number): number {
const value = this.value(key);
if (typeof value !== 'number' || !Number.isInteger(value) || value < low || value > high) {
fail(`${this.what}: ${key} must be an integer in ${low}..=${high}`);
}
return value;
}
finite(key: string): number {
const value = this.value(key);
if (typeof value !== 'number' || !Number.isFinite(value)) {
fail(`${this.what}: ${key} must be a finite JSON number`);
}
if (Number.isInteger(value) && Math.abs(value) > Number.MAX_SAFE_INTEGER) {
fail(`${this.what}: ${key} is outside the exact double range`);
}
return value;
}
finiteIn(key: string, low: number, high: number): number {
const value = this.finite(key);
if (value < low || value > high) fail(`${this.what}: ${key} must be in [${low}, ${high}]`);
return value;
}
boolean(key: string): boolean {
const value = this.value(key);
if (typeof value !== 'boolean') fail(`${this.what}: ${key} must be a boolean`);
return value;
}
constantTrue(key: string): true {
if (!this.boolean(key)) fail(`${this.what}: ${key} must be true`);
return true;
}
object(key: string): Record<string, unknown> {
const value = this.value(key);
if (value === null || typeof value !== 'object' || Array.isArray(value)) {
fail(`${this.what}: ${key} must be an object`);
}
return value as Record<string, unknown>;
}
array(key: string, low: number, high: number): unknown[] {
const value = this.value(key);
if (!Array.isArray(value) || value.length < low || value.length > high) {
fail(`${this.what}: ${key} must be an array of ${low}..=${high} items`);
}
return value;
}
list<T>(key: string, low: number, high: number, read: (item: unknown) => T): T[] {
return this.array(key, low, high).map((item, index) => {
try {
return read(item);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
throw new ContractError(`${this.what}: ${key}[${index}]: ${message}`);
}
});
}
idList(key: string, low: number, high: number): Id[] {
return this.list(key, low, high, (item) => {
if (!isId(item)) fail('every entry must be an id');
return item;
});
}
enumeration<T extends string>(key: string, allowed: readonly T[]): T {
const value = this.string(key);
if (!(allowed as readonly string[]).includes(value)) {
fail(`${this.what}: ${key} must be one of ${allowed.join(', ')}`);
}
return value as T;
}
constant<T extends string>(key: string, expected: T): T {
const value = this.string(key);
if (value !== expected) fail(`${this.what}: ${key} must be "${expected}"`);
return expected;
}
boundedString(key: string, maxCodePoints: number): string {
const value = this.string(key);
if ([...value].length > maxCodePoints) {
fail(`${this.what}: ${key} must be at most ${maxCodePoints} code points`);
}
return value;
}
nullableBoundedString(key: string, maxCodePoints: number): string | null {
return this.value(key) === null ? null : this.boundedString(key, maxCodePoints);
}
/** Refuses fields that were not read. */
finish(): void {
for (const key of Object.keys(this.map)) {
if (!this.seen.has(key)) fail(`${this.what}: unknown field "${key}"`);
}
}
}
/** Fails on the first repeated key, naming it. */
export function requireUnique(keys: readonly string[], what: string): void {
const seen = new Set<string>();
for (const key of keys) {
if (seen.has(key)) fail(`${what}: duplicate "${key}"`);
seen.add(key);
}
}
/** Fails unless `actual` is exactly `expected`, in that order. */
export function requireSameOrder(
actual: readonly string[],
expected: readonly string[],
what: string,
): void {
if (actual.length !== expected.length || actual.some((key, index) => key !== expected[index])) {
fail(
`${what}: must list [${expected.join(', ')}] in that order, found [${actual.join(', ')}]`,
);
}
}
/** A bus `ArtifactRef`, read with the bus's own rules. */
export function readArtifactRef(value: unknown): ArtifactRef {
const reader = new Reader(value, 'ArtifactRef');
const storeId = reader.id('storeId');
const artifactId = reader.id('artifactId');
const generation = reader.u64('generation');
const byteLength = reader.u64('byteLength');
const contentType = reader.string('contentType');
const digestValue = reader.value('digest');
reader.finish();
if (contentType.length < 1 || contentType.length > 127) {
fail('ArtifactRef: contentType must be 1..=127 printable ASCII characters');
}
if (digestValue !== null && !isDigest(digestValue)) {
fail('ArtifactRef: digest must be null or 64 lowercase hex digits');
}
return {
storeId,
artifactId,
generation,
byteLength,
contentType,
digest: digestValue as ArtifactRef['digest'],
};
}
/** The canonical JSON byte length of a value. */
export function canonicalLength(value: unknown): number {
return canonicalize(value).length;
}
/** `parseU64` that throws, for places that have already validated the string. */
export function u64(value: U64): bigint {
const parsed = parseU64(value);
if (parsed === undefined) fail(`${value} is not a canonical U64 string`);
return parsed;
}

View file

@ -0,0 +1,140 @@
/** The domain request/reply envelope of ipc-v1 section 3 and the error codes of section 7. */
import { fail, rejectBusIdentities } from './canonical';
import { bodyDigest, readNullableScope } from './common';
import { Reader } from './reader';
import { type DomainRequestId, type Id, type Scope, domainRequestId } from './scalar';
import { MAX_MESSAGE_CODE_POINTS } from './workers';
export const ERROR_CODES = [
'INVALID_ARGUMENT',
'UNSUPPORTED',
'IDENTITY_MISMATCH',
'STALE_EPOCH',
'STALE_STEP',
'FUTURE_STEP',
'INVALID_PHASE',
'CONFLICT',
'IN_PROGRESS',
'BUSY',
'BUFFER_INVALID',
'RESULT_EXPIRED',
'INCOMPATIBLE_STATE',
'BACKEND_FAILURE',
'INTERNAL',
] as const;
export type ErrorCode = (typeof ERROR_CODES)[number];
export const MUTATION_CERTAINTIES = ['none', 'applied', 'unknown'] as const;
export type MutationCertainty = (typeof MUTATION_CERTAINTIES)[number];
/** The codes raised strictly before any mutation, so their certainty is `none`. */
export const BEFORE_MUTATION: readonly ErrorCode[] = [
'INVALID_ARGUMENT',
'UNSUPPORTED',
'IDENTITY_MISMATCH',
'STALE_EPOCH',
'STALE_STEP',
'FUTURE_STEP',
'INVALID_PHASE',
'CONFLICT',
'IN_PROGRESS',
'BUSY',
'BUFFER_INVALID',
'INCOMPATIBLE_STATE',
];
export interface SessionRpcRequest {
requestId: DomainRequestId;
scope: Scope | null;
params: Record<string, unknown>;
}
export interface SessionRpcSuccess {
type: 'result';
requestId: DomainRequestId;
workerId: Id;
incarnationId: Id;
scope: Scope | null;
result: Record<string, unknown>;
}
export interface SessionRpcFailure {
type: 'error';
requestId: DomainRequestId;
workerId: Id;
incarnationId: Id;
scope: Scope | null;
error: { code: ErrorCode; message: string; mutation: MutationCertainty };
}
export type SessionRpcOutcome = SessionRpcSuccess | SessionRpcFailure;
export function readSessionRpcRequest(value: unknown): SessionRpcRequest {
const reader = new Reader(value, 'SessionRpcRequest');
const request: SessionRpcRequest = {
requestId: domainRequestId(reader.string('requestId')),
scope: readNullableScope(reader.value('scope')),
params: reader.object('params'),
};
reader.finish();
rejectBusIdentities(request.params);
return request;
}
/** The canonical body digest of this request under `method` (ipc-v1 section 5). */
export function requestBodyDigest(request: SessionRpcRequest, method: string): string {
return bodyDigest(method, request.scope, request.params);
}
export function readSessionRpcSuccess(value: unknown): SessionRpcSuccess {
const reader = new Reader(value, 'SessionRpcSuccess');
const success: SessionRpcSuccess = {
type: reader.constant('type', 'result'),
requestId: domainRequestId(reader.string('requestId')),
workerId: reader.id('workerId'),
incarnationId: reader.id('incarnationId'),
scope: readNullableScope(reader.value('scope')),
result: reader.object('result'),
};
reader.finish();
return success;
}
export function readSessionRpcFailure(value: unknown): SessionRpcFailure {
const reader = new Reader(value, 'SessionRpcFailure');
const type = reader.constant('type', 'error');
const requestId = domainRequestId(reader.string('requestId'));
const workerId = reader.id('workerId');
const incarnationId = reader.id('incarnationId');
const scope = readNullableScope(reader.value('scope'));
const errorReader = new Reader(reader.value('error'), 'SessionRpcFailure.error');
const error = {
code: errorReader.enumeration('code', ERROR_CODES),
message: errorReader.boundedString('message', MAX_MESSAGE_CODE_POINTS),
mutation: errorReader.enumeration('mutation', MUTATION_CERTAINTIES),
};
errorReader.finish();
reader.finish();
if (BEFORE_MUTATION.includes(error.code) && error.mutation !== 'none') {
fail(`SessionRpcFailure: ${error.code} is raised before mutation, so mutation is "none"`);
}
return { type, requestId, workerId, incarnationId, scope, error };
}
export function readSessionRpcOutcome(value: unknown): SessionRpcOutcome {
const type = (value as { type?: unknown } | null)?.type;
if (type === 'result') return readSessionRpcSuccess(value);
if (type === 'error') return readSessionRpcFailure(value);
return fail('SessionRpcOutcome: type must be "result" or "error"');
}
/** Replies echo the original scope (ipc-v1 section 3). */
export function echoes(outcome: SessionRpcOutcome, request: SessionRpcRequest): boolean {
const sameScope =
outcome.scope === null || request.scope === null
? outcome.scope === request.scope
: outcome.scope.sessionId === request.scope.sessionId &&
outcome.scope.epoch === request.scope.epoch &&
outcome.scope.step === request.scope.step;
return outcome.requestId === request.requestId && sameScope;
}

View file

@ -0,0 +1,262 @@
/**
* The domain scalars of ipc-v1 section 2, and the four identities that must never be confused.
*
* `Id`, `U64` and `Digest` are the bus encodings (bus-v1 section 3 defers to ipc-v1 for them),
* and `tests/encodings.test.ts` pins the same edge cases the Rust crate pins.
*/
import { fail } from './canonical';
/** `^[a-z0-9][a-z0-9._-]{0,63}$`. */
export type Id = string;
/** `"0"` or `[1-9][0-9]*`, at most 18446744073709551615. A counter, never a JSON number. */
export type U64 = string;
/** 64 lowercase hexadecimal digits (SHA-256). */
export type Digest = string;
const ID = /^[a-z0-9][a-z0-9._-]{0,63}$/;
const U64_TEXT = /^(0|[1-9][0-9]*)$/;
const DIGEST = /^[0-9a-f]{64}$/;
/** The largest U64, as a bigint. */
export const U64_MAX = 18446744073709551615n;
export function isId(value: unknown): value is Id {
return typeof value === 'string' && ID.test(value);
}
export function isDigest(value: unknown): value is Digest {
return typeof value === 'string' && DIGEST.test(value);
}
/** The value a `U64` string denotes, or `undefined` if it is not canonical. */
export function parseU64(value: unknown): bigint | undefined {
if (typeof value !== 'string' || !U64_TEXT.test(value)) return undefined;
const parsed = BigInt(value);
return parsed <= U64_MAX ? parsed : undefined;
}
export function requireU64(value: unknown, what: string): U64 {
if (parseU64(value) === undefined) fail(`${what} is not a canonical U64 string`);
return value as U64;
}
/** RPC method: 1..=128 printable ASCII characters (bus-v1 section 5). */
export function isMethod(value: unknown): boolean {
return (
typeof value === 'string' &&
value.length >= 1 &&
value.length <= 128 &&
[...value].every((character) => {
const point = character.codePointAt(0) ?? 0;
return point >= 0x20 && point <= 0x7e;
})
);
}
// ---------------------------------------------------------------------------------------------
// The four identities
declare const brand: unique symbol;
/** A branded string: assignable only through its own parser. */
type Branded<Name extends string> = string & { readonly [brand]: Name };
/** A bus RPC correlation id, `call-<U64>`. Not a domain operation id. */
export type BusCallId = Branded<'BusCallId'>;
/** A domain operation id, `req-<U64>`. A safe retry keeps it and gets a new `BusCallId`. */
export type DomainRequestId = Branded<'DomainRequestId'>;
/** A delivery (`dlv-<U64>`) or explicit-hold (`own-<U64>`) owner token. Connection-private. */
export type OwnerToken = Branded<'OwnerToken'>;
export type OwnerKind = 'delivery' | 'hold';
function serial(prefix: string, value: unknown): bigint | undefined {
if (typeof value !== 'string' || !value.startsWith(`${prefix}-`)) return undefined;
return parseU64(value.slice(prefix.length + 1));
}
export function isBusCallId(value: unknown): value is BusCallId {
return serial('call', value) !== undefined;
}
export function isDomainRequestId(value: unknown): value is DomainRequestId {
return serial('req', value) !== undefined;
}
export function ownerTokenKind(value: unknown): OwnerKind | undefined {
if (serial('dlv', value) !== undefined) return 'delivery';
if (serial('own', value) !== undefined) return 'hold';
return undefined;
}
export function busCallId(value: unknown): BusCallId {
if (!isBusCallId(value)) fail('a bus callId must be canonical call-<U64>');
return value;
}
export function domainRequestId(value: unknown): DomainRequestId {
if (!isDomainRequestId(value)) fail('a domain requestId must be canonical req-<U64>');
return value;
}
export function ownerToken(value: unknown): OwnerToken {
if (ownerTokenKind(value) === undefined) {
fail('an owner token must be canonical dlv-<U64> or own-<U64>');
}
return value as OwnerToken;
}
/** The naming half of a bus `ArtifactRef`: what identifies the bytes. */
export interface ArtifactIdentity {
storeId: Id;
artifactId: Id;
generation: U64;
}
/** A transient bus artifact reference (bus-v1 section 4). Never an `AssetRef`. */
export interface ArtifactRef {
storeId: Id;
artifactId: Id;
generation: U64;
byteLength: U64;
contentType: string;
digest: Digest | null;
}
export function artifactIdentity(reference: ArtifactRef): ArtifactIdentity {
return {
storeId: reference.storeId,
artifactId: reference.artifactId,
generation: reference.generation,
};
}
// ---------------------------------------------------------------------------------------------
// RationalNs
/** A nanosecond rational: reduced, positive denominator, zero encoded `0/1`. */
export interface RationalNs {
numerator: U64;
denominator: U64;
}
export const RATIONAL_ZERO: RationalNs = { numerator: '0', denominator: '1' };
function gcd(a: bigint, b: bigint): bigint {
let left = a;
let right = b;
while (right !== 0n) {
const rest = left % right;
left = right;
right = rest;
}
return left;
}
function parts(value: RationalNs, what: string): [bigint, bigint] {
const numerator = parseU64(value.numerator);
const denominator = parseU64(value.denominator);
if (numerator === undefined || denominator === undefined) {
fail(`${what}: numerator and denominator are U64 strings`);
}
return [numerator, denominator];
}
/** The canonical-form rules: positive denominator, `0/1` zero, reduced fraction. */
export function validateRational(value: RationalNs, what = 'RationalNs'): void {
const [numerator, denominator] = parts(value, what);
if (denominator === 0n) fail(`${what}: denominator must be positive`);
if (numerator === 0n && denominator !== 1n) fail(`${what}: zero is encoded 0/1`);
if (numerator !== 0n && gcd(numerator, denominator) !== 1n) {
fail(`${what}: fraction must be reduced`);
}
}
/** Reduces, then validates: the constructor for arithmetic results. */
export function reduced(numerator: bigint, denominator: bigint): RationalNs {
if (denominator <= 0n) fail('RationalNs: denominator must be positive');
let n = numerator;
let d = denominator;
if (n === 0n) {
d = 1n;
} else {
const divisor = gcd(n, d);
n /= divisor;
d /= divisor;
}
if (n > U64_MAX || d > U64_MAX) fail('RationalNs: reduced value does not fit U64');
return { numerator: n.toString(), denominator: d.toString() };
}
export function isRationalZero(value: RationalNs): boolean {
return parseU64(value.numerator) === 0n;
}
export function requirePositiveRational(value: RationalNs, what: string): void {
if (isRationalZero(value)) fail(`${what}: duration must be positive`);
}
export function addRational(left: RationalNs, right: RationalNs): RationalNs {
const [ln, ld] = parts(left, 'RationalNs');
const [rn, rd] = parts(right, 'RationalNs');
return reduced(ln * rd + rn * ld, ld * rd);
}
export function subtractRational(left: RationalNs, right: RationalNs): RationalNs {
const [ln, ld] = parts(left, 'RationalNs');
const [rn, rd] = parts(right, 'RationalNs');
const a = ln * rd;
const b = rn * ld;
if (b > a) fail('RationalNs: subtraction would be negative');
return reduced(a - b, ld * rd);
}
export function multiplyRational(value: RationalNs, factor: bigint): RationalNs {
const [numerator, denominator] = parts(value, 'RationalNs');
return reduced(numerator * factor, denominator);
}
export function compareRational(left: RationalNs, right: RationalNs): -1 | 0 | 1 {
const [ln, ld] = parts(left, 'RationalNs');
const [rn, rd] = parts(right, 'RationalNs');
const a = ln * rd;
const b = rn * ld;
return a < b ? -1 : a > b ? 1 : 0;
}
/**
* The step-v1 section 5 accumulator: `ticks = floor(value / tick)` and the remainder
* `value - ticks * tick`, which is always `>= 0` and `< tick`.
*/
export function divideFloor(value: RationalNs, tick: RationalNs): { ticks: U64; remainder: RationalNs } {
requirePositiveRational(tick, 'RationalNs divideFloor tick');
const [vn, vd] = parts(value, 'RationalNs');
const [tn, td] = parts(tick, 'RationalNs');
const ticks = (vn * td) / (vd * tn);
if (ticks > U64_MAX) fail('RationalNs: tick count does not fit U64');
const remainder = subtractRational(value, multiplyRational(tick, ticks));
return { ticks: ticks.toString(), remainder };
}
// ---------------------------------------------------------------------------------------------
// Scope, SchemaRef, TypedValue
/** The simulation timeline identity. Never a bus route or store incarnation. */
export interface Scope {
sessionId: Id;
epoch: Id;
step: U64;
}
export interface SchemaRef {
id: Id;
version: number;
digest: Digest;
}
/** A schema identity plus an object, capped at 32 KiB of canonical JSON. */
export interface TypedValue {
schema: SchemaRef;
value: Record<string, unknown>;
}
/** The canonical-JSON size limit of one `TypedValue`. */
export const MAX_TYPED_VALUE_BYTES = 32 * 1024;

View file

@ -0,0 +1,57 @@
/**
* `seed-derivation-v1`: independent per-agent seeds from one recorded master seed.
*
* The specification is `docs/design/session-framework/seed-derivation-v1.md`, and
* `fixtures/seed-vectors.json` its test vectors, which the Rust crate reproduces.
*/
import { createHash } from 'node:crypto';
import { fail } from './canonical';
import { requireUnique } from './reader';
import { isId, parseU64 } from './scalar';
export const ALGORITHM = 'seed-derivation-v1';
export const PREFIX = 'flybrain/seed-derivation-v1';
/** The exact bytes hashed: prefix, master seed and agent id, each followed by one newline. */
export function material(masterSeed: bigint, agentId: string): Uint8Array {
if (!isId(agentId)) fail('seed derivation: agentId is not a valid id');
if (masterSeed < 0n || masterSeed > 18446744073709551615n) {
fail('seed derivation: the master seed is a U64');
}
return new TextEncoder().encode(`${PREFIX}\n${masterSeed.toString()}\n${agentId}\n`);
}
export function materialDigest(masterSeed: bigint, agentId: string): string {
return createHash('sha256').update(material(masterSeed, agentId)).digest('hex');
}
/**
* The signed 32-bit seed `Agent.Initialize` takes for `agentId`: the first nonzero big-endian
* `u32` lane of the digest, as a two's-complement `i32`.
*/
export function agentSeed(masterSeed: bigint, agentId: string): number {
let bytes = Buffer.from(material(masterSeed, agentId));
for (let round = 0; round < 4; round += 1) {
if (round > 0) bytes = Buffer.concat([bytes, Buffer.from(`${round}\n`, 'utf8')]);
const digest = createHash('sha256').update(bytes).digest();
for (let offset = 0; offset < digest.length; offset += 4) {
const word = digest.readUInt32BE(offset);
if (word !== 0) return word | 0;
}
}
return fail('seed derivation: every lane of four digests was zero');
}
/** The seeds of a whole composition, in the order the agent ids are given. */
export function compositionSeeds(masterSeed: bigint, agentIds: readonly string[]): number[] {
requireUnique(agentIds, 'seed derivation: agentIds');
return agentIds.map((agentId) => agentSeed(masterSeed, agentId));
}
/** Parses a master seed from its `U64` decimal string. */
export function masterSeed(text: string): bigint {
const parsed = parseU64(text);
if (parsed === undefined) fail('seed derivation: the master seed is a canonical U64 string');
return parsed;
}

View file

@ -0,0 +1,268 @@
/**
* The trace format of step-v1 section 8, split into behaviour and operational metadata.
*
* `behaviourEquals` compares the first half only, which is the comparison section 8 asks for:
* sequential, concurrent and reversed runs must agree "excluding wall time, request ids and
* other explicitly operational metadata".
*/
import { canonicalize, digestOf, fail } from './canonical';
import { readRational, readScope } from './common';
import { MAX_VIEWS } from './media';
import { Reader, requireUnique, u64 } from './reader';
import {
type BusCallId,
type Digest,
type DomainRequestId,
type Id,
type OwnerToken,
type RationalNs,
type Scope,
type U64,
busCallId,
domainRequestId,
ownerToken,
} from './scalar';
import { MAX_AGENTS, MAX_RATE_ROLES } from './workers';
export interface TraceAgent {
agentId: Id;
profileDigest: Digest;
ticksAdvanced: U64;
brainTicks: U64;
remainder: RationalNs;
decisionDigest: Digest;
committedStep: U64;
}
export interface TraceObservation {
viewId: Id;
producedStep: U64;
}
export interface TraceBehaviour {
scope: Scope;
agents: TraceAgent[];
batchId: Id;
controlDigest: Digest;
acknowledgedBoundary: U64;
observationBoundaries: TraceObservation[];
outcomeIds: Id[];
eventIds: Id[];
publishedBoundary: U64;
}
export interface TraceRequest {
agentId: Id;
requestId: DomainRequestId;
}
export interface TraceOperational {
wallTimeNs: U64;
prepareRequestIds: TraceRequest[];
advanceRequestId: DomainRequestId;
commitRequestIds: TraceRequest[];
busCallIds: BusCallId[];
deliveryIds: OwnerToken[];
}
export interface TransitionTrace {
behaviour: TraceBehaviour;
operational: TraceOperational;
}
export function readTraceBehaviour(value: unknown): TraceBehaviour {
const reader = new Reader(value, 'TraceBehaviour');
const scope = readScope(reader.value('scope'));
const agents = reader.list('agents', 1, MAX_AGENTS, (item) => {
const agent = new Reader(item, 'TraceBehaviour.agents');
const entry: TraceAgent = {
agentId: agent.id('agentId'),
profileDigest: agent.digest('profileDigest'),
ticksAdvanced: agent.u64('ticksAdvanced'),
brainTicks: agent.u64('brainTicks'),
remainder: readRational(agent.value('remainder')),
decisionDigest: agent.digest('decisionDigest'),
committedStep: agent.u64('committedStep'),
};
agent.finish();
return entry;
});
const batchId = reader.id('batchId');
const controlDigest = reader.digest('controlDigest');
const acknowledgedBoundary = reader.u64('acknowledgedBoundary');
const observationBoundaries = reader.list(
'observationBoundaries',
0,
MAX_VIEWS * 2,
(item) => {
const observation = new Reader(item, 'TraceBehaviour.observationBoundaries');
const entry: TraceObservation = {
viewId: observation.id('viewId'),
producedStep: observation.u64('producedStep'),
};
observation.finish();
return entry;
},
);
const outcomeIds = reader.idList('outcomeIds', 0, MAX_RATE_ROLES);
const eventIds = reader.idList('eventIds', 0, MAX_RATE_ROLES);
const publishedBoundary = reader.u64('publishedBoundary');
reader.finish();
requireUnique(
agents.map((agent) => agent.agentId),
'TraceBehaviour.agents',
);
requireUnique(
observationBoundaries.map((observation) => observation.viewId),
'TraceBehaviour.observationBoundaries',
);
requireUnique(eventIds, 'TraceBehaviour.eventIds');
requireUnique(outcomeIds, 'TraceBehaviour.outcomeIds');
const next = u64(scope.step) + 1n;
for (const agent of agents) {
if (u64(agent.committedStep) !== next) {
fail("TraceBehaviour: every commit acknowledgment is the transition's next boundary");
}
}
if (u64(acknowledgedBoundary) !== next) {
fail('TraceBehaviour: the acknowledged boundary is scope.step + 1');
}
if (u64(publishedBoundary) !== u64(acknowledgedBoundary)) {
fail('TraceBehaviour: the published boundary is the boundary every agent committed');
}
return {
scope,
agents,
batchId,
controlDigest,
acknowledgedBoundary,
observationBoundaries,
outcomeIds,
eventIds,
publishedBoundary,
};
}
export function readTraceOperational(value: unknown): TraceOperational {
const reader = new Reader(value, 'TraceOperational');
const readRequests = (item: unknown): TraceRequest => {
const request = new Reader(item, 'TraceOperational request');
const entry: TraceRequest = {
agentId: request.id('agentId'),
requestId: domainRequestId(request.string('requestId')),
};
request.finish();
return entry;
};
const operational: TraceOperational = {
wallTimeNs: reader.u64('wallTimeNs'),
prepareRequestIds: reader.list('prepareRequestIds', 1, MAX_AGENTS, readRequests),
advanceRequestId: domainRequestId(reader.string('advanceRequestId')),
commitRequestIds: reader.list('commitRequestIds', 1, MAX_AGENTS, readRequests),
busCallIds: reader.list('busCallIds', 0, 64, busCallId),
deliveryIds: reader.list('deliveryIds', 0, 64, ownerToken),
};
reader.finish();
requireUnique(
operational.prepareRequestIds.map((request) => request.agentId),
'TraceOperational.prepareRequestIds',
);
requireUnique(
operational.commitRequestIds.map((request) => request.agentId),
'TraceOperational.commitRequestIds',
);
requireUnique(operational.busCallIds, 'TraceOperational.busCallIds');
requireUnique(operational.deliveryIds, 'TraceOperational.deliveryIds');
return operational;
}
export function readTransitionTrace(value: unknown): TransitionTrace {
const reader = new Reader(value, 'TransitionTrace');
const trace: TransitionTrace = {
behaviour: readTraceBehaviour(reader.value('behaviour')),
operational: readTraceOperational(reader.value('operational')),
};
reader.finish();
const agents = trace.behaviour.agents.map((agent) => agent.agentId);
for (const phase of [trace.operational.prepareRequestIds, trace.operational.commitRequestIds]) {
for (const request of phase) {
if (!agents.includes(request.agentId)) {
fail(
`TransitionTrace: request recorded for "${request.agentId}", which is not in the transition`,
);
}
}
}
return trace;
}
/** Sorts the order-free collections, so completion order cannot change the comparison. */
export function normalizeBehaviour(behaviour: TraceBehaviour): TraceBehaviour {
return {
...behaviour,
agents: [...behaviour.agents].sort((left, right) =>
left.agentId < right.agentId ? -1 : left.agentId > right.agentId ? 1 : 0,
),
observationBoundaries: [...behaviour.observationBoundaries].sort((left, right) =>
left.viewId < right.viewId ? -1 : left.viewId > right.viewId ? 1 : 0,
),
};
}
export function behaviourDigest(behaviour: TraceBehaviour): string {
return digestOf(normalizeBehaviour(behaviour));
}
export function behaviourEquals(left: TransitionTrace, right: TransitionTrace): boolean {
return (
canonicalize(normalizeBehaviour(left.behaviour)) ===
canonicalize(normalizeBehaviour(right.behaviour))
);
}
/** The behaviour fields that differ, named. Empty when `behaviourEquals` holds. */
export function behaviourDiff(left: TransitionTrace, right: TransitionTrace): string[] {
const a = normalizeBehaviour(left.behaviour);
const b = normalizeBehaviour(right.behaviour);
const out: string[] = [];
const differs = (first: unknown, second: unknown): boolean =>
canonicalize(first) !== canonicalize(second);
if (differs(a.scope, b.scope)) out.push(`scope: ${canonicalize(a.scope)} vs ${canonicalize(b.scope)}`);
if (a.batchId !== b.batchId) out.push(`batchId: ${a.batchId} vs ${b.batchId}`);
if (a.controlDigest !== b.controlDigest) out.push('controlDigest differs');
if (a.acknowledgedBoundary !== b.acknowledgedBoundary) {
out.push(`acknowledgedBoundary: ${a.acknowledgedBoundary} vs ${b.acknowledgedBoundary}`);
}
if (a.publishedBoundary !== b.publishedBoundary) {
out.push(`publishedBoundary: ${a.publishedBoundary} vs ${b.publishedBoundary}`);
}
if (differs(a.observationBoundaries, b.observationBoundaries)) {
out.push('observationBoundaries differ');
}
if (differs(a.outcomeIds, b.outcomeIds)) out.push('outcomeIds differ');
if (differs(a.eventIds, b.eventIds)) out.push('eventIds differ');
const idsA = a.agents.map((agent) => agent.agentId);
const idsB = b.agents.map((agent) => agent.agentId);
if (differs(idsA, idsB)) {
out.push(`agents: [${idsA.join(', ')}] vs [${idsB.join(', ')}]`);
} else {
a.agents.forEach((agent, index) => {
if (differs(agent, b.agents[index])) {
out.push(`agent ${agent.agentId}: behaviour differs`);
}
});
}
return out;
}
/** Two whole runs agree on behaviour, transition by transition. */
export function runsEqual(
left: readonly TransitionTrace[],
right: readonly TransitionTrace[],
): boolean {
return (
left.length === right.length &&
left.every((trace, index) => behaviourEquals(trace, right[index] as TransitionTrace))
);
}

View file

@ -0,0 +1,996 @@
/** The closed enums and method payloads of workers-v1, plus the `Worker.*` common methods. */
import { fail } from './canonical';
import { readRational, readScope } from './common';
import {
type AudioDescriptor,
type AudioRef,
MAX_VIEWS,
type ViewDescriptor,
type ViewRef,
readAudioDescriptor,
readAudioList,
readViewDescriptor,
readViewList,
validateAudioAgainst,
validateViewAgainst,
} from './media';
import { Reader, requireSameOrder, requireUnique, u64 } from './reader';
import {
type Digest,
type DomainRequestId,
type Id,
type RationalNs,
type SchemaRef,
type Scope,
type TypedValue,
type U64,
addRational,
compareRational,
domainRequestId,
isRationalZero,
requirePositiveRational,
} from './scalar';
import { readSchemaRef, readTypedValue, readNullableTypedValue } from './common';
export const MAX_AGENTS = 4;
export const MAX_PORTS = 4;
export const MAX_RATE_ROLES = 64;
export const MAX_STIMULI = 64;
export const MAX_REWARDS = 64;
export const MAX_BUTTONS = 32;
export const MAX_AXES = 16;
export const MAX_ACKNOWLEDGE = 16;
export const MAX_ENGINE_FRAME_LEN = 64;
/** Not stated by a document; this crate's choice, published in the schema set. */
export const MAX_CAPABILITIES = 32;
export const MAX_SUPPORTED_MAJORS = 8;
export const MAX_MESSAGE_CODE_POINTS = 512;
export const ROLES = ['agent', 'environment', 'coordinator'] as const;
export type Role = (typeof ROLES)[number];
export const WORKER_STATES = [
'uninitialized',
'ready',
'preparing',
'prepared',
'advancing',
'committing',
'capturing',
'staged-restore',
'restoring',
'failed',
'stopping',
] as const;
export type WorkerState = (typeof WORKER_STATES)[number];
export const RECOVERY = ['exact-checkpoint', 'episode-restart'] as const;
export type Recovery = (typeof RECOVERY)[number];
export const DETERMINISM = ['fixed-build', 'unverified'] as const;
export type Determinism = (typeof DETERMINISM)[number];
export const AXIS_RANGES = ['bipolar', 'unit'] as const;
export type AxisRange = (typeof AXIS_RANGES)[number];
export function axisBounds(range: AxisRange): [number, number] {
return range === 'bipolar' ? [-1, 1] : [0, 1];
}
export interface AssetRef {
id: Id;
digest: Digest;
byteLength: U64;
format: Id;
}
export interface SensoryInput {
boundary: U64;
views: ViewRef[];
structured: TypedValue | null;
}
export interface Stimulus {
id: Id;
kindId: Id;
durationMs: number;
}
export interface Reward {
eventId: Id;
ruleId: Id;
value: number;
}
export interface AgentTelemetry {
brainTicks: U64;
populationRateHz: number;
rates: { roleId: Id; hz: number }[];
learning: { enabled: boolean; updates: U64; changed: U64; signal: number };
}
export function readAssetRef(value: unknown): AssetRef {
const reader = new Reader(value, 'AssetRef');
const asset: AssetRef = {
id: reader.id('id'),
digest: reader.digest('digest'),
byteLength: reader.u64('byteLength'),
format: reader.id('format'),
};
reader.finish();
if (u64(asset.byteLength) === 0n) fail('AssetRef: byteLength must be positive');
return asset;
}
export function readSensoryInput(value: unknown): SensoryInput {
const reader = new Reader(value, 'SensoryInput');
const input: SensoryInput = {
boundary: reader.u64('boundary'),
views: readViewList(reader, 'views'),
structured: readNullableTypedValue(reader.value('structured')),
};
reader.finish();
for (const view of input.views) {
if (u64(view.producedStep) > u64(input.boundary)) {
fail(`SensoryInput: view "${view.viewId}" was produced after the observed boundary`);
}
}
return input;
}
/** Required sensory views against the descriptors that declared them. */
export function validateSensoryInputAgainst(
input: SensoryInput,
descriptors: readonly ViewDescriptor[],
): void {
for (const view of input.views) {
const descriptor = descriptors.find((candidate) => candidate.viewId === view.viewId);
if (!descriptor) {
fail(`SensoryInput: view "${view.viewId}" is not declared by the environment`);
}
validateViewAgainst(view, descriptor, u64(input.boundary));
}
}
/** A pixel-only profile rejects non-null structured input (workers-v1 section 1). */
export function validateSensoryInputForProfile(
input: SensoryInput,
structuredSensing: boolean,
): void {
if (input.structured !== null && !structuredSensing) {
fail('SensoryInput: a pixel-only profile rejects non-null structured input');
}
}
export function readStimulus(value: unknown): Stimulus {
const reader = new Reader(value, 'Stimulus');
const stimulus: Stimulus = {
id: reader.id('id'),
kindId: reader.id('kindId'),
durationMs: reader.finite('durationMs'),
};
reader.finish();
if (stimulus.durationMs <= 0) fail('Stimulus: durationMs must be finite and > 0');
return stimulus;
}
export function readReward(value: unknown): Reward {
const reader = new Reader(value, 'Reward');
const reward: Reward = {
eventId: reader.id('eventId'),
ruleId: reader.id('ruleId'),
value: reader.finite('value'),
};
reader.finish();
return reward;
}
export function readStimulusList(reader: Reader, key: string): Stimulus[] {
const items = reader.list(key, 0, MAX_STIMULI, readStimulus);
requireUnique(
items.map((item) => item.id),
key,
);
return items;
}
export function readRewardList(reader: Reader, key: string): Reward[] {
const items = reader.list(key, 0, MAX_REWARDS, readReward);
requireUnique(
items.map((item) => item.eventId),
key,
);
return items;
}
export function readAgentTelemetry(value: unknown): AgentTelemetry {
const reader = new Reader(value, 'AgentTelemetry');
const brainTicks = reader.u64('brainTicks');
const populationRateHz = reader.finiteIn('populationRateHz', 0, Number.MAX_VALUE);
const rates = reader.list(('rates'), 0, MAX_RATE_ROLES, (item) => {
const rate = new Reader(item, 'AgentTelemetry.rates');
const entry = { roleId: rate.id('roleId'), hz: rate.finiteIn('hz', 0, Number.MAX_VALUE) };
rate.finish();
return entry;
});
const learningReader = new Reader(reader.value('learning'), 'AgentTelemetry.learning');
const learning = {
enabled: learningReader.boolean('enabled'),
updates: learningReader.u64('updates'),
changed: learningReader.u64('changed'),
signal: learningReader.finite('signal'),
};
learningReader.finish();
reader.finish();
requireUnique(
rates.map((rate) => rate.roleId),
'AgentTelemetry.rates',
);
if (u64(learning.changed) > u64(learning.updates)) {
fail('AgentTelemetry: learning.changed cannot exceed learning.updates');
}
return { brainTicks, populationRateHz, rates, learning };
}
/** Rates are in profile-defined order (workers-v1 section 1). */
export function validateTelemetryRoles(
telemetry: AgentTelemetry,
roleOrder: readonly string[],
): void {
requireSameOrder(
telemetry.rates.map((rate) => rate.roleId),
roleOrder,
'AgentTelemetry.rates',
);
}
// ------------------------------------------------------------------------------ agent methods
export interface AgentInitializeParams {
agentId: Id;
profile: AssetRef;
seed: number;
initialInput: SensoryInput;
initialDecisionContext: TypedValue;
workerThreads: number;
}
export interface AgentInitializeResult {
agentId: Id;
profileDigest: Digest;
tickDuration: RationalNs;
warmupTicks: U64;
committedStep: U64;
decisionContextDigest: Digest;
telemetry: AgentTelemetry;
}
export interface PrepareParams {
agentId: Id;
profileDigest: Digest;
interval: RationalNs;
decisionContextDigest: Digest;
preStepStimulations: Stimulus[];
}
export interface PreparedDecision {
agentId: Id;
ticksAdvanced: U64;
brainTicks: U64;
remainder: RationalNs;
decision: TypedValue;
}
export interface CommitParams {
agentId: Id;
preparedRequestId: DomainRequestId;
nextInput: SensoryInput;
nextDecisionContext: TypedValue;
rewards: Reward[];
taskStimulations: Stimulus[];
}
export interface AgentCommitResult {
agentId: Id;
committedStep: U64;
decisionContextDigest: Digest;
telemetry: AgentTelemetry;
}
export function readAgentInitializeParams(value: unknown): AgentInitializeParams {
const reader = new Reader(value, 'AgentInitializeParams');
const params: AgentInitializeParams = {
agentId: reader.id('agentId'),
profile: readAssetRef(reader.value('profile')),
seed: reader.int('seed', -2_147_483_648, 2_147_483_647),
initialInput: readSensoryInput(reader.value('initialInput')),
initialDecisionContext: readTypedValue(reader.value('initialDecisionContext')),
workerThreads: reader.int('workerThreads', 1, 4_096),
};
reader.finish();
return params;
}
export function readAgentInitializeResult(value: unknown): AgentInitializeResult {
const reader = new Reader(value, 'AgentInitializeResult');
const result: AgentInitializeResult = {
agentId: reader.id('agentId'),
profileDigest: reader.digest('profileDigest'),
tickDuration: readRational(reader.value('tickDuration')),
warmupTicks: reader.u64('warmupTicks'),
committedStep: reader.u64('committedStep'),
decisionContextDigest: reader.digest('decisionContextDigest'),
telemetry: readAgentTelemetry(reader.value('telemetry')),
};
reader.finish();
requirePositiveRational(result.tickDuration, 'AgentInitializeResult.tickDuration');
if (u64(result.committedStep) !== 0n) {
fail('AgentInitializeResult: committedStep must be "0"');
}
return result;
}
export function readPrepareParams(value: unknown): PrepareParams {
const reader = new Reader(value, 'PrepareParams');
const params: PrepareParams = {
agentId: reader.id('agentId'),
profileDigest: reader.digest('profileDigest'),
interval: readRational(reader.value('interval')),
decisionContextDigest: reader.digest('decisionContextDigest'),
preStepStimulations: readStimulusList(reader, 'preStepStimulations'),
};
reader.finish();
requirePositiveRational(params.interval, 'PrepareParams.interval');
return params;
}
export function readPreparedDecision(value: unknown): PreparedDecision {
const reader = new Reader(value, 'PreparedDecision');
const decision: PreparedDecision = {
agentId: reader.id('agentId'),
ticksAdvanced: reader.u64('ticksAdvanced'),
brainTicks: reader.u64('brainTicks'),
remainder: readRational(reader.value('remainder')),
decision: readTypedValue(reader.value('decision')),
};
reader.finish();
if (u64(decision.ticksAdvanced) > u64(decision.brainTicks)) {
fail('PreparedDecision: ticksAdvanced cannot exceed the total brainTicks');
}
return decision;
}
/** The remainder is always `>= 0` and `< one model tick` (step-v1 section 5). */
export function validateRemainder(decision: PreparedDecision, tickDuration: RationalNs): void {
requirePositiveRational(tickDuration, 'tickDuration');
if (compareRational(decision.remainder, tickDuration) >= 0) {
fail('PreparedDecision: remainder must be less than one model tick');
}
}
export function readCommitParams(value: unknown): CommitParams {
const reader = new Reader(value, 'CommitParams');
const params: CommitParams = {
agentId: reader.id('agentId'),
preparedRequestId: domainRequestId(reader.string('preparedRequestId')),
nextInput: readSensoryInput(reader.value('nextInput')),
nextDecisionContext: readTypedValue(reader.value('nextDecisionContext')),
rewards: readRewardList(reader, 'rewards'),
taskStimulations: readStimulusList(reader, 'taskStimulations'),
};
reader.finish();
return params;
}
/** The commit of transition `k -> k+1` carries `scope.step = k` and the input for `k+1`. */
export function validateCommitAgainstScope(params: CommitParams, scope: Scope): void {
const expected = u64(scope.step) + 1n;
if (u64(params.nextInput.boundary) !== expected) {
fail(`CommitParams: nextInput.boundary must be ${expected} for scope.step ${scope.step}`);
}
}
export function readAgentCommitResult(value: unknown): AgentCommitResult {
const reader = new Reader(value, 'AgentCommitResult');
const result: AgentCommitResult = {
agentId: reader.id('agentId'),
committedStep: reader.u64('committedStep'),
decisionContextDigest: reader.digest('decisionContextDigest'),
telemetry: readAgentTelemetry(reader.value('telemetry')),
};
reader.finish();
return result;
}
// ------------------------------------------------------------------- environment methods
export interface AxisSchema {
id: Id;
range: AxisRange;
neutral: number;
}
export interface ControllerSchema {
schema: SchemaRef;
buttons: Id[];
axes: AxisSchema[];
}
export interface PortControl {
portId: Id;
buttons: { id: Id; down: boolean }[];
axes: { id: Id; value: number }[];
}
export interface PortDescriptor {
portId: Id;
controls: ControllerSchema;
}
export interface EnvironmentDescriptor {
backendDigest: Digest;
contentDigest: Digest;
configurationDigest: Digest;
stepDuration: RationalNs;
ports: PortDescriptor[];
inspectionSchema: SchemaRef;
views: ViewDescriptor[];
audio: AudioDescriptor[];
recovery: Recovery;
determinism: Determinism;
}
export interface WorldObservation {
boundary: U64;
worldTime: RationalNs;
engineFrame: string | null;
sensoryViews: ViewRef[];
inspection: TypedValue;
broadcastViews: ViewRef[];
audio: AudioRef[];
}
export interface EnvironmentInitializeParams {
backendConfig: AssetRef;
taskConfig: AssetRef;
episodeId: Id;
portBindings: { portId: Id; agentId: Id }[];
}
export interface EnvironmentInitializeResult {
descriptor: EnvironmentDescriptor;
observation: WorldObservation;
}
export interface AdvanceParams {
batchId: Id;
controls: PortControl[];
}
export interface StepResult {
batchId: Id;
appliedFromStep: U64;
nextStep: U64;
appliedControlsDigest: Digest;
observation: WorldObservation;
}
export function readControllerSchema(value: unknown): ControllerSchema {
const reader = new Reader(value, 'ControllerSchema');
const controls: ControllerSchema = {
schema: readSchemaRef(reader.value('schema')),
buttons: reader.idList('buttons', 0, MAX_BUTTONS),
axes: reader.list('axes', 0, MAX_AXES, (item) => {
const axis = new Reader(item, 'ControllerSchema.axes');
const id = axis.id('id');
const range = axis.enumeration('range', AXIS_RANGES);
const [low, high] = axisBounds(range);
const neutral = axis.finiteIn('neutral', low, high);
axis.finish();
return { id, range, neutral };
}),
};
reader.finish();
requireUnique(controls.buttons, 'ControllerSchema.buttons');
requireUnique(
controls.axes.map((axis) => axis.id),
'ControllerSchema.axes',
);
return controls;
}
export function readPortControl(value: unknown): PortControl {
const reader = new Reader(value, 'PortControl');
const control: PortControl = {
portId: reader.id('portId'),
buttons: reader.list('buttons', 0, MAX_BUTTONS, (item) => {
const button = new Reader(item, 'PortControl.buttons');
const entry = { id: button.id('id'), down: button.boolean('down') };
button.finish();
return entry;
}),
axes: reader.list('axes', 0, MAX_AXES, (item) => {
const axis = new Reader(item, 'PortControl.axes');
const entry = { id: axis.id('id'), value: axis.finite('value') };
axis.finish();
return entry;
}),
};
reader.finish();
requireUnique(
control.buttons.map((button) => button.id),
'PortControl.buttons',
);
requireUnique(
control.axes.map((axis) => axis.id),
'PortControl.axes',
);
return control;
}
/**
* Every declared button and axis, in descriptor order, in range. An out-of-range value is
* refused, never clamped (workers-v1 section 3).
*/
export function validatePortControlAgainst(
control: PortControl,
controls: ControllerSchema,
): void {
requireSameOrder(
control.buttons.map((button) => button.id),
controls.buttons,
'PortControl.buttons',
);
requireSameOrder(
control.axes.map((axis) => axis.id),
controls.axes.map((axis) => axis.id),
'PortControl.axes',
);
control.axes.forEach((axis, index) => {
const schema = controls.axes[index] as AxisSchema;
const [low, high] = axisBounds(schema.range);
if (axis.value < low || axis.value > high) {
fail(
`PortControl: axis "${axis.id}" value ${axis.value} is outside its ${schema.range} range and is refused, not clamped`,
);
}
});
}
export function readEnvironmentDescriptor(value: unknown): EnvironmentDescriptor {
const reader = new Reader(value, 'EnvironmentDescriptor');
const descriptor: EnvironmentDescriptor = {
backendDigest: reader.digest('backendDigest'),
contentDigest: reader.digest('contentDigest'),
configurationDigest: reader.digest('configurationDigest'),
stepDuration: readRational(reader.value('stepDuration')),
ports: reader.list('ports', 1, MAX_PORTS, (item) => {
const port = new Reader(item, 'EnvironmentDescriptor.ports');
const entry = {
portId: port.id('portId'),
controls: readControllerSchema(port.value('controls')),
};
port.finish();
return entry;
}),
inspectionSchema: readSchemaRef(reader.value('inspectionSchema')),
views: reader.list('views', 0, MAX_VIEWS, readViewDescriptor),
audio: reader.list('audio', 0, 8, readAudioDescriptor),
recovery: reader.enumeration('recovery', RECOVERY),
determinism: reader.enumeration('determinism', DETERMINISM),
};
reader.finish();
requirePositiveRational(descriptor.stepDuration, 'EnvironmentDescriptor.stepDuration');
requireUnique(
descriptor.ports.map((port) => port.portId),
'EnvironmentDescriptor.ports',
);
requireUnique(
descriptor.views.map((view) => view.viewId),
'EnvironmentDescriptor.views',
);
requireUnique(
descriptor.audio.map((stream) => stream.streamId),
'EnvironmentDescriptor.audio',
);
return descriptor;
}
export function findPort(
descriptor: EnvironmentDescriptor,
portId: string,
): PortDescriptor | undefined {
return descriptor.ports.find((port) => port.portId === portId);
}
/** One complete batch: every configured port once, in descriptor order. */
export function validateBatch(
descriptor: EnvironmentDescriptor,
controls: readonly PortControl[],
): void {
requireSameOrder(
controls.map((control) => control.portId),
descriptor.ports.map((port) => port.portId),
'Environment.Advance controls',
);
controls.forEach((control, index) => {
validatePortControlAgainst(control, (descriptor.ports[index] as PortDescriptor).controls);
});
}
export function readWorldObservation(value: unknown): WorldObservation {
const reader = new Reader(value, 'WorldObservation');
const observation: WorldObservation = {
boundary: reader.u64('boundary'),
worldTime: readRational(reader.value('worldTime')),
engineFrame: reader.nullableBoundedString('engineFrame', MAX_ENGINE_FRAME_LEN),
sensoryViews: readViewList(reader, 'sensoryViews'),
inspection: readTypedValue(reader.value('inspection')),
broadcastViews: readViewList(reader, 'broadcastViews'),
audio: readAudioList(reader, 'audio'),
};
reader.finish();
for (const view of [...observation.sensoryViews, ...observation.broadcastViews]) {
if (u64(view.producedStep) > u64(observation.boundary)) {
fail('WorldObservation: a view cannot be produced after the boundary');
}
}
return observation;
}
export function validateObservationAgainst(
observation: WorldObservation,
descriptor: EnvironmentDescriptor,
): void {
const expected = descriptor.inspectionSchema;
const found = observation.inspection.schema;
if (
found.id !== expected.id ||
found.version !== expected.version ||
found.digest !== expected.digest
) {
fail("WorldObservation: inspection must use the descriptor's inspectionSchema");
}
for (const view of [...observation.sensoryViews, ...observation.broadcastViews]) {
const declared = descriptor.views.find((candidate) => candidate.viewId === view.viewId);
if (!declared) {
fail(`WorldObservation: view "${view.viewId}" is not declared by the descriptor`);
}
validateViewAgainst(view, declared, u64(observation.boundary));
}
for (const chunk of observation.audio) {
const declared = descriptor.audio.find((candidate) => candidate.streamId === chunk.streamId);
if (!declared) {
fail(`WorldObservation: audio stream "${chunk.streamId}" is not declared by the descriptor`);
}
validateAudioAgainst(chunk, declared);
}
}
export function readEnvironmentInitializeParams(value: unknown): EnvironmentInitializeParams {
const reader = new Reader(value, 'EnvironmentInitializeParams');
const params: EnvironmentInitializeParams = {
backendConfig: readAssetRef(reader.value('backendConfig')),
taskConfig: readAssetRef(reader.value('taskConfig')),
episodeId: reader.id('episodeId'),
portBindings: reader.list('portBindings', 1, MAX_PORTS, (item) => {
const binding = new Reader(item, 'EnvironmentInitializeParams.portBindings');
const entry = { portId: binding.id('portId'), agentId: binding.id('agentId') };
binding.finish();
return entry;
}),
};
reader.finish();
requireUnique(
params.portBindings.map((binding) => binding.portId),
'EnvironmentInitializeParams.portBindings portId',
);
requireUnique(
params.portBindings.map((binding) => binding.agentId),
'EnvironmentInitializeParams.portBindings agentId',
);
return params;
}
export function readEnvironmentInitializeResult(value: unknown): EnvironmentInitializeResult {
const reader = new Reader(value, 'EnvironmentInitializeResult');
const result: EnvironmentInitializeResult = {
descriptor: readEnvironmentDescriptor(reader.value('descriptor')),
observation: readWorldObservation(reader.value('observation')),
};
reader.finish();
if (u64(result.observation.boundary) !== 0n) {
fail('EnvironmentInitializeResult: the initial observation is boundary 0');
}
if (!isRationalZero(result.observation.worldTime)) {
fail('EnvironmentInitializeResult: initial worldTime is zero (0/1)');
}
validateObservationAgainst(result.observation, result.descriptor);
return result;
}
export function readAdvanceParams(value: unknown): AdvanceParams {
const reader = new Reader(value, 'AdvanceParams');
const params: AdvanceParams = {
batchId: reader.id('batchId'),
controls: reader.list('controls', 1, MAX_PORTS, readPortControl),
};
reader.finish();
requireUnique(
params.controls.map((control) => control.portId),
'AdvanceParams.controls',
);
return params;
}
export function readStepResult(value: unknown): StepResult {
const reader = new Reader(value, 'StepResult');
const result: StepResult = {
batchId: reader.id('batchId'),
appliedFromStep: reader.u64('appliedFromStep'),
nextStep: reader.u64('nextStep'),
appliedControlsDigest: reader.digest('appliedControlsDigest'),
observation: readWorldObservation(reader.value('observation')),
};
reader.finish();
if (u64(result.nextStep) !== u64(result.appliedFromStep) + 1n) {
fail('StepResult: nextStep must be appliedFromStep + 1; one result is one step');
}
if (u64(result.observation.boundary) !== u64(result.nextStep)) {
fail('StepResult: the observation boundary must be nextStep');
}
return result;
}
/** Exactly boundary `k+1`, with world time advanced by exactly one `stepDuration`. */
export function validateStepResultAgainst(
result: StepResult,
descriptor: EnvironmentDescriptor,
previous: WorldObservation,
): void {
validateObservationAgainst(result.observation, descriptor);
const expected = addRational(previous.worldTime, descriptor.stepDuration);
if (compareRational(result.observation.worldTime, expected) !== 0) {
fail('StepResult: worldTime must advance by exactly one stepDuration');
}
if (u64(result.observation.boundary) !== u64(previous.boundary) + 1n) {
fail('StepResult: the observation must be exactly the next boundary');
}
}
// --------------------------------------------------------------------- common worker methods
export interface HelloParams {
sessionId: Id;
expectedWorkerId: Id;
role: Role;
supportedMajors: number[];
}
export interface HelloResult {
selectedMajor: 1;
selectedMinor: 0;
workerId: Id;
incarnationId: Id;
role: Role;
buildDigest: Digest;
contractDigest: Digest;
capabilities: Id[];
limits: { maxAgents: number; maxPorts: number };
}
export interface StatusResult {
state: WorkerState;
currentScope: Scope | null;
activeRequestId: DomainRequestId | null;
lastCompletedRequestId: DomainRequestId | null;
lastBatchId: Id | null;
progressCounter: U64;
}
export interface AcknowledgeParams {
requestIds: DomainRequestId[];
}
export interface AcknowledgeResult {
acknowledged: DomainRequestId[];
}
export interface ShutdownParams {
reason: Id;
}
export interface ShutdownResult {
stopping: true;
}
export interface TaskEvent {
id: Id;
kindId: Id;
sourceStep: U64;
agentId: Id | null;
payload: TypedValue;
}
export interface EpisodeRequest {
kind: 'terminal';
reason: Id;
outcome: TypedValue;
}
/** Required capabilities are agent-step-v1 and world-step-v1 for their roles. */
export function requiredCapability(role: Role): string | null {
if (role === 'agent') return 'agent-step-v1';
if (role === 'environment') return 'world-step-v1';
return null;
}
export function readHelloParams(value: unknown): HelloParams {
const reader = new Reader(value, 'HelloParams');
const params: HelloParams = {
sessionId: reader.id('sessionId'),
expectedWorkerId: reader.id('expectedWorkerId'),
role: reader.enumeration('role', ROLES),
supportedMajors: reader.list('supportedMajors', 1, MAX_SUPPORTED_MAJORS, (item) => {
if (typeof item !== 'number' || !Number.isInteger(item) || item < 1 || item > 65_535) {
fail('every supported major must be an integer 1..=65535');
}
return item;
}),
};
reader.finish();
requireUnique(
params.supportedMajors.map(String),
'HelloParams.supportedMajors',
);
return params;
}
export function readHelloResult(value: unknown): HelloResult {
const reader = new Reader(value, 'HelloResult');
const selectedMajor = reader.int('selectedMajor', 1, 1) as 1;
const selectedMinor = reader.int('selectedMinor', 0, 0) as 0;
const workerId = reader.id('workerId');
const incarnationId = reader.id('incarnationId');
const role = reader.enumeration('role', ROLES);
const buildDigest = reader.digest('buildDigest');
const contractDigest = reader.digest('contractDigest');
const capabilities = reader.idList('capabilities', 0, MAX_CAPABILITIES);
const limitsReader = new Reader(reader.value('limits'), 'HelloResult.limits');
const limits = {
maxAgents: limitsReader.int('maxAgents', 1, MAX_AGENTS),
maxPorts: limitsReader.int('maxPorts', 1, MAX_PORTS),
};
limitsReader.finish();
reader.finish();
requireUnique(capabilities, 'HelloResult.capabilities');
const required = requiredCapability(role);
if (required !== null && !capabilities.includes(required)) {
fail(`HelloResult: a ${role} worker must advertise ${required}`);
}
return {
selectedMajor,
selectedMinor,
workerId,
incarnationId,
role,
buildDigest,
contractDigest,
capabilities,
limits,
};
}
export function readStatusResult(value: unknown): StatusResult {
const reader = new Reader(value, 'StatusResult');
const state = reader.enumeration('state', WORKER_STATES);
const currentScopeValue = reader.value('currentScope');
const currentScope = currentScopeValue === null ? null : readScope(currentScopeValue);
const active = reader.value('activeRequestId');
const completed = reader.value('lastCompletedRequestId');
const result: StatusResult = {
state,
currentScope,
activeRequestId: active === null ? null : domainRequestId(active),
lastCompletedRequestId: completed === null ? null : domainRequestId(completed),
lastBatchId: reader.nullableId('lastBatchId'),
progressCounter: reader.u64('progressCounter'),
};
reader.finish();
if (result.state === 'uninitialized' && result.currentScope !== null) {
fail('StatusResult: an uninitialized worker has a null currentScope');
}
return result;
}
export function readAcknowledgeParams(value: unknown): AcknowledgeParams {
const reader = new Reader(value, 'AcknowledgeParams');
const params: AcknowledgeParams = {
requestIds: reader.list('requestIds', 1, MAX_ACKNOWLEDGE, domainRequestId),
};
reader.finish();
requireUnique(params.requestIds, 'AcknowledgeParams.requestIds');
return params;
}
export function readAcknowledgeResult(value: unknown): AcknowledgeResult {
const reader = new Reader(value, 'AcknowledgeResult');
const result: AcknowledgeResult = {
acknowledged: reader.list('acknowledged', 0, MAX_ACKNOWLEDGE, domainRequestId),
};
reader.finish();
requireUnique(result.acknowledged, 'AcknowledgeResult.acknowledged');
return result;
}
export function readShutdownParams(value: unknown): ShutdownParams {
const reader = new Reader(value, 'ShutdownParams');
const params = { reason: reader.id('reason') };
reader.finish();
return params;
}
export function readShutdownResult(value: unknown): ShutdownResult {
const reader = new Reader(value, 'ShutdownResult');
const result: ShutdownResult = { stopping: reader.constantTrue('stopping') };
reader.finish();
return result;
}
export function readTaskEvent(value: unknown): TaskEvent {
const reader = new Reader(value, 'TaskEvent');
const event: TaskEvent = {
id: reader.id('id'),
kindId: reader.id('kindId'),
sourceStep: reader.u64('sourceStep'),
agentId: reader.nullableId('agentId'),
payload: readTypedValue(reader.value('payload')),
};
reader.finish();
return event;
}
export function readEpisodeRequest(value: unknown): EpisodeRequest {
const reader = new Reader(value, 'EpisodeRequest');
const request: EpisodeRequest = {
kind: reader.constant('kind', 'terminal'),
reason: reader.id('reason'),
outcome: readTypedValue(reader.value('outcome')),
};
reader.finish();
return request;
}
export interface ActivateRestoreResult {
committedStep: U64;
checkpointId: Id;
observation: WorldObservation | null;
}
export function readActivateRestoreResult(value: unknown): ActivateRestoreResult {
const reader = new Reader(value, 'ActivateRestoreResult');
const observationValue = reader.value('observation');
const result: ActivateRestoreResult = {
committedStep: reader.u64('committedStep'),
checkpointId: reader.id('checkpointId'),
observation: observationValue === null ? null : readWorldObservation(observationValue),
};
reader.finish();
if (
result.observation !== null &&
u64(result.observation.boundary) !== u64(result.committedStep)
) {
fail('ActivateRestoreResult: the observation boundary must be the committed step');
}
return result;
}
/** An environment returns its restored observation; an agent returns null. */
export function validateActivateRestoreForRole(result: ActivateRestoreResult, role: Role): void {
if (role === 'environment' && result.observation === null) {
fail('ActivateRestoreResult: an environment must return its restored observation');
}
if (role === 'agent' && result.observation !== null) {
fail('ActivateRestoreResult: an agent returns a null observation');
}
}

View file

@ -0,0 +1,150 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import {
MAX_ENVELOPE_BYTES,
canonicalize,
digestOf,
parseStrict,
requireEnvelopeFit,
} from '../src/canonical';
import { bodyDigest, operationKeyDigest, readScope } from '../src/common';
import * as fixtures from '../src/fixtures';
const ASTRAL = String.fromCodePoint(0x10400);
const FULLWIDTH_A = String.fromCodePoint(0xff21);
const E_ACUTE = String.fromCodePoint(0xe9);
test('object keys are sorted by UTF-16 code unit', () => {
const value: Record<string, number> = { b: 1, a: 2, A: 3 };
value[E_ACUTE] = 4;
value[ASTRAL] = 5;
value[FULLWIDTH_A] = 6;
assert.equal(
canonicalize(value),
`{"A":3,"a":2,"b":1,"${E_ACUTE}":4,"${ASTRAL}":5,"${FULLWIDTH_A}":6}`,
'an astral key, whose leading surrogate is D801, sorts before U+FF21',
);
});
test('numbers print the way ECMAScript prints them', () => {
const file = fixtures.load('boundaries.json');
for (const item of fixtures.section(file, 'doubles')) {
const record = item as Record<string, unknown>;
const value = record.value as number;
if (record.accept === true) {
assert.equal(canonicalize(value), record.canonical, `${value} prints as its canonical form`);
} else {
assert.throws(() => canonicalize(value), `${value} must be refused`);
}
}
});
test('strings are escaped the way JSON.stringify escapes them', () => {
const bell = String.fromCharCode(7);
const del = String.fromCharCode(0x7f);
const value = { s: ['q"', 'b\\', 't\t', 'n\n', bell, del].join(' ') };
assert.equal(
canonicalize(value),
JSON.stringify(value),
'for a single-key object the two agree exactly, escape for escape',
);
assert.ok(canonicalize(value).includes('\\u0007'), 'a control character uses lowercase \\u');
assert.ok(canonicalize(value).includes(del), 'DEL is not an escape in JSON');
});
test('canonical form does not depend on the input formatting', () => {
const compact = '{"b":[1,2,{"y":true,"x":null}],"a":"z"}';
const pretty = '{\n "a" : "z",\n "b": [ 1, 2, { "x": null, "y": true } ]\n}';
assert.equal(canonicalize(parseStrict(compact)), canonicalize(parseStrict(pretty)));
assert.equal(digestOf(parseStrict(compact)), digestOf(parseStrict(pretty)));
});
test('duplicate keys and invalid UTF-8 never parse', () => {
assert.throws(() => parseStrict('{"a":1,"a":2}'), /duplicate key/);
assert.throws(() => parseStrict('{"a":{"b":1,"b":2}}'), /duplicate key/);
assert.throws(() => parseStrict(new Uint8Array([0x7b, 0x22, 0xff, 0x22, 0x7d])), /UTF-8/);
assert.throws(() => parseStrict('{"a":1} {"b":2}'), /trailing data/);
assert.throws(() => parseStrict('{"a":NaN}'));
assert.throws(() => parseStrict('{"a":Infinity}'));
assert.throws(() => parseStrict('{"a":1'));
assert.throws(() => parseStrict(''));
});
test('an envelope over 64 KiB is refused', () => {
assert.throws(() => requireEnvelopeFit({ pad: 'a'.repeat(MAX_ENVELOPE_BYTES) }, 0));
const small = { pad: 'a' };
const length = canonicalize(small).length;
assert.equal(requireEnvelopeFit(small, MAX_ENVELOPE_BYTES - length), MAX_ENVELOPE_BYTES);
assert.throws(() => requireEnvelopeFit(small, MAX_ENVELOPE_BYTES - length + 1));
});
test('operation keys match the fixture and separate the operations they should', () => {
const file = fixtures.load('operations.json');
const digests: [string, string][] = [];
for (const item of fixtures.section(file, 'keys')) {
const name = fixtures.field(item, 'name');
const digest = operationKeyDigest({
scope: readScope(fixtures.member(item, 'scope')),
method: fixtures.field(item, 'method'),
workerId: fixtures.field(item, 'workerId'),
});
assert.equal(digest, fixtures.field(item, 'digest'), `${name}: operation key digest`);
digests.push([name, digest]);
}
digests.forEach(([name, digest], index) => {
for (const [otherName, other] of digests.slice(index + 1)) {
assert.notEqual(digest, other, `${name} and ${otherName} are different operations`);
}
});
});
test('canonical bodies match the fixture', () => {
const file = fixtures.load('operations.json');
for (const item of fixtures.section(file, 'bodies')) {
const scopeValue = fixtures.member(item, 'scope');
const digest = bodyDigest(
fixtures.field(item, 'method'),
scopeValue === null ? null : readScope(scopeValue),
fixtures.member(item, 'params'),
);
assert.equal(digest, fixtures.field(item, 'digest'), fixtures.field(item, 'name'));
}
});
test('operation pairs agree with the fixture about sameness', () => {
const file = fixtures.load('operations.json');
for (const item of fixtures.section(file, 'pairs')) {
const name = fixtures.field(item, 'name');
const reason = fixtures.field(item, 'reason');
const worker = fixtures.field(item, 'workerId');
const rightWorker = fixtures.optionalField(item, 'rightWorkerId') ?? worker;
const side = (key: string, workerId: string): [string, string] => {
const value = fixtures.member(item, key);
const method = fixtures.field(value, 'method');
const scope = readScope(fixtures.member(value, 'scope'));
const params = fixtures.member(value, 'params');
return [operationKeyDigest({ scope, method, workerId }), bodyDigest(method, scope, params)];
};
const [leftKey, leftBody] = side('left', worker);
const [rightKey, rightBody] = side('right', rightWorker);
const record = item as Record<string, unknown>;
assert.equal(leftKey === rightKey, record.sameKey, `${name}: key sameness. ${reason}`);
assert.equal(leftBody === rightBody, record.sameBody, `${name}: body sameness. ${reason}`);
}
});
test('a domain body can never carry a bus identity', () => {
const file = fixtures.load('operations.json');
for (const item of fixtures.section(file, 'rejected')) {
assert.throws(
() =>
bodyDigest(
fixtures.field(item, 'method'),
readScope(fixtures.member(item, 'scope')),
fixtures.member(item, 'params'),
),
`${fixtures.field(item, 'name')} must be refused`,
);
}
});

View file

@ -0,0 +1,126 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { canonicalize, parseStrict } from '../src/canonical';
import * as checkpoint from '../src/checkpoint';
import * as fixtures from '../src/fixtures';
function envelopeBytes(): Uint8Array {
const file = fixtures.load('checkpoint-envelope.json');
const envelope = fixtures.member(file, 'envelope');
return fixtures.decodeBase64(fixtures.field(envelope, 'base64'));
}
test('the fixture envelope decodes to its recorded layout', () => {
const file = fixtures.load('checkpoint-envelope.json');
const bytes = envelopeBytes();
const envelope = checkpoint.decode(bytes);
checkpoint.validateManifest(envelope);
const layout = fixtures.member(fixtures.member(file, 'envelope'), 'layout') as Record<
string,
unknown
>;
assert.equal(Buffer.from(bytes.subarray(0, 8)).toString('ascii'), checkpoint.MAGIC);
assert.equal(String(bytes.length), layout.totalBytes);
assert.equal(String(envelope.layout.tableOffset), layout.tableOffset);
assert.equal(envelope.layout.manifestBytes, layout.manifestBytes);
const entries = layout.entries as Record<string, unknown>[];
assert.equal(envelope.layout.entries.length, entries.length);
envelope.layout.entries.forEach((entry, index) => {
const recorded = entries[index]!;
assert.equal(entry.name, recorded.name);
assert.equal(String(entry.offset), recorded.offset);
assert.equal(String(entry.byteLength), recorded.byteLength);
assert.equal(entry.digest, recorded.digest);
assert.equal(entry.offset % 8, 0, 'payloads start on an eight-byte boundary');
});
for (const payload of fixtures.section(file, 'payloads')) {
const name = fixtures.field(payload, 'name');
const expected = fixtures.decodeBase64(fixtures.field(payload, 'base64'));
const found = envelope.payloads.find((candidate) => candidate.name === name);
assert.ok(found, `payload ${name} must be present`);
assert.deepEqual(found.bytes, expected, `payload ${name} must come back byte for byte`);
}
assert.equal(canonicalize(envelope.manifest), canonicalize(fixtures.member(file, 'manifest')));
});
test('every recorded corruption is refused', () => {
const file = fixtures.load('checkpoint-envelope.json');
const bytes = envelopeBytes();
for (const item of fixtures.section(file, 'corruption')) {
const name = fixtures.field(item, 'name');
const offset = (item as Record<string, unknown>).offset as number;
const corrupted = Uint8Array.from(bytes);
corrupted[offset] = (corrupted[offset]! ^ 0x01) & 0xff;
assert.throws(() => checkpoint.decode(corrupted), `${name} must be refused`);
}
assert.throws(() => checkpoint.decode(bytes.subarray(0, bytes.length - 1)));
assert.throws(() => checkpoint.decode(bytes.subarray(0, 8)));
});
test('a FLYSIM01 envelope is not read as a session checkpoint', () => {
const manifest = Buffer.from('{"schemaVersion":2,"chunks":["agent"]}', 'utf8');
const length = Buffer.alloc(4);
length.writeUInt32LE(manifest.length, 0);
const chunkLength = Buffer.alloc(4);
chunkLength.writeUInt32LE(64, 0);
const legacy = Buffer.concat([
Buffer.from('FLYSIM01', 'ascii'),
length,
manifest,
chunkLength,
Buffer.alloc(64),
Buffer.alloc(4), // the CRC32 footer
]);
assert.throws(() => checkpoint.decode(legacy), /magic/);
});
test('the layout is deterministic and the manifest is canonical', () => {
const manifest = parseStrict('{"b":2,"a":1}');
const payloads = [
{ name: 'one', bytes: new TextEncoder().encode('first') },
{ name: 'two', bytes: new Uint8Array(9) },
];
const bytes = checkpoint.encode(manifest, payloads);
assert.deepEqual(bytes, checkpoint.encode(manifest, payloads));
const envelope = checkpoint.decode(bytes);
const start = checkpoint.HEADER_BYTES;
const end = start + envelope.layout.manifestBytes;
assert.equal(Buffer.from(bytes.subarray(start, end)).toString('utf8'), '{"a":1,"b":2}');
assert.throws(() =>
checkpoint.encode(manifest, [
{ name: 'one', bytes: new Uint8Array() },
{ name: 'one', bytes: new Uint8Array() },
]),
);
assert.throws(() => checkpoint.encode(manifest, [{ name: 'One', bytes: new Uint8Array() }]));
});
test('an envelope written here is read by the same rules the Rust crate wrote its fixture with', () => {
const file = fixtures.load('checkpoint-envelope.json');
const manifest = fixtures.member(file, 'manifest');
const payloads = fixtures
.section(file, 'payloads')
.map((payload) => ({
name: fixtures.field(payload, 'name'),
bytes: fixtures.decodeBase64(fixtures.field(payload, 'base64')),
}));
assert.deepEqual(
Uint8Array.from(checkpoint.encode(manifest, payloads)),
Uint8Array.from(envelopeBytes()),
'the two implementations produce the same bytes for the same inputs',
);
});
test('a manifest missing a required field is not a complete checkpoint', () => {
const file = fixtures.load('checkpoint-envelope.json');
const full = fixtures.member(file, 'manifest') as Record<string, unknown>;
for (const field of checkpoint.REQUIRED_MANIFEST_FIELDS) {
const manifest = { ...full };
delete manifest[field];
const envelope = checkpoint.decode(checkpoint.encode(manifest, []));
assert.throws(() => checkpoint.validateManifest(envelope), `without ${field}`);
}
});

View file

@ -0,0 +1,85 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import * as fixtures from '../src/fixtures';
import { readCommittedSnapshot, readSessionDescriptor, validateSnapshotAgainst } from '../src/publishing';
import {
findPort,
readEnvironmentDescriptor,
readPortControl,
readSensoryInput,
readStepResult,
readWorldObservation,
validateBatch,
validateObservationAgainst,
validatePortControlAgainst,
validateSensoryInputAgainst,
validateStepResultAgainst,
} from '../src/workers';
import { requiredProducedStep, frameBytes } from '../src/media';
test('every descriptor check lands the way the fixture says', () => {
const file = fixtures.load('descriptor-checks.json');
const descriptor = readEnvironmentDescriptor(fixtures.member(file, 'descriptor'));
const delayed = readEnvironmentDescriptor(fixtures.member(file, 'delayedDescriptor'));
const session = readSessionDescriptor(fixtures.member(file, 'sessionDescriptor'));
const previous = readWorldObservation(fixtures.member(file, 'stepResultPrevious'));
for (const item of fixtures.cases(file)) {
const name = fixtures.field(item, 'name');
const kind = fixtures.field(item, 'kind');
const reason = fixtures.optionalField(item, 'reason') ?? '';
const expectAccept = fixtures.field(item, 'expect') === 'accept';
const value = fixtures.member(item, 'value');
const attempt = () => {
switch (kind) {
case 'portControl': {
const control = readPortControl(value);
const port = findPort(descriptor, control.portId);
if (!port) throw new Error('no such port');
validatePortControlAgainst(control, port.controls);
return;
}
case 'advanceControls': {
const controls = (value as unknown[]).map(readPortControl);
validateBatch(descriptor, controls);
return;
}
case 'sensoryInput':
validateSensoryInputAgainst(readSensoryInput(value), descriptor.views);
return;
case 'sensoryInputDelayed':
validateSensoryInputAgainst(readSensoryInput(value), delayed.views);
return;
case 'worldObservation':
validateObservationAgainst(readWorldObservation(value), descriptor);
return;
case 'stepResult':
validateStepResultAgainst(readStepResult(value), descriptor, previous);
return;
case 'snapshot':
validateSnapshotAgainst(readCommittedSnapshot(value), session);
return;
default:
throw new Error(`unknown descriptor check kind "${kind}"`);
}
};
if (expectAccept) {
assert.doesNotThrow(attempt, `${name} must be accepted. ${reason}`);
} else {
assert.throws(attempt, `${name} must be refused. ${reason}`);
}
}
});
test('the required producing boundary saturates at zero', () => {
const file = fixtures.load('descriptor-checks.json');
const delayed = readEnvironmentDescriptor(fixtures.member(file, 'delayedDescriptor'));
const view = delayed.views[0]!;
assert.equal(view.observationDelaySteps, 2);
assert.equal(requiredProducedStep(view, 0n), 0n);
assert.equal(requiredProducedStep(view, 1n), 0n);
assert.equal(requiredProducedStep(view, 2n), 0n);
assert.equal(requiredProducedStep(view, 3n), 1n);
assert.equal(frameBytes(view), 160 * 4 * 144);
});

View file

@ -0,0 +1,70 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import * as fixtures from '../src/fixtures';
import { readArtifactRef } from '../src/reader';
import {
artifactIdentity,
isBusCallId,
isDigest,
isDomainRequestId,
isId,
ownerTokenKind,
parseU64,
} from '../src/scalar';
import { readAssetRef } from '../src/workers';
test('U64 boundaries reject from the fixture', () => {
const file = fixtures.load('boundaries.json');
for (const item of fixtures.section(file, 'u64')) {
const record = item as Record<string, unknown>;
assert.equal(
parseU64(record.text) !== undefined,
record.accept,
`${String(record.text)}: ${String(record.reason)}`,
);
}
});
test('the Id and Digest encodings are the ones the bus uses', () => {
for (const id of ['a', 'fly-a', '0', 'a.b_c-d', 'a'.repeat(64)]) {
assert.ok(isId(id), `${id} is an Id`);
}
for (const id of ['', 'A', '-a', '.a', 'a b', 'fly/a', 'a'.repeat(65)]) {
assert.ok(!isId(id), `${id} is not an Id`);
}
assert.ok(isDigest('a'.repeat(64)));
assert.ok(!isDigest('A'.repeat(64)), 'digests are lowercase');
assert.ok(!isDigest('a'.repeat(63)), 'digests are 64 hex digits');
assert.ok(!isDigest('g'.repeat(64)), 'digests are hexadecimal');
});
test('the four identities never accept each other spellings', () => {
const file = fixtures.load('identities.json');
for (const item of fixtures.cases(file)) {
const record = item as Record<string, unknown>;
const text = record.text as string;
assert.equal(isBusCallId(text), record.busCallId, `busCallId ${text}`);
assert.equal(isDomainRequestId(text), record.domainRequestId, `domainRequestId ${text}`);
assert.equal(ownerTokenKind(text) ?? null, record.ownerToken, `owner token ${text}`);
const accepted = [
isBusCallId(text),
isDomainRequestId(text),
ownerTokenKind(text) !== undefined,
].filter(Boolean).length;
assert.ok(accepted <= 1, `${text} is accepted by more than one identity type`);
}
});
test('an artifact identity is the naming half of an ArtifactRef, and an asset is neither', () => {
const file = fixtures.load('identities.json');
const artifact = fixtures.member(file, 'artifact');
const reference = readArtifactRef(fixtures.member(artifact, 'ref'));
assert.deepEqual(artifactIdentity(reference), fixtures.member(artifact, 'identity'));
const asset = readAssetRef(fixtures.member(file, 'asset'));
assert.notEqual(
asset.id,
reference.artifactId,
'the fixture asset and artifact are deliberately different things',
);
});

View file

@ -0,0 +1,136 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { canonicalize, parseStrict, requireEnvelopeFit, sha256Hex } from '../src/canonical';
import { MAX_TYPED_VALUE_BYTES } from '../src/scalar';
import { readTypedValue } from '../src/common';
import * as fixtures from '../src/fixtures';
import { READERS, roundTrip } from './readers';
test('every valid case round trips and canonicalizes to its recorded bytes', () => {
const file = fixtures.load('valid.json');
const cases = fixtures.cases(file);
for (const item of cases) {
const name = fixtures.field(item, 'name');
const typeName = fixtures.field(item, 'type');
const value = fixtures.member(item, 'value');
let written: unknown;
try {
written = roundTrip(typeName, value);
} catch (error) {
assert.fail(`${name} (${typeName}) must be accepted: ${String(error)}`);
}
const canonicalIn = canonicalize(value);
assert.equal(
canonicalize(written),
canonicalIn,
`${name}: reading and writing must preserve every field`,
);
assert.equal(canonicalIn, fixtures.field(item, 'canonical'), `${name}: canonical JSON`);
assert.equal(sha256Hex(canonicalIn), fixtures.field(item, 'digest'), `${name}: digest`);
}
assert.ok(cases.length >= 70, 'the valid fixture should stay broad');
});
test('every type this package reads appears in the valid fixture', () => {
const covered = fixtures
.cases(fixtures.load('valid.json'))
.map((item) => fixtures.field(item, 'type'));
const missing = Object.keys(READERS).filter((typeName) => !covered.includes(typeName));
assert.deepEqual(missing, [], 'every readable type needs at least one accepted fixture');
});
test('every invalid case is refused', () => {
const file = fixtures.load('invalid.json');
const cases = fixtures.cases(file);
for (const item of cases) {
const name = fixtures.field(item, 'name');
const typeName = fixtures.field(item, 'type');
const reason = fixtures.field(item, 'reason');
assert.throws(
() => roundTrip(typeName, fixtures.member(item, 'value')),
`${name} (${typeName}) must be refused: ${reason}`,
);
}
assert.ok(cases.length >= 80, 'the invalid fixture should stay broad');
});
test('every raw byte case is refused before or during validation', () => {
for (const item of fixtures.cases(fixtures.load('raw.json'))) {
const name = fixtures.field(item, 'name');
const typeName = fixtures.field(item, 'type');
const reason = fixtures.field(item, 'reason');
const bytes = fixtures.decodeBase64(fixtures.field(item, 'base64'));
// parseStrict is the only door into the readers, so a byte sequence that does not parse
// never reaches validation.
assert.throws(
() => roundTrip(typeName, parseStrict(bytes)),
`${name} must be refused: ${reason}`,
);
}
});
test('generated boundary cases land on the right side of every limit', () => {
const file = fixtures.load('generated.json');
const padSchema = fixtures.member(file, 'padSchema');
for (const item of fixtures.cases(file)) {
const name = fixtures.field(item, 'name');
const kind = fixtures.field(item, 'kind');
const expectAccept = fixtures.field(item, 'expect') === 'accept';
const record = item as Record<string, unknown>;
const attempt = () => {
switch (kind) {
case 'padded-typed-value': {
const pad = 'a'.repeat(record.padCharacters as number);
readTypedValue({ schema: padSchema, value: { pad } });
return;
}
case 'padded-request': {
const pad = 'a'.repeat(record.padCharacters as number);
const total = record.envelopeTotal as number;
const body = roundTrip('SessionRpcRequest', {
requestId: 'req-1',
scope: null,
params: { pad },
});
requireEnvelopeFit(body, total - canonicalize(body).length);
return;
}
case 'error-message':
case 'error-message-astral': {
const character = kind === 'error-message' ? 'x' : '\u{10400}';
const message = character.repeat(record.codePoints as number);
roundTrip('SessionRpcFailure', {
type: 'error',
requestId: 'req-41',
workerId: 'fly-a',
incarnationId: 'inc-1',
scope: null,
error: { code: 'INTERNAL', message, mutation: 'unknown' },
});
return;
}
default:
throw new Error(`unknown generated case kind "${kind}"`);
}
};
if (expectAccept) {
assert.doesNotThrow(attempt, `${name} must be accepted`);
} else {
assert.throws(attempt, `${name} must be refused`);
}
}
});
test('a typed value at the cap is accepted and one byte more is not', () => {
const schema = { id: 'pad.v1', version: 1, digest: sha256Hex('pad.v1') };
const overhead = canonicalize({ schema, value: { pad: '' } }).length;
const atCap = readTypedValue({
schema,
value: { pad: 'a'.repeat(MAX_TYPED_VALUE_BYTES - overhead) },
});
assert.equal(canonicalize(atCap).length, MAX_TYPED_VALUE_BYTES);
assert.throws(() =>
readTypedValue({ schema, value: { pad: 'a'.repeat(MAX_TYPED_VALUE_BYTES - overhead + 1) } }),
);
});

View file

@ -0,0 +1,98 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { readRational } from '../src/common';
import * as fixtures from '../src/fixtures';
import {
RATIONAL_ZERO,
U64_MAX,
addRational,
compareRational,
divideFloor,
multiplyRational,
reduced,
requirePositiveRational,
subtractRational,
validateRational,
} from '../src/scalar';
test('the accumulator produces the fixture tick counts and remainders', () => {
const file = fixtures.load('rational.json');
for (const item of fixtures.section(file, 'accumulator')) {
const name = fixtures.field(item, 'name');
const step = readRational(fixtures.member(item, 'stepDuration'));
const tick = readRational(fixtures.member(item, 'tickDuration'));
let accumulator = RATIONAL_ZERO;
let total = 0n;
fixtures.section(item, 'steps').forEach((expected, index) => {
accumulator = addRational(accumulator, step);
const { ticks, remainder } = divideFloor(accumulator, tick);
accumulator = remainder;
total += BigInt(ticks);
assert.equal(ticks, fixtures.field(expected, 'ticks'), `${name}: ticks at step ${index}`);
assert.deepEqual(
remainder,
readRational(fixtures.member(expected, 'remainder')),
`${name}: remainder at step ${index}`,
);
assert.ok(compareRational(remainder, tick) < 0, `${name}: remainder below one tick`);
});
assert.equal(total.toString(), fixtures.field(item, 'totalTicks'), `${name}: total ticks`);
}
});
test('checked arithmetic reduces or refuses', () => {
const file = fixtures.load('rational.json');
for (const item of fixtures.section(file, 'add')) {
const left = readRational(fixtures.member(item, 'a'));
const right = readRational(fixtures.member(item, 'b'));
const record = item as Record<string, unknown>;
if (record.sum !== undefined) {
assert.deepEqual(addRational(left, right), readRational(record.sum as never));
} else {
assert.throws(() => addRational(left, right));
}
}
for (const item of fixtures.section(file, 'subtract')) {
const left = readRational(fixtures.member(item, 'a'));
const right = readRational(fixtures.member(item, 'b'));
const record = item as Record<string, unknown>;
if (record.difference !== undefined) {
assert.deepEqual(subtractRational(left, right), readRational(record.difference as never));
} else {
assert.throws(() => subtractRational(left, right));
}
}
for (const item of fixtures.section(file, 'multiply')) {
const value = readRational(fixtures.member(item, 'a'));
const factor = BigInt(fixtures.field(item, 'k'));
const record = item as Record<string, unknown>;
if (record.product !== undefined) {
assert.deepEqual(multiplyRational(value, factor), readRational(record.product as never));
} else {
assert.throws(() => multiplyRational(value, factor));
}
}
for (const item of fixtures.section(file, 'compare')) {
const left = readRational(fixtures.member(item, 'a'));
const right = readRational(fixtures.member(item, 'b'));
const expected = { less: -1, equal: 0, greater: 1 }[fixtures.field(item, 'ordering')];
assert.equal(compareRational(left, right), expected);
}
});
test('zero has exactly one encoding and durations must be positive', () => {
validateRational(RATIONAL_ZERO);
assert.throws(() => validateRational({ numerator: '0', denominator: '2' }), /0\/1/);
assert.throws(() => validateRational({ numerator: '1', denominator: '0' }), /positive/);
assert.throws(() => validateRational({ numerator: '2', denominator: '4' }), /reduced/);
assert.throws(() => requirePositiveRational(RATIONAL_ZERO, 'worldTime'));
assert.throws(() => divideFloor(RATIONAL_ZERO, RATIONAL_ZERO), /positive/);
});
test('reduction refuses a result that does not fit U64', () => {
const big = { numerator: U64_MAX.toString(), denominator: '1' };
assert.throws(() => multiplyRational(big, 2n), /does not fit U64/);
assert.throws(() => addRational(big, big), /does not fit U64/);
assert.deepEqual(reduced(U64_MAX * 2n, 2n), big);
});

View file

@ -0,0 +1,122 @@
/** One place that knows how to read every type named by a fixture. */
import type { Json } from '../src/canonical';
import {
readRational,
readSchemaRef,
readScope,
readTypedValue,
} from '../src/common';
import {
readActivateRestoreParams,
readAudioDescriptor,
readAudioRef,
readCaptureParams,
readCaptureResult,
readStageRestoreParams,
readStageRestoreResult,
readViewDescriptor,
readViewRef,
} from '../src/media';
import { readCommittedSnapshot, readSessionDescriptor } from '../src/publishing';
import {
readSessionRpcFailure,
readSessionRpcRequest,
readSessionRpcSuccess,
} from '../src/rpc';
import {
readTraceBehaviour,
readTraceOperational,
readTransitionTrace,
} from '../src/trace';
import {
readAcknowledgeParams,
readAcknowledgeResult,
readActivateRestoreResult,
readAdvanceParams,
readAgentCommitResult,
readAgentInitializeParams,
readAgentInitializeResult,
readAgentTelemetry,
readAssetRef,
readCommitParams,
readControllerSchema,
readEnvironmentDescriptor,
readEnvironmentInitializeParams,
readEnvironmentInitializeResult,
readEpisodeRequest,
readHelloParams,
readHelloResult,
readPortControl,
readPrepareParams,
readPreparedDecision,
readReward,
readSensoryInput,
readShutdownParams,
readShutdownResult,
readStatusResult,
readStepResult,
readStimulus,
readTaskEvent,
readWorldObservation,
} from '../src/workers';
/** Every type the fixtures name, and the reader that validates it. */
export const READERS: Record<string, (value: unknown) => unknown> = {
Scope: readScope,
RationalNs: readRational,
SchemaRef: readSchemaRef,
TypedValue: readTypedValue,
SessionRpcRequest: readSessionRpcRequest,
SessionRpcSuccess: readSessionRpcSuccess,
SessionRpcFailure: readSessionRpcFailure,
AssetRef: readAssetRef,
SensoryInput: readSensoryInput,
Stimulus: readStimulus,
Reward: readReward,
AgentTelemetry: readAgentTelemetry,
AgentInitializeParams: readAgentInitializeParams,
AgentInitializeResult: readAgentInitializeResult,
PrepareParams: readPrepareParams,
PreparedDecision: readPreparedDecision,
CommitParams: readCommitParams,
AgentCommitResult: readAgentCommitResult,
ControllerSchema: readControllerSchema,
PortControl: readPortControl,
EnvironmentDescriptor: readEnvironmentDescriptor,
EnvironmentInitializeParams: readEnvironmentInitializeParams,
EnvironmentInitializeResult: readEnvironmentInitializeResult,
WorldObservation: readWorldObservation,
AdvanceParams: readAdvanceParams,
StepResult: readStepResult,
HelloParams: readHelloParams,
HelloResult: readHelloResult,
StatusResult: readStatusResult,
AcknowledgeParams: readAcknowledgeParams,
AcknowledgeResult: readAcknowledgeResult,
ShutdownParams: readShutdownParams,
ShutdownResult: readShutdownResult,
TaskEvent: readTaskEvent,
EpisodeRequest: readEpisodeRequest,
ViewDescriptor: readViewDescriptor,
ViewRef: readViewRef,
AudioDescriptor: readAudioDescriptor,
AudioRef: readAudioRef,
CaptureParams: readCaptureParams,
CaptureResult: readCaptureResult,
StageRestoreParams: readStageRestoreParams,
StageRestoreResult: readStageRestoreResult,
ActivateRestoreParams: readActivateRestoreParams,
ActivateRestoreResult: readActivateRestoreResult,
SessionDescriptor: readSessionDescriptor,
CommittedSnapshot: readCommittedSnapshot,
TraceBehaviour: readTraceBehaviour,
TraceOperational: readTraceOperational,
TransitionTrace: readTransitionTrace,
};
/** Reads the value as `typeName` and hands back what the reader reconstructed. */
export function roundTrip(typeName: string, value: Json): unknown {
const reader = READERS[typeName];
if (!reader) throw new Error(`no fixture reader for type "${typeName}"`);
return reader(value);
}

View file

@ -0,0 +1,100 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { canonicalize, digestOf, parseStrict, sha256Hex } from '../src/canonical';
import * as fixtures from '../src/fixtures';
/**
* The schema set is generated by the Rust crate; this side hashes the checked-in file with its
* own canonical JSON and digest, which is the cross-language half of `contractDigest`.
*/
test('the contract digest is the digest of the checked-in schema set', () => {
const bytes = fixtures.loadBytes('schema-set.json');
const text = Buffer.from(bytes).toString('utf8');
assert.ok(text.endsWith('\n'), 'the file is the canonical set plus one newline');
const canonical = text.slice(0, -1);
const recorded = fixtures.load('contract-digest.json') as Record<string, unknown>;
assert.equal(sha256Hex(canonical), recorded.contractDigest);
assert.equal(canonical.length, recorded.schemaSetBytes);
// ... and the file really is canonical JSON: reserializing it changes nothing.
assert.equal(canonicalize(parseStrict(canonical)), canonical);
assert.equal(digestOf(parseStrict(canonical)), recorded.contractDigest);
});
test('the contract digest survives reformatting and changes when a schema changes', () => {
const set = parseStrict(
Buffer.from(fixtures.loadBytes('schema-set.json')).toString('utf8').slice(0, -1),
) as Record<string, unknown>;
const recorded = fixtures.load('contract-digest.json') as Record<string, unknown>;
const pretty = parseStrict(JSON.stringify(set, null, 4));
assert.equal(digestOf(pretty), recorded.contractDigest, 'pretty printing is not a change');
const types = set.types as Record<string, unknown>[];
const renamed = structuredClone(set);
((renamed.types as Record<string, unknown>[])[0]!.fields as Record<string, unknown>[])[0]!.name =
'sessionIdentifier';
assert.notEqual(digestOf(renamed), recorded.contractDigest, 'a renamed field is a change');
const widened = structuredClone(set);
for (const limit of widened.limits as Record<string, unknown>[]) {
if (limit.name === 'maxAgents') limit.value = 8;
}
assert.notEqual(digestOf(widened), recorded.contractDigest, 'a widened bound is a change');
const dropped = structuredClone(set);
(dropped.types as unknown[]).pop();
assert.notEqual(digestOf(dropped), recorded.contractDigest, 'a dropped type is a change');
assert.ok(types.length >= 50, 'the schema set should stay broad');
});
test('the schema set publishes the limits this package enforces', async () => {
const set = parseStrict(
Buffer.from(fixtures.loadBytes('schema-set.json')).toString('utf8').slice(0, -1),
) as Record<string, unknown>;
const limits = new Map(
(set.limits as Record<string, unknown>[]).map((limit) => [
limit.name as string,
limit.value as number,
]),
);
const workers = await import('../src/workers');
const media = await import('../src/media');
const scalar = await import('../src/scalar');
const canonical = await import('../src/canonical');
assert.equal(limits.get('maxAgents'), workers.MAX_AGENTS);
assert.equal(limits.get('maxPorts'), workers.MAX_PORTS);
assert.equal(limits.get('maxRateRoles'), workers.MAX_RATE_ROLES);
assert.equal(limits.get('maxStimuliPerOperation'), workers.MAX_STIMULI);
assert.equal(limits.get('maxRewardsPerOperation'), workers.MAX_REWARDS);
assert.equal(limits.get('maxButtons'), workers.MAX_BUTTONS);
assert.equal(limits.get('maxAxes'), workers.MAX_AXES);
assert.equal(limits.get('maxAcknowledge'), workers.MAX_ACKNOWLEDGE);
assert.equal(limits.get('maxMessageCodePoints'), workers.MAX_MESSAGE_CODE_POINTS);
assert.equal(limits.get('maxViews'), media.MAX_VIEWS);
assert.equal(limits.get('maxViewDimension'), media.MAX_VIEW_DIMENSION);
assert.equal(limits.get('maxSampleFrames'), media.MAX_SAMPLE_FRAMES);
assert.equal(limits.get('maxAudioStreams'), media.MAX_AUDIO_STREAMS);
assert.equal(limits.get('maxTypedValueBytes'), scalar.MAX_TYPED_VALUE_BYTES);
assert.equal(limits.get('maxEnvelopeBytes'), canonical.MAX_ENVELOPE_BYTES);
});
test('the closed enums this package knows are the ones the schema set declares', async () => {
const set = parseStrict(
Buffer.from(fixtures.loadBytes('schema-set.json')).toString('utf8').slice(0, -1),
) as Record<string, unknown>;
const enums = new Map(
(set.enums as Record<string, unknown>[]).map((entry) => [
entry.name as string,
entry.members as string[],
]),
);
const workers = await import('../src/workers');
const rpc = await import('../src/rpc');
assert.deepEqual(enums.get('ErrorCode'), [...rpc.ERROR_CODES]);
assert.deepEqual(enums.get('MutationCertainty'), [...rpc.MUTATION_CERTAINTIES]);
assert.deepEqual(enums.get('Role'), [...workers.ROLES]);
assert.deepEqual(enums.get('WorkerState'), [...workers.WORKER_STATES]);
assert.deepEqual(enums.get('Recovery'), [...workers.RECOVERY]);
assert.deepEqual(enums.get('Determinism'), [...workers.DETERMINISM]);
assert.deepEqual(enums.get('AxisRange'), [...workers.AXIS_RANGES]);
});

View file

@ -0,0 +1,61 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import * as fixtures from '../src/fixtures';
import * as seed from '../src/seed';
test('every vector derives its recorded seed', () => {
const file = fixtures.load('seed-vectors.json');
assert.equal((file as Record<string, unknown>).algorithm, seed.ALGORITHM);
const vectors = fixtures.section(file, 'vectors');
for (const item of vectors) {
const master = seed.masterSeed(fixtures.field(item, 'masterSeed'));
const agentId = fixtures.field(item, 'agentId');
assert.equal(
Buffer.from(seed.material(master, agentId)).toString('utf8'),
fixtures.field(item, 'material'),
'the hashed material is part of the specification',
);
assert.equal(seed.materialDigest(master, agentId), fixtures.field(item, 'materialDigest'));
assert.equal(
seed.agentSeed(master, agentId),
(item as Record<string, unknown>).seed,
`seed for ${agentId} under master ${master}`,
);
}
assert.ok(vectors.length >= 20, 'keep the vector table broad');
});
test('one composition gets independent seeds', () => {
const file = fixtures.load('seed-vectors.json');
const composition = fixtures.member(file, 'composition');
const master = seed.masterSeed(fixtures.field(composition, 'masterSeed'));
const ids = fixtures.section(composition, 'agentIds') as string[];
const seeds = seed.compositionSeeds(master, ids);
assert.deepEqual(seeds, fixtures.section(composition, 'seeds'));
assert.equal(new Set(seeds).size, seeds.length, 'per-agent seeds are independent');
assert.ok(
seeds.every((value) => value !== 0),
'a zero seed would stall an xorshift generator',
);
});
test('a different master seed or agent id derives a different seed', () => {
assert.notEqual(seed.agentSeed(0n, 'fly-a'), seed.agentSeed(1n, 'fly-a'));
assert.notEqual(seed.agentSeed(0n, 'fly-a'), seed.agentSeed(0n, 'fly-b'));
assert.equal(seed.agentSeed(7n, 'fly-a'), seed.agentSeed(7n, 'fly-a'));
});
test('invalid inputs are refused rather than normalized', () => {
const file = fixtures.load('seed-vectors.json');
for (const item of fixtures.section(file, 'invalid')) {
const master = seed.masterSeed(fixtures.field(item, 'masterSeed'));
const agentId = fixtures.optionalField(item, 'agentId');
if (agentId !== undefined) {
assert.throws(() => seed.agentSeed(master, agentId), `${agentId} must be refused`);
} else {
const ids = fixtures.section(item, 'agentIds') as string[];
assert.throws(() => seed.compositionSeeds(master, ids));
}
}
});

View file

@ -0,0 +1,80 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import * as fixtures from '../src/fixtures';
import {
behaviourDiff,
behaviourDigest,
behaviourEquals,
readTransitionTrace,
runsEqual,
} from '../src/trace';
test('every variant compares the way the fixture says', () => {
const file = fixtures.load('traces.json');
const baseline = readTransitionTrace(fixtures.member(file, 'baseline'));
for (const item of fixtures.section(file, 'variants')) {
const name = fixtures.field(item, 'name');
const variant = readTransitionTrace(fixtures.member(item, 'trace'));
const expected = (item as Record<string, unknown>).behaviourEquals as boolean;
const diff = behaviourDiff(baseline, variant);
assert.equal(
behaviourEquals(baseline, variant),
expected,
`${name}: behaviour equality. differences: ${JSON.stringify(diff)}`,
);
assert.equal(diff.length === 0, expected, `${name}: the diff is empty exactly when equal`);
const needle = fixtures.optionalField(item, 'diffContains');
if (needle !== undefined) {
assert.ok(
diff.some((line) => line.includes(needle)),
`${name}: the diff should name ${needle}, got ${JSON.stringify(diff)}`,
);
}
if (expected) {
assert.equal(
behaviourDigest(baseline.behaviour),
behaviourDigest(variant.behaviour),
`${name}: equal behaviour has one digest`,
);
}
}
});
test('a whole run compares transition by transition', () => {
const file = fixtures.load('traces.json');
const baseline = readTransitionTrace(fixtures.member(file, 'baseline'));
const variants = fixtures.section(file, 'variants');
const reversed = readTransitionTrace(fixtures.member(variants[0] as never, 'trace'));
const changed = readTransitionTrace(
fixtures.member(
variants.find((item) => fixtures.field(item, 'name') === 'one extra neural tick') as never,
'trace',
),
);
assert.ok(runsEqual([baseline, baseline], [reversed, baseline]));
assert.ok(!runsEqual([baseline], [changed]));
assert.ok(!runsEqual([baseline], [baseline, baseline]));
});
test('operational metadata is recorded and excluded', () => {
const file = fixtures.load('traces.json');
const baseline = readTransitionTrace(fixtures.member(file, 'baseline'));
assert.equal(baseline.operational.busCallIds.length, 3);
assert.equal(baseline.operational.prepareRequestIds.length, 2);
assert.equal(baseline.operational.deliveryIds.length, 2);
const retried = readTransitionTrace(
fixtures.member(
fixtures
.section(file, 'variants')
.find(
(item) =>
fixtures.field(item, 'name') ===
'a safe retry with fresh bus callIds, delivery ids and wall time',
) as never,
'trace',
),
);
assert.notDeepEqual(baseline.operational, retried.operational);
assert.ok(behaviourEquals(baseline, retried));
});

View file

@ -0,0 +1,19 @@
{
"compilerOptions": {
"target": "ES2022",
"useDefineForClassFields": true,
"module": "ESNext",
"lib": ["ES2022", "DOM", "DOM.Iterable", "WebWorker"],
"types": ["node"],
"skipLibCheck": true,
"moduleResolution": "Bundler",
"isolatedModules": true,
"moduleDetection": "force",
"noEmit": true,
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"resolveJsonModule": true
},
"include": ["src", "tests"]
}

View file

@ -102,22 +102,28 @@
"reason": "an integral double prints without a fraction" "reason": "an integral double prints without a fraction"
}, },
{ {
"value": 0.1, "value": -17.0,
"canonical": "0.1", "canonical": "-17",
"accept": true, "accept": true,
"reason": "" "reason": ""
}, },
{ {
"value": 1e+21, "value": 0.1,
"canonical": "1e+21", "canonical": "0.1",
"accept": true, "accept": true,
"reason": "ECMAScript switches to exponent notation at 1e21" "reason": "the shortest round-tripping form"
},
{
"value": 0.30000000000000004,
"canonical": "0.30000000000000004",
"accept": true,
"reason": "shortest round-tripping form, not a rounded one"
}, },
{ {
"value": 1e-07, "value": 1e-07,
"canonical": "1e-7", "canonical": "1e-7",
"accept": true, "accept": true,
"reason": "" "reason": "ECMAScript switches to exponent notation below 1e-6"
}, },
{ {
"value": 5e-324, "value": 5e-324,
@ -126,10 +132,10 @@
"reason": "the smallest subnormal double" "reason": "the smallest subnormal double"
}, },
{ {
"value": 1.7976931348623157e+308, "value": 1234.5678,
"canonical": "1.7976931348623157e+308", "canonical": "1234.5678",
"accept": true, "accept": true,
"reason": "the largest finite double" "reason": "a fractional value of any magnitude is canonicalizable"
}, },
{ {
"value": 9007199254740991, "value": 9007199254740991,
@ -137,17 +143,35 @@
"accept": true, "accept": true,
"reason": "the largest exactly representable integer" "reason": "the largest exactly representable integer"
}, },
{
"value": -9007199254740991,
"canonical": "-9007199254740991",
"accept": true,
"reason": ""
},
{ {
"value": 9007199254740993, "value": 9007199254740993,
"canonical": null, "canonical": null,
"accept": false, "accept": false,
"reason": "past the exact integer range, canonical JSON refuses it" "reason": "an integral value past the exact range; counters are U64 strings"
}, },
{ {
"value": 0.30000000000000004, "value": 1e+21,
"canonical": "0.30000000000000004", "canonical": null,
"accept": true, "accept": false,
"reason": "shortest round-tripping form, not a rounded one" "reason": "integral and far past the exact range; JSON.parse cannot tell it from the same digits written out"
},
{
"value": 1.7976931348623157e+308,
"canonical": null,
"accept": false,
"reason": "integral and far past the exact range"
},
{
"value": 1.5e+20,
"canonical": null,
"accept": false,
"reason": "1.5e20 is integral as a double, so it falls under the same rule as 1e21"
} }
] ]
} }

View file

@ -5,9 +5,12 @@
//! sorted tree produces the same bytes), strings escaped the way `JSON.stringify` escapes //! sorted tree produces the same bytes), strings escaped the way `JSON.stringify` escapes
//! them, no insignificant whitespace. A digest is the SHA-256 of those bytes, lowercase hex. //! them, no insignificant whitespace. A digest is the SHA-256 of those bytes, lowercase hex.
//! //!
//! Numbers outside the exactly representable double range are refused rather than rounded: //! A JSON number is canonicalizable when it is finite and, if it is integral, no larger in
//! every counter and clock in these contracts is a `U64` decimal string, so a JSON number //! magnitude than 2^53-1. Integers past that range are refused rather than rounded: every
//! larger than 2^53-1 is a schema error, not something to canonicalize approximately. //! counter and clock in these contracts is a `U64` decimal string, so a large JSON number is
//! a schema error. The rule is stated on the value, not on how it was written, because
//! `JSON.parse` cannot tell `1e21` from `1000000000000000000000`, and two implementations
//! that disagree about one number do not agree about any digest.
use serde_json::{Number, Value}; use serde_json::{Number, Value};
use sha2::{Digest as _, Sha256}; use sha2::{Digest as _, Sha256};
@ -20,15 +23,14 @@ pub const MAX_EXACT_INTEGER: i64 = 9_007_199_254_740_991;
/// The bus envelope ceiling every domain message must also fit (bus-v1 section 4). /// The bus envelope ceiling every domain message must also fit (bus-v1 section 4).
pub const MAX_ENVELOPE_BYTES: usize = flybus::wire::MAX_ENVELOPE_BYTES; pub const MAX_ENVELOPE_BYTES: usize = flybus::wire::MAX_ENVELOPE_BYTES;
/// The `f64` a JSON number denotes, or `None` if it is not a finite exactly representable one. /// The `f64` a JSON number denotes, or `None` if it is not canonicalizable: not finite, or an
/// integral value outside the exactly representable integer range.
pub fn finite_double(n: &Number) -> Option<f64> { pub fn finite_double(n: &Number) -> Option<f64> {
if let Some(u) = n.as_u64() { let value = n.as_f64().filter(|v| v.is_finite())?;
return (u <= MAX_EXACT_INTEGER as u64).then_some(u as f64); if value.fract() == 0.0 && value.abs() > MAX_EXACT_INTEGER as f64 {
return None;
} }
if let Some(i) = n.as_i64() { Some(value)
return (i >= -MAX_EXACT_INTEGER).then_some(i as f64);
}
n.as_f64().filter(|v| v.is_finite())
} }
/// `String(number)` for a finite double, the ECMAScript algorithm RFC 8785 requires. /// `String(number)` for a finite double, the ECMAScript algorithm RFC 8785 requires.