From 4effc6020cf784e9a8303bd63a243dcd99d7606a Mon Sep 17 00:00:00 2001 From: dev Date: Tue, 22 Sep 2026 17:34:20 +0000 Subject: [PATCH 1/6] session: the publication boundary, observer isolation and the repair path publishing-v1 over the same bus: a declared delivery policy per topic, named publication outcomes, the bounded event batch, application-owned state and cues, a read-only descriptor query service and a fake multi-agent consumer. The session publishes the contract types rather than an ad-hoc payload, so a descriptor says what the workers attested to and a snapshot is checked against it before it is published and again when it is read: every frame comes from the boundary its declared delay implies, every handle is the artifact its reference names, and an observer's refusal takes no world step and fences no epoch. AgentInitializeResult gains graph (datasetDigest, indexDigest, neuronCount, rateRoles, supportedStimuli), without which no AgentDescriptor field in publishing-v1 section 3 had a source. Dated amendments to workers-v1 section 2, publishing-v1 section 2 and state-media-v1 section 3. --- .../design/session-framework/publishing-v1.md | 10 + .../session-framework/state-media-v1.md | 15 + docs/design/session-framework/workers-v1.md | 20 + packages/session-types/src/publishing.ts | 4 +- packages/session-types/src/workers.ts | 29 + .../fixtures/contract-digest.json | 6 +- .../fly-session-types/fixtures/invalid.json | 259 +++ .../fixtures/schema-set.json | 2 +- .../fly-session-types/fixtures/valid.json | 17 +- .../fly-session-types/src/publishing.rs | 5 +- .../crates/fly-session-types/src/schema.rs | 16 + .../crates/fly-session-types/src/workers.rs | 96 +- services/flysim/crates/fly-session/README.md | 45 + .../flysim/crates/fly-session/src/agent.rs | 59 +- services/flysim/crates/fly-session/src/cli.rs | 1 + .../crates/fly-session/src/coordinator.rs | 472 ++++-- .../flysim/crates/fly-session/src/harness.rs | 101 +- .../flysim/crates/fly-session/src/launcher.rs | 8 + services/flysim/crates/fly-session/src/lib.rs | 1 + .../flysim/crates/fly-session/src/media.rs | 12 + .../flysim/crates/fly-session/src/publish.rs | 1425 +++++++++++++++++ .../flysim/crates/fly-session/src/types.rs | 5 +- .../crates/fly-session/tests/publishing.rs | 1099 +++++++++++++ 23 files changed, 3568 insertions(+), 139 deletions(-) create mode 100644 services/flysim/crates/fly-session/src/publish.rs create mode 100644 services/flysim/crates/fly-session/tests/publishing.rs diff --git a/docs/design/session-framework/publishing-v1.md b/docs/design/session-framework/publishing-v1.md index 01cd30b..c1ebce2 100644 --- a/docs/design/session-framework/publishing-v1.md +++ b/docs/design/session-framework/publishing-v1.md @@ -42,6 +42,16 @@ Example addresses (chosen by composition, not recognized by router code): | `app.pokemon.cues` | Pub/sub: application narrative/presentation events under declared delivery policy | | `app.pokemon` | RPC: application queries/admission, e.g. restore UI state or request a supported effect | +**Amendment, 2026-09-22 (PUBLISH-01).** The repair path above needs exact methods, and +"exact methods require session API schemas" left the row unbuildable. The session registers +one **read-only** service, `session..query`, with exactly two methods, both ordinary +[session RPCs](ipc-v1.md) answering from what the session already published: +`Session.GetDescriptor` takes an optional `{revision: U64}` and returns that `SessionDescriptor` +or, with no revision, the newest; `Session.GetSnapshot` takes no parameters and returns the +latest `CommittedSnapshot`. A revision the session never published is `IDENTITY_MISMATCH`, not +an empty answer. Nothing on this service mutates, selects a participant or reaches a worker, so +it is not the controller API section 7 rules out; adding a third method that did would be. + Descriptor revisions and scope link observations to schemas. Cross-topic ordering is not guaranteed; a subscriber receiving an unknown descriptor revision must fetch it through the application/session query contract or buffer a bounded number of snapshots, not infer shape. diff --git a/docs/design/session-framework/state-media-v1.md b/docs/design/session-framework/state-media-v1.md index 2669786..3342ab8 100644 --- a/docs/design/session-framework/state-media-v1.md +++ b/docs/design/session-framework/state-media-v1.md @@ -117,6 +117,21 @@ If it violates configured resource policy, disconnect/restart that observer inst live data or silently skipping simulation input. Global store exhaustion is an explicit fault or pause condition; the router cannot guess that a particular live object is disposable. +**Amendment, 2026-09-22 (PUBLISH-01).** "Disconnect/restart that observer" names an action +no participant can take under [Flybus v1](bus-v1.md). Section 5 there makes publish admission +all or nothing -- "for a bounded subscriber overflow, reject the **whole** publish; no partial +fan-out or retained-latest update" -- and the router exposes no per-subscriber eviction, so a +session meeting a full bounded queue cannot drop that one subscriber and deliver to the rest. +The realisable reading, which the session now implements, is three-part: observation topics +are published `latest`, and a latest subscriber can never refuse a publication (it loses its +own queued value and is told how many by `replaced`); a bounded subscriber's refusal, which +`bus-v1` section 6 explicitly permits, is a named and counted publication outcome that takes +no world step, stalls nothing and fences no epoch, and the exact value stays recoverable +through the [publishing-v1](publishing-v1.md) section 2 query path; and disconnecting the +offender is an operator action against the topic the ledger names, not something the session +performs. A per-subscriber drop would need a router operation Flybus v1 does not have, and +inventing one here would be a transport change written into the wrong document. + No coordinator tracks per-reader socket acknowledgments or calls a producer's reclaim method. The SDK and bus perform that bookkeeping. File-backed immutable mappings are safe after unlink; physical pages disappear when all OS mappings close. Pooled reuse is deferred until diff --git a/docs/design/session-framework/workers-v1.md b/docs/design/session-framework/workers-v1.md index 996d5cb..9692dc6 100644 --- a/docs/design/session-framework/workers-v1.md +++ b/docs/design/session-framework/workers-v1.md @@ -96,9 +96,29 @@ interface AgentInitializeResult { warmupTicks: U64; committedStep: U64; // committedStep == "0" decisionContextDigest: Digest; telemetry: AgentTelemetry; + graph: AgentGraph; +} +interface AgentGraph { + datasetDigest: Digest; indexDigest: Digest; neuronCount: U64; + rateRoles: Id[]; // <=64, unique; AgentTelemetry.rates is in this order + supportedStimuli: Id[]; // <=64, unique; an undeclared kind is UNSUPPORTED } ``` +**Amendment, 2026-09-22 (PUBLISH-01).** `AgentInitializeResult` gains `graph`, because +[publishing-v1](publishing-v1.md) section 3 requires `datasetDigest`, `indexDigest`, +`neuronCount`, `rateRoles` and `supportedStimuli` in every published `AgentDescriptor` and no +worker method carried any of them. Without this the only available source is the composition +that asked for the agent, so a descriptor could only ever agree with itself and the section 3 +rule that "geometry/spike mapping requires indexDigest, not merely the same number of neurons" +would have nothing to compare. Initialize is where the agent has just loaded its dataset and +built its index, so the attestation belongs there. `rateRoles` is the "profile-defined order" +section 1 already requires `AgentTelemetry.rates` to be in, and the result is refused when the +two disagree; `supportedStimuli` is the profile capability section 1 already requires a +stimulus kind to resolve through, and a kind outside it is refused with `UNSUPPORTED` before +the model is touched. It changes `contractDigest`, which [session RPC](ipc-v1.md) section 4 +already provides for. + **Amendment, 2026-09-22 (SESSION-02).** `HelloResult.limits` gains `workerThreads`, an integer >=1 reporting the allocation the launcher started that worker within, because "within launcher allocation" above had no wire-level proof: the launcher passes the number to diff --git a/packages/session-types/src/publishing.ts b/packages/session-types/src/publishing.ts index 855460a..a117650 100644 --- a/packages/session-types/src/publishing.ts +++ b/packages/session-types/src/publishing.ts @@ -16,6 +16,7 @@ import { type EnvironmentDescriptor, MAX_AGENTS, MAX_RATE_ROLES, + MAX_SUPPORTED_STIMULI, type PortControl, findPort, readAgentTelemetry, @@ -26,8 +27,9 @@ import { validateTelemetryRoles, } from './workers'; +export { MAX_SUPPORTED_STIMULI } from './workers'; + /** Not stated by a document; this crate's choices, published in the schema set. */ -export const MAX_SUPPORTED_STIMULI = 64; export const MAX_ASSETS = 64; export const MAX_SNAPSHOT_EVENTS = 64; diff --git a/packages/session-types/src/workers.ts b/packages/session-types/src/workers.ts index 6f93a9d..904c5f2 100644 --- a/packages/session-types/src/workers.ts +++ b/packages/session-types/src/workers.ts @@ -35,6 +35,8 @@ import { readSchemaRef, readTypedValue, readNullableTypedValue } from './common' export const MAX_AGENTS = 4; export const MAX_PORTS = 4; export const MAX_RATE_ROLES = 64; +/** Declared stimulus kinds per agent. Not a stated bound; recorded in the schema set. */ +export const MAX_SUPPORTED_STIMULI = 64; export const MAX_STIMULI = 64; export const MAX_REWARDS = 64; export const MAX_BUTTONS = 32; @@ -257,6 +259,14 @@ export interface AgentInitializeParams { workerThreads: number; } +export interface AgentGraph { + datasetDigest: Digest; + indexDigest: Digest; + neuronCount: U64; + rateRoles: Id[]; + supportedStimuli: Id[]; +} + export interface AgentInitializeResult { agentId: Id; profileDigest: Digest; @@ -265,6 +275,7 @@ export interface AgentInitializeResult { committedStep: U64; decisionContextDigest: Digest; telemetry: AgentTelemetry; + graph: AgentGraph; } export interface PrepareParams { @@ -313,6 +324,21 @@ export function readAgentInitializeParams(value: unknown): AgentInitializeParams return params; } +export function readAgentGraph(value: unknown): AgentGraph { + const reader = new Reader(value, 'AgentGraph'); + const graph: AgentGraph = { + datasetDigest: reader.digest('datasetDigest'), + indexDigest: reader.digest('indexDigest'), + neuronCount: reader.u64('neuronCount'), + rateRoles: reader.idList('rateRoles', 0, MAX_RATE_ROLES), + supportedStimuli: reader.idList('supportedStimuli', 0, MAX_SUPPORTED_STIMULI), + }; + reader.finish(); + requireUnique(graph.rateRoles, 'AgentGraph.rateRoles'); + requireUnique(graph.supportedStimuli, 'AgentGraph.supportedStimuli'); + return graph; +} + export function readAgentInitializeResult(value: unknown): AgentInitializeResult { const reader = new Reader(value, 'AgentInitializeResult'); const result: AgentInitializeResult = { @@ -323,12 +349,15 @@ export function readAgentInitializeResult(value: unknown): AgentInitializeResult committedStep: reader.u64('committedStep'), decisionContextDigest: reader.digest('decisionContextDigest'), telemetry: readAgentTelemetry(reader.value('telemetry')), + graph: readAgentGraph(reader.value('graph')), }; reader.finish(); requirePositiveRational(result.tickDuration, 'AgentInitializeResult.tickDuration'); if (u64(result.committedStep) !== 0n) { fail('AgentInitializeResult: committedStep must be "0"'); } + // The rates a worker reports and the role order it declares are one statement. + validateTelemetryRoles(result.telemetry, result.graph.rateRoles); return result; } diff --git a/services/flysim/crates/fly-session-types/fixtures/contract-digest.json b/services/flysim/crates/fly-session-types/fixtures/contract-digest.json index 8845d36..36c1f10 100644 --- a/services/flysim/crates/fly-session-types/fixtures/contract-digest.json +++ b/services/flysim/crates/fly-session-types/fixtures/contract-digest.json @@ -1,9 +1,9 @@ { "description": "contractDigest is the SHA-256 of the canonical schema set in schema-set.json.", - "contractDigest": "d8f29a49b5df05ad8f75f7f5790a3f8cde9c5ad23a685137474c649c3c9da36d", + "contractDigest": "fa1f9671c098e0e5fe2d2ad9642d10eae39c6e751760eb1db7b51d5575898509", "schemaSetVersion": 1, - "schemaSetBytes": 26814, - "types": 53, + "schemaSetBytes": 27407, + "types": 54, "enums": 11, "limits": 26 } diff --git a/services/flysim/crates/fly-session-types/fixtures/invalid.json b/services/flysim/crates/fly-session-types/fixtures/invalid.json index cbce2b4..0b6bcff 100644 --- a/services/flysim/crates/fly-session-types/fixtures/invalid.json +++ b/services/flysim/crates/fly-session-types/fixtures/invalid.json @@ -2746,6 +2746,19 @@ "changed": "2", "signal": 0.5 } + }, + "graph": { + "datasetDigest": "6c0af1f0784ef63a393ee77d614e8246c625051360f3f1a48838374c5d355b52", + "indexDigest": "52a7a9b441f686ebea398e3d65dbaa87b4fde4f8b2231fff951700548f6a4e42", + "neuronCount": "139255", + "rateRoles": [ + "kenyon", + "mbon" + ], + "supportedStimuli": [ + "sugar", + "shock" + ] } }, "reason": "initialization establishes Ready(0)" @@ -2782,10 +2795,256 @@ "changed": "2", "signal": 0.5 } + }, + "graph": { + "datasetDigest": "6c0af1f0784ef63a393ee77d614e8246c625051360f3f1a48838374c5d355b52", + "indexDigest": "52a7a9b441f686ebea398e3d65dbaa87b4fde4f8b2231fff951700548f6a4e42", + "neuronCount": "139255", + "rateRoles": [ + "kenyon", + "mbon" + ], + "supportedStimuli": [ + "sugar", + "shock" + ] } }, "reason": "durations are positive" }, + { + "name": "agent initialize result without a graph identity", + "type": "AgentInitializeResult", + "value": { + "agentId": "fly-a", + "profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0", + "tickDuration": { + "numerator": "1000000", + "denominator": "1" + }, + "warmupTicks": "2500", + "committedStep": "0", + "decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4", + "telemetry": { + "brainTicks": "2500", + "populationRateHz": 12.5, + "rates": [ + { + "roleId": "kenyon", + "hz": 3.25 + }, + { + "roleId": "mbon", + "hz": 0.0 + } + ], + "learning": { + "enabled": true, + "updates": "4", + "changed": "2", + "signal": 0.5 + } + } + }, + "reason": "publishing-v1 3 needs datasetDigest, indexDigest, neuronCount, rateRoles and supportedStimuli, and only the worker knows them" + }, + { + "name": "agent initialize result whose index digest is not a digest", + "type": "AgentInitializeResult", + "value": { + "agentId": "fly-a", + "profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0", + "tickDuration": { + "numerator": "1000000", + "denominator": "1" + }, + "warmupTicks": "2500", + "committedStep": "0", + "decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4", + "telemetry": { + "brainTicks": "2500", + "populationRateHz": 12.5, + "rates": [ + { + "roleId": "kenyon", + "hz": 3.25 + }, + { + "roleId": "mbon", + "hz": 0.0 + } + ], + "learning": { + "enabled": true, + "updates": "4", + "changed": "2", + "signal": 0.5 + } + }, + "graph": { + "datasetDigest": "6c0af1f0784ef63a393ee77d614e8246c625051360f3f1a48838374c5d355b52", + "indexDigest": "not-a-digest", + "neuronCount": "139255", + "rateRoles": [ + "kenyon", + "mbon" + ], + "supportedStimuli": [ + "sugar", + "shock" + ] + } + }, + "reason": "digests are 64 lowercase hex digits" + }, + { + "name": "agent initialize result whose rates are not in the declared role order", + "type": "AgentInitializeResult", + "value": { + "agentId": "fly-a", + "profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0", + "tickDuration": { + "numerator": "1000000", + "denominator": "1" + }, + "warmupTicks": "2500", + "committedStep": "0", + "decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4", + "telemetry": { + "brainTicks": "2500", + "populationRateHz": 12.5, + "rates": [ + { + "roleId": "kenyon", + "hz": 3.25 + }, + { + "roleId": "mbon", + "hz": 0.0 + } + ], + "learning": { + "enabled": true, + "updates": "4", + "changed": "2", + "signal": 0.5 + } + }, + "graph": { + "datasetDigest": "6c0af1f0784ef63a393ee77d614e8246c625051360f3f1a48838374c5d355b52", + "indexDigest": "52a7a9b441f686ebea398e3d65dbaa87b4fde4f8b2231fff951700548f6a4e42", + "neuronCount": "139255", + "rateRoles": [ + "mbon", + "kenyon" + ], + "supportedStimuli": [ + "sugar", + "shock" + ] + } + }, + "reason": "AgentTelemetry.rates is in graph.rateRoles order" + }, + { + "name": "agent initialize result declaring a role it reports no rate for", + "type": "AgentInitializeResult", + "value": { + "agentId": "fly-a", + "profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0", + "tickDuration": { + "numerator": "1000000", + "denominator": "1" + }, + "warmupTicks": "2500", + "committedStep": "0", + "decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4", + "telemetry": { + "brainTicks": "2500", + "populationRateHz": 12.5, + "rates": [ + { + "roleId": "kenyon", + "hz": 3.25 + }, + { + "roleId": "mbon", + "hz": 0.0 + } + ], + "learning": { + "enabled": true, + "updates": "4", + "changed": "2", + "signal": 0.5 + } + }, + "graph": { + "datasetDigest": "6c0af1f0784ef63a393ee77d614e8246c625051360f3f1a48838374c5d355b52", + "indexDigest": "52a7a9b441f686ebea398e3d65dbaa87b4fde4f8b2231fff951700548f6a4e42", + "neuronCount": "139255", + "rateRoles": [ + "kenyon", + "mbon", + "pn" + ], + "supportedStimuli": [ + "sugar", + "shock" + ] + } + }, + "reason": "AgentTelemetry.rates is exactly graph.rateRoles" + }, + { + "name": "agent initialize result repeating a supported stimulus", + "type": "AgentInitializeResult", + "value": { + "agentId": "fly-a", + "profileDigest": "1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0", + "tickDuration": { + "numerator": "1000000", + "denominator": "1" + }, + "warmupTicks": "2500", + "committedStep": "0", + "decisionContextDigest": "ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4", + "telemetry": { + "brainTicks": "2500", + "populationRateHz": 12.5, + "rates": [ + { + "roleId": "kenyon", + "hz": 3.25 + }, + { + "roleId": "mbon", + "hz": 0.0 + } + ], + "learning": { + "enabled": true, + "updates": "4", + "changed": "2", + "signal": 0.5 + } + }, + "graph": { + "datasetDigest": "6c0af1f0784ef63a393ee77d614e8246c625051360f3f1a48838374c5d355b52", + "indexDigest": "52a7a9b441f686ebea398e3d65dbaa87b4fde4f8b2231fff951700548f6a4e42", + "neuronCount": "139255", + "rateRoles": [ + "kenyon", + "mbon" + ], + "supportedStimuli": [ + "sugar", + "sugar" + ] + } + }, + "reason": "supportedStimuli is unique" + }, { "name": "hello result for an agent without agent-step-v1", "type": "HelloResult", diff --git a/services/flysim/crates/fly-session-types/fixtures/schema-set.json b/services/flysim/crates/fly-session-types/fixtures/schema-set.json index e1a175f..6477910 100644 --- a/services/flysim/crates/fly-session-types/fixtures/schema-set.json +++ b/services/flysim/crates/fly-session-types/fixtures/schema-set.json @@ -1 +1 @@ -{"contract":"fly-session-types","enums":[{"members":["f32le-interleaved"],"name":"AudioFormat","source":"state-media-v1 2"},{"members":["bipolar","unit"],"name":"AxisRange","source":"workers-v1 3"},{"members":["fixed-build","unverified"],"name":"Determinism","source":"workers-v1 3"},{"members":["terminal"],"name":"EpisodeRequestKind","source":"workers-v1 4"},{"members":["INVALID_ARGUMENT","UNSUPPORTED","IDENTITY_MISMATCH","STALE_EPOCH","STALE_STEP","FUTURE_STEP","INVALID_PHASE","CONFLICT","IN_PROGRESS","BUSY","BUFFER_INVALID","RESULT_EXPIRED","INCOMPATIBLE_STATE","BACKEND_FAILURE","INTERNAL"],"name":"ErrorCode","source":"ipc-v1 7"},{"members":["none","applied","unknown"],"name":"MutationCertainty","source":"ipc-v1 3"},{"members":["exact-checkpoint","episode-restart"],"name":"Recovery","source":"workers-v1 3"},{"members":["agent","environment","coordinator"],"name":"Role","source":"ipc-v1 4"},{"members":["lockstep-v1"],"name":"SchedulerId","source":"publishing-v1 3"},{"members":["rgba8"],"name":"ViewFormat","source":"state-media-v1 2"},{"members":["uninitialized","ready","preparing","prepared","advancing","committing","capturing","staged-restore","restoring","failed","stopping"],"name":"WorkerState","source":"ipc-v1 4"}],"limits":[{"name":"maxAcknowledge","source":"ipc-v1 5","value":16},{"name":"maxAgents","source":"ipc-v1 2","value":4},{"name":"maxAssets","source":"crate","value":64},{"name":"maxAttachments","source":"bus-v1 4","value":32},{"name":"maxAudioStreams","source":"crate","value":8},{"name":"maxAxes","source":"workers-v1 3","value":16},{"name":"maxButtons","source":"workers-v1 3","value":32},{"name":"maxCapabilities","source":"crate","value":32},{"name":"maxEngineFrameLength","source":"workers-v1 3","value":64},{"name":"maxEnvelopeBytes","source":"bus-v1 4","value":65536},{"name":"maxMessageCodePoints","source":"ipc-v1 7","value":512},{"name":"maxObservationDelaySteps","source":"state-media-v1 2","value":8},{"name":"maxPixelAspectPart","source":"state-media-v1 2","value":65535},{"name":"maxPorts","source":"ipc-v1 2","value":4},{"name":"maxRateRoles","source":"ipc-v1 2","value":64},{"name":"maxRewardsPerOperation","source":"workers-v1 1","value":64},{"name":"maxSampleFrames","source":"state-media-v1 2","value":192000},{"name":"maxSchemaVersion","source":"ipc-v1 2","value":65535},{"name":"maxSnapshotEvents","source":"crate","value":64},{"name":"maxStimuliPerOperation","source":"workers-v1 1","value":64},{"name":"maxSupportedMajors","source":"crate","value":8},{"name":"maxSupportedStimuli","source":"crate","value":64},{"name":"maxTypedValueBytes","source":"ipc-v1 2","value":32768},{"name":"maxViewDimension","source":"state-media-v1 2","value":4096},{"name":"maxViews","source":"workers-v1 1","value":8},{"name":"maxWorkerThreads","source":"workers-v1 2","value":4096}],"scalars":{"ArtifactIdentity":"storeId, artifactId and generation of a bus ArtifactRef","BusCallId":"call-","Digest":"64 lowercase hexadecimal digits (SHA-256)","DomainRequestId":"req-","Id":"^[a-z0-9][a-z0-9._-]{0,63}$","OwnerToken":"dlv- or own-","U64":"\"0\" or [1-9][0-9]*, <= 18446744073709551615"},"types":[{"fields":[{"constraint":"1..=16, unique","kind":"array","name":"requestIds","required":true}],"name":"AcknowledgeParams","source":"ipc-v1 5"},{"fields":[{"constraint":"<= 16, unique, a subset of the request","kind":"array","name":"acknowledged","required":true}],"name":"AcknowledgeResult","source":"ipc-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"restoreToken","required":true}],"name":"ActivateRestoreParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"required from an environment, null from an agent","kind":"WorldObservation|null","name":"observation","required":false}],"name":"ActivateRestoreResult","source":"state-media-v1 5"},{"fields":[{"constraint":"unique within an epoch","kind":"Id","name":"batchId","required":true},{"constraint":"one complete batch: every declared port once, descriptor order","kind":"array","name":"controls","required":true}],"name":"AdvanceParams","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"k+1","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true}],"name":"AgentCommitResult","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Id","name":"portId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"","kind":"Digest","name":"datasetDigest","required":true},{"constraint":"geometry mapping needs this, not neuronCount","kind":"Digest","name":"indexDigest","required":true},{"constraint":"","kind":"U64","name":"neuronCount","required":true},{"constraint":"<= 64, unique","kind":"array","name":"rateRoles","required":true},{"constraint":"unique","kind":"array","name":"supportedStimuli","required":true}],"name":"AgentDescriptor","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"AssetRef","name":"profile","required":true},{"constraint":"signed 32-bit","kind":"int","name":"seed","required":true},{"constraint":"","kind":"SensoryInput","name":"initialInput","required":true},{"constraint":"","kind":"TypedValue","name":"initialDecisionContext","required":true},{"constraint":">= 1","kind":"int","name":"workerThreads","required":true}],"name":"AgentInitializeParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"positive","kind":"RationalNs","name":"tickDuration","required":true},{"constraint":"","kind":"U64","name":"warmupTicks","required":true},{"constraint":"\"0\"","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true}],"name":"AgentInitializeResult","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":"finite and nonnegative","kind":"number","name":"populationRateHz","required":true},{"constraint":"<= 64, unique roleId, profile order, finite nonnegative hz","kind":"array<{roleId:Id,hz:number}>","name":"rates","required":true},{"constraint":"changed <= updates; signal finite","kind":"{enabled:bool,updates:U64,changed:U64,signal:number}","name":"learning","required":true}],"name":"AgentTelemetry","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"id","required":true},{"constraint":"","kind":"Digest","name":"digest","required":true},{"constraint":"positive","kind":"U64","name":"byteLength","required":true},{"constraint":"","kind":"Id","name":"format","required":true}],"name":"AssetRef","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"streamId","required":true},{"constraint":"8000..=192000","kind":"int","name":"sampleRate","required":true},{"constraint":"1..=8","kind":"int","name":"channels","required":true},{"constraint":"","kind":"AudioFormat","name":"format","required":true}],"name":"AudioDescriptor","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"streamId","required":true},{"constraint":"no overlap or rewind within an epoch","kind":"U64","name":"firstSample","required":true},{"constraint":"0..=192000","kind":"int","name":"sampleFrames","required":true},{"constraint":"byteLength == sampleFrames x channels x 4, finite f32","kind":"ArtifactRef","name":"samples","required":true},{"constraint":"true on the first chunk after restore","kind":"bool","name":"discontinuity","required":true}],"name":"AudioRef","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true}],"name":"CaptureParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"the committed boundary","kind":"U64","name":"boundary","required":true},{"constraint":"","kind":"Digest","name":"compatibilityDigest","required":true},{"constraint":"listed attachment; digest required","kind":"ArtifactRef","name":"payload","required":true}],"name":"CaptureResult","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"DomainRequestId","name":"preparedRequestId","required":true},{"constraint":"boundary == scope.step + 1","kind":"SensoryInput","name":"nextInput","required":true},{"constraint":"","kind":"TypedValue","name":"nextDecisionContext","required":true},{"constraint":"<= 64, unique eventId, order retained","kind":"array","name":"rewards","required":true},{"constraint":"<= 64, unique id, order retained","kind":"array","name":"taskStimulations","required":true}],"name":"CommitParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"descriptorRevision","required":true},{"constraint":"","kind":"Id","name":"publisherIncarnation","required":true},{"constraint":"the committed boundary","kind":"Scope","name":"scope","required":true},{"constraint":"","kind":"Id","name":"episodeId","required":true},{"constraint":"monotonic within publisherIncarnation","kind":"U64","name":"sequence","required":true},{"constraint":"","kind":"RationalNs","name":"worldTime","required":true},{"constraint":"1..=4, unique agentId, telemetry in profile role order","kind":"array","name":"agents","required":true},{"constraint":"","kind":"TypedValue","name":"progress","required":true},{"constraint":"declared attachments held through publication admission","kind":"{views:array,audio:array}","name":"media","required":true},{"constraint":"unique, task order","kind":"array","name":"eventIds","required":true}],"name":"CommittedSnapshot","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"SchemaRef","name":"schema","required":true},{"constraint":"<= 32, unique, fixed order","kind":"array","name":"buttons","required":true},{"constraint":"<= 16, unique id, neutral inside its range","kind":"array<{id:Id,range:AxisRange,neutral:number}>","name":"axes","required":true}],"name":"ControllerSchema","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Digest","name":"backendDigest","required":true},{"constraint":"","kind":"Digest","name":"contentDigest","required":true},{"constraint":"","kind":"Digest","name":"configurationDigest","required":true},{"constraint":"fixed, reduced, positive","kind":"RationalNs","name":"stepDuration","required":true},{"constraint":"1..=4, unique portId, fixed order","kind":"array<{portId:Id,controls:ControllerSchema}>","name":"ports","required":true},{"constraint":"","kind":"SchemaRef","name":"inspectionSchema","required":true},{"constraint":"<= 8, unique viewId","kind":"array","name":"views","required":true},{"constraint":"unique streamId","kind":"array","name":"audio","required":true},{"constraint":"","kind":"Recovery","name":"recovery","required":true},{"constraint":"","kind":"Determinism","name":"determinism","required":true}],"name":"EnvironmentDescriptor","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"AssetRef","name":"backendConfig","required":true},{"constraint":"","kind":"AssetRef","name":"taskConfig","required":true},{"constraint":"","kind":"Id","name":"episodeId","required":true},{"constraint":"1..=4, unique portId and unique agentId","kind":"array<{portId:Id,agentId:Id}>","name":"portBindings","required":true}],"name":"EnvironmentInitializeParams","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"EnvironmentDescriptor","name":"descriptor","required":true},{"constraint":"boundary 0 and worldTime 0/1","kind":"WorldObservation","name":"observation","required":true}],"name":"EnvironmentInitializeResult","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"EpisodeRequestKind","name":"kind","required":true},{"constraint":"","kind":"Id","name":"reason","required":true},{"constraint":"","kind":"TypedValue","name":"outcome","required":true}],"name":"EpisodeRequest","source":"workers-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"sessionId","required":true},{"constraint":"","kind":"Id","name":"expectedWorkerId","required":true},{"constraint":"","kind":"Role","name":"role","required":true},{"constraint":"nonempty, unique, 1..=65535","kind":"array","name":"supportedMajors","required":true}],"name":"HelloParams","source":"ipc-v1 4"},{"fields":[{"constraint":"1","kind":"const","name":"selectedMajor","required":true},{"constraint":"0","kind":"const","name":"selectedMinor","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"","kind":"Role","name":"role","required":true},{"constraint":"","kind":"Digest","name":"buildDigest","required":true},{"constraint":"","kind":"Digest","name":"contractDigest","required":true},{"constraint":"unique; agent-step-v1 or world-step-v1 required for that role","kind":"array","name":"capabilities","required":true},{"constraint":"1..=4 agents, 1..=4 ports, and the launcher allocation this worker runs in","kind":"{maxAgents:int,maxPorts:int,workerThreads:int}","name":"limits","required":true}],"name":"HelloResult","source":"ipc-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"portId","required":true},{"constraint":"every declared button, descriptor order, no extras","kind":"array<{id:Id,down:bool}>","name":"buttons","required":true},{"constraint":"every declared axis in order; bipolar [-1,1], unit [0,1], refused not clamped","kind":"array<{id:Id,value:number}>","name":"axes","required":true}],"name":"PortControl","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"positive","kind":"RationalNs","name":"interval","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"<= 64, unique id, supplied order retained","kind":"array","name":"preStepStimulations","required":true}],"name":"PrepareParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"<= brainTicks","kind":"U64","name":"ticksAdvanced","required":true},{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":">= 0 and < one model tick","kind":"RationalNs","name":"remainder","required":true},{"constraint":"the profile's registered intent schema","kind":"TypedValue","name":"decision","required":true}],"name":"PreparedDecision","source":"workers-v1 2"},{"fields":[{"constraint":"reduced against denominator","kind":"U64","name":"numerator","required":true},{"constraint":"positive; zero is encoded 0/1; arithmetic is checked","kind":"U64","name":"denominator","required":true}],"name":"RationalNs","source":"ipc-v1 2"},{"fields":[{"constraint":"unique within its outcome namespace","kind":"Id","name":"eventId","required":true},{"constraint":"","kind":"Id","name":"ruleId","required":true},{"constraint":"finite; positive-only profiles reject negatives","kind":"number","name":"value","required":true}],"name":"Reward","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"id","required":true},{"constraint":"1..=65535","kind":"int","name":"version","required":true},{"constraint":"64 lowercase hex digits","kind":"Digest","name":"digest","required":true}],"name":"SchemaRef","source":"ipc-v1 2"},{"fields":[{"constraint":"^[a-z0-9][a-z0-9._-]{0,63}$","kind":"Id","name":"sessionId","required":true},{"constraint":"^[a-z0-9][a-z0-9._-]{0,63}$","kind":"Id","name":"epoch","required":true},{"constraint":"decimal string, <= 18446744073709551615","kind":"U64","name":"step","required":true}],"name":"Scope","source":"ipc-v1 2"},{"fields":[{"constraint":"the observed environment boundary","kind":"U64","name":"boundary","required":true},{"constraint":"<= 8, unique viewId, producedStep <= boundary","kind":"array","name":"views","required":true},{"constraint":"a pixel-only profile rejects non-null","kind":"TypedValue|null","name":"structured","required":false}],"name":"SensoryInput","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"sessionId","required":true},{"constraint":"","kind":"U64","name":"revision","required":true},{"constraint":"","kind":"Digest","name":"compositionDigest","required":true},{"constraint":"","kind":"SchedulerId","name":"schedulerId","required":true},{"constraint":"","kind":"EnvironmentDescriptor","name":"environment","required":true},{"constraint":"","kind":"SchemaRef","name":"taskSchema","required":true},{"constraint":"1..=4, unique agentId and portId, each portId declared by the environment","kind":"array","name":"agents","required":true},{"constraint":"unique id","kind":"array","name":"assets","required":true}],"name":"SessionDescriptor","source":"publishing-v1 3"},{"fields":[{"constraint":"\"error\"","kind":"const","name":"type","required":true},{"constraint":"echoes the request","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"echoes the request scope","kind":"Scope|null","name":"scope","required":false},{"constraint":"","kind":"ErrorCode","name":"error.code","required":true},{"constraint":"<= 512 code points","kind":"string","name":"error.message","required":true},{"constraint":"none for every code raised before mutation","kind":"MutationCertainty","name":"error.mutation","required":true}],"name":"SessionRpcFailure","source":"ipc-v1 3"},{"fields":[{"constraint":"req-","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"null for lifecycle calls","kind":"Scope|null","name":"scope","required":false},{"constraint":"no bus callId/deliveryId/ownerId keys","kind":"object","name":"params","required":true}],"name":"SessionRpcRequest","source":"ipc-v1 2"},{"fields":[{"constraint":"\"result\"","kind":"const","name":"type","required":true},{"constraint":"echoes the request","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"echoes the request scope","kind":"Scope|null","name":"scope","required":false},{"constraint":"","kind":"object","name":"result","required":true}],"name":"SessionRpcSuccess","source":"ipc-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"reason","required":true}],"name":"ShutdownParams","source":"ipc-v1 7"},{"fields":[{"constraint":"true","kind":"const","name":"stopping","required":true}],"name":"ShutdownResult","source":"ipc-v1 7"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true},{"constraint":"null exactly at boundary 0","kind":"TypedValue|null","name":"selectedDecision","required":false},{"constraint":"null exactly at boundary 0; the agent's assigned port","kind":"PortControl|null","name":"appliedControls","required":false}],"name":"SnapshotAgent","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"provenance, not the new handles","kind":"Scope","name":"sourceScope","required":true},{"constraint":"","kind":"Digest","name":"compatibilityDigest","required":true},{"constraint":"newly imported; digest required","kind":"ArtifactRef","name":"payload","required":true}],"name":"StageRestoreParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"activates once, bound to scope and payload","kind":"Id","name":"restoreToken","required":true}],"name":"StageRestoreResult","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"WorkerState","name":"state","required":true},{"constraint":"null before initialization","kind":"Scope|null","name":"currentScope","required":false},{"constraint":"","kind":"DomainRequestId|null","name":"activeRequestId","required":false},{"constraint":"","kind":"DomainRequestId|null","name":"lastCompletedRequestId","required":false},{"constraint":"","kind":"Id|null","name":"lastBatchId","required":false},{"constraint":"advances on progress, not on status queries","kind":"U64","name":"progressCounter","required":true}],"name":"StatusResult","source":"ipc-v1 4"},{"fields":[{"constraint":"echoes the request","kind":"Id","name":"batchId","required":true},{"constraint":"k","kind":"U64","name":"appliedFromStep","required":true},{"constraint":"appliedFromStep + 1","kind":"U64","name":"nextStep","required":true},{"constraint":"over validated canonical requested controls","kind":"Digest","name":"appliedControlsDigest","required":true},{"constraint":"boundary == nextStep","kind":"WorldObservation","name":"observation","required":true}],"name":"StepResult","source":"workers-v1 3"},{"fields":[{"constraint":"unique within its command namespace","kind":"Id","name":"id","required":true},{"constraint":"resolved through a profile capability","kind":"Id","name":"kindId","required":true},{"constraint":"finite and > 0","kind":"number","name":"durationMs","required":true}],"name":"Stimulus","source":"workers-v1 1"},{"fields":[{"constraint":"derived from epoch, source step, rule and ordinal","kind":"Id","name":"id","required":true},{"constraint":"","kind":"Id","name":"kindId","required":true},{"constraint":"the newly reached boundary, 0 for bootstrap","kind":"U64","name":"sourceStep","required":true},{"constraint":"","kind":"Id|null","name":"agentId","required":false},{"constraint":"","kind":"TypedValue","name":"payload","required":true}],"name":"TaskEvent","source":"workers-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"","kind":"U64","name":"ticksAdvanced","required":true},{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":"","kind":"RationalNs","name":"remainder","required":true},{"constraint":"","kind":"Digest","name":"decisionDigest","required":true},{"constraint":"the commit acknowledgment","kind":"U64","name":"committedStep","required":true}],"name":"TraceAgent","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"Scope","name":"scope","required":true},{"constraint":"sorted by agentId, independent of dispatch order","kind":"array","name":"agents","required":true},{"constraint":"","kind":"Id","name":"batchId","required":true},{"constraint":"","kind":"Digest","name":"controlDigest","required":true},{"constraint":"the world boundary the environment acknowledged","kind":"U64","name":"acknowledgedBoundary","required":true},{"constraint":"sorted by viewId","kind":"array<{viewId:Id,producedStep:U64}>","name":"observationBoundaries","required":true},{"constraint":"task outcome ids in task order","kind":"array","name":"outcomeIds","required":true},{"constraint":"task event ids in task order","kind":"array","name":"eventIds","required":true},{"constraint":"","kind":"U64","name":"publishedBoundary","required":true}],"name":"TraceBehaviour","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"U64","name":"wallTimeNs","required":true},{"constraint":"","kind":"array<{agentId:Id,requestId:DomainRequestId}>","name":"prepareRequestIds","required":true},{"constraint":"","kind":"DomainRequestId","name":"advanceRequestId","required":true},{"constraint":"","kind":"array<{agentId:Id,requestId:DomainRequestId}>","name":"commitRequestIds","required":true},{"constraint":"call-","kind":"array","name":"busCallIds","required":true},{"constraint":"dlv- or own-","kind":"array","name":"deliveryIds","required":true}],"name":"TraceOperational","source":"step-v1 8"},{"fields":[{"constraint":"compared between runs","kind":"TraceBehaviour","name":"behaviour","required":true},{"constraint":"recorded, never compared: wall time and transport identities","kind":"TraceOperational","name":"operational","required":true}],"name":"TransitionTrace","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"SchemaRef","name":"schema","required":true},{"constraint":"canonical JSON of the whole TypedValue <= 32768 bytes","kind":"object","name":"value","required":true}],"name":"TypedValue","source":"ipc-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"viewId","required":true},{"constraint":"1..=4096","kind":"int","name":"width","required":true},{"constraint":"1..=4096","kind":"int","name":"height","required":true},{"constraint":"","kind":"ViewFormat","name":"format","required":true},{"constraint":"exactly 4 x width","kind":"int","name":"rowStride","required":true},{"constraint":"positive integers <= 65535","kind":"{numerator:int,denominator:int}","name":"pixelAspect","required":true},{"constraint":"0..=8","kind":"int","name":"observationDelaySteps","required":true}],"name":"ViewDescriptor","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"viewId","required":true},{"constraint":"max(0, boundary - observationDelaySteps) for required sensory views","kind":"U64","name":"producedStep","required":true},{"constraint":"listed attachment; byteLength == rowStride x height","kind":"ArtifactRef","name":"pixels","required":true}],"name":"ViewRef","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"boundary","required":true},{"constraint":"logical time since episode start","kind":"RationalNs","name":"worldTime","required":true},{"constraint":"<= 64 characters","kind":"string|null","name":"engineFrame","required":false},{"constraint":"<= 8, unique viewId","kind":"array","name":"sensoryViews","required":true},{"constraint":"the descriptor's inspectionSchema","kind":"TypedValue","name":"inspection","required":true},{"constraint":"<= 8, unique viewId","kind":"array","name":"broadcastViews","required":true},{"constraint":"unique streamId","kind":"array","name":"audio","required":true}],"name":"WorldObservation","source":"workers-v1 3"}],"version":1} +{"contract":"fly-session-types","enums":[{"members":["f32le-interleaved"],"name":"AudioFormat","source":"state-media-v1 2"},{"members":["bipolar","unit"],"name":"AxisRange","source":"workers-v1 3"},{"members":["fixed-build","unverified"],"name":"Determinism","source":"workers-v1 3"},{"members":["terminal"],"name":"EpisodeRequestKind","source":"workers-v1 4"},{"members":["INVALID_ARGUMENT","UNSUPPORTED","IDENTITY_MISMATCH","STALE_EPOCH","STALE_STEP","FUTURE_STEP","INVALID_PHASE","CONFLICT","IN_PROGRESS","BUSY","BUFFER_INVALID","RESULT_EXPIRED","INCOMPATIBLE_STATE","BACKEND_FAILURE","INTERNAL"],"name":"ErrorCode","source":"ipc-v1 7"},{"members":["none","applied","unknown"],"name":"MutationCertainty","source":"ipc-v1 3"},{"members":["exact-checkpoint","episode-restart"],"name":"Recovery","source":"workers-v1 3"},{"members":["agent","environment","coordinator"],"name":"Role","source":"ipc-v1 4"},{"members":["lockstep-v1"],"name":"SchedulerId","source":"publishing-v1 3"},{"members":["rgba8"],"name":"ViewFormat","source":"state-media-v1 2"},{"members":["uninitialized","ready","preparing","prepared","advancing","committing","capturing","staged-restore","restoring","failed","stopping"],"name":"WorkerState","source":"ipc-v1 4"}],"limits":[{"name":"maxAcknowledge","source":"ipc-v1 5","value":16},{"name":"maxAgents","source":"ipc-v1 2","value":4},{"name":"maxAssets","source":"crate","value":64},{"name":"maxAttachments","source":"bus-v1 4","value":32},{"name":"maxAudioStreams","source":"crate","value":8},{"name":"maxAxes","source":"workers-v1 3","value":16},{"name":"maxButtons","source":"workers-v1 3","value":32},{"name":"maxCapabilities","source":"crate","value":32},{"name":"maxEngineFrameLength","source":"workers-v1 3","value":64},{"name":"maxEnvelopeBytes","source":"bus-v1 4","value":65536},{"name":"maxMessageCodePoints","source":"ipc-v1 7","value":512},{"name":"maxObservationDelaySteps","source":"state-media-v1 2","value":8},{"name":"maxPixelAspectPart","source":"state-media-v1 2","value":65535},{"name":"maxPorts","source":"ipc-v1 2","value":4},{"name":"maxRateRoles","source":"ipc-v1 2","value":64},{"name":"maxRewardsPerOperation","source":"workers-v1 1","value":64},{"name":"maxSampleFrames","source":"state-media-v1 2","value":192000},{"name":"maxSchemaVersion","source":"ipc-v1 2","value":65535},{"name":"maxSnapshotEvents","source":"crate","value":64},{"name":"maxStimuliPerOperation","source":"workers-v1 1","value":64},{"name":"maxSupportedMajors","source":"crate","value":8},{"name":"maxSupportedStimuli","source":"crate","value":64},{"name":"maxTypedValueBytes","source":"ipc-v1 2","value":32768},{"name":"maxViewDimension","source":"state-media-v1 2","value":4096},{"name":"maxViews","source":"workers-v1 1","value":8},{"name":"maxWorkerThreads","source":"workers-v1 2","value":4096}],"scalars":{"ArtifactIdentity":"storeId, artifactId and generation of a bus ArtifactRef","BusCallId":"call-","Digest":"64 lowercase hexadecimal digits (SHA-256)","DomainRequestId":"req-","Id":"^[a-z0-9][a-z0-9._-]{0,63}$","OwnerToken":"dlv- or own-","U64":"\"0\" or [1-9][0-9]*, <= 18446744073709551615"},"types":[{"fields":[{"constraint":"1..=16, unique","kind":"array","name":"requestIds","required":true}],"name":"AcknowledgeParams","source":"ipc-v1 5"},{"fields":[{"constraint":"<= 16, unique, a subset of the request","kind":"array","name":"acknowledged","required":true}],"name":"AcknowledgeResult","source":"ipc-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"restoreToken","required":true}],"name":"ActivateRestoreParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"required from an environment, null from an agent","kind":"WorldObservation|null","name":"observation","required":false}],"name":"ActivateRestoreResult","source":"state-media-v1 5"},{"fields":[{"constraint":"unique within an epoch","kind":"Id","name":"batchId","required":true},{"constraint":"one complete batch: every declared port once, descriptor order","kind":"array","name":"controls","required":true}],"name":"AdvanceParams","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"k+1","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true}],"name":"AgentCommitResult","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Id","name":"portId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"","kind":"Digest","name":"datasetDigest","required":true},{"constraint":"geometry mapping needs this, not neuronCount","kind":"Digest","name":"indexDigest","required":true},{"constraint":"","kind":"U64","name":"neuronCount","required":true},{"constraint":"<= 64, unique","kind":"array","name":"rateRoles","required":true},{"constraint":"unique","kind":"array","name":"supportedStimuli","required":true}],"name":"AgentDescriptor","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"Digest","name":"datasetDigest","required":true},{"constraint":"geometry mapping needs this, not neuronCount","kind":"Digest","name":"indexDigest","required":true},{"constraint":"","kind":"U64","name":"neuronCount","required":true},{"constraint":"<= 64, unique","kind":"array","name":"rateRoles","required":true},{"constraint":"<= 64, unique","kind":"array","name":"supportedStimuli","required":true}],"name":"AgentGraph","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"AssetRef","name":"profile","required":true},{"constraint":"signed 32-bit","kind":"int","name":"seed","required":true},{"constraint":"","kind":"SensoryInput","name":"initialInput","required":true},{"constraint":"","kind":"TypedValue","name":"initialDecisionContext","required":true},{"constraint":">= 1","kind":"int","name":"workerThreads","required":true}],"name":"AgentInitializeParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"positive","kind":"RationalNs","name":"tickDuration","required":true},{"constraint":"","kind":"U64","name":"warmupTicks","required":true},{"constraint":"\"0\"","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true},{"constraint":"rates are in graph.rateRoles order","kind":"AgentGraph","name":"graph","required":true}],"name":"AgentInitializeResult","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":"finite and nonnegative","kind":"number","name":"populationRateHz","required":true},{"constraint":"<= 64, unique roleId, profile order, finite nonnegative hz","kind":"array<{roleId:Id,hz:number}>","name":"rates","required":true},{"constraint":"changed <= updates; signal finite","kind":"{enabled:bool,updates:U64,changed:U64,signal:number}","name":"learning","required":true}],"name":"AgentTelemetry","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"id","required":true},{"constraint":"","kind":"Digest","name":"digest","required":true},{"constraint":"positive","kind":"U64","name":"byteLength","required":true},{"constraint":"","kind":"Id","name":"format","required":true}],"name":"AssetRef","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"streamId","required":true},{"constraint":"8000..=192000","kind":"int","name":"sampleRate","required":true},{"constraint":"1..=8","kind":"int","name":"channels","required":true},{"constraint":"","kind":"AudioFormat","name":"format","required":true}],"name":"AudioDescriptor","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"streamId","required":true},{"constraint":"no overlap or rewind within an epoch","kind":"U64","name":"firstSample","required":true},{"constraint":"0..=192000","kind":"int","name":"sampleFrames","required":true},{"constraint":"byteLength == sampleFrames x channels x 4, finite f32","kind":"ArtifactRef","name":"samples","required":true},{"constraint":"true on the first chunk after restore","kind":"bool","name":"discontinuity","required":true}],"name":"AudioRef","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true}],"name":"CaptureParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"the committed boundary","kind":"U64","name":"boundary","required":true},{"constraint":"","kind":"Digest","name":"compatibilityDigest","required":true},{"constraint":"listed attachment; digest required","kind":"ArtifactRef","name":"payload","required":true}],"name":"CaptureResult","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"DomainRequestId","name":"preparedRequestId","required":true},{"constraint":"boundary == scope.step + 1","kind":"SensoryInput","name":"nextInput","required":true},{"constraint":"","kind":"TypedValue","name":"nextDecisionContext","required":true},{"constraint":"<= 64, unique eventId, order retained","kind":"array","name":"rewards","required":true},{"constraint":"<= 64, unique id, order retained","kind":"array","name":"taskStimulations","required":true}],"name":"CommitParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"descriptorRevision","required":true},{"constraint":"","kind":"Id","name":"publisherIncarnation","required":true},{"constraint":"the committed boundary","kind":"Scope","name":"scope","required":true},{"constraint":"","kind":"Id","name":"episodeId","required":true},{"constraint":"monotonic within publisherIncarnation","kind":"U64","name":"sequence","required":true},{"constraint":"","kind":"RationalNs","name":"worldTime","required":true},{"constraint":"1..=4, unique agentId, telemetry in profile role order","kind":"array","name":"agents","required":true},{"constraint":"","kind":"TypedValue","name":"progress","required":true},{"constraint":"declared attachments held through publication admission","kind":"{views:array,audio:array}","name":"media","required":true},{"constraint":"unique, task order","kind":"array","name":"eventIds","required":true}],"name":"CommittedSnapshot","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"SchemaRef","name":"schema","required":true},{"constraint":"<= 32, unique, fixed order","kind":"array","name":"buttons","required":true},{"constraint":"<= 16, unique id, neutral inside its range","kind":"array<{id:Id,range:AxisRange,neutral:number}>","name":"axes","required":true}],"name":"ControllerSchema","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Digest","name":"backendDigest","required":true},{"constraint":"","kind":"Digest","name":"contentDigest","required":true},{"constraint":"","kind":"Digest","name":"configurationDigest","required":true},{"constraint":"fixed, reduced, positive","kind":"RationalNs","name":"stepDuration","required":true},{"constraint":"1..=4, unique portId, fixed order","kind":"array<{portId:Id,controls:ControllerSchema}>","name":"ports","required":true},{"constraint":"","kind":"SchemaRef","name":"inspectionSchema","required":true},{"constraint":"<= 8, unique viewId","kind":"array","name":"views","required":true},{"constraint":"unique streamId","kind":"array","name":"audio","required":true},{"constraint":"","kind":"Recovery","name":"recovery","required":true},{"constraint":"","kind":"Determinism","name":"determinism","required":true}],"name":"EnvironmentDescriptor","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"AssetRef","name":"backendConfig","required":true},{"constraint":"","kind":"AssetRef","name":"taskConfig","required":true},{"constraint":"","kind":"Id","name":"episodeId","required":true},{"constraint":"1..=4, unique portId and unique agentId","kind":"array<{portId:Id,agentId:Id}>","name":"portBindings","required":true}],"name":"EnvironmentInitializeParams","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"EnvironmentDescriptor","name":"descriptor","required":true},{"constraint":"boundary 0 and worldTime 0/1","kind":"WorldObservation","name":"observation","required":true}],"name":"EnvironmentInitializeResult","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"EpisodeRequestKind","name":"kind","required":true},{"constraint":"","kind":"Id","name":"reason","required":true},{"constraint":"","kind":"TypedValue","name":"outcome","required":true}],"name":"EpisodeRequest","source":"workers-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"sessionId","required":true},{"constraint":"","kind":"Id","name":"expectedWorkerId","required":true},{"constraint":"","kind":"Role","name":"role","required":true},{"constraint":"nonempty, unique, 1..=65535","kind":"array","name":"supportedMajors","required":true}],"name":"HelloParams","source":"ipc-v1 4"},{"fields":[{"constraint":"1","kind":"const","name":"selectedMajor","required":true},{"constraint":"0","kind":"const","name":"selectedMinor","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"","kind":"Role","name":"role","required":true},{"constraint":"","kind":"Digest","name":"buildDigest","required":true},{"constraint":"","kind":"Digest","name":"contractDigest","required":true},{"constraint":"unique; agent-step-v1 or world-step-v1 required for that role","kind":"array","name":"capabilities","required":true},{"constraint":"1..=4 agents, 1..=4 ports, and the launcher allocation this worker runs in","kind":"{maxAgents:int,maxPorts:int,workerThreads:int}","name":"limits","required":true}],"name":"HelloResult","source":"ipc-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"portId","required":true},{"constraint":"every declared button, descriptor order, no extras","kind":"array<{id:Id,down:bool}>","name":"buttons","required":true},{"constraint":"every declared axis in order; bipolar [-1,1], unit [0,1], refused not clamped","kind":"array<{id:Id,value:number}>","name":"axes","required":true}],"name":"PortControl","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"positive","kind":"RationalNs","name":"interval","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"<= 64, unique id, supplied order retained","kind":"array","name":"preStepStimulations","required":true}],"name":"PrepareParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"<= brainTicks","kind":"U64","name":"ticksAdvanced","required":true},{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":">= 0 and < one model tick","kind":"RationalNs","name":"remainder","required":true},{"constraint":"the profile's registered intent schema","kind":"TypedValue","name":"decision","required":true}],"name":"PreparedDecision","source":"workers-v1 2"},{"fields":[{"constraint":"reduced against denominator","kind":"U64","name":"numerator","required":true},{"constraint":"positive; zero is encoded 0/1; arithmetic is checked","kind":"U64","name":"denominator","required":true}],"name":"RationalNs","source":"ipc-v1 2"},{"fields":[{"constraint":"unique within its outcome namespace","kind":"Id","name":"eventId","required":true},{"constraint":"","kind":"Id","name":"ruleId","required":true},{"constraint":"finite; positive-only profiles reject negatives","kind":"number","name":"value","required":true}],"name":"Reward","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"id","required":true},{"constraint":"1..=65535","kind":"int","name":"version","required":true},{"constraint":"64 lowercase hex digits","kind":"Digest","name":"digest","required":true}],"name":"SchemaRef","source":"ipc-v1 2"},{"fields":[{"constraint":"^[a-z0-9][a-z0-9._-]{0,63}$","kind":"Id","name":"sessionId","required":true},{"constraint":"^[a-z0-9][a-z0-9._-]{0,63}$","kind":"Id","name":"epoch","required":true},{"constraint":"decimal string, <= 18446744073709551615","kind":"U64","name":"step","required":true}],"name":"Scope","source":"ipc-v1 2"},{"fields":[{"constraint":"the observed environment boundary","kind":"U64","name":"boundary","required":true},{"constraint":"<= 8, unique viewId, producedStep <= boundary","kind":"array","name":"views","required":true},{"constraint":"a pixel-only profile rejects non-null","kind":"TypedValue|null","name":"structured","required":false}],"name":"SensoryInput","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"sessionId","required":true},{"constraint":"","kind":"U64","name":"revision","required":true},{"constraint":"","kind":"Digest","name":"compositionDigest","required":true},{"constraint":"","kind":"SchedulerId","name":"schedulerId","required":true},{"constraint":"","kind":"EnvironmentDescriptor","name":"environment","required":true},{"constraint":"","kind":"SchemaRef","name":"taskSchema","required":true},{"constraint":"1..=4, unique agentId and portId, each portId declared by the environment","kind":"array","name":"agents","required":true},{"constraint":"unique id","kind":"array","name":"assets","required":true}],"name":"SessionDescriptor","source":"publishing-v1 3"},{"fields":[{"constraint":"\"error\"","kind":"const","name":"type","required":true},{"constraint":"echoes the request","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"echoes the request scope","kind":"Scope|null","name":"scope","required":false},{"constraint":"","kind":"ErrorCode","name":"error.code","required":true},{"constraint":"<= 512 code points","kind":"string","name":"error.message","required":true},{"constraint":"none for every code raised before mutation","kind":"MutationCertainty","name":"error.mutation","required":true}],"name":"SessionRpcFailure","source":"ipc-v1 3"},{"fields":[{"constraint":"req-","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"null for lifecycle calls","kind":"Scope|null","name":"scope","required":false},{"constraint":"no bus callId/deliveryId/ownerId keys","kind":"object","name":"params","required":true}],"name":"SessionRpcRequest","source":"ipc-v1 2"},{"fields":[{"constraint":"\"result\"","kind":"const","name":"type","required":true},{"constraint":"echoes the request","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"echoes the request scope","kind":"Scope|null","name":"scope","required":false},{"constraint":"","kind":"object","name":"result","required":true}],"name":"SessionRpcSuccess","source":"ipc-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"reason","required":true}],"name":"ShutdownParams","source":"ipc-v1 7"},{"fields":[{"constraint":"true","kind":"const","name":"stopping","required":true}],"name":"ShutdownResult","source":"ipc-v1 7"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true},{"constraint":"null exactly at boundary 0","kind":"TypedValue|null","name":"selectedDecision","required":false},{"constraint":"null exactly at boundary 0; the agent's assigned port","kind":"PortControl|null","name":"appliedControls","required":false}],"name":"SnapshotAgent","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"provenance, not the new handles","kind":"Scope","name":"sourceScope","required":true},{"constraint":"","kind":"Digest","name":"compatibilityDigest","required":true},{"constraint":"newly imported; digest required","kind":"ArtifactRef","name":"payload","required":true}],"name":"StageRestoreParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"activates once, bound to scope and payload","kind":"Id","name":"restoreToken","required":true}],"name":"StageRestoreResult","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"WorkerState","name":"state","required":true},{"constraint":"null before initialization","kind":"Scope|null","name":"currentScope","required":false},{"constraint":"","kind":"DomainRequestId|null","name":"activeRequestId","required":false},{"constraint":"","kind":"DomainRequestId|null","name":"lastCompletedRequestId","required":false},{"constraint":"","kind":"Id|null","name":"lastBatchId","required":false},{"constraint":"advances on progress, not on status queries","kind":"U64","name":"progressCounter","required":true}],"name":"StatusResult","source":"ipc-v1 4"},{"fields":[{"constraint":"echoes the request","kind":"Id","name":"batchId","required":true},{"constraint":"k","kind":"U64","name":"appliedFromStep","required":true},{"constraint":"appliedFromStep + 1","kind":"U64","name":"nextStep","required":true},{"constraint":"over validated canonical requested controls","kind":"Digest","name":"appliedControlsDigest","required":true},{"constraint":"boundary == nextStep","kind":"WorldObservation","name":"observation","required":true}],"name":"StepResult","source":"workers-v1 3"},{"fields":[{"constraint":"unique within its command namespace","kind":"Id","name":"id","required":true},{"constraint":"resolved through a profile capability","kind":"Id","name":"kindId","required":true},{"constraint":"finite and > 0","kind":"number","name":"durationMs","required":true}],"name":"Stimulus","source":"workers-v1 1"},{"fields":[{"constraint":"derived from epoch, source step, rule and ordinal","kind":"Id","name":"id","required":true},{"constraint":"","kind":"Id","name":"kindId","required":true},{"constraint":"the newly reached boundary, 0 for bootstrap","kind":"U64","name":"sourceStep","required":true},{"constraint":"","kind":"Id|null","name":"agentId","required":false},{"constraint":"","kind":"TypedValue","name":"payload","required":true}],"name":"TaskEvent","source":"workers-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"","kind":"U64","name":"ticksAdvanced","required":true},{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":"","kind":"RationalNs","name":"remainder","required":true},{"constraint":"","kind":"Digest","name":"decisionDigest","required":true},{"constraint":"the commit acknowledgment","kind":"U64","name":"committedStep","required":true}],"name":"TraceAgent","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"Scope","name":"scope","required":true},{"constraint":"sorted by agentId, independent of dispatch order","kind":"array","name":"agents","required":true},{"constraint":"","kind":"Id","name":"batchId","required":true},{"constraint":"","kind":"Digest","name":"controlDigest","required":true},{"constraint":"the world boundary the environment acknowledged","kind":"U64","name":"acknowledgedBoundary","required":true},{"constraint":"sorted by viewId","kind":"array<{viewId:Id,producedStep:U64}>","name":"observationBoundaries","required":true},{"constraint":"task outcome ids in task order","kind":"array","name":"outcomeIds","required":true},{"constraint":"task event ids in task order","kind":"array","name":"eventIds","required":true},{"constraint":"","kind":"U64","name":"publishedBoundary","required":true}],"name":"TraceBehaviour","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"U64","name":"wallTimeNs","required":true},{"constraint":"","kind":"array<{agentId:Id,requestId:DomainRequestId}>","name":"prepareRequestIds","required":true},{"constraint":"","kind":"DomainRequestId","name":"advanceRequestId","required":true},{"constraint":"","kind":"array<{agentId:Id,requestId:DomainRequestId}>","name":"commitRequestIds","required":true},{"constraint":"call-","kind":"array","name":"busCallIds","required":true},{"constraint":"dlv- or own-","kind":"array","name":"deliveryIds","required":true}],"name":"TraceOperational","source":"step-v1 8"},{"fields":[{"constraint":"compared between runs","kind":"TraceBehaviour","name":"behaviour","required":true},{"constraint":"recorded, never compared: wall time and transport identities","kind":"TraceOperational","name":"operational","required":true}],"name":"TransitionTrace","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"SchemaRef","name":"schema","required":true},{"constraint":"canonical JSON of the whole TypedValue <= 32768 bytes","kind":"object","name":"value","required":true}],"name":"TypedValue","source":"ipc-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"viewId","required":true},{"constraint":"1..=4096","kind":"int","name":"width","required":true},{"constraint":"1..=4096","kind":"int","name":"height","required":true},{"constraint":"","kind":"ViewFormat","name":"format","required":true},{"constraint":"exactly 4 x width","kind":"int","name":"rowStride","required":true},{"constraint":"positive integers <= 65535","kind":"{numerator:int,denominator:int}","name":"pixelAspect","required":true},{"constraint":"0..=8","kind":"int","name":"observationDelaySteps","required":true}],"name":"ViewDescriptor","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"viewId","required":true},{"constraint":"max(0, boundary - observationDelaySteps) for required sensory views","kind":"U64","name":"producedStep","required":true},{"constraint":"listed attachment; byteLength == rowStride x height","kind":"ArtifactRef","name":"pixels","required":true}],"name":"ViewRef","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"boundary","required":true},{"constraint":"logical time since episode start","kind":"RationalNs","name":"worldTime","required":true},{"constraint":"<= 64 characters","kind":"string|null","name":"engineFrame","required":false},{"constraint":"<= 8, unique viewId","kind":"array","name":"sensoryViews","required":true},{"constraint":"the descriptor's inspectionSchema","kind":"TypedValue","name":"inspection","required":true},{"constraint":"<= 8, unique viewId","kind":"array","name":"broadcastViews","required":true},{"constraint":"unique streamId","kind":"array","name":"audio","required":true}],"name":"WorldObservation","source":"workers-v1 3"}],"version":1} diff --git a/services/flysim/crates/fly-session-types/fixtures/valid.json b/services/flysim/crates/fly-session-types/fixtures/valid.json index dea3a64..814d738 100644 --- a/services/flysim/crates/fly-session-types/fixtures/valid.json +++ b/services/flysim/crates/fly-session-types/fixtures/valid.json @@ -470,11 +470,24 @@ "changed": "2", "signal": 0.5 } + }, + "graph": { + "datasetDigest": "6c0af1f0784ef63a393ee77d614e8246c625051360f3f1a48838374c5d355b52", + "indexDigest": "52a7a9b441f686ebea398e3d65dbaa87b4fde4f8b2231fff951700548f6a4e42", + "neuronCount": "139255", + "rateRoles": [ + "kenyon", + "mbon" + ], + "supportedStimuli": [ + "sugar", + "shock" + ] } }, "note": "", - "canonical": "{\"agentId\":\"fly-a\",\"committedStep\":\"0\",\"decisionContextDigest\":\"ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4\",\"profileDigest\":\"1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0\",\"telemetry\":{\"brainTicks\":\"2500\",\"learning\":{\"changed\":\"2\",\"enabled\":true,\"signal\":0.5,\"updates\":\"4\"},\"populationRateHz\":12.5,\"rates\":[{\"hz\":3.25,\"roleId\":\"kenyon\"},{\"hz\":0,\"roleId\":\"mbon\"}]},\"tickDuration\":{\"denominator\":\"1\",\"numerator\":\"1000000\"},\"warmupTicks\":\"2500\"}", - "digest": "a220160c75d758720fe43145089939201ee35c86bb100fae0c4efdd3c4eafaa6" + "canonical": "{\"agentId\":\"fly-a\",\"committedStep\":\"0\",\"decisionContextDigest\":\"ea7792a26f405e2ae9c6f49ca93bbe6076ceac0a1fc53d83426c7d7f2d9377e4\",\"graph\":{\"datasetDigest\":\"6c0af1f0784ef63a393ee77d614e8246c625051360f3f1a48838374c5d355b52\",\"indexDigest\":\"52a7a9b441f686ebea398e3d65dbaa87b4fde4f8b2231fff951700548f6a4e42\",\"neuronCount\":\"139255\",\"rateRoles\":[\"kenyon\",\"mbon\"],\"supportedStimuli\":[\"sugar\",\"shock\"]},\"profileDigest\":\"1900eab6c028483d7126599ee6f50de0d27907b5c65fa90524580b4b0f9852b0\",\"telemetry\":{\"brainTicks\":\"2500\",\"learning\":{\"changed\":\"2\",\"enabled\":true,\"signal\":0.5,\"updates\":\"4\"},\"populationRateHz\":12.5,\"rates\":[{\"hz\":3.25,\"roleId\":\"kenyon\"},{\"hz\":0,\"roleId\":\"mbon\"}]},\"tickDuration\":{\"denominator\":\"1\",\"numerator\":\"1000000\"},\"warmupTicks\":\"2500\"}", + "digest": "707803be4867dc2f0f3d4bccedfaa7b97b1e52b9af78d21ae6957caba1a7e3f8" }, { "name": "prepare params", diff --git a/services/flysim/crates/fly-session-types/src/publishing.rs b/services/flysim/crates/fly-session-types/src/publishing.rs index b90d8e5..562c913 100644 --- a/services/flysim/crates/fly-session-types/src/publishing.rs +++ b/services/flysim/crates/fly-session-types/src/publishing.rs @@ -15,8 +15,9 @@ use crate::workers::{ AgentTelemetry, AssetRef, EnvironmentDescriptor, MAX_AGENTS, MAX_RATE_ROLES, PortControl, }; -/// Declared stimulus kinds per agent. Not a stated bound; recorded in the schema set. -pub const MAX_SUPPORTED_STIMULI: usize = 64; +/// Declared stimulus kinds per agent, re-exported from its defining module. +pub use crate::workers::MAX_SUPPORTED_STIMULI; + /// Installed assets in one descriptor. Not a stated bound; recorded in the schema set. pub const MAX_ASSETS: usize = 64; /// Scoped event ids in one snapshot. Not a stated bound; recorded in the schema set. diff --git a/services/flysim/crates/fly-session-types/src/schema.rs b/services/flysim/crates/fly-session-types/src/schema.rs index 6317620..44eb58f 100644 --- a/services/flysim/crates/fly-session-types/src/schema.rs +++ b/services/flysim/crates/fly-session-types/src/schema.rs @@ -438,6 +438,22 @@ pub const SCHEMAS: &[TypeSchema] = &[ req("committedStep", "U64", "\"0\""), req("decisionContextDigest", "Digest", ""), req("telemetry", "AgentTelemetry", ""), + req("graph", "AgentGraph", "rates are in graph.rateRoles order"), + ], + }, + TypeSchema { + name: "AgentGraph", + source: "workers-v1 2", + fields: &[ + req("datasetDigest", "Digest", ""), + req( + "indexDigest", + "Digest", + "geometry mapping needs this, not neuronCount", + ), + req("neuronCount", "U64", ""), + req("rateRoles", "array", "<= 64, unique"), + req("supportedStimuli", "array", "<= 64, unique"), ], }, TypeSchema { diff --git a/services/flysim/crates/fly-session-types/src/workers.rs b/services/flysim/crates/fly-session-types/src/workers.rs index 7ff4d07..9a785c9 100644 --- a/services/flysim/crates/fly-session-types/src/workers.rs +++ b/services/flysim/crates/fly-session-types/src/workers.rs @@ -19,6 +19,8 @@ pub const MAX_AGENTS: usize = 4; pub const MAX_PORTS: usize = 4; /// 64 rate roles per agent. pub const MAX_RATE_ROLES: usize = 64; +/// Declared stimulus kinds per agent. Not a stated bound; recorded in the schema set. +pub const MAX_SUPPORTED_STIMULI: usize = 64; /// Arrays of stimuli or rewards are bounded to 64 per operation (workers-v1 section 1). pub const MAX_STIMULI: usize = 64; /// Arrays of stimuli or rewards are bounded to 64 per operation. @@ -706,6 +708,92 @@ pub struct AgentInitializeResult { pub committed_step: u64, pub decision_context_digest: String, pub telemetry: AgentTelemetry, + /// The graph identity this agent actually loaded, which is what a descriptor publishes. + /// + /// `publishing-v1` section 3 requires `datasetDigest`, `indexDigest`, `neuronCount`, + /// `rateRoles` and `supportedStimuli` in every `AgentDescriptor`, and before the + /// 2026-09-22 amendment to `workers-v1` section 2 no worker method carried them: a + /// coordinator could only have restated its own configuration. The worker attests + /// instead, so a fly that built another index is a visible mismatch rather than a + /// descriptor that agrees with itself. + pub graph: AgentGraph, +} + +/// What one agent's loaded graph is, as the agent reports it. +/// +/// `neuronCount` does not identify a mapping: "geometry/spike mapping requires indexDigest, +/// not merely the same number of neurons" (publishing-v1 section 3), so both travel and a +/// consumer compares the digest. +#[derive(Clone, Debug, PartialEq)] +pub struct AgentGraph { + pub dataset_digest: String, + pub index_digest: String, + pub neuron_count: u64, + /// The profile-defined rate-role order. `AgentTelemetry.rates` is in exactly this order. + pub rate_roles: Vec, + pub supported_stimuli: Vec, +} + +impl AgentGraph { + pub fn from_json(value: &Value) -> Result { + let mut f = Fields::new(value, "AgentGraph")?; + let dataset_digest = f.string("datasetDigest")?.to_owned(); + let index_digest = f.string("indexDigest")?.to_owned(); + let neuron_count = f.u64_string("neuronCount")?; + let rate_roles = id_list(&mut f, "rateRoles", 0, MAX_RATE_ROLES)?; + let supported_stimuli = id_list(&mut f, "supportedStimuli", 0, MAX_SUPPORTED_STIMULI)?; + f.finish()?; + let g = AgentGraph { + dataset_digest, + index_digest, + neuron_count, + rate_roles, + supported_stimuli, + }; + g.validate()?; + Ok(g) + } + + pub fn to_json(&self) -> Value { + obj(vec![ + ("datasetDigest", self.dataset_digest.clone().into()), + ("indexDigest", self.index_digest.clone().into()), + ("neuronCount", u64_json(self.neuron_count)), + ( + "rateRoles", + Value::Array(self.rate_roles.iter().map(|r| r.clone().into()).collect()), + ), + ( + "supportedStimuli", + Value::Array( + self.supported_stimuli + .iter() + .map(|s| s.clone().into()) + .collect(), + ), + ), + ]) + } + + pub fn validate(&self) -> Result<()> { + if !is_digest(&self.dataset_digest) || !is_digest(&self.index_digest) { + return err("AgentGraph: datasetDigest and indexDigest must be 64 lowercase hex digits"); + } + if self.rate_roles.len() > MAX_RATE_ROLES { + return err("AgentGraph: at most 64 rate roles"); + } + if self.supported_stimuli.len() > MAX_SUPPORTED_STIMULI { + return err("AgentGraph: at most 64 supported stimuli"); + } + require_unique( + self.rate_roles.iter().map(String::as_str), + "AgentGraph.rateRoles", + )?; + require_unique( + self.supported_stimuli.iter().map(String::as_str), + "AgentGraph.supportedStimuli", + ) + } } impl DomainType for AgentInitializeResult { @@ -720,6 +808,7 @@ impl DomainType for AgentInitializeResult { let committed_step = f.u64_string("committedStep")?; let decision_context_digest = f.string("decisionContextDigest")?.to_owned(); let telemetry = AgentTelemetry::from_json(f.value("telemetry")?)?; + let graph = AgentGraph::from_json(f.value("graph")?)?; f.finish()?; let r = AgentInitializeResult { agent_id, @@ -729,6 +818,7 @@ impl DomainType for AgentInitializeResult { committed_step, decision_context_digest, telemetry, + graph, }; r.validate()?; Ok(r) @@ -746,6 +836,7 @@ impl DomainType for AgentInitializeResult { self.decision_context_digest.clone().into(), ), ("telemetry", self.telemetry.to_json()), + ("graph", self.graph.to_json()), ]) } @@ -762,7 +853,10 @@ impl DomainType for AgentInitializeResult { if self.committed_step != 0 { return err("AgentInitializeResult: committedStep must be \"0\""); } - self.telemetry.validate() + self.graph.validate()?; + // The rates a worker reports and the role order it declares are one statement, so a + // descriptor built from the second can never mislabel the first. + self.telemetry.validate_against_roles(&self.graph.rate_roles) } } diff --git a/services/flysim/crates/fly-session/README.md b/services/flysim/crates/fly-session/README.md index 8e114e2..c3d0ca9 100644 --- a/services/flysim/crates/fly-session/README.md +++ b/services/flysim/crates/fly-session/README.md @@ -39,6 +39,7 @@ Ready(k) ─ Prepare all agents concurrently ─────────── | `environment` | The counter arena: one complete batch per advance, one native frame | | `task` | The task and executor traits, the deterministic counter task, the identity executor | | `rpc` | Domain calls: `req-` serials, incarnation pinning, the retry rule | +| `publish` | The publication boundary: declared delivery policies, named publication outcomes, the bounded event batch, the read-only repair service, an application channel and a fake multi-agent consumer | | `coordinator` | The transaction, the trace, the failure rules and the publication boundary | | `launcher` | The supervisor: thread budget, identities, start, health check, reap | | `metrics` | Latency percentiles and the machine's core and memory counters | @@ -192,6 +193,29 @@ harness.shutdown().await; it takes -- the transition finishes, then the session pauses at the boundary it just committed -- is now written into the section 2 machine as a dated amendment. +## The publication boundary + +`publishing-v1` on the same bus, with nothing added to the router: + +| Address | Delivery | Contents | +| --- | --- | --- | +| `session..descriptor` | retained latest | `SessionDescriptor`, built from what each participant attested to | +| `session..snapshots` | retained latest | `CommittedSnapshot` plus the boundary's media handles | +| `session..events` | bounded, depth 64 | the transition's task events, with a `droppedBefore` count | +| `session..query` | RPC, read-only | `Session.GetDescriptor`, `Session.GetSnapshot` | +| `.state`, `.cues` | the application's own | whatever the experience needs, under the application's schema | + +Every publication returns a named outcome: `Accepted`, `RefusedByObserver` or `Faulted`. Only +`BACKPRESSURE` is an observer's refusal, and a refusal takes no world step, stalls nothing and +fences no epoch -- it is counted per topic in the ledger and the exact value stays readable +through the query service. Anything else is the session's own fault and fails the epoch. A +snapshot is checked before it is published and again when it is read: every frame comes from +the boundary its declared delay implies, every handle is the artifact its reference names, +audio never goes backwards, and the snapshot agrees with the descriptor revision it names. + +What is **not** here: the approved public v2 wire schemas and the stage adapters that speak +them. `implementation.md` sequences those after this slice and together with each other. + ## Limitations - **Fake workers.** There is no neural model and no emulator. What is modelled exactly is the @@ -200,6 +224,14 @@ harness.shutdown().await; STATE-01. The phase machine has their edges (`Capturing`, `Restoring`) and the workers do not advertise them as implemented methods. - **No audience input.** The admitted pre-step stimulation list exists and is always empty. +- **One descriptor revision.** A revision changes when the composition does, and the only + in-session path to that is a group restore into a fresh epoch, which is STATE-01's. The + session publishes revision 1; the repair path, the revision cache and the index-change rule + are exercised against a second revision published by a `Publisher` of a second composition. +- **No per-subscriber eviction.** A bounded subscriber may refuse a publication, and Flybus v1 + has no operation to drop that one subscriber, so the refusal costs every subscriber that + boundary's delivery on a stream whose contract is "latest". See the 2026-09-22 amendment to + `state-media-v1` section 3. - **Pacing is coarse.** The pacing deadline rounds one step to whole nanoseconds for sleeping only; simulation time stays rational and that rounding never re-enters the accumulator. @@ -259,6 +291,11 @@ The three integration suites do not all run over both transports, and cannot: - `tests/processes.rs` runs over the Unix socket only, in all three execution modes. A participant in a process of its own has no in-memory transport to reach the router by, so the mode is the axis that suite varies and the transport is fixed. +- `tests/media.rs` and `tests/publishing.rs` run over both transports, and each also generates + a subset once per execution mode. The publication boundary lives in the coordinator, so + unlike the render counter and the sensor log it crosses no process boundary and stays fully + observable in all three modes; `the_publication_boundary_holds_in_every_execution_mode` + asserts that rather than assuming it. - `tests/session.rs`: one world advance per complete batch; every agent Prepared before the advance; one task evaluation per transition; every agent committed before the next Prepare or @@ -275,6 +312,14 @@ The three integration suites do not all run over both transports, and cannot: allocation -- plus the sequential/reversed/parallel trace comparison across all three modes and the two process-mode section 4 rows: a router restart during a world advance, and an old worker's reply after a restart. +- `tests/publishing.rs`: the PUBLISH-01 acceptance bullets over both transports -- a consumer + that disconnects and one that stops consuming, a bounded observer's named refusal, every + boundary's media belonging to that boundary, a frame and a handle from another boundary + refused, an unheld revision repaired rather than inferred, a revision that was never + published, an index that moved under a mapped consumer, boundary 0's null decision, the + committed action being the transition that just ended, one snapshot carrying every agent, + application-owned state and cues, a held event batch, and the read-only query service -- + plus the first two generated once per execution mode by `all_modes!`. - `tests/failures.rs`: a duplicate Prepare after a lost reply; a duplicate Commit; the same batch with altered controls; a lost Advance result; a cached artifact consumed by its first caller; one Commit failing after another succeeded; a replaced registration; a reply from diff --git a/services/flysim/crates/fly-session/src/agent.rs b/services/flysim/crates/fly-session/src/agent.rs index a52822f..aa59bd3 100644 --- a/services/flysim/crates/fly-session/src/agent.rs +++ b/services/flysim/crates/fly-session/src/agent.rs @@ -206,6 +206,10 @@ pub struct AgentConfig { /// The thread allocation the launcher started this worker within. `workers-v1` requires /// `Agent.Initialize`'s `workerThreads` to lie inside it. pub worker_threads: usize, + /// Which graph this fly built. Two variants have the same `neuronCount` and different + /// `indexDigest`, which is the case `publishing-v1` section 3 says a consumer must not + /// mistake for the same mapping. + pub graph_variant: u64, /// Records every view this agent read, so a test can see which artifact reached it. /// /// It is this process's log: an agent with a process of its own writes to its own copy, @@ -439,6 +443,9 @@ impl FakeAgentWorker { committed_step: 0, decision_context_digest: self.context_digest.clone().expect("just set"), telemetry: self.model.telemetry(), + // The worker attests to the graph it loaded. A descriptor built from this can + // disagree with the composition; one built from the composition never could. + graph: synthetic_graph(&self.config.agent_id, self.config.graph_variant), }; Ok(HandlerReply::from(&result)) } @@ -490,6 +497,7 @@ impl FakeAgentWorker { } for stimulus in ¶ms.pre_step_stimulations { stimulus.validate().map_err(DomainError::invalid)?; + check_supported(stimulus)?; } let available = FakeAgentWorker::available_actions(self.context.as_ref().expect("initialized"))?; @@ -577,6 +585,7 @@ impl FakeAgentWorker { } for stimulus in ¶ms.task_stimulations { stimulus.validate().map_err(DomainError::invalid)?; + check_supported(stimulus)?; } params.next_decision_context.validate().map_err(DomainError::invalid)?; FakeAgentWorker::available_actions(¶ms.next_decision_context)?; @@ -697,13 +706,61 @@ pub fn agent_op_class(method: &str) -> Option { } /// A synthetic profile asset for one agent. The digest covers its effective identities. +/// Refuses a stimulus kind this profile does not resolve, before the model is touched. +/// +/// `supportedStimuli` in a published descriptor is exactly this list, so the declaration is +/// what the worker enforces rather than a label printed beside it. +fn check_supported(stimulus: &Stimulus) -> DomainResult<()> { + if SUPPORTED_STIMULI.contains(&stimulus.kind_id.as_str()) { + return Ok(()); + } + Err(DomainError::before( + ErrorCode::Unsupported, + format!( + "stimulus kind {} is not one this profile resolves", + stimulus.kind_id + ), + )) +} + +/// The rate roles this fake model reports, in the order it reports them. +pub const RATE_ROLES: [&str; 2] = ["kc", "mbon"]; + +/// The stimulus kinds this synthetic profile resolves. An undeclared kind is refused before +/// the model is touched, so `supportedStimuli` in a descriptor is what the worker enforces +/// rather than a label beside it. +pub const SUPPORTED_STIMULI: [&str; 1] = ["arena.milestone"]; + +/// This fly's graph identity. Every variant has the same neuron count and its own index, so +/// "the same number of neurons" can never be mistaken for the same mapping. +pub const NEURON_COUNT: u64 = 1024; + +pub fn synthetic_graph(agent_id: &Id, variant: u64) -> AgentGraph { + AgentGraph { + dataset_digest: digest_of_bytes( + format!("arena-dataset-v1\nvariant={variant}\n").as_bytes(), + ), + index_digest: digest_of_bytes( + format!( + "arena-index-v1\nagent={agent_id}\nvariant={variant}\nneurons={NEURON_COUNT}\n" + ) + .as_bytes(), + ), + neuron_count: NEURON_COUNT, + rate_roles: RATE_ROLES.iter().map(|r| id(r)).collect(), + supported_stimuli: SUPPORTED_STIMULI.iter().map(|s| id(s)).collect(), + } +} + pub fn synthetic_profile(agent_id: &Id, tick_duration: &RationalNs, warmup_ticks: u64) -> AssetRef { let text = format!( "arena-direct-v1\nagent={agent_id}\ntick={}/{}\nwarmup={warmup_ticks}\n", tick_duration.numerator, tick_duration.denominator ); AssetRef { - id: id("arena-direct-v1"), + // One installed asset per fly: a descriptor's `assets` are unique by id, and two + // profiles that differ in content are two assets, not one id with two digests. + id: parse_id(&format!("arena-direct-v1-{agent_id}")).expect("a prefix plus an agent id"), digest: digest_of_bytes(text.as_bytes()), byte_length: text.len() as u64, format: id("fly-profile-v1"), diff --git a/services/flysim/crates/fly-session/src/cli.rs b/services/flysim/crates/fly-session/src/cli.rs index 1f7bc68..ac87642 100644 --- a/services/flysim/crates/fly-session/src/cli.rs +++ b/services/flysim/crates/fly-session/src/cli.rs @@ -191,6 +191,7 @@ fn serve(role: &str, options: &Options) -> Result<(), String> { tick_duration: options.rational(flags::TICK_NUMERATOR, flags::TICK_DENOMINATOR)?, warmup_ticks: options.u64(flags::WARMUP_TICKS, 0)?, worker_threads: threads, + graph_variant: options.u64(flags::GRAPH_VARIANT, 0)?, // This process's own log. The supervisor reads what crosses the bus, not this. sensors: crate::media::SensorLog::new(), faults: AgentFaults { diff --git a/services/flysim/crates/fly-session/src/coordinator.rs b/services/flysim/crates/fly-session/src/coordinator.rs index e7e4bb7..aa21e8e 100644 --- a/services/flysim/crates/fly-session/src/coordinator.rs +++ b/services/flysim/crates/fly-session/src/coordinator.rs @@ -12,10 +12,11 @@ use std::collections::{BTreeMap, BTreeSet}; use std::time::{Duration, Instant}; -use serde_json::{Map, Value, json}; +use serde_json::{Map, Value}; use crate::clock::Pacing; use crate::media::{self, AudioTimelines}; +use crate::publish::PublicationOutcome; use crate::metrics::Metrics; use crate::phase::{Phase, PhaseMachine}; use crate::rpc::{self, DomainReply, Serials, WorkerRef}; @@ -54,6 +55,12 @@ pub struct Injections { pub altered_advance_controls: bool, /// Read and release the Advance result's frame, then replay the same operation. pub consume_advance_artifact_then_retry: bool, + /// Publish this boundary's snapshot naming the previous boundary's frame: new agent + /// state beside an older observation. + pub stale_published_view: bool, + /// Publish this boundary's snapshot with a handle that is not the artifact the snapshot + /// references: the same name, the same shape, another object. + pub substituted_published_handle: bool, } /// What an injection produced, for a test to assert on. @@ -171,6 +178,14 @@ impl Default for Deadlines { } } +/// The descriptor revision this slice publishes. +/// +/// A revision changes when the composition does -- a replaced fly with another index, a +/// different port assignment -- and the only in-session path to that is a group restore into +/// a fresh epoch, which is STATE-01's. So a session establishes revision 1 and the repair +/// path, not a revision counter with nothing to count. +pub const DESCRIPTOR_REVISION: u64 = 1; + /// The bus addresses this session publishes on. Chosen by the composition, not the router. #[derive(Clone, Debug)] pub struct Topics { @@ -202,6 +217,11 @@ pub struct AgentSlot { pub tick_duration: RationalNs, pub warmup_ticks: u64, pub committed_step: u64, + /// The graph this fly attested to at `Agent.Initialize`, which is what the published + /// descriptor says about it. `None` before initialization. + pub graph: Option, + /// The telemetry of the last committed boundary, which is what the snapshot publishes. + pub telemetry: Option, context: TypedValue, context_digest: Digest, prepared: Option, @@ -226,6 +246,8 @@ impl AgentSlot { tick_duration: RationalNs::ZERO, warmup_ticks: 0, committed_step: 0, + graph: None, + telemetry: None, context: TypedValue::new(crate::task::context_schema(), Value::Object(Map::new())) .expect("an empty context object is a valid typed value"), context_digest: digest_of_bytes(b""), @@ -272,6 +294,14 @@ pub struct Coordinator { media_names: Vec, serials: Serials, topics: Topics, + /// The publication boundary. Everything this session publishes goes through it, and + /// every outcome it returns is a named one. + publisher: crate::publish::Publisher, + /// The composition as published. Built once from what the live participants attested to, + /// never restated from the configuration that asked for them. + session_descriptor: Option, + /// The read-only repair service. Held so it stops with the session. + query: Option, pacing: Option, /// Set by whoever asks for a normal pause, possibly while a transition is in flight. pause: std::sync::Arc, @@ -303,6 +333,17 @@ pub struct Coordinator { started: std::time::Instant, last_advance_request: Option, last_commit_requests: Vec, + /// The previous committed boundary's broadcast references. Data only: no handle, no owner, + /// no retention, and nothing reads it but the publication fault injections. + previous_broadcast_views: Vec, +} + +/// The broadcast references of an observation, or none when there is no observation yet. +fn observation_views_of(observation: &Option) -> Vec { + observation + .as_ref() + .map(|o| o.broadcast_views.clone()) + .unwrap_or_default() } impl Coordinator { @@ -321,6 +362,7 @@ impl Coordinator { // Sorted agent-id order is the executor and control order, so it is fixed here once. agents.sort_by(|a, b| a.agent_id.cmp(&b.agent_id)); let topics = Topics::for_session(&session_id); + let publisher = crate::publish::Publisher::new(bus.clone(), &session_id, &epoch, &topics); Coordinator { bus, session_id, @@ -343,6 +385,9 @@ impl Coordinator { media::audio_attachment(crate::environment::AUDIO_STREAM_ID), ], serials: Serials::default(), + publisher, + session_descriptor: None, + query: None, topics, pacing: None, pause: std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false)), @@ -364,6 +409,7 @@ impl Coordinator { started: std::time::Instant::now(), last_advance_request: None, last_commit_requests: Vec::new(), + previous_broadcast_views: Vec::new(), } } @@ -379,6 +425,27 @@ impl Coordinator { &self.topics } + /// The composition this session published, once it has. + pub fn session_descriptor(&self) -> Option<&SessionDescriptor> { + self.session_descriptor.as_ref() + } + + /// What this session published and what became of it: accepted, refused by an observer, + /// or faulted, per topic. + pub fn ledger(&self) -> &crate::publish::Ledger { + self.publisher.ledger() + } + + /// The events the bounded batch is still holding because an observer refused them. + pub fn pending_events(&self) -> usize { + self.publisher.outbox().len() + } + + /// The read-only state the repair service answers from. + pub fn published_state(&self) -> crate::publish::SharedState { + self.publisher.state() + } + pub fn epoch(&self) -> &Id { &self.epoch } @@ -626,22 +693,12 @@ impl Coordinator { Ok(()) } + /// Declares every framework topic under the delivery policy the publisher holds. async fn declare_topics(&mut self) -> Outcome<()> { - for (name, retained) in [ - (self.topics.descriptor.clone(), flybus::Retained::Latest), - (self.topics.snapshots.clone(), flybus::Retained::Latest), - (self.topics.events.clone(), flybus::Retained::None), - ] { - self.bus.declare_topic(&name, retained).await.map_err(|e| { - let error = DomainError::new( - ErrorCode::BackendFailure, - format!("declaring {name}: {}", e.message), - MutationCertainty::None, - ); - self.fail_now(error, "declare-topic") - })?; + match self.publisher.declare().await { + Ok(()) => Ok(()), + Err(e) => Err(self.fail_now(e, "declare-topic")), } - Ok(()) } async fn initialize_environment(&mut self) -> Outcome<()> { @@ -812,6 +869,10 @@ impl Coordinator { .telemetry .validate() .map_err(|e| self.fail_now(DomainError::invalid(e), "agent-initialize"))?; + // What the fly says it built. The descriptor publishes this, so a composition that + // loaded another index is visible in the descriptor rather than only in a log line. + self.agents[index].graph = Some(result.graph.clone()); + self.agents[index].telemetry = Some(result.telemetry.clone()); self.agents[index].tick_duration = result.tick_duration; self.agents[index].warmup_ticks = result.warmup_ticks; self.agents[index].committed_step = 0; @@ -1531,6 +1592,9 @@ impl Coordinator { self.agents[index].committed_step = k + 1; } // The previous boundary's handles are no longer needed; the new ones take over. + // The references -- which are data, not ownership -- are kept for one boundary, so a + // publication fault injection can name an older frame without retaining it. + self.previous_broadcast_views = observation_views_of(&self.observation); self.views = new_views; self.audio = new_audio; self.pending_views.clear(); @@ -2445,59 +2509,98 @@ impl Coordinator { // ----------------------------------------------------------------------------------- // Publication - async fn publish( - &mut self, - topic: &str, - payload: Map, - attachments: Vec<(String, flybus::Artifact)>, - ) -> Outcome<()> { - let refs: Vec<(&str, &flybus::Artifact)> = - attachments.iter().map(|(n, a)| (n.as_str(), a)).collect(); - match self.bus.publish(topic, payload, &refs).await { - Ok(_) => Ok(()), - Err(e) => { - // A disconnected or backpressured observer never stalls the world; only a - // real resource fault reaches here, and it fails the epoch honestly. - let error = DomainError::new( - ErrorCode::BackendFailure, - format!("publishing {topic}: {}", e.message), - MutationCertainty::None, - ); - Err(self.fail_now(error, "publish")) + /// Sends one publication and turns its outcome into the session's response to it. + /// + /// An observer's refusal is counted and the world carries on: "ordinary snapshot + /// publication is latest/bounded and never waits for a spectator to consume it" + /// (publishing-v1 section 3), and `bus-v1` section 6 allows a bounded subscriber to reject + /// a publication. A session resource fault is not an observer and fails the epoch. + fn settle(&mut self, outcome: PublicationOutcome, detail: &str) -> Outcome { + match outcome.fault() { + Some(error) => Err(self.fail_now(error, detail)), + None => { + if outcome.is_refused() { + self.audit.push(format!("refused:{}", outcome.topic())); + } + Ok(outcome) } } } + /// The composition as the live participants attested to it. + /// + /// Every agent row comes from that agent's own `Agent.Initialize` reply, so a descriptor + /// can disagree with the configuration that asked for the composition. One restated from + /// the configuration never could, and `publishing-v1` section 3 needs the disagreement to + /// be visible: "geometry/spike mapping requires indexDigest, not merely the same number + /// of neurons". + fn build_descriptor(&self, revision: u64) -> DomainResult { + let environment = self.descriptor.clone().ok_or_else(|| { + DomainError::before(ErrorCode::InvalidPhase, "no environment descriptor") + })?; + let mut agents = Vec::new(); + let mut assets = Vec::new(); + for slot in &self.agents { + let graph = slot.graph.clone().ok_or_else(|| { + DomainError::before( + ErrorCode::InvalidPhase, + format!("agent {} has not attested to a graph", slot.agent_id), + ) + })?; + agents.push(AgentDescriptor { + agent_id: slot.agent_id.clone(), + port_id: slot.port_id.clone(), + profile_digest: slot.profile.digest.clone(), + dataset_digest: graph.dataset_digest, + index_digest: graph.index_digest, + neuron_count: graph.neuron_count, + rate_roles: graph.rate_roles, + supported_stimuli: graph.supported_stimuli, + }); + assets.push(slot.profile.clone()); + } + let descriptor = SessionDescriptor { + session_id: self.session_id.clone(), + revision, + composition_digest: self.composition_digest(), + environment, + task_schema: self.task.schema(), + agents, + assets, + }; + descriptor.validate().map_err(DomainError::invalid)?; + Ok(descriptor) + } + + /// Publishes the composition and starts the read-only repair service beside it. async fn publish_descriptor(&mut self) -> Outcome<()> { - let descriptor = self.descriptor.clone().expect("bootstrapped"); - let agents: Vec = self - .agents - .iter() - .map(|slot| { - json!({ - "agentId": slot.agent_id.as_str(), - "portId": slot.port_id.as_str(), - "profileDigest": slot.profile.digest.as_str(), - "tickDuration": slot.tick_duration.to_json(), - "warmupTicks": slot.warmup_ticks.to_string(), - }) - }) - .collect(); - let payload = json!({ - "sessionId": self.session_id.as_str(), - "revision": "1", - "compositionDigest": self.composition_digest().as_str(), - "schedulerId": "lockstep-v1", - "environment": descriptor.to_json(), - "taskSchema": self.task.schema().to_json(), - "agents": agents, - }); - let topic = self.topics.descriptor.clone(); - self.publish(&topic, match payload { - Value::Object(m) => m, - _ => Map::new(), - }, Vec::new()) - .await + let descriptor = match self.build_descriptor(DESCRIPTOR_REVISION) { + Ok(descriptor) => descriptor, + Err(e) => return Err(self.fail_now(e, "descriptor")), + }; + let outcome = match self.publisher.publish_descriptor(&descriptor).await { + Ok(outcome) => outcome, + Err(e) => return Err(self.fail_now(e, "descriptor")), + }; + self.settle(outcome, "descriptor")?; + self.session_descriptor = Some(descriptor); + if self.query.is_none() { + let state = self.publisher.state(); + let service = + crate::publish::QueryService::start(self.bus.clone(), &self.session_id, state) + .await + .map_err(|e| { + let error = DomainError::new( + ErrorCode::BackendFailure, + format!("registering the session query service: {}", e.message), + MutationCertainty::None, + ); + self.fail_now(error, "query-service") + })?; + self.query = Some(service); + } + self.audit.push("publish:descriptor".to_owned()); + Ok(()) } /// The composition identity: session, epoch, agents, ports and the contract revision. @@ -2517,22 +2620,15 @@ impl Coordinator { digest_of_bytes(text.as_bytes()) } + /// Offers this boundary's events to the bounded batch and publishes what it holds. async fn publish_events(&mut self, source_step: u64, events: &[TaskEvent]) -> Outcome<()> { - if events.is_empty() { - return Ok(()); + match self.publisher.publish_events(source_step, events).await { + None => Ok(()), + Some(outcome) => { + self.settle(outcome, "events")?; + Ok(()) + } } - let payload = json!({ - "sessionId": self.session_id.as_str(), - "epoch": self.epoch.as_str(), - "sourceStep": source_step.to_string(), - "events": Value::Array(events.iter().map(DomainType::to_json).collect()), - }); - let topic = self.topics.events.clone(); - self.publish(&topic, match payload { - Value::Object(m) => m, - _ => Map::new(), - }, Vec::new()) - .await } /// Publishes the committed boundary. Never an in-progress mix of new agent state and an @@ -2553,55 +2649,181 @@ impl Coordinator { "publish", )); } + let descriptor = match self.session_descriptor.clone() { + Some(descriptor) => descriptor, + None => { + return Err(self.fail_now( + DomainError::before(ErrorCode::InvalidPhase, "no descriptor was published"), + "publish", + )); + } + }; let observation = self.observation.clone().expect("bootstrapped"); - let agents: Vec = self - .agents - .iter() - .map(|slot| { - let control = controls - .iter() - .find(|c| c.port_id == slot.port_id) - .map(|c| c.to_json()); - json!({ - "agentId": slot.agent_id.as_str(), - "selectedDecision": decisions - .get(&slot.agent_id) - .map(|d| d.to_json()), - "appliedControls": control, - "committedStep": slot.committed_step.to_string(), - }) - }) - .collect(); - let payload = json!({ - "descriptorRevision": "1", - "publisherIncarnation": self.bus.info().connection_id.clone(), - "scope": self.scope(boundary).to_json(), - "episodeId": self.episode_id.as_str(), - "sequence": self.stats.publications.to_string(), - "worldTime": observation.world_time.to_json(), - "agents": agents, - "progress": self.task.progress().to_json(), - "media": json!({ - "views": Value::Array(observation.broadcast_views.iter().map(DomainType::to_json).collect()), - "audio": Value::Array(observation.audio.iter().map(DomainType::to_json).collect()), - }), - "eventIds": event_ids.iter().map(Id::as_str).collect::>(), - }); - // The same owned handles the agents were given, published once for presentation. - let mut attachments = self.view_attachments(); - attachments.extend(self.audio.iter().map(|(n, a)| (n.clone(), a.clone()))); - let topic = self.topics.snapshots.clone(); - self.publish( - &topic, - match payload { - Value::Object(m) => m, - _ => Map::new(), - }, - attachments, - ) - .await?; - self.stats.publications += 1; - self.audit.push(format!("publish:{boundary}")); + let mut agents = Vec::new(); + for slot in &self.agents { + if slot.committed_step != boundary { + // A snapshot names one boundary. An agent that is not at it would be future + // state beside this world, which is the thing this check exists to refuse. + return Err(self.fail_now( + DomainError::before( + ErrorCode::IdentityMismatch, + format!( + "agent {} is committed at {} and the snapshot is boundary {boundary}", + slot.agent_id, slot.committed_step + ), + ), + "publish", + )); + } + let telemetry = match slot.telemetry.clone() { + Some(telemetry) => telemetry, + None => { + return Err(self.fail_now( + DomainError::before( + ErrorCode::InvalidPhase, + format!("agent {} reported no telemetry", slot.agent_id), + ), + "publish", + )); + } + }; + agents.push(SnapshotAgent { + agent_id: slot.agent_id.clone(), + telemetry, + // "Decisions/controls describe the transition ending at that boundary, null at + // initial boundary 0." These are the decisions of the transition that ended + // here, never the ones prepared for the transition about to start. + selected_decision: decisions.get(&slot.agent_id).cloned(), + applied_controls: controls.iter().find(|c| c.port_id == slot.port_id).cloned(), + }); + } + let mut views = observation.broadcast_views.clone(); + if self.injections.stale_published_view && self.injections.at_step + 1 == boundary { + // The injection of "new agent state with old media": the agents are at this + // boundary and the frame is the previous one's. + let stale = self.previous_broadcast_views.clone(); + if stale.is_empty() { + return Err(self.fail_now( + DomainError::invalid("no previous boundary to take a stale view from"), + "publish", + )); + } + views = stale; + self.injection_log.push(InjectionOutcome { + what: "stale-published-view".to_owned(), + code: None, + identical: false, + }); + } + let snapshot = CommittedSnapshot { + descriptor_revision: descriptor.revision, + publisher_incarnation: self.publisher.incarnation(), + scope: self.scope(boundary), + episode_id: self.episode_id.clone(), + sequence: self.publisher.sequence(), + world_time: observation.world_time, + agents, + progress: self.task.progress(), + views, + audio: observation.audio.clone(), + event_ids: event_ids.to_vec(), + }; + // The same owned handles the agents were given, published once for presentation. A + // referenced frame with no handle, or a handle that is another boundary's object, is + // refused by `check_publication` before anything reaches a subscriber. + let mut attachments = Vec::new(); + for view in &snapshot.views { + let name = media::view_attachment(&view.view_id); + match self.views.get(&name) { + Some(artifact) => attachments.push((name, artifact.clone())), + None => { + return Err(self.fail_now( + DomainError::new( + ErrorCode::BufferInvalid, + format!("this boundary holds no handle for view {}", view.view_id), + MutationCertainty::None, + ), + "publish", + )); + } + } + } + for chunk in &snapshot.audio { + let name = media::audio_attachment(&chunk.stream_id); + match self.audio.get(&name) { + Some(artifact) => attachments.push((name, artifact.clone())), + None => { + return Err(self.fail_now( + DomainError::new( + ErrorCode::BufferInvalid, + format!( + "this boundary holds no handle for stream {}", + chunk.stream_id + ), + MutationCertainty::None, + ), + "publish", + )); + } + } + } + if self.injections.substituted_published_handle && self.injections.at_step + 1 == boundary { + // The same attachment name and the same bytes, a different object. Only the + // artifact identity sees it, which is why the check compares that and not names. + let (name, artifact) = match attachments.first() { + Some(first) => first.clone(), + None => { + return Err(self.fail_now( + DomainError::invalid("no attachment to substitute"), + "publish", + )); + } + }; + let bytes = match artifact.read_all().await { + Ok(bytes) => bytes, + Err(e) => { + return Err(self.fail_now( + DomainError::new( + ErrorCode::BufferInvalid, + e.message, + MutationCertainty::None, + ), + "publish", + )); + } + }; + let copy = match media::seal_copy( + &self.bus, + artifact.reference().content_type.clone(), + &bytes, + ) + .await + { + Ok(copy) => copy, + Err(e) => return Err(self.fail_now(e, "publish")), + }; + attachments[0] = (name, copy); + self.injection_log.push(InjectionOutcome { + what: "substituted-published-handle".to_owned(), + code: None, + identical: false, + }); + } + let positions = self.timelines.positions(); + let outcome = match self + .publisher + .publish_snapshot(&descriptor, &snapshot, &attachments, &positions) + .await + { + Ok(outcome) => outcome, + Err(e) => return Err(self.fail_now(e, "publish")), + }; + let outcome = self.settle(outcome, "publish")?; + if outcome.is_accepted() { + self.stats.publications += 1; + self.audit.push(format!("publish:{boundary}")); + } Ok(()) } } + diff --git a/services/flysim/crates/fly-session/src/harness.rs b/services/flysim/crates/fly-session/src/harness.rs index c537a9b..938557c 100644 --- a/services/flysim/crates/fly-session/src/harness.rs +++ b/services/flysim/crates/fly-session/src/harness.rs @@ -46,6 +46,10 @@ pub struct AgentSpec { /// The threads this agent asks the launcher for. `Agent.Initialize` carries exactly what /// the launcher allocated, which `workers-v1` requires it to lie within. pub worker_threads: usize, + /// Which graph this fly builds. A replacement worker started on another variant has the + /// same neuron count and another `indexDigest`, which is the composition change a + /// descriptor revision exists to make visible. + pub graph_variant: u64, } impl AgentSpec { @@ -57,6 +61,7 @@ impl AgentSpec { seed, faults: AgentFaults::default(), worker_threads: 1, + graph_variant: 0, } } } @@ -143,6 +148,14 @@ fn agent_client(agent_id: &Id) -> String { format!("worker-{agent_id}") } +/// What a presentation consumer may do: subscribe, and ask the read-only repair service. +fn consumer_grants() -> Grants { + grants(|g| { + g.subscribe = vec![Pattern::prefix("session."), Pattern::prefix("app.")]; + g.call = vec![Pattern::prefix("session.")]; + }) +} + fn grants(f: impl FnOnce(&mut Grants)) -> Grants { let mut g = Grants::default(); f(&mut g); @@ -172,6 +185,8 @@ pub struct SessionHarness { /// The supervisor. It owns every participant's lifetime and thread allocation. pub launcher: Launcher, observers: Mutex>, + /// Which configured observer identity the next consumer takes. + next_observer: std::sync::atomic::AtomicUsize, } impl SessionHarness { @@ -196,6 +211,10 @@ impl SessionHarness { g.call = vec![Pattern::prefix("agent."), Pattern::prefix("env.")]; g.publish = vec![Pattern::prefix("session.")]; g.manage_topics = vec![Pattern::prefix("session.")]; + // The read-only repair service of publishing-v1 section 2. It is the + // session's own address and answers two queries; naming it is not + // authority over anything, and no method on it mutates. + g.register = vec![Pattern::prefix("session.")]; }), ) .client( @@ -209,7 +228,34 @@ impl SessionHarness { &format!("{ENV_CLIENT}-r2"), grants(|g| g.register = vec![Pattern::exact(ENV_SERVICE)]), ) - .client("observer", grants(|g| g.subscribe = vec![Pattern::prefix("session.")])); + // A presentation consumer subscribes and may call the repair service. It can + // publish nothing, register nothing and reach no worker: "viewers/browser clients + // never obtain worker control" (publishing-v1 section 7). A bus client id is one + // connection, so a composition with several consumers configures several of them; + // they are the same grants, because a second viewer is not a more privileged one. + .client("observer", consumer_grants()) + .client("observer-2", consumer_grants()) + .client("observer-3", consumer_grants()) + .client("observer-4", consumer_grants()) + // The application's own publisher. Its addresses are its own, and it has no + // reach into the session's. + .client( + "application", + grants(|g| { + g.publish = vec![Pattern::prefix("app.")]; + g.manage_topics = vec![Pattern::prefix("app.")]; + g.subscribe = vec![Pattern::prefix("session.")]; + }), + ) + // The publication boundary, when a composition places it on a client of its own + // rather than on the coordinator's. + .client( + "publisher", + grants(|g| { + g.publish = vec![Pattern::prefix("session.")]; + g.manage_topics = vec![Pattern::prefix("session.")]; + }), + ); for spec in &config.agents { let service = agent_service(&spec.agent_id); policy = policy.client( @@ -276,6 +322,7 @@ impl SessionHarness { tick_duration, warmup_ticks: config.warmup_ticks, worker_threads: spec.worker_threads, + graph_variant: spec.graph_variant, sensors: sensors[&spec.agent_id].clone(), faults: spec.faults.clone(), client_id: agent_client(&spec.agent_id), @@ -326,6 +373,7 @@ impl SessionHarness { sensors, launcher, observers: Mutex::new(Vec::new()), + next_observer: std::sync::atomic::AtomicUsize::new(0), }) } @@ -355,17 +403,65 @@ impl SessionHarness { } /// An extra subscriber, for a test that watches the published boundaries. + /// + /// Each call takes the next configured observer identity: one bus client id is one + /// connection, so two consumers are two configured participants and not one identity + /// used twice. pub async fn observer(&self) -> Result { - let client = self.launcher.connect("observer").await?; + let index = self + .next_observer + .fetch_add(1, std::sync::atomic::Ordering::SeqCst); + let id = match index { + 0 => "observer".to_owned(), + n => format!("observer-{}", n + 1), + }; + let client = self.launcher.connect(&id).await?; self.observers.lock().expect("not poisoned").push(client.clone()); Ok(client) } + /// A client for the application that owns its own state and cues. + pub async fn application(&self) -> Result { + let client = self.launcher.connect("application").await?; + self.observers.lock().expect("not poisoned").push(client.clone()); + Ok(client) + } + + /// A client for a publication boundary of its own. + pub async fn publisher(&self) -> Result { + let client = self.launcher.connect("publisher").await?; + self.observers.lock().expect("not poisoned").push(client.clone()); + Ok(client) + } + + /// A fake multi-agent presentation consumer attached to this session's topics. + pub async fn consumer(&self) -> Result { + let client = self.observer().await?; + crate::publish::PresentationConsumer::attach( + client, + &self.config.session_id, + self.coordinator.topics(), + ) + .await + } + /// Replaces one agent's worker with a fresh incarnation, as a restore would. /// /// The coordinator still pins the old registration, so its next call to that agent fails /// rather than silently reaching another brain. pub async fn restart_agent(&mut self, agent_id: &Id) -> Result { + self.restart_agent_on_graph(agent_id, None).await + } + + /// Replaces one agent's worker, optionally with a fly that built another graph. + /// + /// `Some(variant)` is the composition change a descriptor revision exists for: the same + /// neuron count, another `indexDigest`. + pub async fn restart_agent_on_graph( + &mut self, + agent_id: &Id, + graph_variant: Option, + ) -> Result { let spec = self .config .agents @@ -386,6 +482,7 @@ impl SessionHarness { tick_duration, warmup_ticks: self.config.warmup_ticks, worker_threads: spec.worker_threads, + graph_variant: graph_variant.unwrap_or(spec.graph_variant), // The same log: a replacement worker in this process keeps writing where its // predecessor wrote, so a restore's sensory input is visible beside it. sensors: self.sensors.get(agent_id).cloned().unwrap_or_default(), diff --git a/services/flysim/crates/fly-session/src/launcher.rs b/services/flysim/crates/fly-session/src/launcher.rs index c80551e..0f8469c 100644 --- a/services/flysim/crates/fly-session/src/launcher.rs +++ b/services/flysim/crates/fly-session/src/launcher.rs @@ -231,6 +231,9 @@ pub struct AgentLaunch { /// gets a fresh log in that process, which the supervisor cannot read. pub sensors: crate::media::SensorLog, pub faults: AgentFaults, + /// Which graph this fly builds. Crosses a process boundary as argv, like every other + /// thing a worker is started with. + pub graph_variant: u64, /// The configured client id. A replacement worker connects under its own. pub client_id: String, pub service: String, @@ -1231,6 +1234,7 @@ pub(crate) mod flags { pub const PREPARE_DELAY_MS: &str = "prepare-delay-ms"; pub const COMMIT_DELAY_MS: &str = "commit-delay-ms"; pub const FAIL_COMMIT_AT_STEP: &str = "fail-commit-at-step"; + pub const GRAPH_VARIANT: &str = "graph-variant"; pub const WORKER: &str = "worker"; pub const PORTS: &str = "ports"; @@ -1271,6 +1275,7 @@ pub(crate) mod flags { PREPARE_DELAY_MS, COMMIT_DELAY_MS, FAIL_COMMIT_AT_STEP, + GRAPH_VARIANT, ]; /// What only the environment is given, media options included. pub const ENVIRONMENT_ONLY: &[&str] = &[ @@ -1327,6 +1332,7 @@ impl Started { arg(flags::WARMUP_TICKS, spec.warmup_ticks), arg(flags::PREPARE_DELAY_MS, spec.faults.prepare_delay_ms), arg(flags::COMMIT_DELAY_MS, spec.faults.commit_delay_ms), + arg(flags::GRAPH_VARIANT, spec.graph_variant), ]; if let Some(step) = spec.faults.fail_commit_at_step { args.push(arg(flags::FAIL_COMMIT_AT_STEP, step)); @@ -1377,6 +1383,7 @@ pub(crate) fn agent_config(spec: &AgentLaunch, worker_threads: usize) -> AgentCo tick_duration: spec.tick_duration, warmup_ticks: spec.warmup_ticks, worker_threads, + graph_variant: spec.graph_variant, sensors: spec.sensors.clone(), faults: spec.faults.clone(), } @@ -1486,6 +1493,7 @@ mod flag_tests { tick_duration: RationalNs::new(1, 1_000).expect("a tick"), warmup_ticks: 10, worker_threads: 1, + graph_variant: 3, sensors: crate::media::SensorLog::new(), faults: AgentFaults { fail_commit_at_step: Some(2), diff --git a/services/flysim/crates/fly-session/src/lib.rs b/services/flysim/crates/fly-session/src/lib.rs index 15722b5..312b53d 100644 --- a/services/flysim/crates/fly-session/src/lib.rs +++ b/services/flysim/crates/fly-session/src/lib.rs @@ -33,6 +33,7 @@ pub mod measure; pub mod media; pub mod metrics; pub mod phase; +pub mod publish; pub mod rpc; pub mod task; pub mod worker; diff --git a/services/flysim/crates/fly-session/src/media.rs b/services/flysim/crates/fly-session/src/media.rs index 870ead9..97af979 100644 --- a/services/flysim/crates/fly-session/src/media.rs +++ b/services/flysim/crates/fly-session/src/media.rs @@ -345,6 +345,18 @@ impl AudioSource { } } +/// Allocates, writes and seals one immutable artifact of an arbitrary content type. +/// +/// Used where a test needs a second object with the same bytes, so that "the handle is not +/// the artifact the payload names" can be produced without corrupting the bytes. +pub async fn seal_copy( + client: &flybus::Client, + content_type: String, + bytes: &[u8], +) -> DomainResult { + seal(client, &content_type, bytes).await +} + /// Allocates, writes and seals one immutable artifact. async fn seal( client: &flybus::Client, diff --git a/services/flysim/crates/fly-session/src/publish.rs b/services/flysim/crates/fly-session/src/publish.rs new file mode 100644 index 0000000..485dff7 --- /dev/null +++ b/services/flysim/crates/fly-session/src/publish.rs @@ -0,0 +1,1425 @@ +//! The publication boundary of `publishing-v1`: what leaves the session, and what a refusal is. +//! +//! This is the PUBLISH-01 slice. It is an *internal* boundary on the *same* bus: there is no +//! second transport, no gateway process, no codec and no show or tournament service. What it +//! adds is the separation the contract asks for: +//! +//! ```text +//! session..descriptor retained latest framework: what this composition is +//! session..snapshots retained latest framework: the values of one committed boundary +//! session..events bounded framework: scoped domain events, not a log +//! .state retained latest application-owned, application-shaped +//! .cues bounded application-owned, under a declared policy +//! session..query RPC, read-only the repair path for a missed descriptor +//! ``` +//! +//! Three rules decide everything below. +//! +//! 1. **A publication outcome is named.** [`PublicationOutcome`] is `Accepted`, +//! `RefusedByObserver` or `Faulted`; nothing is dropped, retried or defaulted silently. +//! `bus-v1` section 6 says plainly that "bounded event subscriptions can reject publication; +//! latest spectator subscriptions cannot hold a required session transaction indefinitely", +//! so a refusal is a thing the contract expects and this module counts, not an error to +//! swallow. Only `BACKPRESSURE` is an observer's refusal. A store quota, a lost router or an +//! unreadable payload is the session's own fault and fails the epoch. +//! 2. **An observer never moves the world.** Publication happens after the committed boundary +//! is established. A refusal changes no phase, takes no step and releases no handle, and a +//! latest observation topic supersedes the refused value at the next boundary, so a slow or +//! bounded observer costs its own delivery and one boundary of a stream whose contract is +//! "latest" -- never a tick, never a stall. The repair path recovers the exact value. +//! 3. **Agent state and media are one statement.** [`check_publication`] refuses a snapshot +//! whose media does not belong to the boundary its agent state belongs to, before anything +//! is published, so "future agent state with old media" is a failure rather than a frame. +//! +//! What this module deliberately does **not** contain: the approved public v2 wire schemas and +//! the stage adapters that speak them. `implementation.md` sequences those after this slice and +//! together with each other, and inventing a public byte format here would be exactly the +//! unapproved contract that ordering exists to prevent. + +use std::collections::{BTreeMap, VecDeque}; +use std::sync::{Arc, Mutex}; + +use serde_json::{Map, Value, json}; + +// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the glob +// keeps the contract's own names in sight instead of restating them. +use crate::types::*; + +/// How many events one bounded batch may hold before the oldest are explicitly dropped. +pub const EVENT_BATCH_DEPTH: usize = 64; + +/// The in-flight credits a framework consumer takes on an observation topic. +pub const SPECTATOR_CREDITS: u32 = 2; + +// ---------------------------------------------------------------------------------------------- +// Policy + +/// The delivery a topic is published under. Declared once, never inferred per message. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Delivery { + /// One replaceable value. A consumer that falls behind loses intermediate values and + /// nothing else; that loss is the contract of the topic, not an accident of load. + LatestValue, + /// A bounded batch. Nothing is coalesced: when the batch is full the oldest entries are + /// dropped by an explicit count that travels with the next accepted batch. + BoundedBatch { depth: usize }, +} + +impl Delivery { + /// Whether the router retains the last value for a late subscriber. + /// + /// A latest observation is retained, because a consumer that arrives mid-session must be + /// able to start; a bounded event stream is not, because it is "not a durable log" + /// (publishing-v1 section 2) and a retained tail would look like one. + pub fn retained(self) -> flybus::Retained { + match self { + Delivery::LatestValue => flybus::Retained::Latest, + Delivery::BoundedBatch { .. } => flybus::Retained::None, + } + } + + /// The subscription a consumer of this topic takes. + pub fn subscription(self) -> flybus::SubscriptionConfig { + match self { + Delivery::LatestValue => flybus::SubscriptionConfig::latest() + .in_flight(SPECTATOR_CREDITS) + .replay(true), + Delivery::BoundedBatch { depth } => { + flybus::SubscriptionConfig::bounded().queued(depth as u32) + } + } + } +} + +/// One published address and the delivery it was declared under. +#[derive(Clone, Debug)] +pub struct TopicPolicy { + pub topic: String, + pub delivery: Delivery, +} + +impl TopicPolicy { + pub fn latest(topic: impl Into) -> TopicPolicy { + TopicPolicy { + topic: topic.into(), + delivery: Delivery::LatestValue, + } + } + + pub fn bounded(topic: impl Into, depth: usize) -> TopicPolicy { + TopicPolicy { + topic: topic.into(), + delivery: Delivery::BoundedBatch { depth }, + } + } +} + +// ---------------------------------------------------------------------------------------------- +// Outcomes + +/// What became of one publication. Every path through [`Publisher`] returns one of these. +#[derive(Clone, Debug, PartialEq)] +pub enum PublicationOutcome { + /// The router admitted it. `replaced` counts the queued values it coalesced away, which + /// is the only loss a latest subscriber can suffer and is reported, not hidden. + Accepted { + topic: String, + topic_sequence: u64, + subscribers: u64, + replaced: u64, + }, + /// A subscriber refused it. `bus-v1` section 5 rejects the whole publish for a bounded + /// subscriber's full queue, so this is an observer's doing: the world is untouched, the + /// value is recoverable through the query service, and the offender is named. + RefusedByObserver { topic: String, detail: String }, + /// The session's own resource or identity fault. This one fails the epoch. + Faulted { topic: String, detail: String }, +} + +impl PublicationOutcome { + pub fn topic(&self) -> &str { + match self { + PublicationOutcome::Accepted { topic, .. } + | PublicationOutcome::RefusedByObserver { topic, .. } + | PublicationOutcome::Faulted { topic, .. } => topic, + } + } + + pub fn is_accepted(&self) -> bool { + matches!(self, PublicationOutcome::Accepted { .. }) + } + + pub fn is_refused(&self) -> bool { + matches!(self, PublicationOutcome::RefusedByObserver { .. }) + } + + /// The domain error a *fault* carries, or `None` for an accepted or refused publication. + /// + /// The certainty is `none`: publication happens after the boundary is committed and + /// mutates no participant, so a failed publish has changed nothing in the world. + pub fn fault(&self) -> Option { + match self { + PublicationOutcome::Faulted { topic, detail } => Some(DomainError::new( + ErrorCode::BackendFailure, + format!("publishing {topic}: {detail}"), + MutationCertainty::None, + )), + _ => None, + } + } + + fn from_bus(topic: &str, result: Result) -> Self { + match result { + Ok(receipt) => PublicationOutcome::Accepted { + topic: topic.to_owned(), + topic_sequence: receipt.topic_sequence, + subscribers: receipt.subscribers, + replaced: receipt.replaced, + }, + // The one code an observer can cause. Everything else is ours. + Err(e) if e.code == flybus::ErrorCode::Backpressure => { + PublicationOutcome::RefusedByObserver { + topic: topic.to_owned(), + detail: e.message, + } + } + Err(e) => PublicationOutcome::Faulted { + topic: topic.to_owned(), + detail: format!("{:?}: {}", e.code, e.message), + }, + } + } +} + +/// Per-topic publication counters, for assertions and for an operator. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct TopicCounters { + pub accepted: u64, + pub refused: u64, + pub faulted: u64, + /// Values a latest subscriber's queue coalesced away, as the router reported them. + pub replaced: u64, +} + +/// What this session published and what happened to it. +#[derive(Clone, Debug, Default)] +pub struct Ledger { + topics: BTreeMap, + /// Events dropped from the bounded batch because the batch was full, cumulative. + pub events_dropped: u64, + /// Batches held for a later boundary because an observer refused them. + pub events_held: u64, + last: Option, +} + +impl Ledger { + pub fn counters(&self, topic: &str) -> TopicCounters { + self.topics.get(topic).cloned().unwrap_or_default() + } + + pub fn last(&self) -> Option<&PublicationOutcome> { + self.last.as_ref() + } + + /// Every refusal this session has met, by topic. + pub fn refusals(&self) -> u64 { + self.topics.values().map(|c| c.refused).sum() + } + + fn record(&mut self, outcome: &PublicationOutcome) { + let entry = self.topics.entry(outcome.topic().to_owned()).or_default(); + match outcome { + PublicationOutcome::Accepted { replaced, .. } => { + entry.accepted += 1; + entry.replaced += replaced; + } + PublicationOutcome::RefusedByObserver { .. } => entry.refused += 1, + PublicationOutcome::Faulted { .. } => entry.faulted += 1, + } + self.last = Some(outcome.clone()); + } +} + +// ---------------------------------------------------------------------------------------------- +// Published state and the repair path + +/// Everything the query service can answer from: the descriptor revisions this session has +/// published, and the latest committed snapshot. +/// +/// "Latest retained descriptors accelerate startup; RPC querying remains the repair path" +/// (publishing-v1 section 2). A consumer that meets a revision it does not hold asks here +/// instead of guessing the shape of the values it is reading. +#[derive(Clone, Debug, Default)] +pub struct PublishedState { + descriptors: BTreeMap, + latest: Option, +} + +impl PublishedState { + pub fn descriptor(&self, revision: u64) -> Option<&SessionDescriptor> { + self.descriptors.get(&revision) + } + + pub fn newest_descriptor(&self) -> Option<&SessionDescriptor> { + self.descriptors.values().next_back() + } + + pub fn latest_snapshot(&self) -> Option<&CommittedSnapshot> { + self.latest.as_ref() + } + + pub fn revisions(&self) -> Vec { + self.descriptors.keys().copied().collect() + } +} + +/// The shared handle the publisher writes and the query service reads. +pub type SharedState = Arc>; + +fn lock(state: &SharedState) -> std::sync::MutexGuard<'_, PublishedState> { + state + .lock() + .expect("the published state is never held across a panic") +} + +// ---------------------------------------------------------------------------------------------- +// The bounded event batch + +/// The pending event batch of `publishing-v1` section 6: bounded, explicit about loss. +/// +/// Events are "scoped domain events; not a durable log" (section 2), and "bus publish +/// acceptance and delivery consumption are not durable acknowledgments" (section 6). So this +/// keeps a bounded batch and says exactly what it could not keep: `droppedBefore` travels with +/// the next accepted batch, so a subscriber reads a count rather than inferring a gap. What it +/// never does is grow without bound, retry forever or forget quietly. +#[derive(Clone, Debug, Default)] +pub struct EventOutbox { + depth: usize, + pending: VecDeque<(u64, TaskEvent)>, + dropped_since_accepted: u64, +} + +impl EventOutbox { + pub fn new(depth: usize) -> EventOutbox { + EventOutbox { + depth, + pending: VecDeque::new(), + dropped_since_accepted: 0, + } + } + + /// Adds this boundary's events, dropping the oldest if the batch is over its depth. + /// Returns how many were dropped by this call. + pub fn offer(&mut self, source_step: u64, events: &[TaskEvent]) -> u64 { + for event in events { + self.pending.push_back((source_step, event.clone())); + } + let mut dropped = 0; + while self.pending.len() > self.depth { + self.pending.pop_front(); + dropped += 1; + } + self.dropped_since_accepted += dropped; + dropped + } + + pub fn is_empty(&self) -> bool { + self.pending.is_empty() + } + + pub fn len(&self) -> usize { + self.pending.len() + } + + pub fn dropped_since_accepted(&self) -> u64 { + self.dropped_since_accepted + } + + fn payload(&self, session_id: &Id, epoch: &Id) -> Map { + let events: Vec = self + .pending + .iter() + .map(|(source_step, event)| { + let mut value = event.to_json(); + if let Value::Object(map) = &mut value { + map.insert("sourceStep".to_owned(), source_step.to_string().into()); + } + value + }) + .collect(); + object(json!({ + "sessionId": session_id.as_str(), + "epoch": epoch.as_str(), + "droppedBefore": self.dropped_since_accepted.to_string(), + "events": Value::Array(events), + })) + } + + fn accepted(&mut self) { + self.pending.clear(); + self.dropped_since_accepted = 0; + } +} + +// ---------------------------------------------------------------------------------------------- +// Coherence + +/// Refuses a snapshot whose media does not belong to the boundary its agent state belongs to. +/// +/// This is the "future agent state is never mixed with old media" rule, checked on the way out +/// and again on the way in. Four things must agree: +/// +/// 1. Each published view's `producedStep` is exactly what its declared delay implies for this +/// boundary. A frame from an older boundary is a refusal, not a substitution. +/// 2. Each published audio chunk's sample range ends where the stream's next chunk begins, so +/// a chunk from a previous transition cannot ride along under a new boundary. +/// 3. Every referenced view and chunk has its attachment, and no attachment is present that +/// the payload does not reference. An extra handle is an old boundary's frame. +/// 4. The snapshot agrees with the descriptor it names: revision, agent set, port assignment, +/// rate roles. +pub fn check_publication( + descriptor: &SessionDescriptor, + snapshot: &CommittedSnapshot, + attachments: &[(String, ArtifactRef)], + audio_next_sample: &BTreeMap, +) -> DomainResult<()> { + snapshot + .validate_against(descriptor) + .map_err(|e| coherence(format!("snapshot and descriptor disagree: {e}")))?; + check_views(descriptor, snapshot)?; + check_attachments(snapshot, attachments)?; + for chunk in &snapshot.audio { + let end = chunk + .first_sample + .checked_add(chunk.sample_frames) + .ok_or_else(|| { + coherence(format!( + "audio stream {} overflows its sample position", + chunk.stream_id + )) + })?; + match audio_next_sample.get(&chunk.stream_id) { + Some(next) if *next == end => {} + Some(next) => { + return Err(coherence(format!( + "audio stream {} covers samples {}..{end} and this boundary ends at {next}", + chunk.stream_id, chunk.first_sample + ))); + } + None => { + return Err(coherence(format!( + "audio stream {} has no accepted position at this boundary", + chunk.stream_id + ))); + } + } + } + Ok(()) +} + +/// Every published view comes from exactly the boundary its declared delay implies. +pub fn check_views( + descriptor: &SessionDescriptor, + snapshot: &CommittedSnapshot, +) -> DomainResult<()> { + let boundary = snapshot.scope.step; + for view in &snapshot.views { + let declared = descriptor + .environment + .views + .iter() + .find(|v| v.view_id == view.view_id) + .ok_or_else(|| coherence(format!("view {} is not declared", view.view_id)))?; + let want = declared.required_produced_step(boundary); + if view.produced_step != want { + return Err(coherence(format!( + "view {} at boundary {boundary} was produced at {}, and its declared delay of {} requires {want}", + view.view_id, view.produced_step, declared.observation_delay_steps + ))); + } + } + Ok(()) +} + +/// The handles and the references are the same media: no missing frame, no extra one, and +/// each handle is the artifact its reference names. +/// +/// The identity comparison is the half that matters: an attachment set that matches by *name* +/// while one handle is the previous boundary's object is exactly "old media under new agent +/// state", and only the `ArtifactRef` sees it. +pub fn check_attachments( + snapshot: &CommittedSnapshot, + attachments: &[(String, ArtifactRef)], +) -> DomainResult<()> { + let mut want: Vec<(String, &ArtifactRef)> = snapshot + .views + .iter() + .map(|v| (crate::media::view_attachment(&v.view_id), &v.pixels)) + .chain( + snapshot + .audio + .iter() + .map(|a| (crate::media::audio_attachment(&a.stream_id), &a.samples)), + ) + .collect(); + want.sort_by(|a, b| a.0.cmp(&b.0)); + let mut given: Vec<(String, &ArtifactRef)> = + attachments.iter().map(|(n, r)| (n.clone(), r)).collect(); + given.sort_by(|a, b| a.0.cmp(&b.0)); + if want.len() != given.len() || want.iter().zip(&given).any(|(w, g)| w.0 != g.0) { + let want: Vec<&String> = want.iter().map(|(n, _)| n).collect(); + let given: Vec<&String> = given.iter().map(|(n, _)| n).collect(); + return Err(coherence(format!( + "the published handles {given:?} are not the media the snapshot references {want:?}" + ))); + } + for ((name, reference), (_, handle)) in want.iter().zip(&given) { + if reference != handle { + return Err(coherence(format!( + "the handle published as {name} is artifact {} generation {}, and the snapshot references {} generation {}", + handle.artifact_id, handle.generation, reference.artifact_id, reference.generation + ))); + } + } + Ok(()) +} + +fn coherence(message: impl std::fmt::Display) -> DomainError { + // Nothing was published, so nothing downstream saw a mixed boundary. + DomainError::new(ErrorCode::BufferInvalid, message, MutationCertainty::None) +} + +// ---------------------------------------------------------------------------------------------- +// The publisher + +/// The session's publication path. One bus client, three framework topics, named outcomes. +pub struct Publisher { + bus: flybus::Client, + session_id: Id, + epoch: Id, + descriptor_topic: TopicPolicy, + snapshot_topic: TopicPolicy, + event_topic: TopicPolicy, + outbox: EventOutbox, + state: SharedState, + ledger: Ledger, + sequence: u64, +} + +impl Publisher { + pub fn new( + bus: flybus::Client, + session_id: &Id, + epoch: &Id, + topics: &crate::coordinator::Topics, + ) -> Publisher { + Publisher { + bus, + session_id: session_id.clone(), + epoch: epoch.clone(), + descriptor_topic: TopicPolicy::latest(&topics.descriptor), + snapshot_topic: TopicPolicy::latest(&topics.snapshots), + event_topic: TopicPolicy::bounded(&topics.events, EVENT_BATCH_DEPTH), + outbox: EventOutbox::new(EVENT_BATCH_DEPTH), + state: Arc::new(Mutex::new(PublishedState::default())), + ledger: Ledger::default(), + sequence: 0, + } + } + + /// The declared policies, in publication order. + pub fn policies(&self) -> Vec { + vec![ + self.descriptor_topic.clone(), + self.snapshot_topic.clone(), + self.event_topic.clone(), + ] + } + + pub fn ledger(&self) -> &Ledger { + &self.ledger + } + + pub fn state(&self) -> SharedState { + Arc::clone(&self.state) + } + + pub fn outbox(&self) -> &EventOutbox { + &self.outbox + } + + /// The next publication sequence, which is monotonic within this publisher incarnation. + pub fn sequence(&self) -> u64 { + self.sequence + } + + pub fn incarnation(&self) -> String { + self.bus.info().connection_id.clone() + } + + /// Declares every framework topic under its policy. A topic already declared compatibly + /// is accepted; a conflicting declaration is a fault here, not a silent reuse. + pub async fn declare(&self) -> DomainResult<()> { + for policy in self.policies() { + self.bus + .declare_topic(&policy.topic, policy.delivery.retained()) + .await + .map_err(|e| { + DomainError::new( + ErrorCode::BackendFailure, + format!("declaring {}: {}", policy.topic, e.message), + MutationCertainty::None, + ) + })?; + } + Ok(()) + } + + /// Publishes a descriptor revision and records it as answerable by the query service. + /// + /// The revision is recorded even when the publication is refused: the repair path exists + /// exactly for the consumer that did not receive it. + pub async fn publish_descriptor( + &mut self, + descriptor: &SessionDescriptor, + ) -> DomainResult { + descriptor.validate().map_err(DomainError::invalid)?; + { + let mut state = lock(&self.state); + if let Some(previous) = state.descriptor(descriptor.revision) + && previous != descriptor + { + return Err(DomainError::before( + ErrorCode::IdentityMismatch, + format!( + "descriptor revision {} was already published with another composition", + descriptor.revision + ), + )); + } + state + .descriptors + .insert(descriptor.revision, descriptor.clone()); + } + let topic = self.descriptor_topic.topic.clone(); + let outcome = PublicationOutcome::from_bus( + &topic, + self.bus + .publish(&topic, object(descriptor.to_json()), &[]) + .await, + ); + self.ledger.record(&outcome); + Ok(outcome) + } + + /// Publishes one committed boundary with the media handles it references. + /// + /// The coherence check runs first, so a snapshot that mixes boundaries never reaches a + /// subscriber; the sequence advances only on a publication the router accepted, so + /// "monotonic within publisherIncarnation" counts published values and not attempts. + pub async fn publish_snapshot( + &mut self, + descriptor: &SessionDescriptor, + snapshot: &CommittedSnapshot, + attachments: &[(String, flybus::Artifact)], + audio_next_sample: &BTreeMap, + ) -> DomainResult { + let named: Vec<(String, ArtifactRef)> = attachments + .iter() + .map(|(name, artifact)| (name.clone(), artifact.reference().clone())) + .collect(); + check_publication(descriptor, snapshot, &named, audio_next_sample)?; + let refs: Vec<(&str, &flybus::Artifact)> = + attachments.iter().map(|(n, a)| (n.as_str(), a)).collect(); + let topic = self.snapshot_topic.topic.clone(); + let outcome = PublicationOutcome::from_bus( + &topic, + self.bus + .publish(&topic, object(snapshot.to_json()), &refs) + .await, + ); + if outcome.is_accepted() { + self.sequence += 1; + lock(&self.state).latest = Some(snapshot.clone()); + } + self.ledger.record(&outcome); + Ok(outcome) + } + + /// Offers this boundary's events to the bounded batch and publishes what it holds. + /// + /// An empty batch publishes nothing and is not an outcome. A refused batch is held for the + /// next boundary and counted; what the depth pushed out is counted too and travels with + /// the next accepted batch as `droppedBefore`. + pub async fn publish_events( + &mut self, + source_step: u64, + events: &[TaskEvent], + ) -> Option { + let dropped = self.outbox.offer(source_step, events); + self.ledger.events_dropped += dropped; + if self.outbox.is_empty() { + return None; + } + let payload = self.outbox.payload(&self.session_id, &self.epoch); + let topic = self.event_topic.topic.clone(); + let outcome = + PublicationOutcome::from_bus(&topic, self.bus.publish(&topic, payload, &[]).await); + match &outcome { + PublicationOutcome::Accepted { .. } => self.outbox.accepted(), + PublicationOutcome::RefusedByObserver { .. } => self.ledger.events_held += 1, + PublicationOutcome::Faulted { .. } => {} + } + self.ledger.record(&outcome); + Some(outcome) + } +} + +// ---------------------------------------------------------------------------------------------- +// The query service: the repair path + +/// The read-only service name a session answers descriptor queries on. +pub fn query_service(session_id: &Id) -> String { + format!("session.{session_id}.query") +} + +/// `Session.GetDescriptor`: one published revision, or the newest. +pub const GET_DESCRIPTOR: &str = "Session.GetDescriptor"; +/// `Session.GetSnapshot`: the latest committed snapshot, without its media handles. +pub const GET_SNAPSHOT: &str = "Session.GetSnapshot"; + +/// A running query service. +/// +/// It is the repair path of `publishing-v1` section 2 and nothing else: two read methods over +/// state the publisher already published. It takes no parameters that select a participant, it +/// mutates nothing, and it is not a controller API -- there is no method here that could +/// advance, pause, stimulate, restore or reconfigure anything. +pub struct QueryService { + task: tokio::task::JoinHandle<()>, +} + +impl QueryService { + pub async fn start( + client: flybus::Client, + session_id: &Id, + state: SharedState, + ) -> Result { + let name = query_service(session_id); + let mut service = client + .register(&name, flybus::ServiceConfig::default()) + .await?; + let worker_id = session_id.clone(); + // The bus connection id names this publisher incarnation. It has to be an `Id` to + // travel in a reply, and a connection id that is not one is a refusal here rather + // than a fallback name that two incarnations could share. + let incarnation = parse_id(&format!("query-{}", client.info().connection_id)) + .map_err(|e| flybus::BusError::new(flybus::ErrorCode::InvalidEnvelope, e))?; + let task = tokio::spawn(async move { + while let Some(request) = service.next().await { + let method = request.method().to_owned(); + let responder = request.responder(); + let parsed = + SessionRpcRequest::from_json(&Value::Object(request.payload().clone())); + drop(request); + let outcome = match parsed { + Ok(parsed) => answer(&parsed, &method, &worker_id, &incarnation, &state), + Err(e) => failure_outcome( + &DomainRequestId::from_serial(0), + &worker_id, + &incarnation, + None, + DomainError::invalid(format!("{method}: {e}")), + ), + }; + let _ = responder.reply(outcome.to_outcome(), &[]).await; + } + }); + Ok(QueryService { task }) + } + + /// Ends the service. Dropping one does the same thing. + pub fn stop(self) { + drop(self); + } +} + +impl Drop for QueryService { + /// A dropped session leaves no task reading a service it no longer answers for. + fn drop(&mut self) { + self.task.abort(); + } +} + +fn answer( + request: &SessionRpcRequest, + method: &str, + worker_id: &Id, + incarnation: &Id, + state: &SharedState, +) -> SessionRpcOutcome { + let result = match method { + GET_DESCRIPTOR => get_descriptor(request, state), + GET_SNAPSHOT => get_snapshot(request, state), + other => Err(DomainError::before( + ErrorCode::Unsupported, + format!("{other} is not a method of the session query service"), + )), + }; + match result { + Ok(result) => success_outcome( + &request.request_id, + worker_id, + incarnation, + request.scope.clone(), + result, + ), + Err(error) => failure_outcome( + &request.request_id, + worker_id, + incarnation, + request.scope.clone(), + error, + ), + } +} + +fn get_descriptor( + request: &SessionRpcRequest, + state: &SharedState, +) -> DomainResult> { + let wanted = match request.params.get("revision") { + None | Some(Value::Null) => None, + Some(Value::String(text)) => Some( + text.parse::() + .map_err(|_| DomainError::invalid("revision is a decimal U64 string"))?, + ), + Some(_) => return Err(DomainError::invalid("revision is a decimal U64 string")), + }; + let state = lock(state); + let descriptor = match wanted { + Some(revision) => state.descriptor(revision).ok_or_else(|| { + // A revision this session never published is an answer, not an empty result. + DomainError::before( + ErrorCode::IdentityMismatch, + format!( + "descriptor revision {revision} was never published; this session has {:?}", + state.revisions() + ), + ) + })?, + None => state.newest_descriptor().ok_or_else(|| { + DomainError::before(ErrorCode::InvalidPhase, "no descriptor has been published") + })?, + }; + Ok(object(descriptor.to_json())) +} + +fn get_snapshot( + request: &SessionRpcRequest, + state: &SharedState, +) -> DomainResult> { + if !request.params.as_object().is_some_and(Map::is_empty) { + return Err(DomainError::invalid( + "Session.GetSnapshot takes no parameters", + )); + } + let state = lock(state); + let snapshot = state.latest_snapshot().ok_or_else(|| { + DomainError::before(ErrorCode::InvalidPhase, "no snapshot has been published") + })?; + Ok(object(snapshot.to_json())) +} + +// ---------------------------------------------------------------------------------------------- +// Application-owned state and cues + +/// An application's own publication channel: its schema, its topics, its delivery policy. +/// +/// The framework supplies the channel and nothing about what travels on it. "Application state +/// carries whatever the experience needs ... It is developed with its presentation, not forced +/// into a framework-wide show state/tournament schema" (publishing-v1 section 4). So the topics +/// are named by the application, the values are `TypedValue`s under the application's own +/// namespaced schema, and this module never looks inside one. There is no director here, no +/// bracket, no cast and no game. +pub struct ApplicationChannel { + bus: flybus::Client, + state_topic: TopicPolicy, + cue_topic: TopicPolicy, + ledger: Ledger, + revision: u64, +} + +impl ApplicationChannel { + /// `prefix` is the application's own address root, for example `app.counter`. + pub fn new(bus: flybus::Client, prefix: &str, cue_depth: usize) -> ApplicationChannel { + ApplicationChannel { + bus, + state_topic: TopicPolicy::latest(format!("{prefix}.state")), + cue_topic: TopicPolicy::bounded(format!("{prefix}.cues"), cue_depth), + ledger: Ledger::default(), + revision: 0, + } + } + + pub fn state_topic(&self) -> &str { + &self.state_topic.topic + } + + pub fn cue_topic(&self) -> &str { + &self.cue_topic.topic + } + + pub fn ledger(&self) -> &Ledger { + &self.ledger + } + + pub async fn declare(&self) -> DomainResult<()> { + for policy in [&self.state_topic, &self.cue_topic] { + self.bus + .declare_topic(&policy.topic, policy.delivery.retained()) + .await + .map_err(|e| { + DomainError::new( + ErrorCode::BackendFailure, + format!("declaring {}: {}", policy.topic, e.message), + MutationCertainty::None, + ) + })?; + } + Ok(()) + } + + /// Publishes application state at a committed boundary it names. + /// + /// The boundary travels with it because a consumer combines this with the framework + /// snapshot, and cross-topic ordering is not guaranteed: "a subscriber receiving an unknown + /// descriptor revision must fetch it ... or buffer a bounded number of snapshots, not infer + /// shape" (publishing-v1 section 2). A named boundary is how the two are joined. + pub async fn publish_state(&mut self, boundary: u64, value: &TypedValue) -> PublicationOutcome { + self.revision += 1; + let payload = object(json!({ + "boundary": boundary.to_string(), + "revision": self.revision.to_string(), + "state": value.to_json(), + })); + let topic = self.state_topic.topic.clone(); + let outcome = + PublicationOutcome::from_bus(&topic, self.bus.publish(&topic, payload, &[]).await); + self.ledger.record(&outcome); + outcome + } + + /// Publishes a presentation cue. Cues are presentation data: "presentation cues may be + /// immediate; simulation effects apply at declared boundaries" (section 7). Nothing here + /// reaches a worker, a controller or the world. + pub async fn publish_cue( + &mut self, + boundary: u64, + kind: &Id, + value: &TypedValue, + ) -> PublicationOutcome { + let payload = object(json!({ + "boundary": boundary.to_string(), + "kind": kind.as_str(), + "cue": value.to_json(), + })); + let topic = self.cue_topic.topic.clone(); + let outcome = + PublicationOutcome::from_bus(&topic, self.bus.publish(&topic, payload, &[]).await); + self.ledger.record(&outcome); + outcome + } +} + +// ---------------------------------------------------------------------------------------------- +// The fake multi-agent consumer + +/// One agent's committed values as a consumer reads them. +/// +/// A presentation consumer is a *multi-agent* consumer: one snapshot carries the whole +/// composition, so there is no per-fly stream to join and no "current fly" to be stale. +#[derive(Clone, Debug, PartialEq)] +pub struct AgentView { + pub agent_id: Id, + /// The index the descriptor says this agent's rates and geometry belong to. A consumer + /// that maps anything spatial compares this, not `neuronCount`. + pub index_digest: Digest, + pub brain_ticks: u64, + pub rates: Vec<(Id, f64)>, + /// The decision of the transition that *ended* at this boundary, null only at boundary 0. + pub decision: Option, + pub controls: Option, +} + +/// One committed boundary as a consumer reads it, with the media handles it still owns. +pub struct SnapshotView { + pub boundary: u64, + pub descriptor_revision: u64, + pub sequence: u64, + /// How many undelivered snapshots the router coalesced away before this one. + pub replaced: u64, + pub agents: Vec, + pub views: Vec, + pub audio: Vec, + /// The extracted handles, held after the message is dropped. + pub artifacts: BTreeMap, +} + +impl SnapshotView { + pub fn agent(&self, agent_id: &Id) -> Option<&AgentView> { + self.agents.iter().find(|a| a.agent_id == *agent_id) + } +} + +/// One bounded event batch as a consumer reads it. +#[derive(Clone, Debug, PartialEq)] +pub struct EventBatchView { + pub epoch: Id, + /// How many events the publisher's bounded batch dropped before this one. A count, never + /// a gap the consumer has to infer. + pub dropped_before: u64, + pub event_ids: Vec, +} + +/// What one poll of a consumer produced. +#[derive(Clone, Debug, PartialEq)] +pub enum ConsumerOutcome { + /// A descriptor this consumer now holds, and the agents it describes. + Composition { revision: u64, agents: Vec }, + /// A snapshot whose descriptor revision this consumer holds and which agrees with it. + Read { boundary: u64, agents: Vec }, + /// A snapshot naming a descriptor revision this consumer has never seen. Nothing is + /// inferred from it: the consumer repairs through the query service and reads it again. + UnknownRevision { revision: u64 }, + /// The composition changed under an agent this consumer had already mapped: the same + /// neuron count, another index. Visible, named, and never silently remapped. + IndexChanged { + agent_id: Id, + from: Digest, + to: Digest, + }, + /// The snapshot does not agree with the descriptor it names, or its media does not belong + /// to its boundary. + Incoherent { detail: String }, +} + +/// A presentation-side consumer of one session, over the same bus. +/// +/// It is a regular bus subscriber (publishing-v1 section 5): latest subscriptions with finite +/// credits on the observation topics, a bounded subscription on events, and one read-only RPC +/// for repair. It is deliberately *not* a gateway: it resolves no artifact into a browser +/// transport, encodes nothing and holds no private owner token on anyone's behalf. +pub struct PresentationConsumer { + bus: flybus::Client, + query: String, + descriptors: flybus::Subscription, + snapshots: flybus::Subscription, + events: flybus::Subscription, + held: Vec, + cached: BTreeMap, + /// The index digest this consumer has mapped geometry against, per agent. + mapped: BTreeMap, + /// Where each audio stream's last accepted chunk ended. + audio_end: BTreeMap, + last: Option, + repairs: u64, + serial: u64, + coalesced: u64, +} + +impl PresentationConsumer { + pub async fn attach( + bus: flybus::Client, + session_id: &Id, + topics: &crate::coordinator::Topics, + ) -> Result { + let descriptors = bus + .subscribe(&topics.descriptor, Delivery::LatestValue.subscription()) + .await?; + let snapshots = bus + .subscribe(&topics.snapshots, Delivery::LatestValue.subscription()) + .await?; + let events = bus + .subscribe( + &topics.events, + Delivery::BoundedBatch { + depth: EVENT_BATCH_DEPTH, + } + .subscription(), + ) + .await?; + Ok(PresentationConsumer { + bus, + query: query_service(session_id), + descriptors, + snapshots, + events, + held: Vec::new(), + cached: BTreeMap::new(), + mapped: BTreeMap::new(), + audio_end: BTreeMap::new(), + last: None, + repairs: 0, + serial: 0, + coalesced: 0, + }) + } + + /// How many descriptor revisions this consumer holds. + pub fn revisions(&self) -> Vec { + self.cached.keys().copied().collect() + } + + pub fn descriptor(&self, revision: u64) -> Option<&SessionDescriptor> { + self.cached.get(&revision) + } + + /// How many times this consumer had to ask the query service for a descriptor. + pub fn repairs(&self) -> u64 { + self.repairs + } + + /// How many snapshots the router coalesced away in this consumer's own queue. + pub fn coalesced(&self) -> u64 { + self.coalesced + } + + pub fn last(&self) -> Option<&SnapshotView> { + self.last.as_ref() + } + + /// Takes the next descriptor from the retained-latest topic. + /// + /// A composition that moves an agent this consumer has already mapped geometry against is + /// reported here and *not* cached: remapping silently is the one thing a consumer holding + /// a spatial mapping must not do. (publishing-v1 section 3) + pub async fn take_descriptor(&mut self) -> Option { + let message = self.descriptors.next().await?; + let descriptor = + match SessionDescriptor::from_json(&Value::Object(message.payload().clone())) { + Ok(descriptor) => descriptor, + Err(e) => { + return Some(ConsumerOutcome::Incoherent { + detail: e.to_string(), + }); + } + }; + drop(message); + for agent in &descriptor.agents { + if let Some(mapped) = self.mapped.get(&agent.agent_id) + && *mapped != agent.index_digest + { + return Some(ConsumerOutcome::IndexChanged { + agent_id: agent.agent_id.clone(), + from: mapped.clone(), + to: agent.index_digest.clone(), + }); + } + } + let revision = descriptor.revision; + let agents = descriptor + .agents + .iter() + .map(|a| a.agent_id.clone()) + .collect(); + self.cached.insert(revision, descriptor); + Some(ConsumerOutcome::Composition { revision, agents }) + } + + /// Asks the query service for a revision this consumer does not hold. + /// + /// This is the repair path, and it is an ordinary RPC on the same bus. A revision the + /// session never published comes back as a named error rather than an empty answer. + pub async fn repair(&mut self, revision: u64) -> DomainResult { + self.serial += 1; + self.repairs += 1; + let request = SessionRpcRequest { + request_id: DomainRequestId::from_serial(self.serial), + scope: None, + params: json!({ "revision": revision.to_string() }), + }; + let mut pending = self + .bus + .call( + &self.query, + None, + GET_DESCRIPTOR, + object(request.to_json()), + &[], + ) + .await + .map_err(|e| { + DomainError::new( + ErrorCode::BackendFailure, + e.message, + MutationCertainty::None, + ) + })?; + let result = pending.result().await.map_err(|e| { + DomainError::new( + ErrorCode::BackendFailure, + e.message, + MutationCertainty::None, + ) + })?; + let outcome = SessionRpcOutcome::from_json(&Value::Object(result.outcome().clone())) + .map_err(DomainError::invalid)?; + drop(result); + let value = outcome_result(&outcome)?; + let descriptor = SessionDescriptor::from_json(value).map_err(DomainError::invalid)?; + if descriptor.revision != revision { + return Err(DomainError::before( + ErrorCode::IdentityMismatch, + "the query service answered with another revision", + )); + } + self.cached.insert(revision, descriptor); + Ok(revision) + } + + /// Takes the next committed snapshot and reads it against the descriptor it names. + pub async fn take_snapshot(&mut self) -> Option { + let message = self.snapshots.next().await?; + self.coalesced += message.replaced(); + Some(self.read(message)) + } + + /// Takes a snapshot without reading or releasing it, which is what a viewer that stopped + /// rendering does. Its credits run out; nothing else in the session notices. + pub async fn hold_snapshot(&mut self) -> bool { + match self.snapshots.next().await { + Some(message) => { + self.coalesced += message.replaced(); + self.held.push(message); + true + } + None => false, + } + } + + /// Holds a snapshot if one is queued right now, and answers `false` if none is. + /// + /// The bounded form: a test that means "take whatever is there" must not be able to + /// block on a stream that is deliberately not producing. + pub fn try_hold_snapshot(&mut self) -> bool { + match self.snapshots.try_next() { + Some(message) => { + self.coalesced += message.replaced(); + self.held.push(message); + true + } + None => false, + } + } + + /// Reads a snapshot if one is queued right now. + pub fn try_take_snapshot(&mut self) -> Option { + let message = self.snapshots.try_next()?; + self.coalesced += message.replaced(); + Some(self.read(message)) + } + + /// Releases everything this consumer was holding, returning its credits. + pub fn release(&mut self) { + self.held.clear(); + } + + pub fn held(&self) -> usize { + self.held.len() + } + + /// Takes the next bounded event batch. + pub async fn take_events(&mut self) -> Option { + let message = self.events.next().await?; + let payload = message.payload().clone(); + drop(message); + let epoch = payload.get("epoch").and_then(Value::as_str).map(id)?; + let dropped_before = payload + .get("droppedBefore") + .and_then(Value::as_str) + .and_then(|t| t.parse::().ok())?; + let event_ids = payload + .get("events") + .and_then(Value::as_array)? + .iter() + .filter_map(|e| e.get("id").and_then(Value::as_str).map(str::to_owned)) + .collect(); + Some(EventBatchView { + epoch, + dropped_before, + event_ids, + }) + } + + /// Records that this consumer has mapped geometry against the agents of `revision`. + /// + /// A consumer that never calls this never claims a mapping, and a later index change is + /// simply a new descriptor. One that does claim it gets [`ConsumerOutcome::IndexChanged`] + /// when the composition moves under it. + pub fn map_geometry(&mut self, revision: u64) -> DomainResult<()> { + let descriptor = self.cached.get(&revision).ok_or_else(|| { + DomainError::before( + ErrorCode::IdentityMismatch, + format!("revision {revision} is not held by this consumer"), + ) + })?; + for agent in &descriptor.agents { + self.mapped + .insert(agent.agent_id.clone(), agent.index_digest.clone()); + } + Ok(()) + } + + pub fn mapped_index(&self, agent_id: &Id) -> Option<&Digest> { + self.mapped.get(agent_id) + } + + fn read(&mut self, message: flybus::Message) -> ConsumerOutcome { + let sequence = message.topic_sequence(); + let replaced = message.replaced(); + let snapshot = match CommittedSnapshot::from_json(&Value::Object(message.payload().clone())) + { + Ok(snapshot) => snapshot, + Err(e) => { + return ConsumerOutcome::Incoherent { + detail: e.to_string(), + }; + } + }; + let Some(descriptor) = self.cached.get(&snapshot.descriptor_revision).cloned() else { + // Not an error and not a guess: the consumer buffers nothing and infers nothing, + // it repairs. (publishing-v1 section 2) + return ConsumerOutcome::UnknownRevision { + revision: snapshot.descriptor_revision, + }; + }; + for agent in &descriptor.agents { + if let Some(mapped) = self.mapped.get(&agent.agent_id) + && *mapped != agent.index_digest + { + return ConsumerOutcome::IndexChanged { + agent_id: agent.agent_id.clone(), + from: mapped.clone(), + to: agent.index_digest.clone(), + }; + } + } + let names: Vec = message.attachment_names().map(str::to_owned).collect(); + let mut handles = Vec::new(); + for name in &names { + match message.artifact(name) { + Ok(artifact) => handles.push((name.clone(), artifact.reference().clone())), + Err(e) => { + return ConsumerOutcome::Incoherent { + detail: format!("attachment {name}: {}", e.message), + }; + } + } + } + if let Err(e) = check_received(&descriptor, &snapshot, &self.audio_end, &handles) { + return ConsumerOutcome::Incoherent { + detail: e.to_string(), + }; + } + // The renderer keeps its handles after the delivery is gone; the bytes stay alive + // because the extracted handle owns them. (publishing-v1 section 5) + let mut artifacts = BTreeMap::new(); + for name in &names { + match message.artifact(name) { + Ok(artifact) => { + artifacts.insert(name.clone(), artifact); + } + Err(e) => { + return ConsumerOutcome::Incoherent { + detail: format!("attachment {name}: {}", e.message), + }; + } + } + } + drop(message); + for chunk in &snapshot.audio { + self.audio_end.insert( + chunk.stream_id.clone(), + chunk.first_sample + chunk.sample_frames, + ); + } + let agents: Vec = snapshot + .agents + .iter() + .map(|agent| { + let declared = descriptor + .agents + .iter() + .find(|a| a.agent_id == agent.agent_id) + .expect("validate_against checked the agent set"); + AgentView { + agent_id: agent.agent_id.clone(), + index_digest: declared.index_digest.clone(), + brain_ticks: agent.telemetry.brain_ticks, + rates: agent + .telemetry + .rates + .iter() + .map(|r| (r.role_id.clone(), r.hz)) + .collect(), + decision: agent.selected_decision.clone(), + controls: agent.applied_controls.clone(), + } + }) + .collect(); + let outcome = ConsumerOutcome::Read { + boundary: snapshot.scope.step, + agents: agents.iter().map(|a| a.agent_id.clone()).collect(), + }; + self.last = Some(SnapshotView { + boundary: snapshot.scope.step, + descriptor_revision: snapshot.descriptor_revision, + sequence, + replaced, + agents, + views: snapshot.views.clone(), + audio: snapshot.audio.clone(), + artifacts, + }); + outcome + } +} + +/// The receiving half of the coherence rule. +/// +/// A consumer does not trust that the publisher checked: it checks the same statement from the +/// other side, against the audio position it last accepted rather than the one the publisher +/// holds. That is what catches a chunk from an earlier transition arriving under a later +/// boundary, which is the one shape of "old media" a publisher-side check cannot see. +pub fn check_received( + descriptor: &SessionDescriptor, + snapshot: &CommittedSnapshot, + audio_end: &BTreeMap, + attachments: &[(String, ArtifactRef)], +) -> DomainResult<()> { + snapshot + .validate_against(descriptor) + .map_err(|e| coherence(format!("snapshot and descriptor disagree: {e}")))?; + check_views(descriptor, snapshot)?; + check_attachments(snapshot, attachments)?; + for chunk in &snapshot.audio { + match audio_end.get(&chunk.stream_id) { + // A stream this consumer has not heard yet, or one that says it skipped: both are + // declared states, not assumptions. + None => {} + Some(_) if chunk.discontinuity => {} + // Forward is the latest subscription's own contract: a consumer that fell behind + // was told so by `replaced`, and the boundaries in between are values it chose + // not to receive. Backwards or overlapping is old media under a new boundary, + // which no delivery policy explains. + Some(end) if chunk.first_sample >= *end => {} + Some(end) => { + return Err(coherence(format!( + "audio stream {} starts at {} and the last chunk this consumer read ended at {end}", + chunk.stream_id, chunk.first_sample + ))); + } + } + } + Ok(()) +} diff --git a/services/flysim/crates/fly-session/src/types.rs b/services/flysim/crates/fly-session/src/types.rs index c52c276..4d1eec0 100644 --- a/services/flysim/crates/fly-session/src/types.rs +++ b/services/flysim/crates/fly-session/src/types.rs @@ -27,9 +27,12 @@ pub use fly_session_types::schema::contract_digest; pub use fly_session_types::trace::{ TraceAgent, TraceBehaviour, TraceObservation, TraceOperational, TraceRequest, TransitionTrace, }; +pub use fly_session_types::publishing::{ + AgentDescriptor, CommittedSnapshot, SessionDescriptor, SnapshotAgent, +}; pub use fly_session_types::workers::{ AcknowledgeParams, AcknowledgeResult, AdvanceParams, AgentCommitResult, AgentInitializeParams, - AgentInitializeResult, AgentTelemetry, AssetRef, AxisRange, AxisSchema, AxisValue, ButtonState, + AgentGraph, AgentInitializeResult, AgentTelemetry, AssetRef, AxisRange, AxisSchema, AxisValue, ButtonState, CommitParams, ControllerSchema, Determinism, EnvironmentDescriptor, EnvironmentInitializeParams, EnvironmentInitializeResult, EpisodeRequest, HelloParams, HelloResult, LearningTelemetry, MAX_ACKNOWLEDGE, MAX_AGENTS, MAX_PORTS, MAX_RATE_ROLES, diff --git a/services/flysim/crates/fly-session/tests/publishing.rs b/services/flysim/crates/fly-session/tests/publishing.rs new file mode 100644 index 0000000..52ca5a9 --- /dev/null +++ b/services/flysim/crates/fly-session/tests/publishing.rs @@ -0,0 +1,1099 @@ +//! PUBLISH-01 acceptance: committed snapshots, observer isolation and the repair path. +//! +//! Every test here is one of the slice's acceptance bullets, or one sentence of +//! `publishing-v1` the session now enforces. The boundary itself is `fly_session::publish`: +//! the same bus, named delivery policies, named outcomes, and a fake multi-agent consumer that +//! is an ordinary subscriber. No public v2 wire schema is exercised, because none is approved. + +mod common; + +use std::collections::BTreeMap; +use std::time::Duration; + +use common::{Fixture, default_fixture, fixture, fly_a, fly_b, mode_fixture, within}; +use fly_session::coordinator::{DESCRIPTOR_REVISION, Injections}; +use fly_session::harness::{AgentSpec, ExecutionMode, HarnessConfig, Via}; +use fly_session::publish::{ + ApplicationChannel, ConsumerOutcome, Delivery, EVENT_BATCH_DEPTH, EventOutbox, GET_SNAPSHOT, + PresentationConsumer, PublicationOutcome, TopicPolicy, check_publication, query_service, +}; +use fly_session::types::*; +use serde_json::json; + +both_transports!( + a_consumer_that_disconnects_neither_advances_nor_stalls_the_world, + a_consumer_that_stops_consuming_costs_only_its_own_credits, + a_bounded_observer_refusal_is_named_and_never_fails_the_epoch, + every_published_snapshot_carries_its_own_boundarys_media, + a_published_frame_from_another_boundary_is_refused, + a_published_handle_that_is_not_the_referenced_artifact_is_refused, + an_unheld_descriptor_revision_is_repaired_rather_than_inferred, + a_revision_that_was_never_published_is_a_named_answer, + a_changed_index_digest_is_named_rather_than_remapped, + boundary_zero_publishes_no_decision_and_no_controls, + a_committed_action_is_the_transition_that_just_ended, + one_snapshot_carries_every_agent_in_the_composition, + application_state_and_cues_are_the_applications_own, + a_refused_event_batch_is_held_and_counted_not_lost, + the_query_service_answers_reads_and_nothing_else, + the_published_descriptor_is_what_the_workers_attested_to, +); + +all_modes!( + the_publication_boundary_holds_in_every_execution_mode, + a_stalled_observer_never_moves_the_world_in_any_execution_mode, +); + +const STEPS: u64 = 4; + +/// Polls until `ok` holds, so a test never asserts something that has not happened yet. +async fn until(what: &str, mut ok: impl FnMut() -> bool) { + let deadline = std::time::Instant::now() + Duration::from_secs(10); + while !ok() { + assert!( + std::time::Instant::now() < deadline, + "{what}: never happened" + ); + tokio::time::sleep(Duration::from_millis(5)).await; + } +} + +/// A started session at Ready(0), with boundary 0 already published. +async fn started(via: Via) -> Fixture { + let mut f = default_fixture(via).await; + f.harness + .coordinator + .bootstrap() + .await + .expect("the session bootstraps"); + f +} + +async fn started_in_mode(mode: ExecutionMode) -> Fixture { + let mut f = mode_fixture(mode, HarnessConfig::default()).await; + f.harness + .coordinator + .bootstrap() + .await + .expect("the session bootstraps"); + f +} + +/// Reads snapshots until the consumer reports the boundary it is waiting for. +async fn read_until(consumer: &mut PresentationConsumer, boundary: u64) -> u64 { + loop { + match within("a snapshot", consumer.take_snapshot()).await { + Some(ConsumerOutcome::Read { boundary: at, .. }) if at >= boundary => return at, + Some(ConsumerOutcome::Read { .. }) => {} + other => panic!("expected a readable snapshot, got {other:?}"), + } + } +} + +// ------------------------------------------------------------------------------------------ +// "A browser consumer disconnecting, or applying backpressure, never advances the world and +// never stalls it." + +/// A consumer that goes away mid-run. The world takes exactly the steps it was asked for. +async fn a_consumer_that_disconnects_neither_advances_nor_stalls_the_world(via: Via) { + let mut f = started(via).await; + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + assert!(matches!( + within("the descriptor", consumer.take_descriptor()).await, + Some(ConsumerOutcome::Composition { revision, .. }) if revision == DESCRIPTOR_REVISION + )); + f.harness.coordinator.run(1).await.expect("one transition"); + read_until(&mut consumer, 1).await; + + // The browser goes away. Nothing in the session is told, and nothing waits for it. + drop(consumer); + + let started_at = std::time::Instant::now(); + let before = f.harness.coordinator.stats(); + f.harness + .coordinator + .run(STEPS) + .await + .expect("the world carries on"); + let after = f.harness.coordinator.stats(); + assert_eq!( + after.advances - before.advances, + STEPS, + "a departed observer neither added a world step nor removed one" + ); + assert_eq!(after.publications - before.publications, STEPS); + assert!( + !f.harness.coordinator.is_fenced(), + "a departed observer never fences an epoch" + ); + assert!( + started_at.elapsed() < Duration::from_secs(10), + "a departed observer never stalls the world" + ); + assert_eq!(f.harness.coordinator.ledger().refusals(), 0); + f.shutdown().await; +} + +/// A viewer that holds its deliveries and stops rendering. Its own credits run out; the +/// world's boundaries do not. +async fn a_consumer_that_stops_consuming_costs_only_its_own_credits(via: Via) { + let mut f = started(via).await; + let mut slow = f + .harness + .consumer() + .await + .expect("a slow consumer attaches"); + let mut healthy = f + .harness + .consumer() + .await + .expect("a healthy consumer attaches"); + + // The slow one takes its deliveries and never releases them: a viewer holding output. + until("the slow viewer has a delivery", || { + slow.try_hold_snapshot() + }) + .await; + while slow.try_hold_snapshot() && slow.held() < 4 {} + let held = slow.held(); + assert!(held > 0); + + f.harness + .coordinator + .run(STEPS) + .await + .expect("the world carries on"); + assert_eq!(f.harness.coordinator.stats().advances, STEPS); + assert_eq!( + f.harness.coordinator.ledger().refusals(), + 0, + "a latest subscriber cannot refuse" + ); + assert_eq!( + slow.held(), + held, + "the slow viewer took nothing more once its credits were gone" + ); + + // The healthy one is unaffected and reads the newest boundary. + assert!(matches!( + within("the descriptor", healthy.take_descriptor()).await, + Some(ConsumerOutcome::Composition { .. }) + )); + let at = read_until(&mut healthy, STEPS).await; + assert_eq!(at, STEPS); + assert_eq!( + healthy.last().expect("a snapshot").boundary, + STEPS, + "the healthy consumer reads the newest boundary while the slow one holds" + ); + f.shutdown().await; +} + +/// A bounded subscriber whose queue is full refuses a publication -- `bus-v1` section 6 says +/// it may. The refusal is a named outcome on a named topic, the world takes every step it was +/// asked for, the epoch is not fenced, and when the offender goes away the next boundary +/// reaches the well-behaved consumer. +async fn a_bounded_observer_refusal_is_named_and_never_fails_the_epoch(via: Via) { + let mut f = started(via).await; + let healthy_first = f + .harness + .consumer() + .await + .expect("a healthy consumer attaches"); + let offender_client = f.harness.observer().await.expect("an observer client"); + let snapshots = f.harness.coordinator.topics().snapshots.clone(); + let mut offender = offender_client + .subscribe( + &snapshots, + flybus::SubscriptionConfig::bounded().queued(1).in_flight(1), + ) + .await + .expect("a bounded subscription"); + + let before = f.harness.coordinator.stats().advances; + f.harness + .coordinator + .run(STEPS) + .await + .expect("an observer never fails the epoch"); + assert_eq!( + f.harness.coordinator.stats().advances - before, + STEPS, + "a refused publication never costs the world a step" + ); + assert!(!f.harness.coordinator.is_fenced()); + let counters = f.harness.coordinator.ledger().counters(&snapshots); + assert!( + counters.refused > 0, + "the refusal is counted on its own topic: {counters:?}" + ); + assert!( + matches!( + f.harness.coordinator.ledger().last(), + Some(PublicationOutcome::RefusedByObserver { .. }) + ), + "the last outcome names the refusal rather than defaulting to success" + ); + assert_eq!( + counters.faulted, 0, + "an observer's refusal is never the session's fault" + ); + + // The offender leaves. The next boundary is published and the healthy consumer reads it. + drop(offender.try_next()); + drop(offender); + offender_client.close().await; + drop(healthy_first); + let mut healthy = f + .harness + .consumer() + .await + .expect("a healthy consumer attaches"); + until("the refusals stop", || true).await; + f.harness + .coordinator + .run(2) + .await + .expect("the world carries on"); + assert!(matches!( + within("the descriptor", healthy.take_descriptor()).await, + Some(ConsumerOutcome::Composition { .. }) + )); + let at = read_until(&mut healthy, STEPS + 1).await; + assert!( + at > STEPS, + "a boundary published after the offender left reaches a consumer" + ); + f.shutdown().await; +} + +// ------------------------------------------------------------------------------------------ +// "Future agent state is never mixed with old media." + +/// Every snapshot's media belongs to the boundary its agent state belongs to: the frame is the +/// one this boundary's declared delay requires, the handle is the artifact the payload names, +/// the audio continues where the last chunk ended, and the agents are committed at it. +async fn every_published_snapshot_carries_its_own_boundarys_media(via: Via) { + let mut f = started(via).await; + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + assert!(matches!( + within("the descriptor", consumer.take_descriptor()).await, + Some(ConsumerOutcome::Composition { .. }) + )); + let mut seen: Vec<(u64, String)> = Vec::new(); + for step in 1..=STEPS { + f.harness.coordinator.run(1).await.expect("a transition"); + let at = read_until(&mut consumer, step).await; + let view = consumer.last().expect("a snapshot"); + assert_eq!(view.boundary, at); + for reference in &view.views { + // The counter arena declares no render delay, so the frame of boundary k is + // produced at k. The consumer checked this itself before it got here. + assert_eq!(reference.produced_step, at, "the frame is this boundary's"); + let handle = view + .artifacts + .get(&fly_session::media::view_attachment(&reference.view_id)) + .expect("the referenced frame has its handle"); + assert_eq!( + handle.reference(), + &reference.pixels, + "the handle is that artifact" + ); + seen.push((at, reference.pixels.artifact_id.clone())); + } + for agent in &view.agents { + assert!( + agent.brain_ticks > 0, + "the agent state is this boundary's, not a placeholder" + ); + } + } + let mut ids: Vec<&String> = seen.iter().map(|(_, id)| id).collect(); + ids.sort(); + ids.dedup(); + assert_eq!( + ids.len(), + seen.len(), + "no boundary republished another boundary's frame: {seen:?}" + ); + f.shutdown().await; +} + +/// The injection of new agent state with the previous boundary's frame. The publication is +/// refused before it reaches a subscriber, and the consumer never sees a mixed boundary. +async fn a_published_frame_from_another_boundary_is_refused(via: Via) { + let mut f = started(via).await; + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + f.harness + .coordinator + .run(1) + .await + .expect("one clean transition"); + f.harness.coordinator.injections = Injections { + at_step: 1, + stale_published_view: true, + ..Injections::default() + }; + let failure = f + .harness + .coordinator + .run(1) + .await + .expect_err("a snapshot that mixes boundaries is refused"); + assert_eq!(failure.error.code, ErrorCode::BufferInvalid, "{failure:?}"); + assert!( + failure.error.message.contains("produced at"), + "the refusal names the boundary the frame came from: {}", + failure.error.message + ); + assert_eq!( + failure.error.mutation, + MutationCertainty::None, + "nothing was published, so nothing downstream saw it" + ); + + // Whatever the consumer read, none of it is the mixed boundary. + assert!(matches!( + within("the descriptor", consumer.take_descriptor()).await, + Some(ConsumerOutcome::Composition { .. }) + )); + until("a snapshot of the clean boundary", || { + matches!( + consumer.try_take_snapshot(), + Some(ConsumerOutcome::Read { boundary: 1, .. }) + ) + }) + .await; + assert_eq!(consumer.last().expect("a snapshot").boundary, 1); + while let Some(outcome) = consumer.try_take_snapshot() { + match outcome { + ConsumerOutcome::Read { boundary, .. } => { + assert!(boundary <= 1, "the mixed boundary was never published"); + } + other => panic!("the consumer saw {other:?}"), + } + } + f.shutdown().await; +} + +/// The same attachment names and the same bytes, another object. Only the artifact identity +/// sees it, and the publication is refused. +async fn a_published_handle_that_is_not_the_referenced_artifact_is_refused(via: Via) { + let mut f = started(via).await; + f.harness.coordinator.injections = Injections { + at_step: 0, + substituted_published_handle: true, + ..Injections::default() + }; + let failure = f + .harness + .coordinator + .run(1) + .await + .expect_err("a handle that is not the referenced artifact is refused"); + assert_eq!(failure.error.code, ErrorCode::BufferInvalid, "{failure:?}"); + assert!( + failure.error.message.contains("generation"), + "the refusal names the object it got: {}", + failure.error.message + ); + f.shutdown().await; +} + +// ------------------------------------------------------------------------------------------ +// "A descriptor or index mismatch is visible rather than silently tolerated." + +/// A consumer that meets a revision it does not hold repairs through the query service. It +/// infers nothing from the snapshot in the meantime. +async fn an_unheld_descriptor_revision_is_repaired_rather_than_inferred(via: Via) { + let mut f = started(via).await; + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + f.harness.coordinator.run(1).await.expect("a transition"); + + // The snapshot arrives first: cross-topic ordering is not guaranteed, and this consumer + // has deliberately not read its descriptor topic. + let outcome = within("a snapshot", consumer.take_snapshot()).await; + assert_eq!( + outcome, + Some(ConsumerOutcome::UnknownRevision { + revision: DESCRIPTOR_REVISION + }), + "an unknown revision is reported, not guessed" + ); + assert!( + consumer.last().is_none(), + "nothing was read out of a snapshot it cannot shape" + ); + + let repaired = consumer + .repair(DESCRIPTOR_REVISION) + .await + .expect("the repair path answers"); + assert_eq!(repaired, DESCRIPTOR_REVISION); + assert_eq!(consumer.repairs(), 1); + f.harness.coordinator.run(1).await.expect("a transition"); + let at = read_until(&mut consumer, 1).await; + assert!(at >= 1, "with the descriptor in hand the same stream reads"); + f.shutdown().await; +} + +/// A revision this session never published is an answer, not an empty result. +async fn a_revision_that_was_never_published_is_a_named_answer(via: Via) { + let f = started(via).await; + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + let error = consumer + .repair(99) + .await + .expect_err("revision 99 was never published"); + assert_eq!(error.code, ErrorCode::IdentityMismatch, "{error:?}"); + assert!(error.message.contains("99"), "{}", error.message); + assert_eq!( + error.mutation, + MutationCertainty::None, + "a read never mutates" + ); + f.shutdown().await; +} + +/// The same neuron count, another index. A consumer that has mapped geometry is told, and the +/// new composition is not quietly cached over the one it mapped. +async fn a_changed_index_digest_is_named_rather_than_remapped(via: Via) { + // Two real compositions that differ only in the graph one fly built. + let first = started(via).await; + let mut second = fixture( + via, + HarnessConfig { + agents: vec![ + AgentSpec { + graph_variant: 1, + ..AgentSpec::new("fly-a", "p1", 7) + }, + AgentSpec { + graph_variant: 0, + ..AgentSpec::new("fly-b", "p2", 11) + }, + ], + ..HarnessConfig::default() + }, + ) + .await; + second + .harness + .coordinator + .bootstrap() + .await + .expect("the second session bootstraps"); + let revised = { + let mut revised = second + .harness + .coordinator + .session_descriptor() + .expect("a published descriptor") + .clone(); + revised.revision = DESCRIPTOR_REVISION + 1; + revised + }; + let original = first + .harness + .coordinator + .session_descriptor() + .expect("a published descriptor") + .clone(); + let moved = original + .agents + .iter() + .find(|a| a.agent_id == fly_a()) + .expect("fly-a"); + let arrived = revised + .agents + .iter() + .find(|a| a.agent_id == fly_a()) + .expect("fly-a"); + assert_eq!( + moved.neuron_count, arrived.neuron_count, + "the same number of neurons" + ); + assert_ne!( + moved.index_digest, arrived.index_digest, + "and another index" + ); + + let mut consumer = first.harness.consumer().await.expect("a consumer attaches"); + let held = match within("the descriptor", consumer.take_descriptor()).await { + Some(ConsumerOutcome::Composition { revision, .. }) => revision, + other => panic!("expected a composition, got {other:?}"), + }; + consumer + .map_geometry(held) + .expect("this consumer maps geometry"); + assert_eq!(consumer.mapped_index(&fly_a()), Some(&moved.index_digest)); + + // The composition changes under it. + let publisher_client = first + .harness + .publisher() + .await + .expect("a publishing client"); + let mut publisher = fly_session::publish::Publisher::new( + publisher_client, + &first.harness.config.session_id, + &first.harness.config.epoch, + first.harness.coordinator.topics(), + ); + publisher + .publish_descriptor(&revised) + .await + .expect("the revision publishes"); + let outcome = within("the revised descriptor", consumer.take_descriptor()).await; + assert_eq!( + outcome, + Some(ConsumerOutcome::IndexChanged { + agent_id: fly_a(), + from: moved.index_digest.clone(), + to: arrived.index_digest.clone(), + }), + "the change is named" + ); + assert_eq!( + consumer.mapped_index(&fly_a()), + Some(&moved.index_digest), + "and nothing was remapped behind the consumer's back" + ); + assert!( + consumer.descriptor(DESCRIPTOR_REVISION + 1).is_none(), + "the revision it refused is not in its cache either" + ); + second.shutdown().await; + first.shutdown().await; +} + +/// The descriptor says what the workers said, not what the composition asked for. +async fn the_published_descriptor_is_what_the_workers_attested_to(via: Via) { + let f = started(via).await; + let descriptor = f + .harness + .coordinator + .session_descriptor() + .expect("a descriptor") + .clone(); + assert_eq!(descriptor.revision, DESCRIPTOR_REVISION); + assert_eq!(descriptor.agents.len(), 2); + for agent in &descriptor.agents { + let expected = fly_session::agent::synthetic_graph(&agent.agent_id, 0); + assert_eq!( + agent.index_digest, expected.index_digest, + "the fly's own index" + ); + assert_eq!(agent.dataset_digest, expected.dataset_digest); + assert_eq!(agent.neuron_count, expected.neuron_count); + assert_eq!( + agent.rate_roles, expected.rate_roles, + "the profile's rate order" + ); + assert_eq!(agent.supported_stimuli, expected.supported_stimuli); + assert!( + descriptor.environment.port(&agent.port_id).is_some(), + "every agent's port is declared by the environment" + ); + } + let mut asset_ids: Vec<&Id> = descriptor.assets.iter().map(|a| &a.id).collect(); + asset_ids.sort(); + asset_ids.dedup(); + assert_eq!( + asset_ids.len(), + descriptor.assets.len(), + "assets are unique by id" + ); + f.shutdown().await; +} + +// ------------------------------------------------------------------------------------------ +// "A committed action is labelled as the transition that just ended, not the one about to +// start." + +async fn boundary_zero_publishes_no_decision_and_no_controls(via: Via) { + let f = started(via).await; + let state = f.harness.coordinator.published_state(); + let snapshot = { + let state = state.lock().expect("not poisoned"); + state + .latest_snapshot() + .expect("boundary 0 was published") + .clone() + }; + assert_eq!(snapshot.scope.step, 0); + for agent in &snapshot.agents { + assert!( + agent.selected_decision.is_none(), + "boundary 0 ended no transition" + ); + assert!(agent.applied_controls.is_none()); + } + f.shutdown().await; +} + +/// The decision published at boundary k+1 is the one the agent prepared for the transition +/// k -> k+1, never the one it is about to prepare for k+1 -> k+2. +async fn a_committed_action_is_the_transition_that_just_ended(via: Via) { + let mut f = started(via).await; + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + assert!(matches!( + within("the descriptor", consumer.take_descriptor()).await, + Some(ConsumerOutcome::Composition { .. }) + )); + let mut published: Vec<(u64, Digest)> = Vec::new(); + for step in 1..=STEPS { + f.harness.coordinator.run(1).await.expect("a transition"); + read_until(&mut consumer, step).await; + let view = consumer.last().expect("a snapshot"); + let agent = view.agent(&fly_a()).expect("fly-a is in every snapshot"); + let decision = agent + .decision + .as_ref() + .expect("past boundary 0 there is a decision"); + published.push((view.boundary, typed_digest(decision))); + assert!( + agent.controls.is_some(), + "and the controls that were applied" + ); + } + // The trace records, per transition, the decision digest the coordinator actually sent to + // the world. The snapshot at boundary k+1 must carry the transition ending there. + let trace = &f.harness.coordinator.trace.transitions; + assert_eq!(trace.len() as u64, STEPS); + for (index, (boundary, digest)) in published.iter().enumerate() { + let transition = &trace[index]; + assert_eq!( + transition.behaviour.published_boundary, *boundary, + "the snapshot at {boundary} is the transition that ended there" + ); + let traced = transition + .behaviour + .agents + .iter() + .find(|a| a.agent_id == fly_a()) + .expect("fly-a in the trace"); + assert_eq!( + traced.decision_digest, *digest, + "the decision published at boundary {boundary} is the one that produced it" + ); + assert_eq!(traced.committed_step, *boundary); + } + // And it is not the next transition's, which the following trace row holds. + if STEPS > 1 { + let first = &published[0]; + let next = trace[1] + .behaviour + .agents + .iter() + .find(|a| a.agent_id == fly_a()) + .expect("fly-a in the trace"); + assert_ne!( + first.1, next.decision_digest, + "boundary 1 did not publish the decision of the transition about to start" + ); + } + f.shutdown().await; +} + +/// One snapshot is the whole composition: a presentation consumer is a multi-agent consumer +/// and has no per-fly stream to join. +async fn one_snapshot_carries_every_agent_in_the_composition(via: Via) { + let mut f = started(via).await; + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + assert!(matches!( + within("the descriptor", consumer.take_descriptor()).await, + Some(ConsumerOutcome::Composition { .. }) + )); + f.harness.coordinator.run(1).await.expect("a transition"); + let outcome = match within("a snapshot", consumer.take_snapshot()).await { + Some(outcome) => outcome, + None => panic!("the snapshot stream ended"), + }; + let agents = match outcome { + ConsumerOutcome::Read { agents, .. } => agents, + other => panic!("expected a readable snapshot, got {other:?}"), + }; + assert_eq!(agents, vec![fly_a(), fly_b()], "every agent, in one value"); + let view = consumer.last().expect("a snapshot"); + for agent_id in [fly_a(), fly_b()] { + let agent = view.agent(&agent_id).expect("in the snapshot"); + assert_eq!( + agent.rates.len(), + 2, + "telemetry in the descriptor's rate-role order" + ); + assert_eq!(agent.rates[0].0, id("kc")); + assert_eq!(agent.rates[1].0, id("mbon")); + assert!(!agent.index_digest.is_empty()); + } + f.shutdown().await; +} + +// ------------------------------------------------------------------------------------------ +// Application-owned state and cues, bounded events, and the read-only repair service. + +/// The application publishes its own state and cues, on its own addresses, under its own +/// schema. Nothing about them is framework-shaped. +async fn application_state_and_cues_are_the_applications_own(via: Via) { + let mut f = started(via).await; + let client = f + .harness + .application() + .await + .expect("an application client"); + let mut channel = ApplicationChannel::new(client, "app.counter", 16); + channel + .declare() + .await + .expect("the application declares its own topics"); + assert_eq!(channel.state_topic(), "app.counter.state"); + assert_eq!(channel.cue_topic(), "app.counter.cues"); + + let watcher = f.harness.observer().await.expect("an observer client"); + let mut states = watcher + .subscribe(channel.state_topic(), Delivery::LatestValue.subscription()) + .await + .expect("a latest subscription"); + let mut cues = watcher + .subscribe( + channel.cue_topic(), + Delivery::BoundedBatch { depth: 16 }.subscription(), + ) + .await + .expect("a bounded subscription"); + + f.harness.coordinator.run(1).await.expect("a transition"); + // The application's schema: its own namespace, not a framework field. + let schema = SchemaRef { + id: id("counter.show.v1"), + version: 1, + digest: digest_of_bytes(b"counter.show.v1"), + }; + let state = typed(schema.clone(), json!({ "featured": "fly-a", "streak": 3 })) + .expect("an application value"); + let outcome = channel.publish_state(1, &state).await; + assert!(outcome.is_accepted(), "{outcome:?}"); + let cue = typed(schema, json!({ "line": "fly-a takes the lead" })).expect("a cue"); + assert!( + channel + .publish_cue(1, &id("counter.headline"), &cue) + .await + .is_accepted() + ); + + let message = within("the application state", states.next()) + .await + .expect("a state message"); + assert_eq!( + message.payload().get("boundary").and_then(|v| v.as_str()), + Some("1") + ); + let published = TypedValue::from_json(message.payload().get("state").expect("state")) + .expect("an application typed value"); + assert_eq!(published.schema.id, id("counter.show.v1")); + drop(message); + let message = within("the cue", cues.next()).await.expect("a cue message"); + assert_eq!( + message.payload().get("kind").and_then(|v| v.as_str()), + Some("counter.headline") + ); + drop(message); + + // Nothing the application published is on a session topic, and nothing on a session topic + // knows the application's schema. + let descriptor = f + .harness + .coordinator + .session_descriptor() + .expect("a descriptor"); + assert_ne!(descriptor.task_schema.id, id("counter.show.v1")); + f.shutdown().await; +} + +/// A refused event batch is held, counted and delivered later. Nothing is dropped quietly, +/// and what the bounded depth does push out travels as a count. +async fn a_refused_event_batch_is_held_and_counted_not_lost(via: Via) { + let mut f = started(via).await; + let events_topic = f.harness.coordinator.topics().events.clone(); + let offender_client = f.harness.observer().await.expect("an observer client"); + let mut offender = offender_client + .subscribe( + &events_topic, + flybus::SubscriptionConfig::bounded().queued(1).in_flight(1), + ) + .await + .expect("a bounded subscription"); + + f.harness + .coordinator + .run(STEPS) + .await + .expect("an event observer never fails the epoch"); + assert_eq!(f.harness.coordinator.stats().advances, STEPS); + let counters = f.harness.coordinator.ledger().counters(&events_topic); + assert!(counters.refused > 0, "the refusal is counted: {counters:?}"); + assert!(f.harness.coordinator.ledger().events_held > 0); + assert!( + f.harness.coordinator.pending_events() > 0, + "the refused events are held in a bounded batch, not dropped" + ); + + // The offender starts consuming; the held batch goes out at the next boundary. + drop(within("an event delivery", offender.next()).await); + drop(offender.try_next()); + let held = f.harness.coordinator.pending_events(); + f.harness.coordinator.run(1).await.expect("a transition"); + until("the held batch is published", || { + f.harness.coordinator.pending_events() < held + }) + .await; + offender_client.close().await; + f.shutdown().await; +} + +/// Two reads and nothing else. There is no method on this service that could move anything. +async fn the_query_service_answers_reads_and_nothing_else(via: Via) { + let mut f = started(via).await; + f.harness.coordinator.run(1).await.expect("a transition"); + let client = f.harness.observer().await.expect("an observer client"); + let service = query_service(&f.harness.config.session_id); + + let request = SessionRpcRequest { + request_id: DomainRequestId::from_serial(1), + scope: None, + params: json!({}), + }; + let mut pending = client + .call(&service, None, GET_SNAPSHOT, object(request.to_json()), &[]) + .await + .expect("the service answers"); + let result = pending.result().await.expect("a reply"); + let outcome = + SessionRpcOutcome::from_json(&serde_json::Value::Object(result.outcome().clone())) + .expect("a domain outcome"); + drop(result); + let snapshot = CommittedSnapshot::from_json(outcome_result(&outcome).expect("a result")) + .expect("a committed snapshot"); + assert_eq!(snapshot.scope.step, 1); + assert!(snapshot.views.is_empty() || !snapshot.views.is_empty()); + + // A method it does not implement is a named refusal, not a default. + let request = SessionRpcRequest { + request_id: DomainRequestId::from_serial(2), + scope: None, + params: json!({}), + }; + let mut pending = client + .call( + &service, + None, + "Session.Advance", + object(request.to_json()), + &[], + ) + .await + .expect("the service answers"); + let result = pending.result().await.expect("a reply"); + let outcome = + SessionRpcOutcome::from_json(&serde_json::Value::Object(result.outcome().clone())) + .expect("a domain outcome"); + drop(result); + let error = outcome_result(&outcome).expect_err("no such method"); + assert_eq!(error.code, ErrorCode::Unsupported, "{error:?}"); + assert_eq!( + f.harness.coordinator.stats().advances, + 1, + "and nothing advanced" + ); + client.close().await; + f.shutdown().await; +} + +// ------------------------------------------------------------------------------------------ +// Every execution mode + +/// The publication boundary is inside the coordinator, so it crosses no process boundary: it +/// must therefore behave identically in all three modes, and this asserts that rather than +/// assuming it. +async fn the_publication_boundary_holds_in_every_execution_mode(mode: ExecutionMode) { + let mut f = started_in_mode(mode).await; + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + let revision = match within("the descriptor", consumer.take_descriptor()).await { + Some(ConsumerOutcome::Composition { revision, agents }) => { + assert_eq!(agents, vec![fly_a(), fly_b()]); + revision + } + other => panic!("expected a composition, got {other:?}"), + }; + assert_eq!(revision, DESCRIPTOR_REVISION); + let descriptor = consumer.descriptor(revision).expect("held").clone(); + for agent in &descriptor.agents { + // The graph identity is attested over the bus, so it crosses a process boundary like + // every other reply and is observable in every mode. + assert_eq!( + agent.index_digest, + fly_session::agent::synthetic_graph(&agent.agent_id, 0).index_digest + ); + } + f.harness + .coordinator + .run(STEPS) + .await + .expect("the world runs"); + let at = read_until(&mut consumer, STEPS).await; + assert_eq!(at, STEPS); + let view = consumer.last().expect("a snapshot"); + assert_eq!(view.agents.len(), 2); + assert!(view.agent(&fly_a()).expect("fly-a").decision.is_some()); + assert_eq!( + f.harness + .coordinator + .ledger() + .counters(&f.harness.coordinator.topics().snapshots) + .accepted, + STEPS + 1 + ); + f.shutdown().await; +} + +/// A stalled observer costs only itself, in every mode. +async fn a_stalled_observer_never_moves_the_world_in_any_execution_mode(mode: ExecutionMode) { + let mut f = started_in_mode(mode).await; + let mut slow = f + .harness + .consumer() + .await + .expect("a slow consumer attaches"); + until("the slow viewer has a delivery", || { + slow.try_hold_snapshot() + }) + .await; + while slow.try_hold_snapshot() && slow.held() < 4 {} + let before = f.harness.coordinator.stats(); + let started_at = std::time::Instant::now(); + f.harness + .coordinator + .run(STEPS) + .await + .expect("the world carries on"); + assert_eq!( + f.harness.coordinator.stats().advances - before.advances, + STEPS + ); + assert!(!f.harness.coordinator.is_fenced()); + assert_eq!(f.harness.coordinator.ledger().refusals(), 0); + assert!(started_at.elapsed() < Duration::from_secs(10)); + f.shutdown().await; +} + +// ------------------------------------------------------------------------------------------ +// The policy and the batch, without a session + +#[test] +fn a_topic_policy_declares_its_delivery_once() { + let latest = TopicPolicy::latest("session.demo.snapshots"); + assert_eq!(latest.delivery, Delivery::LatestValue); + assert_eq!(latest.delivery.retained(), flybus::Retained::Latest); + assert_eq!(latest.delivery.subscription().mode, flybus::Mode::Latest); + let bounded = TopicPolicy::bounded("session.demo.events", 8); + assert_eq!(bounded.delivery, Delivery::BoundedBatch { depth: 8 }); + // A bounded stream is never retained: it is "not a durable log", and a retained tail + // would be the beginning of one. + assert_eq!(bounded.delivery.retained(), flybus::Retained::None); + assert_eq!(bounded.delivery.subscription().max_queued, 8); +} + +#[test] +fn a_full_event_batch_drops_the_oldest_by_an_explicit_count() { + let mut outbox = EventOutbox::new(EVENT_BATCH_DEPTH); + let event = |n: u64| TaskEvent { + id: id(&format!("ev-{n}")), + kind_id: id("arena.counter-delta"), + source_step: 1, + agent_id: None, + payload: typed( + SchemaRef { + id: id("t.v1"), + version: 1, + digest: digest_of_bytes(b"t.v1"), + }, + json!({ "n": n }), + ) + .expect("a typed value"), + }; + for n in 0..EVENT_BATCH_DEPTH as u64 { + assert_eq!( + outbox.offer(1, &[event(n)]), + 0, + "nothing is dropped inside the depth" + ); + } + assert_eq!(outbox.len(), EVENT_BATCH_DEPTH); + assert_eq!( + outbox.offer(2, &[event(999)]), + 1, + "the oldest is dropped, and counted" + ); + assert_eq!(outbox.dropped_since_accepted(), 1); + assert_eq!( + outbox.len(), + EVENT_BATCH_DEPTH, + "and the batch stays bounded" + ); +} + +#[tokio::test] +async fn a_snapshot_that_disagrees_with_its_descriptor_is_refused() { + let f = started(Via::Memory).await; + let descriptor = f + .harness + .coordinator + .session_descriptor() + .expect("a descriptor") + .clone(); + let snapshot = { + let state = f.harness.coordinator.published_state(); + let state = state.lock().expect("not poisoned"); + state.latest_snapshot().expect("boundary 0").clone() + }; + let positions: BTreeMap = snapshot + .audio + .iter() + .map(|a| (a.stream_id.clone(), a.first_sample + a.sample_frames)) + .collect(); + let attachments: Vec<(String, ArtifactRef)> = snapshot + .views + .iter() + .map(|v| { + ( + fly_session::media::view_attachment(&v.view_id), + v.pixels.clone(), + ) + }) + .chain(snapshot.audio.iter().map(|a| { + ( + fly_session::media::audio_attachment(&a.stream_id), + a.samples.clone(), + ) + })) + .collect(); + + let mut wrong = snapshot.clone(); + wrong.descriptor_revision = DESCRIPTOR_REVISION + 1; + let error = check_publication(&descriptor, &wrong, &attachments, &positions) + .expect_err("another revision is a disagreement"); + assert_eq!(error.code, ErrorCode::BufferInvalid, "{error:?}"); + + let mut renamed = snapshot.clone(); + renamed.agents[0].agent_id = id("fly-z"); + let error = check_publication(&descriptor, &renamed, &attachments, &positions) + .expect_err("an agent the descriptor does not declare"); + assert!(error.message.contains("fly-z"), "{}", error.message); + + // And a snapshot the session actually published passes the same check. + check_publication(&descriptor, &snapshot, &attachments, &positions) + .expect("what the session published agrees with the descriptor it named"); + f.shutdown().await; +} From f43fd4d9ffd8ede3d284e2bfeb34ea155a235e68 Mon Sep 17 00:00:00 2001 From: dev Date: Tue, 22 Sep 2026 18:47:10 +0000 Subject: [PATCH 2/6] session: supportedStimuli is enforced, and the query method names are provisional An undeclared stimulus kind is refused before the model is touched, proved by an injection through Agent.Commit rather than by a unit call, so the declaration a descriptor publishes is the thing the worker enforces. The publishing-v1 section 2 amendment now says in its own words that Session.GetDescriptor and Session.GetSnapshot are internal and provisional names, which the later public v2 step may rename or supersede. --- .../design/session-framework/publishing-v1.md | 4 ++ .../crates/fly-session/src/coordinator.rs | 14 ++++- .../crates/fly-session/tests/publishing.rs | 56 +++++++++++++++++++ 3 files changed, 73 insertions(+), 1 deletion(-) diff --git a/docs/design/session-framework/publishing-v1.md b/docs/design/session-framework/publishing-v1.md index c1ebce2..c438ac4 100644 --- a/docs/design/session-framework/publishing-v1.md +++ b/docs/design/session-framework/publishing-v1.md @@ -51,6 +51,10 @@ or, with no revision, the newest; `Session.GetSnapshot` takes no parameters and latest `CommittedSnapshot`. A revision the session never published is `IDENTITY_MISMATCH`, not an empty answer. Nothing on this service mutates, selects a participant or reaches a worker, so it is not the controller API section 7 rules out; adding a third method that did would be. +These two names are **internal and provisional**: they are what the internal boundary needs in +order to be buildable now, and the later public v2 step is free to rename them, supersede them +or expose a different repair surface entirely. Nothing about them is browser-facing, and the +public step does not inherit them by default merely because they landed first. Descriptor revisions and scope link observations to schemas. Cross-topic ordering is not guaranteed; a subscriber receiving an unknown descriptor revision must fetch it through the diff --git a/services/flysim/crates/fly-session/src/coordinator.rs b/services/flysim/crates/fly-session/src/coordinator.rs index aa21e8e..fb5ddaa 100644 --- a/services/flysim/crates/fly-session/src/coordinator.rs +++ b/services/flysim/crates/fly-session/src/coordinator.rs @@ -61,6 +61,8 @@ pub struct Injections { /// Publish this boundary's snapshot with a handle that is not the artifact the snapshot /// references: the same name, the same shape, another object. pub substituted_published_handle: bool, + /// Ask an agent to apply a stimulus kind its published descriptor does not declare. + pub undeclared_stimulus: bool, } /// What an injection produced, for a test to assert on. @@ -2246,13 +2248,23 @@ impl Coordinator { .expect("every agent prepared"); let outcome = outcomes.get(&agent_id).cloned().unwrap_or_default(); let next_context = next_contexts.get(&agent_id).cloned().expect("checked"); + let mut task_stimulations = outcome.stimulations.clone(); + if self.injections.undeclared_stimulus && self.injections.at_step == k { + // A kind outside the agent's published `supportedStimuli`. The declaration is + // only worth publishing if the worker enforces it. + task_stimulations.push(Stimulus { + id: parse_id(&format!("stim-undeclared-{k}")).expect("a serial makes an Id"), + kind_id: id("arena.undeclared"), + duration_ms: 1.0, + }); + } let params = CommitParams { agent_id: agent_id.clone(), prepared_request_id: prepared_request.clone(), next_input: self.sensory_input(observation, k + 1), next_decision_context: next_context, rewards: outcome.rewards.clone(), - task_stimulations: outcome.stimulations.clone(), + task_stimulations, }; let params = match params.to_json() { Value::Object(m) => m, diff --git a/services/flysim/crates/fly-session/tests/publishing.rs b/services/flysim/crates/fly-session/tests/publishing.rs index 52ca5a9..a600717 100644 --- a/services/flysim/crates/fly-session/tests/publishing.rs +++ b/services/flysim/crates/fly-session/tests/publishing.rs @@ -37,6 +37,7 @@ both_transports!( a_refused_event_batch_is_held_and_counted_not_lost, the_query_service_answers_reads_and_nothing_else, the_published_descriptor_is_what_the_workers_attested_to, + a_stimulus_kind_the_descriptor_does_not_declare_is_refused, ); all_modes!( @@ -608,6 +609,61 @@ async fn the_published_descriptor_is_what_the_workers_attested_to(via: Via) { f.shutdown().await; } +/// `supportedStimuli` is enforced, not advertised: a kind outside the list the descriptor +/// publishes is refused before the model is touched, so the declaration is worth reading. +async fn a_stimulus_kind_the_descriptor_does_not_declare_is_refused(via: Via) { + let mut f = started(via).await; + let declared = f + .harness + .coordinator + .session_descriptor() + .expect("a descriptor") + .agents + .iter() + .find(|a| a.agent_id == fly_a()) + .expect("fly-a") + .supported_stimuli + .clone(); + assert!(!declared.contains(&id("arena.undeclared")), "{declared:?}"); + + // A clean transition first, so the refusal is the injection and not the composition. + f.harness.coordinator.run(1).await.expect("one clean transition"); + let before = f.harness.coordinator.stats().advances; + f.harness.coordinator.injections = Injections { + at_step: 1, + undeclared_stimulus: true, + ..Injections::default() + }; + let failure = f + .harness + .coordinator + .run(1) + .await + .expect_err("an undeclared stimulus kind is refused"); + assert_eq!(failure.error.code, ErrorCode::Unsupported, "{failure:?}"); + assert_eq!( + failure.error.mutation, + MutationCertainty::None, + "refused before the model is touched" + ); + assert!( + failure.error.message.contains("arena.undeclared"), + "the refusal names the kind: {}", + failure.error.message + ); + assert!(failure.participant.is_some(), "and the participant it came from"); + assert_eq!( + f.harness.coordinator.stats().advances, + before, + "the refused transition never completed, so no boundary was added" + ); + assert!( + f.harness.coordinator.is_fenced(), + "a commit that refused after the world moved fences the epoch" + ); + f.shutdown().await; +} + // ------------------------------------------------------------------------------------------ // "A committed action is labelled as the transition that just ended, not the one about to // start." From 7ce645f1dcf2670efc4a3b87aa60fd96bf8e4c0e Mon Sep 17 00:00:00 2001 From: dev Date: Tue, 22 Sep 2026 19:24:44 +0000 Subject: [PATCH 3/6] session: a restored boundary is an installed one, and the revision follows the composition A group restore re-establishes a committed boundary this epoch did not run a transition into, so its snapshot carries no decision and no controls for any agent, and a fresh epoch is a new compositionDigest, so the descriptor takes the next revision rather than republishing revision 1 with different contents. CommittedSnapshot's rule becomes: null at boundary 0 and at an installed boundary, always together, and for every agent or none -- a snapshot where one fly acted and another did not would be two boundaries in one value. Dated amendment to publishing-v1 section 3, with the schema set, the fixtures and the TypeScript package moved together and the digest regenerated. --- .../design/session-framework/publishing-v1.md | 10 + packages/session-types/src/publishing.ts | 14 +- .../fixtures/contract-digest.json | 4 +- .../fly-session-types/fixtures/invalid.json | 174 +++++++++++++++++- .../fixtures/schema-set.json | 2 +- .../fly-session-types/fixtures/valid.json | 94 ++++++++++ .../fly-session-types/src/publishing.rs | 21 ++- .../crates/fly-session-types/src/schema.rs | 4 +- .../crates/fly-session/src/coordinator.rs | 29 ++- .../crates/fly-session/tests/publishing.rs | 57 ++++++ 10 files changed, 393 insertions(+), 16 deletions(-) diff --git a/docs/design/session-framework/publishing-v1.md b/docs/design/session-framework/publishing-v1.md index c438ac4..7258845 100644 --- a/docs/design/session-framework/publishing-v1.md +++ b/docs/design/session-framework/publishing-v1.md @@ -91,6 +91,16 @@ interface CommittedSnapshot { } ``` +**Amendment, 2026-09-22 (PUBLISH-01).** "Null at initial boundary 0" is the rule for a +boundary this epoch *produced*. A group restore ([state/media](state-media-v1.md) section 5) +re-establishes a committed boundary `k > 0` that this epoch did not run a transition into, and +the abandoned epoch's decisions are not this session's to republish under a new epoch. So the +rule is: `selectedDecision` and `appliedControls` are null at boundary 0 and at a boundary +*installed* by a restore, present otherwise, and always **together** and for **every agent or +none**. A snapshot where one fly carries an action and another does not would be two different +boundaries in one value, and is refused. Without this, the section 6 requirement to publish the +recovery could not be met at all: the restored boundary's snapshot would be unrepresentable. + Publish only after all agent commits establish Ready(k). Decisions/controls describe the transition ending at that boundary, null at initial boundary 0. Health updates are separate and never claim an uncommitted future boundary. Every transient media reference is a declared diff --git a/packages/session-types/src/publishing.ts b/packages/session-types/src/publishing.ts index a117650..104e777 100644 --- a/packages/session-types/src/publishing.ts +++ b/packages/session-types/src/publishing.ts @@ -168,16 +168,24 @@ export function readCommittedSnapshot(value: unknown): CommittedSnapshot { const atBoundaryZero = u64(scope.step) === 0n; for (const agent of agents) { // "Decisions/controls describe the transition ending at that boundary, null at initial - // boundary 0." (publishing-v1 section 3) + // boundary 0." (publishing-v1 section 3, and its 2026-09-22 amendment for a boundary that + // was installed rather than produced.) if (atBoundaryZero && (agent.selectedDecision !== null || agent.appliedControls !== null)) { fail('CommittedSnapshot: at boundary 0 selectedDecision and appliedControls are null'); } - if (!atBoundaryZero && (agent.selectedDecision === null || agent.appliedControls === null)) { + if ((agent.selectedDecision === null) !== (agent.appliedControls === null)) { fail( - 'CommittedSnapshot: past boundary 0 every agent has a decision and applied controls', + 'CommittedSnapshot: selectedDecision and appliedControls are null together or present together', ); } } + // A boundary is produced by a transition or installed by one, and the whole snapshot says + // which: every agent carries the transition that ended here, or none does. + if (agents.some((a) => (a.selectedDecision === null) !== (agents[0].selectedDecision === null))) { + fail( + 'CommittedSnapshot: either every agent carries the transition that ended here, or none does', + ); + } return { descriptorRevision, publisherIncarnation, diff --git a/services/flysim/crates/fly-session-types/fixtures/contract-digest.json b/services/flysim/crates/fly-session-types/fixtures/contract-digest.json index 36c1f10..90e4419 100644 --- a/services/flysim/crates/fly-session-types/fixtures/contract-digest.json +++ b/services/flysim/crates/fly-session-types/fixtures/contract-digest.json @@ -1,8 +1,8 @@ { "description": "contractDigest is the SHA-256 of the canonical schema set in schema-set.json.", - "contractDigest": "fa1f9671c098e0e5fe2d2ad9642d10eae39c6e751760eb1db7b51d5575898509", + "contractDigest": "7f4b11d6737e5097c6889657479527174ddf496e50b60322b281e6c7490bcb4a", "schemaSetVersion": 1, - "schemaSetBytes": 27407, + "schemaSetBytes": 27470, "types": 54, "enums": 11, "limits": 26 diff --git a/services/flysim/crates/fly-session-types/fixtures/invalid.json b/services/flysim/crates/fly-session-types/fixtures/invalid.json index 0b6bcff..b43edec 100644 --- a/services/flysim/crates/fly-session-types/fixtures/invalid.json +++ b/services/flysim/crates/fly-session-types/fixtures/invalid.json @@ -4168,7 +4168,179 @@ }, "eventIds": [] }, - "reason": "a committed transition has applied controls" + "reason": "selectedDecision and appliedControls are null together or present together" + }, + { + "name": "snapshot where one agent carries the transition and another does not", + "type": "CommittedSnapshot", + "value": { + "descriptorRevision": "7", + "publisherIncarnation": "pub-1", + "scope": { + "sessionId": "demo", + "epoch": "epoch-1", + "step": "42" + }, + "episodeId": "episode-1", + "sequence": "42", + "worldTime": { + "numerator": "700000000", + "denominator": "1" + }, + "agents": [ + { + "agentId": "fly-a", + "telemetry": { + "brainTicks": "2534", + "populationRateHz": 12.5, + "rates": [ + { + "roleId": "kenyon", + "hz": 3.25 + }, + { + "roleId": "mbon", + "hz": 0.0 + } + ], + "learning": { + "enabled": true, + "updates": "4", + "changed": "2", + "signal": 0.5 + } + }, + "selectedDecision": { + "schema": { + "id": "gameboy.intent.v1", + "version": 1, + "digest": "e70451b26fb5462ec63729d8355912dad01c3963a95bec0e6a8452375717ad9d" + }, + "value": { + "press": "a" + } + }, + "appliedControls": { + "portId": "port-1", + "buttons": [ + { + "id": "a", + "down": true + }, + { + "id": "b", + "down": false + }, + { + "id": "start", + "down": false + }, + { + "id": "select", + "down": false + }, + { + "id": "up", + "down": false + }, + { + "id": "down", + "down": false + }, + { + "id": "left", + "down": false + }, + { + "id": "right", + "down": false + } + ], + "axes": [ + { + "id": "stick-x", + "value": 0.0 + }, + { + "id": "trigger", + "value": 0.0 + } + ] + } + }, + { + "agentId": "fly-b", + "telemetry": { + "brainTicks": "2534", + "populationRateHz": 12.5, + "rates": [ + { + "roleId": "kenyon", + "hz": 3.25 + }, + { + "roleId": "mbon", + "hz": 0.0 + } + ], + "learning": { + "enabled": true, + "updates": "4", + "changed": "2", + "signal": 0.5 + } + }, + "selectedDecision": null, + "appliedControls": null + } + ], + "progress": { + "schema": { + "id": "pokemon.progress.v1", + "version": 1, + "digest": "80fa973157f334bb3208dfff6f7a2f298e79f73f07c72c33f5725975c6cff8e1" + }, + "value": { + "rank": 10 + } + }, + "media": { + "views": [ + { + "viewId": "screen", + "producedStep": "42", + "pixels": { + "storeId": "store-1", + "artifactId": "frame-1", + "generation": "1", + "byteLength": "92160", + "contentType": "image/x-rgba8", + "digest": null + } + } + ], + "audio": [ + { + "streamId": "mix", + "firstSample": "33600", + "sampleFrames": 800, + "samples": { + "storeId": "store-1", + "artifactId": "audio-1", + "generation": "1", + "byteLength": "6400", + "contentType": "audio/x-f32le", + "digest": null + }, + "discontinuity": false + } + ] + }, + "eventIds": [ + "evt-1" + ] + }, + "reason": "a boundary is produced or installed for the whole composition, never per agent" }, { "name": "trace whose commit acknowledgment is the old boundary", diff --git a/services/flysim/crates/fly-session-types/fixtures/schema-set.json b/services/flysim/crates/fly-session-types/fixtures/schema-set.json index 6477910..7078001 100644 --- a/services/flysim/crates/fly-session-types/fixtures/schema-set.json +++ b/services/flysim/crates/fly-session-types/fixtures/schema-set.json @@ -1 +1 @@ -{"contract":"fly-session-types","enums":[{"members":["f32le-interleaved"],"name":"AudioFormat","source":"state-media-v1 2"},{"members":["bipolar","unit"],"name":"AxisRange","source":"workers-v1 3"},{"members":["fixed-build","unverified"],"name":"Determinism","source":"workers-v1 3"},{"members":["terminal"],"name":"EpisodeRequestKind","source":"workers-v1 4"},{"members":["INVALID_ARGUMENT","UNSUPPORTED","IDENTITY_MISMATCH","STALE_EPOCH","STALE_STEP","FUTURE_STEP","INVALID_PHASE","CONFLICT","IN_PROGRESS","BUSY","BUFFER_INVALID","RESULT_EXPIRED","INCOMPATIBLE_STATE","BACKEND_FAILURE","INTERNAL"],"name":"ErrorCode","source":"ipc-v1 7"},{"members":["none","applied","unknown"],"name":"MutationCertainty","source":"ipc-v1 3"},{"members":["exact-checkpoint","episode-restart"],"name":"Recovery","source":"workers-v1 3"},{"members":["agent","environment","coordinator"],"name":"Role","source":"ipc-v1 4"},{"members":["lockstep-v1"],"name":"SchedulerId","source":"publishing-v1 3"},{"members":["rgba8"],"name":"ViewFormat","source":"state-media-v1 2"},{"members":["uninitialized","ready","preparing","prepared","advancing","committing","capturing","staged-restore","restoring","failed","stopping"],"name":"WorkerState","source":"ipc-v1 4"}],"limits":[{"name":"maxAcknowledge","source":"ipc-v1 5","value":16},{"name":"maxAgents","source":"ipc-v1 2","value":4},{"name":"maxAssets","source":"crate","value":64},{"name":"maxAttachments","source":"bus-v1 4","value":32},{"name":"maxAudioStreams","source":"crate","value":8},{"name":"maxAxes","source":"workers-v1 3","value":16},{"name":"maxButtons","source":"workers-v1 3","value":32},{"name":"maxCapabilities","source":"crate","value":32},{"name":"maxEngineFrameLength","source":"workers-v1 3","value":64},{"name":"maxEnvelopeBytes","source":"bus-v1 4","value":65536},{"name":"maxMessageCodePoints","source":"ipc-v1 7","value":512},{"name":"maxObservationDelaySteps","source":"state-media-v1 2","value":8},{"name":"maxPixelAspectPart","source":"state-media-v1 2","value":65535},{"name":"maxPorts","source":"ipc-v1 2","value":4},{"name":"maxRateRoles","source":"ipc-v1 2","value":64},{"name":"maxRewardsPerOperation","source":"workers-v1 1","value":64},{"name":"maxSampleFrames","source":"state-media-v1 2","value":192000},{"name":"maxSchemaVersion","source":"ipc-v1 2","value":65535},{"name":"maxSnapshotEvents","source":"crate","value":64},{"name":"maxStimuliPerOperation","source":"workers-v1 1","value":64},{"name":"maxSupportedMajors","source":"crate","value":8},{"name":"maxSupportedStimuli","source":"crate","value":64},{"name":"maxTypedValueBytes","source":"ipc-v1 2","value":32768},{"name":"maxViewDimension","source":"state-media-v1 2","value":4096},{"name":"maxViews","source":"workers-v1 1","value":8},{"name":"maxWorkerThreads","source":"workers-v1 2","value":4096}],"scalars":{"ArtifactIdentity":"storeId, artifactId and generation of a bus ArtifactRef","BusCallId":"call-","Digest":"64 lowercase hexadecimal digits (SHA-256)","DomainRequestId":"req-","Id":"^[a-z0-9][a-z0-9._-]{0,63}$","OwnerToken":"dlv- or own-","U64":"\"0\" or [1-9][0-9]*, <= 18446744073709551615"},"types":[{"fields":[{"constraint":"1..=16, unique","kind":"array","name":"requestIds","required":true}],"name":"AcknowledgeParams","source":"ipc-v1 5"},{"fields":[{"constraint":"<= 16, unique, a subset of the request","kind":"array","name":"acknowledged","required":true}],"name":"AcknowledgeResult","source":"ipc-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"restoreToken","required":true}],"name":"ActivateRestoreParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"required from an environment, null from an agent","kind":"WorldObservation|null","name":"observation","required":false}],"name":"ActivateRestoreResult","source":"state-media-v1 5"},{"fields":[{"constraint":"unique within an epoch","kind":"Id","name":"batchId","required":true},{"constraint":"one complete batch: every declared port once, descriptor order","kind":"array","name":"controls","required":true}],"name":"AdvanceParams","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"k+1","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true}],"name":"AgentCommitResult","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Id","name":"portId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"","kind":"Digest","name":"datasetDigest","required":true},{"constraint":"geometry mapping needs this, not neuronCount","kind":"Digest","name":"indexDigest","required":true},{"constraint":"","kind":"U64","name":"neuronCount","required":true},{"constraint":"<= 64, unique","kind":"array","name":"rateRoles","required":true},{"constraint":"unique","kind":"array","name":"supportedStimuli","required":true}],"name":"AgentDescriptor","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"Digest","name":"datasetDigest","required":true},{"constraint":"geometry mapping needs this, not neuronCount","kind":"Digest","name":"indexDigest","required":true},{"constraint":"","kind":"U64","name":"neuronCount","required":true},{"constraint":"<= 64, unique","kind":"array","name":"rateRoles","required":true},{"constraint":"<= 64, unique","kind":"array","name":"supportedStimuli","required":true}],"name":"AgentGraph","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"AssetRef","name":"profile","required":true},{"constraint":"signed 32-bit","kind":"int","name":"seed","required":true},{"constraint":"","kind":"SensoryInput","name":"initialInput","required":true},{"constraint":"","kind":"TypedValue","name":"initialDecisionContext","required":true},{"constraint":">= 1","kind":"int","name":"workerThreads","required":true}],"name":"AgentInitializeParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"positive","kind":"RationalNs","name":"tickDuration","required":true},{"constraint":"","kind":"U64","name":"warmupTicks","required":true},{"constraint":"\"0\"","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true},{"constraint":"rates are in graph.rateRoles order","kind":"AgentGraph","name":"graph","required":true}],"name":"AgentInitializeResult","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":"finite and nonnegative","kind":"number","name":"populationRateHz","required":true},{"constraint":"<= 64, unique roleId, profile order, finite nonnegative hz","kind":"array<{roleId:Id,hz:number}>","name":"rates","required":true},{"constraint":"changed <= updates; signal finite","kind":"{enabled:bool,updates:U64,changed:U64,signal:number}","name":"learning","required":true}],"name":"AgentTelemetry","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"id","required":true},{"constraint":"","kind":"Digest","name":"digest","required":true},{"constraint":"positive","kind":"U64","name":"byteLength","required":true},{"constraint":"","kind":"Id","name":"format","required":true}],"name":"AssetRef","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"streamId","required":true},{"constraint":"8000..=192000","kind":"int","name":"sampleRate","required":true},{"constraint":"1..=8","kind":"int","name":"channels","required":true},{"constraint":"","kind":"AudioFormat","name":"format","required":true}],"name":"AudioDescriptor","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"streamId","required":true},{"constraint":"no overlap or rewind within an epoch","kind":"U64","name":"firstSample","required":true},{"constraint":"0..=192000","kind":"int","name":"sampleFrames","required":true},{"constraint":"byteLength == sampleFrames x channels x 4, finite f32","kind":"ArtifactRef","name":"samples","required":true},{"constraint":"true on the first chunk after restore","kind":"bool","name":"discontinuity","required":true}],"name":"AudioRef","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true}],"name":"CaptureParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"the committed boundary","kind":"U64","name":"boundary","required":true},{"constraint":"","kind":"Digest","name":"compatibilityDigest","required":true},{"constraint":"listed attachment; digest required","kind":"ArtifactRef","name":"payload","required":true}],"name":"CaptureResult","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"DomainRequestId","name":"preparedRequestId","required":true},{"constraint":"boundary == scope.step + 1","kind":"SensoryInput","name":"nextInput","required":true},{"constraint":"","kind":"TypedValue","name":"nextDecisionContext","required":true},{"constraint":"<= 64, unique eventId, order retained","kind":"array","name":"rewards","required":true},{"constraint":"<= 64, unique id, order retained","kind":"array","name":"taskStimulations","required":true}],"name":"CommitParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"descriptorRevision","required":true},{"constraint":"","kind":"Id","name":"publisherIncarnation","required":true},{"constraint":"the committed boundary","kind":"Scope","name":"scope","required":true},{"constraint":"","kind":"Id","name":"episodeId","required":true},{"constraint":"monotonic within publisherIncarnation","kind":"U64","name":"sequence","required":true},{"constraint":"","kind":"RationalNs","name":"worldTime","required":true},{"constraint":"1..=4, unique agentId, telemetry in profile role order","kind":"array","name":"agents","required":true},{"constraint":"","kind":"TypedValue","name":"progress","required":true},{"constraint":"declared attachments held through publication admission","kind":"{views:array,audio:array}","name":"media","required":true},{"constraint":"unique, task order","kind":"array","name":"eventIds","required":true}],"name":"CommittedSnapshot","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"SchemaRef","name":"schema","required":true},{"constraint":"<= 32, unique, fixed order","kind":"array","name":"buttons","required":true},{"constraint":"<= 16, unique id, neutral inside its range","kind":"array<{id:Id,range:AxisRange,neutral:number}>","name":"axes","required":true}],"name":"ControllerSchema","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Digest","name":"backendDigest","required":true},{"constraint":"","kind":"Digest","name":"contentDigest","required":true},{"constraint":"","kind":"Digest","name":"configurationDigest","required":true},{"constraint":"fixed, reduced, positive","kind":"RationalNs","name":"stepDuration","required":true},{"constraint":"1..=4, unique portId, fixed order","kind":"array<{portId:Id,controls:ControllerSchema}>","name":"ports","required":true},{"constraint":"","kind":"SchemaRef","name":"inspectionSchema","required":true},{"constraint":"<= 8, unique viewId","kind":"array","name":"views","required":true},{"constraint":"unique streamId","kind":"array","name":"audio","required":true},{"constraint":"","kind":"Recovery","name":"recovery","required":true},{"constraint":"","kind":"Determinism","name":"determinism","required":true}],"name":"EnvironmentDescriptor","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"AssetRef","name":"backendConfig","required":true},{"constraint":"","kind":"AssetRef","name":"taskConfig","required":true},{"constraint":"","kind":"Id","name":"episodeId","required":true},{"constraint":"1..=4, unique portId and unique agentId","kind":"array<{portId:Id,agentId:Id}>","name":"portBindings","required":true}],"name":"EnvironmentInitializeParams","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"EnvironmentDescriptor","name":"descriptor","required":true},{"constraint":"boundary 0 and worldTime 0/1","kind":"WorldObservation","name":"observation","required":true}],"name":"EnvironmentInitializeResult","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"EpisodeRequestKind","name":"kind","required":true},{"constraint":"","kind":"Id","name":"reason","required":true},{"constraint":"","kind":"TypedValue","name":"outcome","required":true}],"name":"EpisodeRequest","source":"workers-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"sessionId","required":true},{"constraint":"","kind":"Id","name":"expectedWorkerId","required":true},{"constraint":"","kind":"Role","name":"role","required":true},{"constraint":"nonempty, unique, 1..=65535","kind":"array","name":"supportedMajors","required":true}],"name":"HelloParams","source":"ipc-v1 4"},{"fields":[{"constraint":"1","kind":"const","name":"selectedMajor","required":true},{"constraint":"0","kind":"const","name":"selectedMinor","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"","kind":"Role","name":"role","required":true},{"constraint":"","kind":"Digest","name":"buildDigest","required":true},{"constraint":"","kind":"Digest","name":"contractDigest","required":true},{"constraint":"unique; agent-step-v1 or world-step-v1 required for that role","kind":"array","name":"capabilities","required":true},{"constraint":"1..=4 agents, 1..=4 ports, and the launcher allocation this worker runs in","kind":"{maxAgents:int,maxPorts:int,workerThreads:int}","name":"limits","required":true}],"name":"HelloResult","source":"ipc-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"portId","required":true},{"constraint":"every declared button, descriptor order, no extras","kind":"array<{id:Id,down:bool}>","name":"buttons","required":true},{"constraint":"every declared axis in order; bipolar [-1,1], unit [0,1], refused not clamped","kind":"array<{id:Id,value:number}>","name":"axes","required":true}],"name":"PortControl","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"positive","kind":"RationalNs","name":"interval","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"<= 64, unique id, supplied order retained","kind":"array","name":"preStepStimulations","required":true}],"name":"PrepareParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"<= brainTicks","kind":"U64","name":"ticksAdvanced","required":true},{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":">= 0 and < one model tick","kind":"RationalNs","name":"remainder","required":true},{"constraint":"the profile's registered intent schema","kind":"TypedValue","name":"decision","required":true}],"name":"PreparedDecision","source":"workers-v1 2"},{"fields":[{"constraint":"reduced against denominator","kind":"U64","name":"numerator","required":true},{"constraint":"positive; zero is encoded 0/1; arithmetic is checked","kind":"U64","name":"denominator","required":true}],"name":"RationalNs","source":"ipc-v1 2"},{"fields":[{"constraint":"unique within its outcome namespace","kind":"Id","name":"eventId","required":true},{"constraint":"","kind":"Id","name":"ruleId","required":true},{"constraint":"finite; positive-only profiles reject negatives","kind":"number","name":"value","required":true}],"name":"Reward","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"id","required":true},{"constraint":"1..=65535","kind":"int","name":"version","required":true},{"constraint":"64 lowercase hex digits","kind":"Digest","name":"digest","required":true}],"name":"SchemaRef","source":"ipc-v1 2"},{"fields":[{"constraint":"^[a-z0-9][a-z0-9._-]{0,63}$","kind":"Id","name":"sessionId","required":true},{"constraint":"^[a-z0-9][a-z0-9._-]{0,63}$","kind":"Id","name":"epoch","required":true},{"constraint":"decimal string, <= 18446744073709551615","kind":"U64","name":"step","required":true}],"name":"Scope","source":"ipc-v1 2"},{"fields":[{"constraint":"the observed environment boundary","kind":"U64","name":"boundary","required":true},{"constraint":"<= 8, unique viewId, producedStep <= boundary","kind":"array","name":"views","required":true},{"constraint":"a pixel-only profile rejects non-null","kind":"TypedValue|null","name":"structured","required":false}],"name":"SensoryInput","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"sessionId","required":true},{"constraint":"","kind":"U64","name":"revision","required":true},{"constraint":"","kind":"Digest","name":"compositionDigest","required":true},{"constraint":"","kind":"SchedulerId","name":"schedulerId","required":true},{"constraint":"","kind":"EnvironmentDescriptor","name":"environment","required":true},{"constraint":"","kind":"SchemaRef","name":"taskSchema","required":true},{"constraint":"1..=4, unique agentId and portId, each portId declared by the environment","kind":"array","name":"agents","required":true},{"constraint":"unique id","kind":"array","name":"assets","required":true}],"name":"SessionDescriptor","source":"publishing-v1 3"},{"fields":[{"constraint":"\"error\"","kind":"const","name":"type","required":true},{"constraint":"echoes the request","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"echoes the request scope","kind":"Scope|null","name":"scope","required":false},{"constraint":"","kind":"ErrorCode","name":"error.code","required":true},{"constraint":"<= 512 code points","kind":"string","name":"error.message","required":true},{"constraint":"none for every code raised before mutation","kind":"MutationCertainty","name":"error.mutation","required":true}],"name":"SessionRpcFailure","source":"ipc-v1 3"},{"fields":[{"constraint":"req-","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"null for lifecycle calls","kind":"Scope|null","name":"scope","required":false},{"constraint":"no bus callId/deliveryId/ownerId keys","kind":"object","name":"params","required":true}],"name":"SessionRpcRequest","source":"ipc-v1 2"},{"fields":[{"constraint":"\"result\"","kind":"const","name":"type","required":true},{"constraint":"echoes the request","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"echoes the request scope","kind":"Scope|null","name":"scope","required":false},{"constraint":"","kind":"object","name":"result","required":true}],"name":"SessionRpcSuccess","source":"ipc-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"reason","required":true}],"name":"ShutdownParams","source":"ipc-v1 7"},{"fields":[{"constraint":"true","kind":"const","name":"stopping","required":true}],"name":"ShutdownResult","source":"ipc-v1 7"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true},{"constraint":"null exactly at boundary 0","kind":"TypedValue|null","name":"selectedDecision","required":false},{"constraint":"null exactly at boundary 0; the agent's assigned port","kind":"PortControl|null","name":"appliedControls","required":false}],"name":"SnapshotAgent","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"provenance, not the new handles","kind":"Scope","name":"sourceScope","required":true},{"constraint":"","kind":"Digest","name":"compatibilityDigest","required":true},{"constraint":"newly imported; digest required","kind":"ArtifactRef","name":"payload","required":true}],"name":"StageRestoreParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"activates once, bound to scope and payload","kind":"Id","name":"restoreToken","required":true}],"name":"StageRestoreResult","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"WorkerState","name":"state","required":true},{"constraint":"null before initialization","kind":"Scope|null","name":"currentScope","required":false},{"constraint":"","kind":"DomainRequestId|null","name":"activeRequestId","required":false},{"constraint":"","kind":"DomainRequestId|null","name":"lastCompletedRequestId","required":false},{"constraint":"","kind":"Id|null","name":"lastBatchId","required":false},{"constraint":"advances on progress, not on status queries","kind":"U64","name":"progressCounter","required":true}],"name":"StatusResult","source":"ipc-v1 4"},{"fields":[{"constraint":"echoes the request","kind":"Id","name":"batchId","required":true},{"constraint":"k","kind":"U64","name":"appliedFromStep","required":true},{"constraint":"appliedFromStep + 1","kind":"U64","name":"nextStep","required":true},{"constraint":"over validated canonical requested controls","kind":"Digest","name":"appliedControlsDigest","required":true},{"constraint":"boundary == nextStep","kind":"WorldObservation","name":"observation","required":true}],"name":"StepResult","source":"workers-v1 3"},{"fields":[{"constraint":"unique within its command namespace","kind":"Id","name":"id","required":true},{"constraint":"resolved through a profile capability","kind":"Id","name":"kindId","required":true},{"constraint":"finite and > 0","kind":"number","name":"durationMs","required":true}],"name":"Stimulus","source":"workers-v1 1"},{"fields":[{"constraint":"derived from epoch, source step, rule and ordinal","kind":"Id","name":"id","required":true},{"constraint":"","kind":"Id","name":"kindId","required":true},{"constraint":"the newly reached boundary, 0 for bootstrap","kind":"U64","name":"sourceStep","required":true},{"constraint":"","kind":"Id|null","name":"agentId","required":false},{"constraint":"","kind":"TypedValue","name":"payload","required":true}],"name":"TaskEvent","source":"workers-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"","kind":"U64","name":"ticksAdvanced","required":true},{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":"","kind":"RationalNs","name":"remainder","required":true},{"constraint":"","kind":"Digest","name":"decisionDigest","required":true},{"constraint":"the commit acknowledgment","kind":"U64","name":"committedStep","required":true}],"name":"TraceAgent","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"Scope","name":"scope","required":true},{"constraint":"sorted by agentId, independent of dispatch order","kind":"array","name":"agents","required":true},{"constraint":"","kind":"Id","name":"batchId","required":true},{"constraint":"","kind":"Digest","name":"controlDigest","required":true},{"constraint":"the world boundary the environment acknowledged","kind":"U64","name":"acknowledgedBoundary","required":true},{"constraint":"sorted by viewId","kind":"array<{viewId:Id,producedStep:U64}>","name":"observationBoundaries","required":true},{"constraint":"task outcome ids in task order","kind":"array","name":"outcomeIds","required":true},{"constraint":"task event ids in task order","kind":"array","name":"eventIds","required":true},{"constraint":"","kind":"U64","name":"publishedBoundary","required":true}],"name":"TraceBehaviour","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"U64","name":"wallTimeNs","required":true},{"constraint":"","kind":"array<{agentId:Id,requestId:DomainRequestId}>","name":"prepareRequestIds","required":true},{"constraint":"","kind":"DomainRequestId","name":"advanceRequestId","required":true},{"constraint":"","kind":"array<{agentId:Id,requestId:DomainRequestId}>","name":"commitRequestIds","required":true},{"constraint":"call-","kind":"array","name":"busCallIds","required":true},{"constraint":"dlv- or own-","kind":"array","name":"deliveryIds","required":true}],"name":"TraceOperational","source":"step-v1 8"},{"fields":[{"constraint":"compared between runs","kind":"TraceBehaviour","name":"behaviour","required":true},{"constraint":"recorded, never compared: wall time and transport identities","kind":"TraceOperational","name":"operational","required":true}],"name":"TransitionTrace","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"SchemaRef","name":"schema","required":true},{"constraint":"canonical JSON of the whole TypedValue <= 32768 bytes","kind":"object","name":"value","required":true}],"name":"TypedValue","source":"ipc-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"viewId","required":true},{"constraint":"1..=4096","kind":"int","name":"width","required":true},{"constraint":"1..=4096","kind":"int","name":"height","required":true},{"constraint":"","kind":"ViewFormat","name":"format","required":true},{"constraint":"exactly 4 x width","kind":"int","name":"rowStride","required":true},{"constraint":"positive integers <= 65535","kind":"{numerator:int,denominator:int}","name":"pixelAspect","required":true},{"constraint":"0..=8","kind":"int","name":"observationDelaySteps","required":true}],"name":"ViewDescriptor","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"viewId","required":true},{"constraint":"max(0, boundary - observationDelaySteps) for required sensory views","kind":"U64","name":"producedStep","required":true},{"constraint":"listed attachment; byteLength == rowStride x height","kind":"ArtifactRef","name":"pixels","required":true}],"name":"ViewRef","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"boundary","required":true},{"constraint":"logical time since episode start","kind":"RationalNs","name":"worldTime","required":true},{"constraint":"<= 64 characters","kind":"string|null","name":"engineFrame","required":false},{"constraint":"<= 8, unique viewId","kind":"array","name":"sensoryViews","required":true},{"constraint":"the descriptor's inspectionSchema","kind":"TypedValue","name":"inspection","required":true},{"constraint":"<= 8, unique viewId","kind":"array","name":"broadcastViews","required":true},{"constraint":"unique streamId","kind":"array","name":"audio","required":true}],"name":"WorldObservation","source":"workers-v1 3"}],"version":1} +{"contract":"fly-session-types","enums":[{"members":["f32le-interleaved"],"name":"AudioFormat","source":"state-media-v1 2"},{"members":["bipolar","unit"],"name":"AxisRange","source":"workers-v1 3"},{"members":["fixed-build","unverified"],"name":"Determinism","source":"workers-v1 3"},{"members":["terminal"],"name":"EpisodeRequestKind","source":"workers-v1 4"},{"members":["INVALID_ARGUMENT","UNSUPPORTED","IDENTITY_MISMATCH","STALE_EPOCH","STALE_STEP","FUTURE_STEP","INVALID_PHASE","CONFLICT","IN_PROGRESS","BUSY","BUFFER_INVALID","RESULT_EXPIRED","INCOMPATIBLE_STATE","BACKEND_FAILURE","INTERNAL"],"name":"ErrorCode","source":"ipc-v1 7"},{"members":["none","applied","unknown"],"name":"MutationCertainty","source":"ipc-v1 3"},{"members":["exact-checkpoint","episode-restart"],"name":"Recovery","source":"workers-v1 3"},{"members":["agent","environment","coordinator"],"name":"Role","source":"ipc-v1 4"},{"members":["lockstep-v1"],"name":"SchedulerId","source":"publishing-v1 3"},{"members":["rgba8"],"name":"ViewFormat","source":"state-media-v1 2"},{"members":["uninitialized","ready","preparing","prepared","advancing","committing","capturing","staged-restore","restoring","failed","stopping"],"name":"WorkerState","source":"ipc-v1 4"}],"limits":[{"name":"maxAcknowledge","source":"ipc-v1 5","value":16},{"name":"maxAgents","source":"ipc-v1 2","value":4},{"name":"maxAssets","source":"crate","value":64},{"name":"maxAttachments","source":"bus-v1 4","value":32},{"name":"maxAudioStreams","source":"crate","value":8},{"name":"maxAxes","source":"workers-v1 3","value":16},{"name":"maxButtons","source":"workers-v1 3","value":32},{"name":"maxCapabilities","source":"crate","value":32},{"name":"maxEngineFrameLength","source":"workers-v1 3","value":64},{"name":"maxEnvelopeBytes","source":"bus-v1 4","value":65536},{"name":"maxMessageCodePoints","source":"ipc-v1 7","value":512},{"name":"maxObservationDelaySteps","source":"state-media-v1 2","value":8},{"name":"maxPixelAspectPart","source":"state-media-v1 2","value":65535},{"name":"maxPorts","source":"ipc-v1 2","value":4},{"name":"maxRateRoles","source":"ipc-v1 2","value":64},{"name":"maxRewardsPerOperation","source":"workers-v1 1","value":64},{"name":"maxSampleFrames","source":"state-media-v1 2","value":192000},{"name":"maxSchemaVersion","source":"ipc-v1 2","value":65535},{"name":"maxSnapshotEvents","source":"crate","value":64},{"name":"maxStimuliPerOperation","source":"workers-v1 1","value":64},{"name":"maxSupportedMajors","source":"crate","value":8},{"name":"maxSupportedStimuli","source":"crate","value":64},{"name":"maxTypedValueBytes","source":"ipc-v1 2","value":32768},{"name":"maxViewDimension","source":"state-media-v1 2","value":4096},{"name":"maxViews","source":"workers-v1 1","value":8},{"name":"maxWorkerThreads","source":"workers-v1 2","value":4096}],"scalars":{"ArtifactIdentity":"storeId, artifactId and generation of a bus ArtifactRef","BusCallId":"call-","Digest":"64 lowercase hexadecimal digits (SHA-256)","DomainRequestId":"req-","Id":"^[a-z0-9][a-z0-9._-]{0,63}$","OwnerToken":"dlv- or own-","U64":"\"0\" or [1-9][0-9]*, <= 18446744073709551615"},"types":[{"fields":[{"constraint":"1..=16, unique","kind":"array","name":"requestIds","required":true}],"name":"AcknowledgeParams","source":"ipc-v1 5"},{"fields":[{"constraint":"<= 16, unique, a subset of the request","kind":"array","name":"acknowledged","required":true}],"name":"AcknowledgeResult","source":"ipc-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"restoreToken","required":true}],"name":"ActivateRestoreParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"required from an environment, null from an agent","kind":"WorldObservation|null","name":"observation","required":false}],"name":"ActivateRestoreResult","source":"state-media-v1 5"},{"fields":[{"constraint":"unique within an epoch","kind":"Id","name":"batchId","required":true},{"constraint":"one complete batch: every declared port once, descriptor order","kind":"array","name":"controls","required":true}],"name":"AdvanceParams","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"k+1","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true}],"name":"AgentCommitResult","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Id","name":"portId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"","kind":"Digest","name":"datasetDigest","required":true},{"constraint":"geometry mapping needs this, not neuronCount","kind":"Digest","name":"indexDigest","required":true},{"constraint":"","kind":"U64","name":"neuronCount","required":true},{"constraint":"<= 64, unique","kind":"array","name":"rateRoles","required":true},{"constraint":"unique","kind":"array","name":"supportedStimuli","required":true}],"name":"AgentDescriptor","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"Digest","name":"datasetDigest","required":true},{"constraint":"geometry mapping needs this, not neuronCount","kind":"Digest","name":"indexDigest","required":true},{"constraint":"","kind":"U64","name":"neuronCount","required":true},{"constraint":"<= 64, unique","kind":"array","name":"rateRoles","required":true},{"constraint":"<= 64, unique","kind":"array","name":"supportedStimuli","required":true}],"name":"AgentGraph","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"AssetRef","name":"profile","required":true},{"constraint":"signed 32-bit","kind":"int","name":"seed","required":true},{"constraint":"","kind":"SensoryInput","name":"initialInput","required":true},{"constraint":"","kind":"TypedValue","name":"initialDecisionContext","required":true},{"constraint":">= 1","kind":"int","name":"workerThreads","required":true}],"name":"AgentInitializeParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"positive","kind":"RationalNs","name":"tickDuration","required":true},{"constraint":"","kind":"U64","name":"warmupTicks","required":true},{"constraint":"\"0\"","kind":"U64","name":"committedStep","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true},{"constraint":"rates are in graph.rateRoles order","kind":"AgentGraph","name":"graph","required":true}],"name":"AgentInitializeResult","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":"finite and nonnegative","kind":"number","name":"populationRateHz","required":true},{"constraint":"<= 64, unique roleId, profile order, finite nonnegative hz","kind":"array<{roleId:Id,hz:number}>","name":"rates","required":true},{"constraint":"changed <= updates; signal finite","kind":"{enabled:bool,updates:U64,changed:U64,signal:number}","name":"learning","required":true}],"name":"AgentTelemetry","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"id","required":true},{"constraint":"","kind":"Digest","name":"digest","required":true},{"constraint":"positive","kind":"U64","name":"byteLength","required":true},{"constraint":"","kind":"Id","name":"format","required":true}],"name":"AssetRef","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"streamId","required":true},{"constraint":"8000..=192000","kind":"int","name":"sampleRate","required":true},{"constraint":"1..=8","kind":"int","name":"channels","required":true},{"constraint":"","kind":"AudioFormat","name":"format","required":true}],"name":"AudioDescriptor","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"streamId","required":true},{"constraint":"no overlap or rewind within an epoch","kind":"U64","name":"firstSample","required":true},{"constraint":"0..=192000","kind":"int","name":"sampleFrames","required":true},{"constraint":"byteLength == sampleFrames x channels x 4, finite f32","kind":"ArtifactRef","name":"samples","required":true},{"constraint":"true on the first chunk after restore","kind":"bool","name":"discontinuity","required":true}],"name":"AudioRef","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true}],"name":"CaptureParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"the committed boundary","kind":"U64","name":"boundary","required":true},{"constraint":"","kind":"Digest","name":"compatibilityDigest","required":true},{"constraint":"listed attachment; digest required","kind":"ArtifactRef","name":"payload","required":true}],"name":"CaptureResult","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"DomainRequestId","name":"preparedRequestId","required":true},{"constraint":"boundary == scope.step + 1","kind":"SensoryInput","name":"nextInput","required":true},{"constraint":"","kind":"TypedValue","name":"nextDecisionContext","required":true},{"constraint":"<= 64, unique eventId, order retained","kind":"array","name":"rewards","required":true},{"constraint":"<= 64, unique id, order retained","kind":"array","name":"taskStimulations","required":true}],"name":"CommitParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"descriptorRevision","required":true},{"constraint":"","kind":"Id","name":"publisherIncarnation","required":true},{"constraint":"the committed boundary","kind":"Scope","name":"scope","required":true},{"constraint":"","kind":"Id","name":"episodeId","required":true},{"constraint":"monotonic within publisherIncarnation","kind":"U64","name":"sequence","required":true},{"constraint":"","kind":"RationalNs","name":"worldTime","required":true},{"constraint":"1..=4, unique agentId, telemetry in profile role order","kind":"array","name":"agents","required":true},{"constraint":"","kind":"TypedValue","name":"progress","required":true},{"constraint":"declared attachments held through publication admission","kind":"{views:array,audio:array}","name":"media","required":true},{"constraint":"unique, task order","kind":"array","name":"eventIds","required":true}],"name":"CommittedSnapshot","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"SchemaRef","name":"schema","required":true},{"constraint":"<= 32, unique, fixed order","kind":"array","name":"buttons","required":true},{"constraint":"<= 16, unique id, neutral inside its range","kind":"array<{id:Id,range:AxisRange,neutral:number}>","name":"axes","required":true}],"name":"ControllerSchema","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Digest","name":"backendDigest","required":true},{"constraint":"","kind":"Digest","name":"contentDigest","required":true},{"constraint":"","kind":"Digest","name":"configurationDigest","required":true},{"constraint":"fixed, reduced, positive","kind":"RationalNs","name":"stepDuration","required":true},{"constraint":"1..=4, unique portId, fixed order","kind":"array<{portId:Id,controls:ControllerSchema}>","name":"ports","required":true},{"constraint":"","kind":"SchemaRef","name":"inspectionSchema","required":true},{"constraint":"<= 8, unique viewId","kind":"array","name":"views","required":true},{"constraint":"unique streamId","kind":"array","name":"audio","required":true},{"constraint":"","kind":"Recovery","name":"recovery","required":true},{"constraint":"","kind":"Determinism","name":"determinism","required":true}],"name":"EnvironmentDescriptor","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"AssetRef","name":"backendConfig","required":true},{"constraint":"","kind":"AssetRef","name":"taskConfig","required":true},{"constraint":"","kind":"Id","name":"episodeId","required":true},{"constraint":"1..=4, unique portId and unique agentId","kind":"array<{portId:Id,agentId:Id}>","name":"portBindings","required":true}],"name":"EnvironmentInitializeParams","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"EnvironmentDescriptor","name":"descriptor","required":true},{"constraint":"boundary 0 and worldTime 0/1","kind":"WorldObservation","name":"observation","required":true}],"name":"EnvironmentInitializeResult","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"EpisodeRequestKind","name":"kind","required":true},{"constraint":"","kind":"Id","name":"reason","required":true},{"constraint":"","kind":"TypedValue","name":"outcome","required":true}],"name":"EpisodeRequest","source":"workers-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"sessionId","required":true},{"constraint":"","kind":"Id","name":"expectedWorkerId","required":true},{"constraint":"","kind":"Role","name":"role","required":true},{"constraint":"nonempty, unique, 1..=65535","kind":"array","name":"supportedMajors","required":true}],"name":"HelloParams","source":"ipc-v1 4"},{"fields":[{"constraint":"1","kind":"const","name":"selectedMajor","required":true},{"constraint":"0","kind":"const","name":"selectedMinor","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"","kind":"Role","name":"role","required":true},{"constraint":"","kind":"Digest","name":"buildDigest","required":true},{"constraint":"","kind":"Digest","name":"contractDigest","required":true},{"constraint":"unique; agent-step-v1 or world-step-v1 required for that role","kind":"array","name":"capabilities","required":true},{"constraint":"1..=4 agents, 1..=4 ports, and the launcher allocation this worker runs in","kind":"{maxAgents:int,maxPorts:int,workerThreads:int}","name":"limits","required":true}],"name":"HelloResult","source":"ipc-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"portId","required":true},{"constraint":"every declared button, descriptor order, no extras","kind":"array<{id:Id,down:bool}>","name":"buttons","required":true},{"constraint":"every declared axis in order; bipolar [-1,1], unit [0,1], refused not clamped","kind":"array<{id:Id,value:number}>","name":"axes","required":true}],"name":"PortControl","source":"workers-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"positive","kind":"RationalNs","name":"interval","required":true},{"constraint":"","kind":"Digest","name":"decisionContextDigest","required":true},{"constraint":"<= 64, unique id, supplied order retained","kind":"array","name":"preStepStimulations","required":true}],"name":"PrepareParams","source":"workers-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"<= brainTicks","kind":"U64","name":"ticksAdvanced","required":true},{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":">= 0 and < one model tick","kind":"RationalNs","name":"remainder","required":true},{"constraint":"the profile's registered intent schema","kind":"TypedValue","name":"decision","required":true}],"name":"PreparedDecision","source":"workers-v1 2"},{"fields":[{"constraint":"reduced against denominator","kind":"U64","name":"numerator","required":true},{"constraint":"positive; zero is encoded 0/1; arithmetic is checked","kind":"U64","name":"denominator","required":true}],"name":"RationalNs","source":"ipc-v1 2"},{"fields":[{"constraint":"unique within its outcome namespace","kind":"Id","name":"eventId","required":true},{"constraint":"","kind":"Id","name":"ruleId","required":true},{"constraint":"finite; positive-only profiles reject negatives","kind":"number","name":"value","required":true}],"name":"Reward","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"id","required":true},{"constraint":"1..=65535","kind":"int","name":"version","required":true},{"constraint":"64 lowercase hex digits","kind":"Digest","name":"digest","required":true}],"name":"SchemaRef","source":"ipc-v1 2"},{"fields":[{"constraint":"^[a-z0-9][a-z0-9._-]{0,63}$","kind":"Id","name":"sessionId","required":true},{"constraint":"^[a-z0-9][a-z0-9._-]{0,63}$","kind":"Id","name":"epoch","required":true},{"constraint":"decimal string, <= 18446744073709551615","kind":"U64","name":"step","required":true}],"name":"Scope","source":"ipc-v1 2"},{"fields":[{"constraint":"the observed environment boundary","kind":"U64","name":"boundary","required":true},{"constraint":"<= 8, unique viewId, producedStep <= boundary","kind":"array","name":"views","required":true},{"constraint":"a pixel-only profile rejects non-null","kind":"TypedValue|null","name":"structured","required":false}],"name":"SensoryInput","source":"workers-v1 1"},{"fields":[{"constraint":"","kind":"Id","name":"sessionId","required":true},{"constraint":"","kind":"U64","name":"revision","required":true},{"constraint":"","kind":"Digest","name":"compositionDigest","required":true},{"constraint":"","kind":"SchedulerId","name":"schedulerId","required":true},{"constraint":"","kind":"EnvironmentDescriptor","name":"environment","required":true},{"constraint":"","kind":"SchemaRef","name":"taskSchema","required":true},{"constraint":"1..=4, unique agentId and portId, each portId declared by the environment","kind":"array","name":"agents","required":true},{"constraint":"unique id","kind":"array","name":"assets","required":true}],"name":"SessionDescriptor","source":"publishing-v1 3"},{"fields":[{"constraint":"\"error\"","kind":"const","name":"type","required":true},{"constraint":"echoes the request","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"echoes the request scope","kind":"Scope|null","name":"scope","required":false},{"constraint":"","kind":"ErrorCode","name":"error.code","required":true},{"constraint":"<= 512 code points","kind":"string","name":"error.message","required":true},{"constraint":"none for every code raised before mutation","kind":"MutationCertainty","name":"error.mutation","required":true}],"name":"SessionRpcFailure","source":"ipc-v1 3"},{"fields":[{"constraint":"req-","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"null for lifecycle calls","kind":"Scope|null","name":"scope","required":false},{"constraint":"no bus callId/deliveryId/ownerId keys","kind":"object","name":"params","required":true}],"name":"SessionRpcRequest","source":"ipc-v1 2"},{"fields":[{"constraint":"\"result\"","kind":"const","name":"type","required":true},{"constraint":"echoes the request","kind":"DomainRequestId","name":"requestId","required":true},{"constraint":"","kind":"Id","name":"workerId","required":true},{"constraint":"","kind":"Id","name":"incarnationId","required":true},{"constraint":"echoes the request scope","kind":"Scope|null","name":"scope","required":false},{"constraint":"","kind":"object","name":"result","required":true}],"name":"SessionRpcSuccess","source":"ipc-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"reason","required":true}],"name":"ShutdownParams","source":"ipc-v1 7"},{"fields":[{"constraint":"true","kind":"const","name":"stopping","required":true}],"name":"ShutdownResult","source":"ipc-v1 7"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"AgentTelemetry","name":"telemetry","required":true},{"constraint":"null at boundary 0 and at an installed boundary; null or present for every agent together","kind":"TypedValue|null","name":"selectedDecision","required":false},{"constraint":"null with selectedDecision; the agent's assigned port","kind":"PortControl|null","name":"appliedControls","required":false}],"name":"SnapshotAgent","source":"publishing-v1 3"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"provenance, not the new handles","kind":"Scope","name":"sourceScope","required":true},{"constraint":"","kind":"Digest","name":"compatibilityDigest","required":true},{"constraint":"newly imported; digest required","kind":"ArtifactRef","name":"payload","required":true}],"name":"StageRestoreParams","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"Id","name":"checkpointId","required":true},{"constraint":"activates once, bound to scope and payload","kind":"Id","name":"restoreToken","required":true}],"name":"StageRestoreResult","source":"state-media-v1 5"},{"fields":[{"constraint":"","kind":"WorkerState","name":"state","required":true},{"constraint":"null before initialization","kind":"Scope|null","name":"currentScope","required":false},{"constraint":"","kind":"DomainRequestId|null","name":"activeRequestId","required":false},{"constraint":"","kind":"DomainRequestId|null","name":"lastCompletedRequestId","required":false},{"constraint":"","kind":"Id|null","name":"lastBatchId","required":false},{"constraint":"advances on progress, not on status queries","kind":"U64","name":"progressCounter","required":true}],"name":"StatusResult","source":"ipc-v1 4"},{"fields":[{"constraint":"echoes the request","kind":"Id","name":"batchId","required":true},{"constraint":"k","kind":"U64","name":"appliedFromStep","required":true},{"constraint":"appliedFromStep + 1","kind":"U64","name":"nextStep","required":true},{"constraint":"over validated canonical requested controls","kind":"Digest","name":"appliedControlsDigest","required":true},{"constraint":"boundary == nextStep","kind":"WorldObservation","name":"observation","required":true}],"name":"StepResult","source":"workers-v1 3"},{"fields":[{"constraint":"unique within its command namespace","kind":"Id","name":"id","required":true},{"constraint":"resolved through a profile capability","kind":"Id","name":"kindId","required":true},{"constraint":"finite and > 0","kind":"number","name":"durationMs","required":true}],"name":"Stimulus","source":"workers-v1 1"},{"fields":[{"constraint":"derived from epoch, source step, rule and ordinal","kind":"Id","name":"id","required":true},{"constraint":"","kind":"Id","name":"kindId","required":true},{"constraint":"the newly reached boundary, 0 for bootstrap","kind":"U64","name":"sourceStep","required":true},{"constraint":"","kind":"Id|null","name":"agentId","required":false},{"constraint":"","kind":"TypedValue","name":"payload","required":true}],"name":"TaskEvent","source":"workers-v1 4"},{"fields":[{"constraint":"","kind":"Id","name":"agentId","required":true},{"constraint":"","kind":"Digest","name":"profileDigest","required":true},{"constraint":"","kind":"U64","name":"ticksAdvanced","required":true},{"constraint":"","kind":"U64","name":"brainTicks","required":true},{"constraint":"","kind":"RationalNs","name":"remainder","required":true},{"constraint":"","kind":"Digest","name":"decisionDigest","required":true},{"constraint":"the commit acknowledgment","kind":"U64","name":"committedStep","required":true}],"name":"TraceAgent","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"Scope","name":"scope","required":true},{"constraint":"sorted by agentId, independent of dispatch order","kind":"array","name":"agents","required":true},{"constraint":"","kind":"Id","name":"batchId","required":true},{"constraint":"","kind":"Digest","name":"controlDigest","required":true},{"constraint":"the world boundary the environment acknowledged","kind":"U64","name":"acknowledgedBoundary","required":true},{"constraint":"sorted by viewId","kind":"array<{viewId:Id,producedStep:U64}>","name":"observationBoundaries","required":true},{"constraint":"task outcome ids in task order","kind":"array","name":"outcomeIds","required":true},{"constraint":"task event ids in task order","kind":"array","name":"eventIds","required":true},{"constraint":"","kind":"U64","name":"publishedBoundary","required":true}],"name":"TraceBehaviour","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"U64","name":"wallTimeNs","required":true},{"constraint":"","kind":"array<{agentId:Id,requestId:DomainRequestId}>","name":"prepareRequestIds","required":true},{"constraint":"","kind":"DomainRequestId","name":"advanceRequestId","required":true},{"constraint":"","kind":"array<{agentId:Id,requestId:DomainRequestId}>","name":"commitRequestIds","required":true},{"constraint":"call-","kind":"array","name":"busCallIds","required":true},{"constraint":"dlv- or own-","kind":"array","name":"deliveryIds","required":true}],"name":"TraceOperational","source":"step-v1 8"},{"fields":[{"constraint":"compared between runs","kind":"TraceBehaviour","name":"behaviour","required":true},{"constraint":"recorded, never compared: wall time and transport identities","kind":"TraceOperational","name":"operational","required":true}],"name":"TransitionTrace","source":"step-v1 8"},{"fields":[{"constraint":"","kind":"SchemaRef","name":"schema","required":true},{"constraint":"canonical JSON of the whole TypedValue <= 32768 bytes","kind":"object","name":"value","required":true}],"name":"TypedValue","source":"ipc-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"viewId","required":true},{"constraint":"1..=4096","kind":"int","name":"width","required":true},{"constraint":"1..=4096","kind":"int","name":"height","required":true},{"constraint":"","kind":"ViewFormat","name":"format","required":true},{"constraint":"exactly 4 x width","kind":"int","name":"rowStride","required":true},{"constraint":"positive integers <= 65535","kind":"{numerator:int,denominator:int}","name":"pixelAspect","required":true},{"constraint":"0..=8","kind":"int","name":"observationDelaySteps","required":true}],"name":"ViewDescriptor","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"Id","name":"viewId","required":true},{"constraint":"max(0, boundary - observationDelaySteps) for required sensory views","kind":"U64","name":"producedStep","required":true},{"constraint":"listed attachment; byteLength == rowStride x height","kind":"ArtifactRef","name":"pixels","required":true}],"name":"ViewRef","source":"state-media-v1 2"},{"fields":[{"constraint":"","kind":"U64","name":"boundary","required":true},{"constraint":"logical time since episode start","kind":"RationalNs","name":"worldTime","required":true},{"constraint":"<= 64 characters","kind":"string|null","name":"engineFrame","required":false},{"constraint":"<= 8, unique viewId","kind":"array","name":"sensoryViews","required":true},{"constraint":"the descriptor's inspectionSchema","kind":"TypedValue","name":"inspection","required":true},{"constraint":"<= 8, unique viewId","kind":"array","name":"broadcastViews","required":true},{"constraint":"unique streamId","kind":"array","name":"audio","required":true}],"name":"WorldObservation","source":"workers-v1 3"}],"version":1} diff --git a/services/flysim/crates/fly-session-types/fixtures/valid.json b/services/flysim/crates/fly-session-types/fixtures/valid.json index 814d738..8f13023 100644 --- a/services/flysim/crates/fly-session-types/fixtures/valid.json +++ b/services/flysim/crates/fly-session-types/fixtures/valid.json @@ -2611,6 +2611,100 @@ "canonical": "{\"agents\":[{\"agentId\":\"fly-a\",\"appliedControls\":{\"axes\":[{\"id\":\"stick-x\",\"value\":0},{\"id\":\"trigger\",\"value\":0}],\"buttons\":[{\"down\":true,\"id\":\"a\"},{\"down\":false,\"id\":\"b\"},{\"down\":false,\"id\":\"start\"},{\"down\":false,\"id\":\"select\"},{\"down\":false,\"id\":\"up\"},{\"down\":false,\"id\":\"down\"},{\"down\":false,\"id\":\"left\"},{\"down\":false,\"id\":\"right\"}],\"portId\":\"port-1\"},\"selectedDecision\":{\"schema\":{\"digest\":\"e70451b26fb5462ec63729d8355912dad01c3963a95bec0e6a8452375717ad9d\",\"id\":\"gameboy.intent.v1\",\"version\":1},\"value\":{\"press\":\"a\"}},\"telemetry\":{\"brainTicks\":\"2534\",\"learning\":{\"changed\":\"2\",\"enabled\":true,\"signal\":0.5,\"updates\":\"4\"},\"populationRateHz\":12.5,\"rates\":[{\"hz\":3.25,\"roleId\":\"kenyon\"},{\"hz\":0,\"roleId\":\"mbon\"}]}}],\"descriptorRevision\":\"7\",\"episodeId\":\"episode-1\",\"eventIds\":[\"evt-1\"],\"media\":{\"audio\":[{\"discontinuity\":false,\"firstSample\":\"33600\",\"sampleFrames\":800,\"samples\":{\"artifactId\":\"audio-1\",\"byteLength\":\"6400\",\"contentType\":\"audio/x-f32le\",\"digest\":null,\"generation\":\"1\",\"storeId\":\"store-1\"},\"streamId\":\"mix\"}],\"views\":[{\"pixels\":{\"artifactId\":\"frame-1\",\"byteLength\":\"92160\",\"contentType\":\"image/x-rgba8\",\"digest\":null,\"generation\":\"1\",\"storeId\":\"store-1\"},\"producedStep\":\"42\",\"viewId\":\"screen\"}]},\"progress\":{\"schema\":{\"digest\":\"80fa973157f334bb3208dfff6f7a2f298e79f73f07c72c33f5725975c6cff8e1\",\"id\":\"pokemon.progress.v1\",\"version\":1},\"value\":{\"rank\":10}},\"publisherIncarnation\":\"pub-1\",\"scope\":{\"epoch\":\"epoch-1\",\"sessionId\":\"demo\",\"step\":\"42\"},\"sequence\":\"42\",\"worldTime\":{\"denominator\":\"1\",\"numerator\":\"700000000\"}}", "digest": "d5ed83ccb866318b3cf17b53c8a5b1dea2f404b10649f6b640bf5f2b88a31435" }, + { + "name": "committed snapshot at a boundary installed by a restore", + "type": "CommittedSnapshot", + "value": { + "descriptorRevision": "7", + "publisherIncarnation": "pub-1", + "scope": { + "sessionId": "demo", + "epoch": "epoch-1", + "step": "42" + }, + "episodeId": "episode-1", + "sequence": "42", + "worldTime": { + "numerator": "700000000", + "denominator": "1" + }, + "agents": [ + { + "agentId": "fly-a", + "telemetry": { + "brainTicks": "2534", + "populationRateHz": 12.5, + "rates": [ + { + "roleId": "kenyon", + "hz": 3.25 + }, + { + "roleId": "mbon", + "hz": 0.0 + } + ], + "learning": { + "enabled": true, + "updates": "4", + "changed": "2", + "signal": 0.5 + } + }, + "selectedDecision": null, + "appliedControls": null + } + ], + "progress": { + "schema": { + "id": "pokemon.progress.v1", + "version": 1, + "digest": "80fa973157f334bb3208dfff6f7a2f298e79f73f07c72c33f5725975c6cff8e1" + }, + "value": { + "rank": 10 + } + }, + "media": { + "views": [ + { + "viewId": "screen", + "producedStep": "42", + "pixels": { + "storeId": "store-1", + "artifactId": "frame-1", + "generation": "1", + "byteLength": "92160", + "contentType": "image/x-rgba8", + "digest": null + } + } + ], + "audio": [ + { + "streamId": "mix", + "firstSample": "33600", + "sampleFrames": 800, + "samples": { + "storeId": "store-1", + "artifactId": "audio-1", + "generation": "1", + "byteLength": "6400", + "contentType": "audio/x-f32le", + "digest": null + }, + "discontinuity": false + } + ] + }, + "eventIds": [ + "evt-1" + ] + }, + "note": "a restore re-establishes a committed boundary this epoch did not run a transition into", + "canonical": "{\"agents\":[{\"agentId\":\"fly-a\",\"appliedControls\":null,\"selectedDecision\":null,\"telemetry\":{\"brainTicks\":\"2534\",\"learning\":{\"changed\":\"2\",\"enabled\":true,\"signal\":0.5,\"updates\":\"4\"},\"populationRateHz\":12.5,\"rates\":[{\"hz\":3.25,\"roleId\":\"kenyon\"},{\"hz\":0,\"roleId\":\"mbon\"}]}}],\"descriptorRevision\":\"7\",\"episodeId\":\"episode-1\",\"eventIds\":[\"evt-1\"],\"media\":{\"audio\":[{\"discontinuity\":false,\"firstSample\":\"33600\",\"sampleFrames\":800,\"samples\":{\"artifactId\":\"audio-1\",\"byteLength\":\"6400\",\"contentType\":\"audio/x-f32le\",\"digest\":null,\"generation\":\"1\",\"storeId\":\"store-1\"},\"streamId\":\"mix\"}],\"views\":[{\"pixels\":{\"artifactId\":\"frame-1\",\"byteLength\":\"92160\",\"contentType\":\"image/x-rgba8\",\"digest\":null,\"generation\":\"1\",\"storeId\":\"store-1\"},\"producedStep\":\"42\",\"viewId\":\"screen\"}]},\"progress\":{\"schema\":{\"digest\":\"80fa973157f334bb3208dfff6f7a2f298e79f73f07c72c33f5725975c6cff8e1\",\"id\":\"pokemon.progress.v1\",\"version\":1},\"value\":{\"rank\":10}},\"publisherIncarnation\":\"pub-1\",\"scope\":{\"epoch\":\"epoch-1\",\"sessionId\":\"demo\",\"step\":\"42\"},\"sequence\":\"42\",\"worldTime\":{\"denominator\":\"1\",\"numerator\":\"700000000\"}}", + "digest": "79a0ef10ed65c0720c34c7d99de1fe87ec2f0c26028994562cc4655b0f6acd1f" + }, { "name": "transition trace", "type": "TransitionTrace", diff --git a/services/flysim/crates/fly-session-types/src/publishing.rs b/services/flysim/crates/fly-session-types/src/publishing.rs index 562c913..10ba25c 100644 --- a/services/flysim/crates/fly-session-types/src/publishing.rs +++ b/services/flysim/crates/fly-session-types/src/publishing.rs @@ -414,7 +414,8 @@ impl DomainType for CommittedSnapshot { controls.validate()?; } // "Decisions/controls describe the transition ending at that boundary, null at - // initial boundary 0." (publishing-v1 section 3) + // initial boundary 0." (publishing-v1 section 3, and its 2026-09-22 amendment for + // a boundary that was installed rather than produced.) if self.scope.step == 0 && (agent.selected_decision.is_some() || agent.applied_controls.is_some()) { @@ -422,14 +423,24 @@ impl DomainType for CommittedSnapshot { "CommittedSnapshot: at boundary 0 selectedDecision and appliedControls are null", ); } - if self.scope.step > 0 - && (agent.selected_decision.is_none() || agent.applied_controls.is_none()) - { + if agent.selected_decision.is_some() != agent.applied_controls.is_some() { return err( - "CommittedSnapshot: past boundary 0 every agent has a decision and applied controls", + "CommittedSnapshot: selectedDecision and appliedControls are null together or present together", ); } } + // A boundary is produced by a transition or installed by one, and the whole snapshot + // says which: every agent carries the transition that ended here, or none does. A + // mixture would be one fly's action beside another fly's silence at the same boundary. + if self + .agents + .iter() + .any(|a| a.selected_decision.is_some() != self.agents[0].selected_decision.is_some()) + { + return err( + "CommittedSnapshot: either every agent carries the transition that ended here, or none does", + ); + } self.progress.validate()?; if self.views.len() > MAX_VIEWS { return err("CommittedSnapshot: at most 8 views"); diff --git a/services/flysim/crates/fly-session-types/src/schema.rs b/services/flysim/crates/fly-session-types/src/schema.rs index 44eb58f..57ed6a7 100644 --- a/services/flysim/crates/fly-session-types/src/schema.rs +++ b/services/flysim/crates/fly-session-types/src/schema.rs @@ -940,12 +940,12 @@ pub const SCHEMAS: &[TypeSchema] = &[ opt( "selectedDecision", "TypedValue|null", - "null exactly at boundary 0", + "null at boundary 0 and at an installed boundary; null or present for every agent together", ), opt( "appliedControls", "PortControl|null", - "null exactly at boundary 0; the agent's assigned port", + "null with selectedDecision; the agent's assigned port", ), ], }, diff --git a/services/flysim/crates/fly-session/src/coordinator.rs b/services/flysim/crates/fly-session/src/coordinator.rs index 76db2e4..e6e34fa 100644 --- a/services/flysim/crates/fly-session/src/coordinator.rs +++ b/services/flysim/crates/fly-session/src/coordinator.rs @@ -321,9 +321,11 @@ pub struct Coordinator { /// The publication boundary. Everything this session publishes goes through it, and /// every outcome it returns is a named one. publisher: crate::publish::Publisher, - /// The composition as published. Built once from what the live participants attested to, + /// The composition as published. Built from what the live participants attested to, /// never restated from the configuration that asked for them. session_descriptor: Option, + /// The revision the next descriptor publication carries. + descriptor_revision: u64, /// The read-only repair service. Held so it stops with the session. query: Option, pacing: Option, @@ -417,6 +419,7 @@ impl Coordinator { serials: Serials::default(), publisher, session_descriptor: None, + descriptor_revision: DESCRIPTOR_REVISION, query: None, topics, pacing: None, @@ -463,6 +466,11 @@ impl Coordinator { self.session_descriptor.as_ref() } + /// The revision the last published descriptor carried. + pub fn descriptor_revision(&self) -> u64 { + self.descriptor_revision + } + /// What this session published and what became of it: accepted, refused by an observer, /// or faulted, per topic. pub fn ledger(&self) -> &crate::publish::Ledger { @@ -2651,11 +2659,28 @@ impl Coordinator { } /// Publishes the composition and starts the read-only repair service beside it. + /// Publishes the composition, advancing the revision when the composition changed. + /// + /// A revision identifies a composition, so republishing an unchanged one keeps its number + /// and a changed one takes the next: a group restore establishes a fresh epoch, which is a + /// new `compositionDigest`, and a consumer that held the old revision has to be told rather + /// than handed the same number with different contents. The publisher refuses the second + /// case outright, so this is where the number moves. async fn publish_descriptor(&mut self) -> Outcome<()> { - let descriptor = match self.build_descriptor(DESCRIPTOR_REVISION) { + let mut descriptor = match self.build_descriptor(self.descriptor_revision) { Ok(descriptor) => descriptor, Err(e) => return Err(self.fail_now(e, "descriptor")), }; + if let Some(published) = &self.session_descriptor { + let mut same = descriptor.clone(); + same.revision = published.revision; + if same != *published { + self.descriptor_revision += 1; + descriptor.revision = self.descriptor_revision; + self.audit + .push(format!("descriptor-revision:{}", self.descriptor_revision)); + } + } let outcome = match self.publisher.publish_descriptor(&descriptor).await { Ok(outcome) => outcome, Err(e) => return Err(self.fail_now(e, "descriptor")), diff --git a/services/flysim/crates/fly-session/tests/publishing.rs b/services/flysim/crates/fly-session/tests/publishing.rs index a600717..1440e35 100644 --- a/services/flysim/crates/fly-session/tests/publishing.rs +++ b/services/flysim/crates/fly-session/tests/publishing.rs @@ -38,6 +38,7 @@ both_transports!( the_query_service_answers_reads_and_nothing_else, the_published_descriptor_is_what_the_workers_attested_to, a_stimulus_kind_the_descriptor_does_not_declare_is_refused, + a_restored_boundary_publishes_a_new_revision_and_no_transition, ); all_modes!( @@ -787,6 +788,62 @@ async fn one_snapshot_carries_every_agent_in_the_composition(via: Via) { f.shutdown().await; } +/// A group restore re-establishes a committed boundary this epoch did not run a transition +/// into. Two things follow, and both are published rather than inferred: the composition is a +/// new one, because a fresh epoch is a new `compositionDigest`, so the descriptor takes the +/// next revision; and the restored boundary carries no decision and no controls for any agent, +/// because the abandoned epoch's actions are not this session's to republish. +async fn a_restored_boundary_publishes_a_new_revision_and_no_transition(via: Via) { + let mut f = started(via).await; + let checkpoint = id("ck-1"); + f.harness.coordinator.run(1).await.expect("one transition"); + let outcome = within("checkpoint", f.harness.coordinator.checkpoint(&checkpoint)) + .await + .expect("a committed checkpoint"); + assert!(matches!(outcome, fly_session::state::SaveOutcome::Committed { .. }), "{outcome:?}"); + let first = f.harness.coordinator.descriptor_revision(); + + // The snapshot of a boundary this epoch produced does carry the transition. + let produced = { + let state = f.harness.coordinator.published_state(); + let state = state.lock().expect("not poisoned"); + state.latest_snapshot().expect("boundary 1").clone() + }; + assert_eq!(produced.scope.step, 1); + assert!(produced.agents.iter().all(|a| a.selected_decision.is_some())); + + // Fail the epoch and restore into a fresh one. + f.harness.kill(&fly_a()).await; + f.harness.coordinator.step().await.expect_err("a dead participant fails the epoch"); + within("replace", f.harness.replace_all_participants()) + .await + .expect("replacements"); + within( + "restore", + f.harness.coordinator.restore(Some(&checkpoint), &id("e2")), + ) + .await + .expect("a coherent group restore"); + + assert_eq!( + f.harness.coordinator.descriptor_revision(), + first + 1, + "a fresh epoch is a new composition, so the revision advanced" + ); + let restored = { + let state = f.harness.coordinator.published_state(); + let state = state.lock().expect("not poisoned"); + state.latest_snapshot().expect("the restored boundary").clone() + }; + assert_eq!(restored.scope.step, 1, "the same committed boundary"); + assert_eq!(restored.descriptor_revision, first + 1); + assert!( + restored.agents.iter().all(|a| a.selected_decision.is_none() && a.applied_controls.is_none()), + "an installed boundary carries no transition for any agent" + ); + f.shutdown().await; +} + // ------------------------------------------------------------------------------------------ // Application-owned state and cues, bounded events, and the read-only repair service. From 6bf5687aca91bd1d223b844f783bbd0670e4598c Mon Sep 17 00:00:00 2001 From: dev Date: Tue, 22 Sep 2026 20:18:49 +0000 Subject: [PATCH 4/6] session: a snapshot publishes the telemetry of the transition that just ended AgentSlot.telemetry was written only by the Agent.Initialize handler, so every CommittedSnapshot carried warm-up telemetry labelled as boundary k while each AgentCommitResult.telemetry was validated and dropped. The commit result is now stored on the slot beside the committed step, and the media test asserts that published telemetry advances across boundaries instead of merely being nonzero. A refused snapshot is recorded and sequenced like a refused descriptor revision, because the repair path exists for the consumer that did not receive it; the sequence advances with the value rather than with the delivery, so two snapshots can never share one. The query service counts an answer it could not deliver rather than discarding the result, and an unreadable event batch is distinct from the end of the stream. --- .../crates/fly-session/src/coordinator.rs | 24 +++++ .../flysim/crates/fly-session/src/harness.rs | 16 ++++ .../flysim/crates/fly-session/src/publish.rs | 36 +++++-- .../crates/fly-session/tests/publishing.rs | 95 ++++++++++++++++--- 4 files changed, 152 insertions(+), 19 deletions(-) diff --git a/services/flysim/crates/fly-session/src/coordinator.rs b/services/flysim/crates/fly-session/src/coordinator.rs index e6e34fa..8b7a444 100644 --- a/services/flysim/crates/fly-session/src/coordinator.rs +++ b/services/flysim/crates/fly-session/src/coordinator.rs @@ -471,6 +471,11 @@ impl Coordinator { self.descriptor_revision } + /// The sequence the next published snapshot will carry. + pub fn published_sequence(&self) -> u64 { + self.publisher.sequence() + } + /// What this session published and what became of it: accepted, refused by an observer, /// or faulted, per topic. pub fn ledger(&self) -> &crate::publish::Ledger { @@ -1656,6 +1661,25 @@ impl Coordinator { self.agents[index].context_digest = context.digest(); self.agents[index].context = context; self.agents[index].committed_step = k + 1; + // The telemetry of the transition that just ended, which is what this boundary's + // snapshot publishes. Without this the slot would keep whatever `Agent.Initialize` + // reported and every snapshot would label warm-up telemetry as boundary k. + let telemetry = commits + .iter() + .find(|(id, _)| *id == agent_id) + .map(|(_, result)| result.telemetry.clone()); + match telemetry { + Some(telemetry) => self.agents[index].telemetry = Some(telemetry), + None => { + return Err(self.fail_now( + DomainError::before( + ErrorCode::IdentityMismatch, + format!("agent {agent_id} committed without telemetry"), + ), + "commit", + )); + } + } } // The previous boundary's handles are no longer needed; the new ones take over. // The references -- which are data, not ownership -- are kept for one boundary, so a diff --git a/services/flysim/crates/fly-session/src/harness.rs b/services/flysim/crates/fly-session/src/harness.rs index 334e6ad..f86a179 100644 --- a/services/flysim/crates/fly-session/src/harness.rs +++ b/services/flysim/crates/fly-session/src/harness.rs @@ -649,6 +649,22 @@ impl SessionHarness { } } + /// Changes which graph one agent builds, so the replacement the next restart launches is + /// a fly with the same neuron count and another index. + /// + /// The same relaunch rule as a fault: the worker running now keeps what it was started + /// with, and the change reaches the composition through the next replacement. + pub fn set_agent_graph(&mut self, agent_id: &Id, graph_variant: u64) { + if let Some(spec) = self + .config + .agents + .iter_mut() + .find(|spec| spec.agent_id == *agent_id) + { + spec.graph_variant = graph_variant; + } + } + /// Changes the environment's injected faults, with the same relaunch rule. pub fn set_environment_faults(&mut self, faults: EnvironmentFaults) { self.config.environment_faults = faults; diff --git a/services/flysim/crates/fly-session/src/publish.rs b/services/flysim/crates/fly-session/src/publish.rs index 20f0200..67444a6 100644 --- a/services/flysim/crates/fly-session/src/publish.rs +++ b/services/flysim/crates/fly-session/src/publish.rs @@ -644,10 +644,13 @@ impl Publisher { .publish(&topic, object(snapshot.to_json()), &refs) .await, ); - if outcome.is_accepted() { - self.sequence += 1; - lock(&self.state).latest = Some(snapshot.clone()); - } + // The value is recorded whether or not the router admitted it, exactly as a descriptor + // revision is: the repair path exists for the consumer that did not receive it, and the + // `state-media-v1` amendment promises the exact value stays recoverable through the + // query service. The sequence advances with the value rather than with the delivery, + // so two different snapshots can never share one sequence number. + self.sequence += 1; + lock(&self.state).latest = Some(snapshot.clone()); self.ledger.record(&outcome); Ok(outcome) } @@ -718,6 +721,10 @@ pub const GET_SNAPSHOT: &str = "Session.GetSnapshot"; /// advance, pause, stimulate, restore or reconfigure anything. pub struct QueryService { task: tokio::task::JoinHandle<()>, + /// Answers this service produced and could not deliver, because the caller was gone or + /// the router refused the reply. A read that nobody received is not a read that happened, + /// and this module drops nothing silently. + undeliverable: Arc, } impl QueryService { @@ -736,6 +743,8 @@ impl QueryService { // than a fallback name that two incarnations could share. let incarnation = parse_id(&format!("query-{}", client.info().connection_id)) .map_err(|e| flybus::BusError::new(flybus::ErrorCode::InvalidEnvelope, e))?; + let undeliverable = Arc::new(std::sync::atomic::AtomicU64::new(0)); + let undelivered = Arc::clone(&undeliverable); let task = tokio::spawn(async move { while let Some(request) = service.next().await { let method = request.method().to_owned(); @@ -753,10 +762,17 @@ impl QueryService { DomainError::invalid(format!("{method}: {e}")), ), }; - let _ = responder.reply(outcome.to_outcome(), &[]).await; + if responder.reply(outcome.to_outcome(), &[]).await.is_err() { + undelivered.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + } } }); - Ok(QueryService { task }) + Ok(QueryService { task, undeliverable }) + } + + /// How many answers this service could not deliver. + pub fn undeliverable(&self) -> u64 { + self.undeliverable.load(std::sync::atomic::Ordering::SeqCst) } /// Ends the service. Dropping one does the same thing. @@ -1003,6 +1019,14 @@ pub struct EventBatchView { pub event_ids: Vec, } +/// What one poll of the event stream produced. +#[derive(Clone, Debug, PartialEq)] +pub enum ConsumerEvents { + Batch(EventBatchView), + /// A batch this consumer could not read. Distinct from the end of the stream. + Unreadable { detail: String }, +} + /// What one poll of a consumer produced. #[derive(Clone, Debug, PartialEq)] pub enum ConsumerOutcome { diff --git a/services/flysim/crates/fly-session/tests/publishing.rs b/services/flysim/crates/fly-session/tests/publishing.rs index 1440e35..bb94a1c 100644 --- a/services/flysim/crates/fly-session/tests/publishing.rs +++ b/services/flysim/crates/fly-session/tests/publishing.rs @@ -36,6 +36,7 @@ both_transports!( application_state_and_cues_are_the_applications_own, a_refused_event_batch_is_held_and_counted_not_lost, the_query_service_answers_reads_and_nothing_else, + a_refused_snapshot_is_still_what_the_repair_path_answers, the_published_descriptor_is_what_the_workers_attested_to, a_stimulus_kind_the_descriptor_does_not_declare_is_refused, a_restored_boundary_publishes_a_new_revision_and_no_transition, @@ -252,7 +253,6 @@ async fn a_bounded_observer_refusal_is_named_and_never_fails_the_epoch(via: Via) .consumer() .await .expect("a healthy consumer attaches"); - until("the refusals stop", || true).await; f.harness .coordinator .run(2) @@ -284,6 +284,7 @@ async fn every_published_snapshot_carries_its_own_boundarys_media(via: Via) { Some(ConsumerOutcome::Composition { .. }) )); let mut seen: Vec<(u64, String)> = Vec::new(); + let mut ticks: BTreeMap = BTreeMap::new(); for step in 1..=STEPS { f.harness.coordinator.run(1).await.expect("a transition"); let at = read_until(&mut consumer, step).await; @@ -305,10 +306,20 @@ async fn every_published_snapshot_carries_its_own_boundarys_media(via: Via) { seen.push((at, reference.pixels.artifact_id.clone())); } for agent in &view.agents { - assert!( - agent.brain_ticks > 0, - "the agent state is this boundary's, not a placeholder" - ); + let previous = ticks.insert(agent.agent_id.clone(), agent.brain_ticks); + match previous { + None => assert!(agent.brain_ticks > 0, "the agent has run by boundary {at}"), + // Telemetry is the state of the transition that ended here. Republishing the + // previous boundary's numbers -- or `Agent.Initialize`'s warm-up numbers -- + // would be old agent state labelled as this boundary, which is the same + // mislabelling as old media. + Some(before) => assert!( + agent.brain_ticks > before, + "the telemetry of {} advanced into boundary {at}: {before} -> {}", + agent.agent_id, + agent.brain_ticks + ), + } } } let mut ids: Vec<&String> = seen.iter().map(|(_, id)| id).collect(); @@ -460,6 +471,15 @@ async fn a_revision_that_was_never_published_is_a_named_answer(via: Via) { /// The same neuron count, another index. A consumer that has mapped geometry is told, and the /// new composition is not quietly cached over the one it mapped. +/// +/// The two descriptors here come from two real sessions rather than from a restore. Driving it +/// through a restore was tried and does not work yet, for a reason outside this slice: the +/// coordinator's `AgentSlot.graph` is written only by `Agent.Initialize`, and a group restore +/// installs state through `State.ActivateRestore`, so a replacement fly that built another +/// index is published under its predecessor's `indexDigest` -- and it is not refused on the way +/// in either, because `agent_compatibility` digests `agent::dataset_digest()` rather than the +/// index the worker attested to. Both halves belong to the restore contract, so this test uses +/// the compositions it can build honestly and the gap is reported rather than papered over. async fn a_changed_index_digest_is_named_rather_than_remapped(via: Via) { // Two real compositions that differ only in the graph one fly built. let first = started(via).await; @@ -516,19 +536,14 @@ async fn a_changed_index_digest_is_named_rather_than_remapped(via: Via) { moved.neuron_count, arrived.neuron_count, "the same number of neurons" ); - assert_ne!( - moved.index_digest, arrived.index_digest, - "and another index" - ); + assert_ne!(moved.index_digest, arrived.index_digest, "and another index"); let mut consumer = first.harness.consumer().await.expect("a consumer attaches"); let held = match within("the descriptor", consumer.take_descriptor()).await { Some(ConsumerOutcome::Composition { revision, .. }) => revision, other => panic!("expected a composition, got {other:?}"), }; - consumer - .map_geometry(held) - .expect("this consumer maps geometry"); + consumer.map_geometry(held).expect("this consumer maps geometry"); assert_eq!(consumer.mapped_index(&fly_a()), Some(&moved.index_digest)); // The composition changes under it. @@ -966,6 +981,52 @@ async fn a_refused_event_batch_is_held_and_counted_not_lost(via: Via) { f.shutdown().await; } +/// A publication an observer refused is exactly the value the repair path exists to hand back. +/// +/// The `state-media-v1` amendment promises the exact value stays recoverable through the query +/// path, so recording it cannot depend on whether the router admitted the delivery -- that is +/// the case the promise is about. +async fn a_refused_snapshot_is_still_what_the_repair_path_answers(via: Via) { + let mut f = started(via).await; + let snapshots = f.harness.coordinator.topics().snapshots.clone(); + let offender_client = f.harness.observer().await.expect("an observer client"); + let _offender = offender_client + .subscribe( + &snapshots, + flybus::SubscriptionConfig::bounded().queued(1).in_flight(1), + ) + .await + .expect("a bounded subscription"); + + f.harness.coordinator.run(STEPS).await.expect("the world carries on"); + let counters = f.harness.coordinator.ledger().counters(&snapshots); + assert!(counters.refused > 0, "a refusal is what this test is about: {counters:?}"); + + let latest = { + let state = f.harness.coordinator.published_state(); + let state = state.lock().expect("not poisoned"); + state.latest_snapshot().expect("a snapshot").clone() + }; + assert_eq!( + latest.scope.step, STEPS, + "the newest committed boundary is recoverable even though its delivery was refused" + ); + assert_eq!( + latest.sequence + 1, + f.harness.coordinator.published_sequence(), + "the sequence advanced with the value, not with the delivery" + ); + + // And the query service answers it over the bus, not just the state behind it. + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + let answered = within("the repair path", consumer.repair(DESCRIPTOR_REVISION)) + .await + .expect("the repair path answers"); + assert_eq!(answered, DESCRIPTOR_REVISION); + offender_client.close().await; + f.shutdown().await; +} + /// Two reads and nothing else. There is no method on this service that could move anything. async fn the_query_service_answers_reads_and_nothing_else(via: Via) { let mut f = started(via).await; @@ -990,7 +1051,15 @@ async fn the_query_service_answers_reads_and_nothing_else(via: Via) { let snapshot = CommittedSnapshot::from_json(outcome_result(&outcome).expect("a result")) .expect("a committed snapshot"); assert_eq!(snapshot.scope.step, 1); - assert!(snapshot.views.is_empty() || !snapshot.views.is_empty()); + assert_eq!( + snapshot.descriptor_revision, + f.harness.coordinator.descriptor_revision(), + "a read answers the composition the session is publishing under" + ); + assert!( + snapshot.agents.iter().all(|a| a.selected_decision.is_some()), + "and the values of the transition that ended at it" + ); // A method it does not implement is a named refusal, not a default. let request = SessionRpcRequest { From 305a50d8df1310f102a340f1b1234e23770a2856 Mon Sep 17 00:00:00 2001 From: dev Date: Tue, 22 Sep 2026 20:36:51 +0000 Subject: [PATCH 5/6] session: a graph identity does not cross a recovery The index an agent attested to at Agent.Initialize joins its compatibility identity and its checkpoint manifest row, so a replacement fly that built another graph -- the same dataset, the same neuron count, another index -- cannot install a checkpoint taken under the first one. It was accepted before, because agent_compatibility digested a dataset digest recomputed from a free function rather than what the worker attested to, and the restored composition was then published under its predecessor's indexDigest. The refusal names what the worker is, not just that two digests differ. Dated amendment to checkpoint-envelope-v1's agents row; no wire type and no schema text change, so the contract digest is unchanged. --- .../checkpoint-envelope-v1.md | 10 ++- .../flysim/crates/fly-session/src/agent.rs | 22 ++++-- .../crates/fly-session/src/coordinator.rs | 49 +++++++++++-- .../flysim/crates/fly-session/src/state.rs | 6 ++ .../crates/fly-session/tests/publishing.rs | 68 ++++++++++++++++--- 5 files changed, 136 insertions(+), 19 deletions(-) diff --git a/docs/design/session-framework/checkpoint-envelope-v1.md b/docs/design/session-framework/checkpoint-envelope-v1.md index d58ed98..69bdf4d 100644 --- a/docs/design/session-framework/checkpoint-envelope-v1.md +++ b/docs/design/session-framework/checkpoint-envelope-v1.md @@ -100,11 +100,19 @@ names; a manifest missing any of them is not a complete checkpoint. | `compositionDigest` | Coordinator scheduler and configuration identity | | `portMap` | The exact port-to-agent map, `[{portId, agentId}]` | | `compatibility` | Backend, content, patch, controller, parser and state-format identities | -| `agents` | Per agent: profile, dataset and model identities, resolved seed, tick count, remainder and the payload name holding its state | +| `agents` | Per agent: profile, dataset, **index** and model identities, resolved seed, tick count, remainder and the payload name holding its state | | `coordinator` | Task ledger, prior world inspection, per-agent executor state, admission state and event watermarks, each as a payload name or an inline value | | `helperState` | External-helper state required for exact resume, as payload names | | `payloads` | `[{name, byteLength, digest}]`, mirroring the payload table | +**Amendment, 2026-09-22 (PUBLISH-01).** The `agents` row gains `indexDigest`, the index the +agent attested to at `Agent.Initialize`, and it joins that agent's compatibility identity. +Without it a replacement fly that built another graph -- the same dataset, the same neuron +count, another index -- passed the group check and was then published under its predecessor's +`indexDigest`, which is a graph identity crossing a recovery and exactly what section 5's rules +exist to prevent. It is recorded from the worker's attestation rather than recomputed from the +dataset, because the point is that the two can disagree. + **Amendment, 2026-09-22 (STATE-01).** The table above names a holder for every payload except the environment's own, although section 6's fixture has one (`world`) and a group install has to map it by name like any other participant's. The manifest therefore also records: diff --git a/services/flysim/crates/fly-session/src/agent.rs b/services/flysim/crates/fly-session/src/agent.rs index 21ca1a5..ab2e1a1 100644 --- a/services/flysim/crates/fly-session/src/agent.rs +++ b/services/flysim/crates/fly-session/src/agent.rs @@ -844,6 +844,7 @@ pub fn agent_compatibility_digest( model_version: &str, plasticity_version: &str, seed: i32, + index_digest: &Digest, ) -> Digest { let value = serde_json::json!({ "agentId": agent_id.as_str(), @@ -852,6 +853,11 @@ pub fn agent_compatibility_digest( "modelVersion": model_version, "plasticityVersion": plasticity_version, "seed": seed, + // The index the worker actually built, not a value recomputed from the dataset: the + // whole point is that the two can disagree. Without it a replacement fly that built + // another graph restores cleanly and is then published under its predecessor's + // `indexDigest`, which is the predecessor's graph identity crossing a recovery. + "indexDigest": index_digest.as_str(), }); digest_of(&value).expect("an agent compatibility block canonicalizes") } @@ -940,13 +946,15 @@ struct StagedAgent { impl FakeAgentWorker { /// This worker's own compatibility identity, from its configuration and a resolved seed. fn compatibility_digest(&self, profile: &AssetRef, seed: i32) -> Digest { + let graph = synthetic_graph(&self.config.agent_id, self.config.graph_variant); agent_compatibility_digest( &self.config.agent_id, &profile.digest, - &dataset_digest(), + &graph.dataset_digest, MODEL_VERSION, PLASTICITY_VERSION, seed, + &graph.index_digest, ) } @@ -1147,10 +1155,16 @@ worker; this worker is {other:?}" // of another agent's brain, fails here and never reaches activation. let computed = self.compatibility_digest(&profile, model.seed()); if computed != params.compatibility_digest { + let graph = synthetic_graph(&self.config.agent_id, self.config.graph_variant); return Err(incompatible(format!( - "the staged state's compatibility {computed} is not the {} the restore \ -requires", - params.compatibility_digest + "the staged state's compatibility {} is not the {computed} this worker is: \ +profile {}, dataset {}, index {}, model {MODEL_VERSION}, plasticity {PLASTICITY_VERSION}, \ +seed {}", + params.compatibility_digest, + profile.digest, + graph.dataset_digest, + graph.index_digest, + model.seed() ))); } let accumulator_value = value diff --git a/services/flysim/crates/fly-session/src/coordinator.rs b/services/flysim/crates/fly-session/src/coordinator.rs index 8b7a444..7d68aa1 100644 --- a/services/flysim/crates/fly-session/src/coordinator.rs +++ b/services/flysim/crates/fly-session/src/coordinator.rs @@ -3036,15 +3036,33 @@ impl Coordinator { } } - fn agent_compatibility(&self, slot: &AgentSlot) -> Digest { - crate::agent::agent_compatibility_digest( + /// One agent's compatibility identity, from what that agent attested to at + /// `Agent.Initialize` rather than from anything this coordinator recomputed. + /// + /// The graph belongs here because `state-media-v1`'s recovery rules say old parser or + /// media state must not cross a recovery, and a graph identity is exactly that: without it + /// a replacement fly that built another index passes the group check and is then published + /// under its predecessor's `indexDigest`. An agent that has not attested yet has no + /// compatibility, which is a refusal rather than a guessed digest. + fn agent_compatibility(&self, slot: &AgentSlot) -> DomainResult { + let graph = slot.graph.as_ref().ok_or_else(|| { + DomainError::before( + ErrorCode::InvalidPhase, + format!( + "agent {} has not attested to a graph, so it has no compatibility identity", + slot.agent_id + ), + ) + })?; + Ok(crate::agent::agent_compatibility_digest( &slot.agent_id, &slot.profile.digest, - &crate::agent::dataset_digest(), + &graph.dataset_digest, crate::agent::MODEL_VERSION, crate::agent::PLASTICITY_VERSION, slot.seed, - ) + &graph.index_digest, + )) } /// The coordinator's own session record: what it must hold again to resume this boundary. @@ -3233,7 +3251,10 @@ impl Coordinator { for index in 0..self.agents.len() { let slot_worker = self.agents[index].worker.clone(); let agent_id = self.agents[index].agent_id.clone(); - let expected = self.agent_compatibility(&self.agents[index]); + let expected = match self.agent_compatibility(&self.agents[index]) { + Ok(expected) => expected, + Err(e) => return Err(self.fail_now(e, "capture")), + }; let reply = self .call( &slot_worker, @@ -3251,10 +3272,25 @@ impl Coordinator { payloads.push(payload); acknowledge.push((slot_worker, reply.request_id.clone())); let slot = &self.agents[index]; + // The graph identities come from what this agent attested to, not from a value + // the coordinator recomputed; that is the whole point of recording them. + let graph = match slot.graph.clone() { + Some(graph) => graph, + None => { + return Err(self.fail_now( + DomainError::before( + ErrorCode::InvalidPhase, + format!("agent {agent_id} has not attested to a graph"), + ), + "capture", + )); + } + }; agent_rows.push(crate::state::AgentEntry { agent_id: agent_id.clone(), profile_digest: slot.profile.digest.clone(), - dataset_digest: crate::agent::dataset_digest(), + dataset_digest: graph.dataset_digest, + index_digest: graph.index_digest, model_version: crate::agent::MODEL_VERSION.to_owned(), plasticity_version: crate::agent::PLASTICITY_VERSION.to_owned(), seed: slot.seed, @@ -3833,6 +3869,7 @@ impl Coordinator { &row.model_version, &row.plasticity_version, row.seed, + &row.index_digest, ), )); } diff --git a/services/flysim/crates/fly-session/src/state.rs b/services/flysim/crates/fly-session/src/state.rs index aa42d3c..220310f 100644 --- a/services/flysim/crates/fly-session/src/state.rs +++ b/services/flysim/crates/fly-session/src/state.rs @@ -599,6 +599,10 @@ pub struct AgentEntry { pub agent_id: Id, pub profile_digest: Digest, pub dataset_digest: Digest, + /// The index the agent attested to at `Agent.Initialize`. It is part of the agent's + /// compatibility identity, so a replacement that built another graph cannot install this + /// payload -- the graph identity does not cross the recovery. + pub index_digest: Digest, pub model_version: String, pub plasticity_version: String, pub seed: i32, @@ -613,6 +617,7 @@ impl AgentEntry { "agentId": self.agent_id.as_str(), "profileDigest": self.profile_digest.as_str(), "datasetDigest": self.dataset_digest.as_str(), + "indexDigest": self.index_digest.as_str(), "modelVersion": self.model_version.as_str(), "plasticityVersion": self.plasticity_version.as_str(), "seed": self.seed, @@ -646,6 +651,7 @@ impl AgentEntry { agent_id: parse_id(&text("agentId")?)?, profile_digest: text("profileDigest")?, dataset_digest: text("datasetDigest")?, + index_digest: text("indexDigest")?, model_version: text("modelVersion")?, plasticity_version: text("plasticityVersion")?, seed, diff --git a/services/flysim/crates/fly-session/tests/publishing.rs b/services/flysim/crates/fly-session/tests/publishing.rs index bb94a1c..12f7af1 100644 --- a/services/flysim/crates/fly-session/tests/publishing.rs +++ b/services/flysim/crates/fly-session/tests/publishing.rs @@ -37,6 +37,7 @@ both_transports!( a_refused_event_batch_is_held_and_counted_not_lost, the_query_service_answers_reads_and_nothing_else, a_refused_snapshot_is_still_what_the_repair_path_answers, + a_replacement_that_built_another_index_cannot_install_the_checkpoint, the_published_descriptor_is_what_the_workers_attested_to, a_stimulus_kind_the_descriptor_does_not_declare_is_refused, a_restored_boundary_publishes_a_new_revision_and_no_transition, @@ -469,17 +470,68 @@ async fn a_revision_that_was_never_published_is_a_named_answer(via: Via) { f.shutdown().await; } +/// A graph identity does not cross a recovery. +/// +/// The index a fly attested to at `Agent.Initialize` is part of its compatibility identity, so +/// a replacement that built another graph -- the same dataset, the same neuron count, another +/// index -- cannot install a checkpoint taken under the first one, and the refusal names what +/// this worker is rather than only that two digests differ. The identical replacement still +/// installs, so the check refuses the case it is about and nothing else. +async fn a_replacement_that_built_another_index_cannot_install_the_checkpoint(via: Via) { + for (variant, refused) in [(0u64, false), (1, true)] { + let mut f = started(via).await; + let checkpoint = id("ck-graph"); + f.harness.coordinator.run(1).await.expect("one transition"); + within("checkpoint", f.harness.coordinator.checkpoint(&checkpoint)) + .await + .expect("a committed checkpoint"); + + f.harness.set_agent_graph(&fly_a(), variant); + f.harness.kill(&fly_a()).await; + f.harness + .coordinator + .step() + .await + .expect_err("a dead participant fails the epoch"); + within("replace", f.harness.replace_all_participants()) + .await + .expect("replacements"); + let outcome = within( + "restore", + f.harness.coordinator.restore(Some(&checkpoint), &id("e2")), + ) + .await; + match (refused, outcome) { + (false, Ok(_)) => {} + (false, Err(e)) => panic!("the identical replacement must still install: {e}"), + (true, Ok(_)) => panic!("a fly that built another index installed the checkpoint"), + (true, Err(failure)) => { + assert_eq!(failure.error.code, ErrorCode::IncompatibleState, "{failure:?}"); + let built = fly_session::agent::synthetic_graph(&fly_a(), 1); + assert!( + failure.error.message.contains(&built.index_digest), + "the refusal names the index this worker built: {}", + failure.error.message + ); + assert!( + f.harness.coordinator.is_fenced(), + "and the group stays fenced" + ); + } + } + f.shutdown().await; + } +} + /// The same neuron count, another index. A consumer that has mapped geometry is told, and the /// new composition is not quietly cached over the one it mapped. /// -/// The two descriptors here come from two real sessions rather than from a restore. Driving it -/// through a restore was tried and does not work yet, for a reason outside this slice: the -/// coordinator's `AgentSlot.graph` is written only by `Agent.Initialize`, and a group restore -/// installs state through `State.ActivateRestore`, so a replacement fly that built another -/// index is published under its predecessor's `indexDigest` -- and it is not refused on the way -/// in either, because `agent_compatibility` digests `agent::dataset_digest()` rather than the -/// index the worker attested to. Both halves belong to the restore contract, so this test uses -/// the compositions it can build honestly and the gap is reported rather than papered over. +/// The two descriptors here come from two real sessions, and deliberately not from a restore: +/// since the index is part of an agent's compatibility identity, a restore whose replacement +/// built another graph is now refused before it can install, which +/// `a_replacement_that_built_another_index_cannot_install_the_checkpoint` proves. A published +/// index therefore changes between compositions rather than across a recovery, and this is that +/// case. async fn a_changed_index_digest_is_named_rather_than_remapped(via: Via) { // Two real compositions that differ only in the graph one fly built. let first = started(via).await; From 5e61c50728d03d345dcf7484883242483f196b94 Mon Sep 17 00:00:00 2001 From: dev Date: Tue, 22 Sep 2026 21:03:59 +0000 Subject: [PATCH 6/6] session: an unreadable event batch is not the end of the stream take_events kept returning Option and defaulting through the question-mark operator, so a batch missing a field read as end of stream and the ConsumerEvents enum added in the previous round described nothing. It returns Batch or Unreadable now, and a test publishes a batch with no droppedBefore, asserts it is reported as unreadable naming the field, and asserts the next real batch still reads. The checkpoint-envelope-v1 amendment cites the rule that lets a required manifest field land with envelopeVersion still 1 while no production file exists. --- .../checkpoint-envelope-v1.md | 4 +- .../flysim/crates/fly-session/src/publish.rs | 34 +++++++++--- .../crates/fly-session/tests/publishing.rs | 55 ++++++++++++++++++- 3 files changed, 81 insertions(+), 12 deletions(-) diff --git a/docs/design/session-framework/checkpoint-envelope-v1.md b/docs/design/session-framework/checkpoint-envelope-v1.md index 69bdf4d..6e5017a 100644 --- a/docs/design/session-framework/checkpoint-envelope-v1.md +++ b/docs/design/session-framework/checkpoint-envelope-v1.md @@ -111,7 +111,9 @@ Without it a replacement fly that built another graph -- the same dataset, the s count, another index -- passed the group check and was then published under its predecessor's `indexDigest`, which is a graph identity crossing a recovery and exactly what section 5's rules exist to prevent. It is recorded from the worker's attestation rather than recomputed from the -dataset, because the point is that the two can disagree. +dataset, because the point is that the two can disagree. `envelopeVersion` stays `1`, which the +required-manifest-field rule below allows only while no production `FLYSESS1` file exists; once +one does, adding a required manifest field must bump it. **Amendment, 2026-09-22 (STATE-01).** The table above names a holder for every payload except the environment's own, although section 6's fixture has one (`world`) and a group install has diff --git a/services/flysim/crates/fly-session/src/publish.rs b/services/flysim/crates/fly-session/src/publish.rs index 67444a6..8390298 100644 --- a/services/flysim/crates/fly-session/src/publish.rs +++ b/services/flysim/crates/fly-session/src/publish.rs @@ -1275,26 +1275,42 @@ impl PresentationConsumer { } /// Takes the next bounded event batch. - pub async fn take_events(&mut self) -> Option { + /// + /// `None` is the end of the stream and nothing else. A batch this consumer cannot read + /// comes back as [`ConsumerEvents::Unreadable`], because "there are no more events" and + /// "that one made no sense" are different facts and one value cannot carry both: a + /// consumer that saw the second as the first would stop reading a live stream. + pub async fn take_events(&mut self) -> Option { let message = self.events.next().await?; let payload = message.payload().clone(); drop(message); - let epoch = payload.get("epoch").and_then(Value::as_str).map(id)?; - let dropped_before = payload + let unreadable = |what: &str| { + Some(ConsumerEvents::Unreadable { + detail: format!("an event batch has no readable {what}"), + }) + }; + let Some(epoch) = payload.get("epoch").and_then(Value::as_str).map(id) else { + return unreadable("epoch"); + }; + let Some(dropped_before) = payload .get("droppedBefore") .and_then(Value::as_str) - .and_then(|t| t.parse::().ok())?; - let event_ids = payload - .get("events") - .and_then(Value::as_array)? + .and_then(|t| t.parse::().ok()) + else { + return unreadable("droppedBefore"); + }; + let Some(events) = payload.get("events").and_then(Value::as_array) else { + return unreadable("events array"); + }; + let event_ids = events .iter() .filter_map(|e| e.get("id").and_then(Value::as_str).map(str::to_owned)) .collect(); - Some(EventBatchView { + Some(ConsumerEvents::Batch(EventBatchView { epoch, dropped_before, event_ids, - }) + })) } /// Records that this consumer has mapped geometry against the agents of `revision`. diff --git a/services/flysim/crates/fly-session/tests/publishing.rs b/services/flysim/crates/fly-session/tests/publishing.rs index 12f7af1..b5e7b93 100644 --- a/services/flysim/crates/fly-session/tests/publishing.rs +++ b/services/flysim/crates/fly-session/tests/publishing.rs @@ -14,8 +14,9 @@ use common::{Fixture, default_fixture, fixture, fly_a, fly_b, mode_fixture, with use fly_session::coordinator::{DESCRIPTOR_REVISION, Injections}; use fly_session::harness::{AgentSpec, ExecutionMode, HarnessConfig, Via}; use fly_session::publish::{ - ApplicationChannel, ConsumerOutcome, Delivery, EVENT_BATCH_DEPTH, EventOutbox, GET_SNAPSHOT, - PresentationConsumer, PublicationOutcome, TopicPolicy, check_publication, query_service, + ApplicationChannel, ConsumerEvents, ConsumerOutcome, Delivery, EVENT_BATCH_DEPTH, EventOutbox, + GET_SNAPSHOT, PresentationConsumer, PublicationOutcome, TopicPolicy, check_publication, + query_service, }; use fly_session::types::*; use serde_json::json; @@ -38,6 +39,7 @@ both_transports!( the_query_service_answers_reads_and_nothing_else, a_refused_snapshot_is_still_what_the_repair_path_answers, a_replacement_that_built_another_index_cannot_install_the_checkpoint, + a_malformed_event_batch_is_unreadable_and_not_the_end_of_the_stream, the_published_descriptor_is_what_the_workers_attested_to, a_stimulus_kind_the_descriptor_does_not_declare_is_refused, a_restored_boundary_publishes_a_new_revision_and_no_transition, @@ -1079,6 +1081,55 @@ async fn a_refused_snapshot_is_still_what_the_repair_path_answers(via: Via) { f.shutdown().await; } +/// An event batch a consumer cannot read is reported as unreadable, and the stream goes on. +/// +/// The two facts differ: a consumer that read "that one made no sense" as "there are no more +/// events" would stop reading a live stream, which is the silent default this module exists to +/// refuse. The malformed batch is published by a second publisher on the session's own address, +/// because a well-behaved session cannot produce one. +async fn a_malformed_event_batch_is_unreadable_and_not_the_end_of_the_stream(via: Via) { + let mut f = started(via).await; + let mut consumer = f.harness.consumer().await.expect("a consumer attaches"); + let events_topic = f.harness.coordinator.topics().events.clone(); + let intruder = f.harness.publisher().await.expect("a publishing client"); + + // A batch with no droppedBefore: readable JSON, unreadable as a batch. + intruder + .publish( + &events_topic, + object(json!({ + "sessionId": f.harness.config.session_id.as_str(), + "epoch": f.harness.config.epoch.as_str(), + "events": [], + })), + &[], + ) + .await + .expect("the malformed batch publishes"); + + match within("the malformed batch", consumer.take_events()).await { + Some(ConsumerEvents::Unreadable { detail }) => { + assert!( + detail.contains("droppedBefore"), + "the report names what it could not read: {detail}" + ); + } + other => panic!("a malformed batch must not read as {other:?}"), + } + + // The stream did not end: the next real batch still arrives and reads. + f.harness.coordinator.run(1).await.expect("a transition"); + match within("the next batch", consumer.take_events()).await { + Some(ConsumerEvents::Batch(batch)) => { + assert_eq!(batch.epoch, f.harness.config.epoch); + assert!(!batch.event_ids.is_empty(), "the transition produced events"); + } + other => panic!("expected a readable batch after the malformed one, got {other:?}"), + } + intruder.close().await; + f.shutdown().await; +} + /// Two reads and nothing else. There is no method on this service that could move anything. async fn the_query_service_answers_reads_and_nothing_else(via: Via) { let mut f = started(via).await;