e27306f171
# Conflicts: # services/flysim/crates/fly-session/README.md # services/flysim/crates/fly-session/src/coordinator.rs # services/flysim/crates/fly-session/src/harness.rs # services/flysim/crates/fly-session/src/launcher.rs
266 lines
16 KiB
Markdown
266 lines
16 KiB
Markdown
# Session artifacts, native media and recovery
|
||
|
||
Status: **draft 2**, 2026-09-18. [Flybus](bus-v1.md) owns generic artifact storage, delivery
|
||
ownership, retention and garbage collection. This document specifies the **domain meaning**
|
||
of those artifacts: native observations, clock association and coherent session checkpoints.
|
||
Read [session RPC](ipc-v1.md), [step ordering](step-v1.md) and [worker interfaces](workers-v1.md).
|
||
|
||
## 1. Use the bus ArtifactRef
|
||
|
||
Large payload fields use `ArtifactRef` from bus-v1, and every referenced artifact is listed
|
||
in the surrounding bus attachments. There is no separate BinaryRef, buffer-region registry,
|
||
coordinator lease endpoint, or Buffer.Release/Reclaim protocol. Profile/dataset release assets
|
||
use `AssetRef` (a persistent content identity); transient bus ArtifactRefs are not those assets.
|
||
|
||
The environment publishes one immutable native image. The coordinator may forward the same
|
||
owned handle to multiple agent Commit calls and publish it for presentation. Flybus creates
|
||
destination ownership before releasing the source. It does not send another copy of the
|
||
pixel bytes per recipient through its sockets.
|
||
|
||
An agent consumes/encodes pixels during Initialize/Commit and drops its handle when no longer
|
||
used. A renderer may keep its extracted handle after dropping the message; the DeliveryGuard
|
||
keeps the artifact alive until rendering has finished. A domain cached RPC result keeps its
|
||
own handles so replay remains valid after the original recipient consumes its delivery.
|
||
|
||
Content digests are optional on transient live frames, mandatory on checkpoint payloads and
|
||
persistent asset import. Ownership/index/byte-shape validation is always required. A digest
|
||
does not replace epoch or observation-time identity.
|
||
|
||
## 2. Native observation types
|
||
|
||
```ts
|
||
interface ViewDescriptor {
|
||
viewId: Id;
|
||
width: number; height: number;
|
||
format: "rgba8"; rowStride: number;
|
||
pixelAspect: { numerator: number; denominator: number };
|
||
observationDelaySteps: number;
|
||
}
|
||
interface ViewRef { viewId: Id; producedStep: U64; pixels: ArtifactRef }
|
||
interface AudioDescriptor {
|
||
streamId: Id; sampleRate: number; channels: number;
|
||
format: "f32le-interleaved";
|
||
}
|
||
interface AudioRef {
|
||
streamId: Id; firstSample: U64; sampleFrames: number;
|
||
samples: ArtifactRef; discontinuity: boolean;
|
||
}
|
||
```
|
||
|
||
Epoch is inherited from the domain observation; the bus treats it as opaque payload. View
|
||
dimensions are integers 1..4096, rowStride exactly 4×width, no padded rows in v1. Pixel aspect
|
||
numerator/denominator are positive integers <=65535; observationDelaySteps is integer 0..8.
|
||
Pixels are top-left RGBA8 and artifact length equals rowStride×height. Other formats require
|
||
a media-schema change, not special-case code inside the router.
|
||
|
||
Required sensory views have producedStep equal to
|
||
`max(0, observation.boundary - observationDelaySteps)`. Bootstrap may repeat O[0] until the
|
||
declared pipeline delay fills. Beyond that, missing/extra-delay sensory input is a step
|
||
failure, not an arbitrary latest frame. Observer publication may omit/coalesce frames while
|
||
preserving each artifact's actual producing boundary.
|
||
|
||
Audio sampleRate is integer 8000..192000, channels 1..8, sampleFrames 0..192000 per chunk.
|
||
Samples are finite f32; artifact length is sampleFrames×channels×4. firstSample identifies
|
||
the sample position relative to the episode's configured audio origin, with intended PTS
|
||
firstSample/sampleRate. Crash restore preserves sample position under a new epoch; first
|
||
chunk marks discontinuity. Within an epoch, chunks cannot overlap or go backwards.
|
||
|
||
**Amendment, 2026-09-22 (MEDIA-01).** Two readings of the paragraphs above, made explicit
|
||
because they are now enforced:
|
||
|
||
- The bootstrap window is exactly the boundaries where `max(0, boundary - observationDelaySteps)`
|
||
is zero, that is `boundary <= observationDelaySteps`. Inside it the repeated `O[0]` is the
|
||
**same artifact**, not a fresh render of the same scene; outside it the producing boundary
|
||
advances one per step, and a frame from any other boundary -- older or newer -- is a step
|
||
failure. A producer therefore keeps a queue of `observationDelaySteps + 1` frames and nothing
|
||
more, so there is no older frame available to substitute.
|
||
- Within an epoch, `discontinuity` marks a range the stream actually skipped. The first chunk
|
||
after a restore marks it, and a later chunk may mark it when it starts past where the previous
|
||
chunk ended; a chunk that continues the previous one exactly is continuous by construction and
|
||
its flag is refused. Without that reading the restore rule is advisory, because a stream could
|
||
set the flag on every chunk and satisfy it by accident. The requirement is one-directional: a
|
||
fresh epoch's first chunk **may** mark a discontinuity, because section 6's recovery
|
||
establishes a fresh timeline and publishes one.
|
||
|
||
The environment provides **native game output**. Sensor transformations belong to the agent
|
||
profile. Resizing for viewers, overlays, composition, audio mixing/resampling, encoding,
|
||
browser delivery and streaming belong to the application/presentation layer. No bus or
|
||
generic session configuration assumes a 1080p show or Twitch output.
|
||
|
||
640×480 RGBA at 60 fps produces 73.728 MB/s of raw image data. Artifact fan-out references
|
||
one stored object; reads and any staging/seal copy still consume memory bandwidth. This is
|
||
reasonable to measure before introducing codecs or pooled GPU buffers. Native dimensions
|
||
come from the backend, not a hardcoded GameCube or broadcast resolution.
|
||
|
||
## 3. Domain retention and backpressure
|
||
|
||
Use the same bus call/publish API for observations and artifacts. Bus ownership tracks bytes;
|
||
the session decides which observations are required and when they have been used.
|
||
|
||
| Use | Rule |
|
||
| --- | --- |
|
||
| Required agent input | Retain through encoding/Commit; no coalescing or overwrite |
|
||
| Step-result replay | Retain in the endpoint's current/previous-step cache until domain eviction |
|
||
| Spectator snapshot | Latest subscription, finite in-flight credits; release after actual use |
|
||
| Long rendering/storage job | Explicit artifact hold with a finite byte/count budget |
|
||
| Hot checkpoint | Coalesce only queued replaceable captures, releasing their holds |
|
||
| Durable checkpoint | Acknowledge after durable commit; reject/defer before capture when saturated |
|
||
|
||
Initial session defaults: two outstanding coherent captures and at most the bus-configured
|
||
latest/in-flight frame credits per observer. Presentation audio can target 250 ms and cap at
|
||
one second, but that is a presentation policy, not a bus or brain-clock requirement.
|
||
|
||
Budget cached step observations, active agent deliveries, retained latest and spectator holds
|
||
together. The producer dropping its handle does not free cached/queued/in-use objects.
|
||
A slow spectator exhausts its own credits; new latest messages replace its queued value.
|
||
If it violates configured resource policy, disconnect/restart that observer instead of freeing
|
||
live data or silently skipping simulation input. Global store exhaustion is an explicit fault
|
||
or pause condition; the router cannot guess that a particular live object is disposable.
|
||
|
||
**Amendment, 2026-09-22 (PUBLISH-01).** "Disconnect/restart that observer" names an action
|
||
no participant can take under [Flybus v1](bus-v1.md). Section 5 there makes publish admission
|
||
all or nothing -- "for a bounded subscriber overflow, reject the **whole** publish; no partial
|
||
fan-out or retained-latest update" -- and the router exposes no per-subscriber eviction, so a
|
||
session meeting a full bounded queue cannot drop that one subscriber and deliver to the rest.
|
||
The realisable reading, which the session now implements, is three-part: observation topics
|
||
are published `latest`, and a latest subscriber can never refuse a publication (it loses its
|
||
own queued value and is told how many by `replaced`); a bounded subscriber's refusal, which
|
||
`bus-v1` section 6 explicitly permits, is a named and counted publication outcome that takes
|
||
no world step, stalls nothing and fences no epoch, and the exact value stays recoverable
|
||
through the [publishing-v1](publishing-v1.md) section 2 query path; and disconnecting the
|
||
offender is an operator action against the topic the ledger names, not something the session
|
||
performs. A per-subscriber drop would need a router operation Flybus v1 does not have, and
|
||
inventing one here would be a transport change written into the wrong document.
|
||
|
||
No coordinator tracks per-reader socket acknowledgments or calls a producer's reclaim method.
|
||
The SDK and bus perform that bookkeeping. File-backed immutable mappings are safe after
|
||
unlink; physical pages disappear when all OS mappings close. Pooled reuse is deferred until
|
||
it provides equivalent safety. Bare handles in history are not persistent saved bytes.
|
||
|
||
## 4. Checkpoint identity and content
|
||
|
||
Exact-checkpoint sessions capture at Ready(k) or Paused(k), after all Agent.Commit replies,
|
||
with no Advance in flight. Block next Prepare until all participants supply immutable captures.
|
||
|
||
The manifest records:
|
||
|
||
- Envelope version, checkpoint ID and source session/epoch/step/episode/world time.
|
||
- Coordinator scheduler/configuration identity and exact port-to-agent map.
|
||
- Backend/content/patch/controller/parser/state-format compatibility.
|
||
- Per-agent profile/dataset/model identities, seed, tick count/remainder and payload digests.
|
||
- Task ledger, prior world inspection, per-agent executor state, next sensory/decision state
|
||
or reproducible reconstruction inputs, admission state and event watermarks.
|
||
- Payload names, lengths and hashes, including external-helper state required for exact resume.
|
||
|
||
Use a new envelope version; specify exact byte layout before production files. The historical
|
||
letter-only chunk-name constraint is not silently widened, and FLYSIM01 remains separately
|
||
readable. Persist payload **bytes and durable content identity**, not transient bus storeId,
|
||
artifact IDs, ownership tokens, mappings or pointers.
|
||
|
||
A checkpoint writer owns the bus Artifact handles until bytes are committed or the job fails.
|
||
It then drops them; durable files are outside Flybus's ephemeral GC. On restore, the durable
|
||
store imports fresh immutable bus artifacts. sourceScope is provenance, while the new handles
|
||
belong to the current router/store. Broker retention is never a substitute for a checkpoint.
|
||
|
||
## 5. State RPCs over the bus
|
||
|
||
Common to workers advertising checkpoint-v1:
|
||
|
||
```ts
|
||
interface CaptureParams { checkpointId: Id }
|
||
interface CaptureResult {
|
||
checkpointId: Id; boundary: U64;
|
||
compatibilityDigest: Digest;
|
||
payload: ArtifactRef; // listed attachment; digest required
|
||
}
|
||
interface StageRestoreParams {
|
||
checkpointId: Id; sourceScope: Scope;
|
||
compatibilityDigest: Digest;
|
||
payload: ArtifactRef; // newly imported owned attachment
|
||
}
|
||
interface StageRestoreResult { checkpointId: Id; restoreToken: Id }
|
||
interface ActivateRestoreParams { restoreToken: Id }
|
||
interface ActivateRestoreResult {
|
||
committedStep: U64; checkpointId: Id;
|
||
observation: WorldObservation | null; // environment required, agent null
|
||
}
|
||
```
|
||
|
||
State.Capture uses the committed scope. It completes after immutable capture exists, not when
|
||
a backend save was requested. The cached reply retains its artifact until Worker.Acknowledge.
|
||
The coordinator/writer obtains its own live ownership before acknowledging that cache.
|
||
|
||
State.StageRestore uses a proposed **new epoch** at source boundary k and is allowed only
|
||
on an uninitialized replacement or a quiescent worker. Launcher configuration supplies the
|
||
expected profile/backend identities; no implicit warm-up/reinitialization changes the saved
|
||
brain. It validates into replacement state, without exposing mutations to the live session.
|
||
|
||
After every participant and coordinator state validates, State.ActivateRestore installs
|
||
each staged token under that new scope without advancing a tick. Tokens are bound to scope/
|
||
payload/checkpoint and can activate only once; duplicate domain requests replay the cached
|
||
reply, while a fresh request trying to reuse an activated token is a conflict.
|
||
|
||
The environment returns the coherent restored observation with fresh artifact references and
|
||
restored time. It cannot advance gameplay to manufacture it. Capture/reconstruction therefore
|
||
covers render/inspection state and any pending sensor pipeline. Agent state agrees with it;
|
||
do not replay reward or recalibrate merely to fill missing cached data.
|
||
|
||
**Amendment, 2026-09-22 (STATE-01).** Three readings of this section, made explicit because
|
||
they are now enforced:
|
||
|
||
- `compatibilityDigest` on `CaptureResult` and `StageRestoreParams` is the **participant's**
|
||
capture compatibility digest of [worker interfaces](workers-v1.md) section 2 -- profile,
|
||
resolved seed, numerical model version and effective instance configuration for an agent;
|
||
backend, content, patch, controller and parser identity for an environment. It is not the
|
||
manifest's `compatibility` block of section 4, which is the composition's and which the
|
||
coordinator compares before anything is asked to stage. Both exist because they answer
|
||
different questions, and a restore that passed the second could still be handing an agent
|
||
another agent's brain.
|
||
- The observation `ActivateRestore` returns ran no transition, so it carries **no audio
|
||
chunk**, and one in it is refused. Section 2's chunk is the audio of an interval and this
|
||
observation covers none; MEDIA-01 implemented that rule as "boundary 0 carries no chunk",
|
||
which is true of the only such observation that slice could produce and false of this one.
|
||
The rule is about provenance, not about the boundary number.
|
||
- A participant that staged into a group install the coordinator then abandoned must be
|
||
**replaced** before another restore, exactly as one that activated must. It is holding a
|
||
validated replacement state that nothing installed, and [session RPC](ipc-v1.md) section 6
|
||
already refuses to silently reattach such a participant to an active epoch. Without this the
|
||
group's second attempt meets its own leftovers and calls them a conflict.
|
||
|
||
If emulator validation requires mutation, stage a stopped replacement emulator. If that cannot
|
||
provide externally atomic resume, advertise episode-restart, not exact-checkpoint. After all
|
||
activation acknowledgments, install the coordinator's staged task/executor/admission state
|
||
and establish Paused(new epoch,k). Failure during activation never permits half a group to run.
|
||
|
||
## 6. Durable commit, router failure and recovery
|
||
|
||
Write payload/envelope temporary generation, fsync, rename, fsync directory, then atomically
|
||
write/fsync/rename the store manifest and fsync its directory. **Manifest commit is the durable
|
||
commit point.** Unreferenced temporary generations are not automatic restore candidates.
|
||
|
||
Publish distinct captured/queued/committed/failed/superseded events over the same bus. Only
|
||
durable completion produces a saved acknowledgment/high-water mark. Failed writes release
|
||
owned ephemeral captures according to retry policy, without reporting false durability.
|
||
|
||
After participant/coordinator/router failure:
|
||
|
||
1. Stop steps, abandon the epoch and fence old participants/routes.
|
||
2. Connect to a live router and select a complete compatible durable checkpoint.
|
||
3. Import its payloads as new artifacts; stage/activate every participant and coordinator.
|
||
4. Verify identity/boundary, flush old media/parser queues and publish recovery/discontinuity.
|
||
5. Establish Paused(k), then resume only after the group invariant holds.
|
||
|
||
A router restart loses ephemeral topics, queues, roots and correlations. Continuing with
|
||
old handles is invalid even if some mapped bytes survived. Reconnect is not transparent
|
||
mid-step recovery. The durable log records old/new epochs and abandoned step ranges; rollback
|
||
can lose post-checkpoint work. Durable input replay requires a separate application/session
|
||
journal policy, not an exactly-once claim about Flybus.
|
||
|
||
## 7. Episode reset
|
||
|
||
Reset differs from crash restore. The application selects a policy; the coordinator records
|
||
the old episode's result/abort and creates a new epoch/episode at step zero. World initial
|
||
state and retained/fresh brain components are explicit. Gain retention, eligibility/hold
|
||
clearing, calibration and first sensory input are part of the policy, tested independently.
|
||
Legacy Pokémon ratchet behavior remains in the legacy composition. Shared competitive worlds
|
||
never restore one player's environment independently of the other players.
|