alex/flybrain
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

refactor(session): build the session slice on the shared fly-session-types crate

Browse source 
CONTRACT-01 landed its crate, so the local stand-in module goes away and the domain
scalars, method payloads, their validation, the canonical digests and the trace format
all come from the contract. `src/types.rs` is a facade over that crate plus the few
things a coordinator needs that are not part of the type contract: a session-side
DomainError, the synthetic composition's schema and event-id derivations, and the
coordinator-local ControllerIntent, PortBinding and AgentOutcome that never cross the
bus.

Consequences worth naming:

- Payloads are read and written through DomainType::from_json / to_json instead of
  serde derives, so a misspelled required field fails where the contract says it
  should. serde, sha2 and ryu-js leave this crate's dependencies with them.
- The step-v1 section 8 trace is the contract's TransitionTrace, with behaviour and
  operational metadata already separated; the dispatch-order comparison now runs over
  the contract's own behaviour encoding.
- Two things the migration found. IN_PROGRESS is raised strictly before any mutation,
  so its certainty is "none", not "unknown"; the local module had it wrong.
  WorldObservation::validate_against checks the views a result carries but does not
  require every declared view to be present, so requiring them is made explicit in the
  coordinator's phase C check, where step-v1 puts it.
- The lost-Advance-result injection now waits for the worker to report the operation
  before abandoning the call, so the case it injects really is a loss after dispatch
  rather than a cancellation before it.

Gates: cargo test -p fly-session (61 tests, both transports), cargo clippy
--all-targets clean, and the runnable example produces the same behaviour trace over
both transports.
This commit is contained in:
acamilo 2026-09-22 12:30:41 +00:00
parent 825b584320
commit 52ad46ecb0
22 changed files with 1176 additions and 2347 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
services/flysim/crates/fly-session/Cargo.toml
View file

@ -12,12 +12,12 @@ name = "fly_session"
path = "src/lib.rs"
[dependencies]
# The domain contract (scalars, payloads, canonical digests, the trace format) and the bus.
# Everything else this crate needs is std or Tokio.
fly-session-types = { path = "../fly-session-types" }
flybus = { path = "../flybus" }
ryu-js = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
sha2 = { workspace = true }
tokio = { version = "1", features = ["rt", "sync", "time", "macros"] }
[dev-dependencies]

20
services/flysim/crates/fly-session/README.md
View file

@ -8,6 +8,12 @@ sequential transaction of `step-v1`, driven over the Flybus router, with small f
standing in for a brain and an emulator. It contains no public controller API, no implicit
best-effort retry, no real emulator and no real brain.
The domain scalars, method payloads, their validation, the canonical digests and the trace
format all come from [`fly-session-types`](../fly-session-types), the CONTRACT-01 crate. This
crate adds only what is not part of the type contract: a session-side error value, the
synthetic composition's schema and event-id derivations, and the coordinator-local
`ControllerIntent`, `PortBinding` and `AgentOutcome` that never cross the bus.
```text
Ready(k) ─ Prepare all agents concurrently ────────────> every agent Prepared(k)
─ one executor per agent, sorted agent-id order
@ -22,7 +28,7 @@ Ready(k) ─ Prepare all agents concurrently ───────────
| Module | Contents |
| --- | --- |
| `fly_session_types` | The CONTRACT-01 domain types, as a local stand-in until that crate exists |
| `types` | A facade over the [`fly-session-types`](../fly-session-types) crate, plus the session-side additions a coordinator needs |
| `clock` | The `step-v1` section 5 rational tick accumulator and the coordinator's pacing |
| `phase` | The `step-v1` section 2 state machine as an explicit edge table |
| `dedup` | The `ipc-v1` section 5 operation keys, result caches and retention |
@ -98,6 +104,18 @@ harness.shutdown().await;
event ids derived from epoch, source step, rule and ordinal.
- **Executors.** The stateless identity executor only, as v1 specifies.
## Where this crate narrows or adds to the contract crate
- **Required views.** `WorldObservation::validate_against` checks the views a result carries
against their descriptors. Requiring every *declared* view to be there at all is the
coordinator's Phase C check, so `verify_step_result` makes it: a missing required sensory
view fails the transition with `BUFFER_INVALID` rather than being replaced by an older frame.
- **`ControllerIntent`.** `workers-v1` section 4 calls the task and executor interfaces local
libraries, so their types live here rather than in the payload contract. An intent is a
`PortControl` without its port, and only the coordinator adds the port.
- **The phase machine.** `step-v1` section 2 is this crate's, not the contract crate's; the
trace's phase path is recorded beside the contract's `TransitionTrace`.
## Limitations
- **Fake workers.** There is no neural model and no emulator. What is modelled exactly is the

8
services/flysim/crates/fly-session/examples/session.rs
View file

@ -7,6 +7,7 @@
//! Two fake agents, one counter arena, one coordinator, one router. Nothing here needs a ROM,
//! a dataset, a GPU or a network.
use fly_session::types::*;
use fly_session::harness::{HarnessConfig, SessionHarness, Via};
#[tokio::main(flavor = "multi_thread", worker_threads = 4)]
@ -22,6 +23,7 @@ async fn main() {
for (k, transition) in harness.coordinator.trace.transitions.iter().enumerate() {
let ticks: Vec<String> = transition
.behaviour
.agents
.iter()
.map(|a| format!("{}={} ticks", a.agent_id, a.ticks_advanced))
@ -29,9 +31,9 @@ async fn main() {
println!(
"step {k}: {} batch={} boundary={} events={}",
ticks.join(" "),
transition.batch_id,
transition.acknowledged_boundary,
transition.task_event_ids.len()
transition.behaviour.batch_id,
transition.behaviour.acknowledged_boundary,
transition.behaviour.event_ids.len()
);
}
let progress = harness.coordinator.task_progress();

88
services/flysim/crates/fly-session/src/agent.rs
View file

@ -14,12 +14,9 @@ use serde_json::Value;
use crate::clock::TickAccumulator;
use crate::dedup::OpClass;
use crate::task::{context_schema, decision_schema};
use crate::types::{
AgentCommitResult, AgentInitializeParams, AgentInitializeResult, AgentTelemetry, AssetRef,
AxisValue, ButtonState, CommitParams, Digest, DomainError, DomainResult, ErrorCode, Id,
LearningTelemetry, MAX_EVENT_ARRAY, Mutation, PrepareParams, PreparedDecision, RateSample,
RationalNs, Role, Scope, SensoryInput, Stimulus, TypedValue, U64, WorkerState,
};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
// glob keeps the contract's own names in sight instead of restating them.
use crate::types::*;
use crate::worker::{BoxFuture, HandlerCtx, HandlerReply, StatusCell, WorkerEndpoint};
/// The fake numerical model: a seeded stream and a count of everything that mutated it.
@ -114,8 +111,8 @@ impl FakeModel {
/// names a neuron or a drive value.
fn stimulate(&mut self, stimulus: &Stimulus) {
let kind = u64::from_le_bytes({
let d = Digest::of(stimulus.kind_id.as_str().as_bytes());
let bytes = d.as_str().as_bytes();
let d = digest_of_bytes(stimulus.kind_id.as_bytes());
let bytes = d.as_bytes();
let mut out = [0u8; 8];
out.copy_from_slice(&bytes[..8]);
out
@ -170,16 +167,16 @@ impl FakeModel {
fn telemetry(&self) -> AgentTelemetry {
let draw = self.state >> 29;
AgentTelemetry {
brain_ticks: U64(self.ticks),
brain_ticks: self.ticks,
population_rate_hz: (draw % 1000) as f64 / 10.0,
rates: vec![
RateSample { role_id: Id::lit("kc"), hz: (draw % 700) as f64 / 10.0 },
RateSample { role_id: Id::lit("mbon"), hz: (draw % 310) as f64 / 10.0 },
RateSample { role_id: id("kc"), hz: (draw % 700) as f64 / 10.0 },
RateSample { role_id: id("mbon"), hz: (draw % 310) as f64 / 10.0 },
],
learning: LearningTelemetry {
enabled: self.learning_enabled,
updates: U64(self.learning_updates),
changed: U64(self.learning_changed),
updates: self.learning_updates,
changed: self.learning_changed,
signal: self.last_signal,
},
}
@ -228,7 +225,7 @@ pub struct FakeAgentWorker {
model: FakeModel,
context: Option<TypedValue>,
context_digest: Option<Digest>,
prepared: Option<(Id, PreparedDecision)>,
prepared: Option<(DomainRequestId, PreparedDecision)>,
}
impl FakeAgentWorker {
@ -281,7 +278,7 @@ impl FakeAgentWorker {
for view in &input.views {
let name = format!("view.{}", view.view_id);
let artifact = ctx.artifact(&name)?;
if artifact.reference() != &view.pixels.0 {
if artifact.reference() != &view.pixels {
return Err(DomainError::before(
ErrorCode::BufferInvalid,
format!("attachment {name} is not the artifact the payload names"),
@ -293,7 +290,7 @@ impl FakeAgentWorker {
format!("view {} could not be read: {}", view.view_id, e.message),
)
})?;
if bytes.len() as u64 != view.pixels.0.byte_length {
if bytes.len() as u64 != view.pixels.byte_length {
return Err(DomainError::before(
ErrorCode::BufferInvalid,
format!("view {} is the wrong length", view.view_id),
@ -330,18 +327,15 @@ impl FakeAgentWorker {
fn decision(&self, available: &[String]) -> TypedValue {
let (inc, dec, bias) = self.model.readout(available);
let intent = crate::types::ControllerIntent {
let intent = ControllerIntent {
buttons: vec![
ButtonState { id: Id::lit("inc"), down: inc },
ButtonState { id: Id::lit("dec"), down: dec },
ButtonState { id: id("inc"), down: inc },
ButtonState { id: id("dec"), down: dec },
],
axes: vec![AxisValue { id: Id::lit("bias"), value: bias }],
axes: vec![AxisValue { id: id("bias"), value: bias }],
};
let value = match serde_json::to_value(&intent).expect("an intent serializes") {
Value::Object(m) => m,
_ => unreachable!(),
};
TypedValue::new(decision_schema(), value)
TypedValue::new(decision_schema(), intent.to_json())
.expect("a direct-control decision fits the contract")
}
async fn initialize(&mut self, ctx: &HandlerCtx<'_>) -> DomainResult<HandlerReply> {
@ -353,7 +347,7 @@ impl FakeAgentWorker {
state interface",
));
}
if scope.step.0 != 0 {
if scope.step != 0 {
return Err(DomainError::before(
ErrorCode::FutureStep,
"Agent.Initialize uses the new epoch at step 0",
@ -379,7 +373,7 @@ impl FakeAgentWorker {
let available = FakeAgentWorker::available_actions(&params.initial_decision_context)?;
// Everything is validated before the model is constructed.
let encoded = self.encode(ctx, &params.initial_input).await?;
if params.initial_input.boundary.0 != 0 {
if params.initial_input.boundary != 0 {
return Err(DomainError::invalid("the initial input must observe boundary 0"));
}
@ -390,7 +384,7 @@ impl FakeAgentWorker {
// Warm-up runs with learning disabled and produces no gameplay reward or control.
model.advance(self.config.warmup_ticks);
accumulator.warm_up(self.config.warmup_ticks).map_err(|e| {
DomainError::new(ErrorCode::Internal, e, Mutation::Applied)
DomainError::new(ErrorCode::Internal, e, MutationCertainty::Applied)
})?;
// Calibration happens on settled rates, after warm-up. The readout is a pure read of
// the model, so calibrating it mutates nothing.
@ -412,8 +406,8 @@ impl FakeAgentWorker {
agent_id: self.config.agent_id.clone(),
profile_digest: params.profile.digest.clone(),
tick_duration: self.config.tick_duration,
warmup_ticks: U64(self.config.warmup_ticks),
committed_step: U64(0),
warmup_ticks: self.config.warmup_ticks,
committed_step: 0,
decision_context_digest: self.context_digest.clone().expect("just set"),
telemetry: self.model.telemetry(),
};
@ -429,13 +423,13 @@ impl FakeAgentWorker {
format!("Agent.Prepare needs Ready(k); this worker is {:?}", self.phase),
));
};
if scope.step.0 < k {
if scope.step < k {
return Err(DomainError::before(
ErrorCode::StaleStep,
"Agent.Prepare names a step this worker has left",
));
}
if scope.step.0 > k {
if scope.step > k {
return Err(DomainError::before(
ErrorCode::FutureStep,
"Agent.Prepare names a step beyond this worker's committed boundary",
@ -462,7 +456,7 @@ impl FakeAgentWorker {
));
}
params.interval.validate().map_err(DomainError::invalid)?;
if params.pre_step_stimulations.len() > MAX_EVENT_ARRAY {
if params.pre_step_stimulations.len() > MAX_STIMULI {
return Err(DomainError::invalid("at most 64 pre-step stimulations"));
}
for stimulus in &params.pre_step_stimulations {
@ -487,7 +481,7 @@ impl FakeAgentWorker {
let ticks = {
let accumulator = self.accumulator.as_mut().expect("initialized");
accumulator.advance(&params.interval).map_err(|e| {
DomainError::new(ErrorCode::InvalidArgument, e, Mutation::Applied)
DomainError::new(ErrorCode::InvalidArgument, e, MutationCertainty::Applied)
})?
};
self.model.advance(ticks);
@ -499,7 +493,7 @@ impl FakeAgentWorker {
};
let prepared = PreparedDecision {
agent_id: self.config.agent_id.clone(),
ticks_advanced: U64(ticks),
ticks_advanced: ticks,
brain_ticks,
remainder,
decision,
@ -520,9 +514,9 @@ impl FakeAgentWorker {
format!("Agent.Commit needs Prepared(k); this worker is {:?}", self.phase),
));
};
if scope.step.0 != k {
if scope.step != k {
return Err(DomainError::before(
if scope.step.0 < k { ErrorCode::StaleStep } else { ErrorCode::FutureStep },
if scope.step < k { ErrorCode::StaleStep } else { ErrorCode::FutureStep },
"Agent.Commit must carry the step of its transition, not the new boundary",
));
}
@ -540,12 +534,12 @@ impl FakeAgentWorker {
"Agent.Commit does not match this worker's Prepare request",
));
}
if params.next_input.boundary.0 != k + 1 {
if params.next_input.boundary != k + 1 {
return Err(DomainError::invalid(
"the next sensory input must observe boundary k+1",
));
}
if params.rewards.len() > MAX_EVENT_ARRAY || params.task_stimulations.len() > MAX_EVENT_ARRAY
if params.rewards.len() > MAX_REWARDS || params.task_stimulations.len() > MAX_STIMULI
{
return Err(DomainError::invalid("at most 64 rewards and 64 stimulations"));
}
@ -579,7 +573,7 @@ impl FakeAgentWorker {
return Err(DomainError::new(
ErrorCode::BackendFailure,
"injected commit failure after the next input was installed",
Mutation::Applied,
MutationCertainty::Applied,
));
}
// 2. apply task-derived stimulation in returned event order
@ -598,12 +592,12 @@ impl FakeAgentWorker {
self.prepared = None;
self.phase = AgentPhase::Ready(k + 1);
self.status.set_state(WorkerState::Ready);
self.status.set_scope(Some(Scope::new(&scope.session_id, &scope.epoch, k + 1)));
self.status.set_scope(Some(scope_at(&scope.session_id, &scope.epoch, k + 1)));
self.status.advance_to(self.model.mutations());
let result = AgentCommitResult {
agent_id: self.config.agent_id.clone(),
committed_step: U64(k + 1),
committed_step: k + 1,
decision_context_digest: self.context_digest.clone().expect("just set"),
telemetry: self.model.telemetry(),
};
@ -634,7 +628,7 @@ impl WorkerEndpoint for FakeAgentWorker {
}
fn capabilities(&self) -> Vec<Id> {
vec![Id::lit("agent-step-v1"), Id::lit("pixel-observation-v1")]
vec![id("agent-step-v1"), id("pixel-observation-v1")]
}
fn status_cell(&self) -> StatusCell {
@ -676,10 +670,10 @@ pub fn synthetic_profile(agent_id: &Id, tick_duration: &RationalNs, warmup_ticks
tick_duration.numerator, tick_duration.denominator
);
AssetRef {
id: Id::lit("arena-direct-v1"),
digest: Digest::of(text.as_bytes()),
byte_length: U64(text.len() as u64),
format: Id::lit("fly-profile-v1"),
id: id("arena-direct-v1"),
digest: digest_of_bytes(text.as_bytes()),
byte_length: text.len() as u64,
format: id("fly-profile-v1"),
}
}

75
services/flysim/crates/fly-session/src/clock.rs
View file

@ -1,10 +1,11 @@
//! Time and pacing: `step-v1` section 5.
//!
//! The accumulator is exact rational arithmetic, never rounded nanoseconds. A 60 Hz world with
//! a 1 ms model tick advances 16, 17, 17 ticks over its first three steps and comes back to a
//! remainder of exactly zero; rounding to microseconds does not.
//! The accumulator is the contract crate's exact rational arithmetic, never rounded
//! nanoseconds. A 60 Hz world with a 1 ms model tick advances 16, 17, 17 ticks over its first
//! three steps and comes back to a remainder of exactly zero; rounding to microseconds does
//! not.
use crate::types::{DomainError, ErrorCode, RationalNs, U64};
use crate::types::{DomainError, DomainType, ErrorCode, MutationCertainty, RationalNs};
/// One agent's tick accumulator: its tick duration, its remainder and its executed count.
#[derive(Clone, Debug)]
@ -18,13 +19,11 @@ pub struct TickAccumulator {
impl TickAccumulator {
/// A fresh accumulator. The tick duration must be positive.
pub fn new(tick_duration: RationalNs) -> Result<TickAccumulator, String> {
tick_duration.validate()?;
if !tick_duration.is_positive() {
return Err("tick duration must be positive".to_owned());
}
tick_duration.validate().map_err(|e| e.0)?;
tick_duration.require_positive("tick duration").map_err(|e| e.0)?;
Ok(TickAccumulator {
tick_duration,
remainder: RationalNs::zero(),
remainder: RationalNs::ZERO,
executed_ticks: 0,
warmup_offset: 0,
})
@ -71,17 +70,13 @@ impl TickAccumulator {
/// accumulator -= ticks * model tick duration
/// ```
pub fn advance(&mut self, interval: &RationalNs) -> Result<u64, String> {
interval.validate()?;
if !interval.is_positive() {
return Err("environment interval must be positive".to_owned());
}
let accumulated = self.remainder.checked_add(interval)?;
let ticks = accumulated.checked_div_floor(&self.tick_duration)?;
let consumed = self.tick_duration.checked_mul_u64(ticks)?;
let remainder = accumulated.checked_sub(&consumed)?;
debug_assert_eq!(
remainder.cmp_value(&self.tick_duration),
std::cmp::Ordering::Less,
interval.validate().map_err(|e| e.0)?;
interval.require_positive("environment interval").map_err(|e| e.0)?;
let accumulated = self.remainder.checked_add(interval).map_err(|e| e.0)?;
let (ticks, remainder) =
accumulated.divide_floor(&self.tick_duration).map_err(|e| e.0)?;
debug_assert!(
remainder < self.tick_duration,
"the remainder must stay below one model tick"
);
self.remainder = remainder;
@ -92,8 +87,8 @@ impl TickAccumulator {
Ok(ticks)
}
pub fn brain_ticks(&self) -> U64 {
U64(self.executed_ticks)
pub fn brain_ticks(&self) -> u64 {
self.executed_ticks
}
}
@ -123,7 +118,8 @@ impl Pacing {
/// The wall-clock period of one step, rounded for sleeping only. Simulation time stays
/// rational; this value is never fed back into the accumulator.
fn period(&self) -> std::time::Duration {
let ns = u128::from(self.step_duration.numerator.0) / u128::from(self.step_duration.denominator.0).max(1);
let ns = u128::from(self.step_duration.numerator)
/ u128::from(self.step_duration.denominator).max(1);
std::time::Duration::from_nanos(u64::try_from(ns).unwrap_or(u64::MAX))
}
@ -162,24 +158,26 @@ pub fn ticks_to_legacy_millis(ticks: u64, tick_duration: &RationalNs) -> Result<
// skipping representable ticks, so the run is refused rather than silently rounded.
const EXACT_F64_INTEGERS: u64 = 1 << 53;
if ticks >= EXACT_F64_INTEGERS {
return Err(DomainError::before(
return Err(DomainError::new(
ErrorCode::InvalidArgument,
"tick count exceeds the range the legacy millisecond clock represents exactly",
MutationCertainty::None,
));
}
let per_tick_ms = tick_duration.numerator.0 as f64
/ (tick_duration.denominator.0 as f64 * 1_000_000.0);
let per_tick_ms =
tick_duration.numerator as f64 / (tick_duration.denominator as f64 * 1_000_000.0);
Ok(ticks as f64 * per_tick_ms)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::{hz, millis};
#[test]
fn a_60_hz_world_with_a_1_ms_tick_runs_16_17_17() {
let step = RationalNs::from_hz(60).unwrap();
let mut acc = TickAccumulator::new(RationalNs::from_millis(1).unwrap()).unwrap();
let step = hz(60).unwrap();
let mut acc = TickAccumulator::new(millis(1).unwrap()).unwrap();
let ticks: Vec<u64> = (0..3).map(|_| acc.advance(&step).unwrap()).collect();
assert_eq!(ticks, vec![16, 17, 17]);
assert_eq!(ticks.iter().sum::<u64>(), 50);
@ -188,12 +186,12 @@ mod tests {
#[test]
fn the_remainder_stays_below_one_tick_and_never_goes_negative() {
let step = RationalNs::from_hz(60).unwrap();
let tick = RationalNs::from_millis(1).unwrap();
let step = hz(60).unwrap();
let tick = millis(1).unwrap();
let mut acc = TickAccumulator::new(tick).unwrap();
for _ in 0..600 {
acc.advance(&step).unwrap();
assert_eq!(acc.remainder().cmp_value(&tick), std::cmp::Ordering::Less);
assert!(acc.remainder() < tick);
}
// 600 steps of 1/60 s is exactly 10 s, which is 10,000 whole milliseconds.
assert_eq!(acc.executed_ticks(), 10_000);
@ -202,17 +200,24 @@ mod tests {
#[test]
fn warm_up_ticks_do_not_touch_the_remainder() {
let mut acc = TickAccumulator::new(RationalNs::from_millis(1).unwrap()).unwrap();
let mut acc = TickAccumulator::new(millis(1).unwrap()).unwrap();
acc.warm_up(25).unwrap();
assert_eq!(acc.executed_ticks(), 25);
assert_eq!(acc.warmup_offset(), 25);
assert!(acc.remainder().is_zero());
assert_eq!(acc.advance(&RationalNs::from_hz(60).unwrap()).unwrap(), 16);
assert_eq!(acc.advance(&hz(60).unwrap()).unwrap(), 16);
}
#[test]
fn a_zero_interval_is_refused_rather_than_silently_producing_no_ticks() {
let mut acc = TickAccumulator::new(RationalNs::from_millis(1).unwrap()).unwrap();
assert!(acc.advance(&RationalNs::zero()).is_err());
let mut acc = TickAccumulator::new(millis(1).unwrap()).unwrap();
assert!(acc.advance(&RationalNs::ZERO).is_err());
}
#[test]
fn the_legacy_millisecond_clock_refuses_a_run_beyond_its_exact_range() {
let tick = millis(1).unwrap();
assert_eq!(ticks_to_legacy_millis(50, &tick).unwrap(), 50.0);
assert!(ticks_to_legacy_millis(1 << 53, &tick).is_err());
}
}

397
services/flysim/crates/fly-session/src/coordinator.rs
View file

@ -17,15 +17,9 @@ use crate::clock::Pacing;
use crate::phase::{Phase, PhaseMachine};
use crate::rpc::{self, DomainReply, Serials, WorkerRef};
use crate::task::{ActionExecutor, Task};
use crate::types::{
AgentCommitResult, AgentInitializeParams, AgentInitializeResult, AgentOutcome, AssetRef,
CommitParams, Digest, DomainError, EnvironmentDescriptor, EnvironmentInitializeParams,
EnvironmentInitializeResult, EpisodeRequest, ErrorCode, HelloParams, HelloResult, Id, Mutation,
PortBinding, PortControl, PrepareParams, PreparedDecision, RationalNs, RequestId, Role, Scope,
SensoryInput, StatusResult, StepResult, Stimulus, TypedValue, U64, WorldObservation,
contract_digest, controls_digest,
};
use crate::types::{AgentTransitionTrace, TraceLog, TransitionTrace, ViewProvenance};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
// glob keeps the contract's own names in sight instead of restating them.
use crate::types::*;
/// The order the coordinator dispatches and awaits its per-agent phases in.
///
@ -128,12 +122,12 @@ pub struct AgentSlot {
pub profile: AssetRef,
pub seed: i32,
pub tick_duration: RationalNs,
pub warmup_ticks: U64,
pub warmup_ticks: u64,
pub committed_step: u64,
context: TypedValue,
context_digest: Digest,
prepared: Option<PreparedDecision>,
prepare_request: Option<RequestId>,
prepare_request: Option<DomainRequestId>,
}
impl AgentSlot {
@ -150,11 +144,12 @@ impl AgentSlot {
port_id,
profile,
seed,
tick_duration: RationalNs::zero(),
warmup_ticks: U64(0),
tick_duration: RationalNs::ZERO,
warmup_ticks: 0,
committed_step: 0,
context: TypedValue::new(crate::task::context_schema(), Map::new()),
context_digest: Digest::of(b""),
context: TypedValue::new(crate::task::context_schema(), Value::Object(Map::new()))
.expect("an empty context object is a valid typed value"),
context_digest: digest_of_bytes(b""),
prepared: None,
prepare_request: None,
}
@ -193,7 +188,7 @@ pub struct Coordinator {
/// Set by whoever asks for a normal pause, possibly while a transition is in flight.
pause: std::sync::Arc<std::sync::atomic::AtomicBool>,
episode: Option<EpisodeRequest>,
lifecycle_acks: Vec<(WorkerRef, Id)>,
lifecycle_acks: Vec<(WorkerRef, DomainRequestId)>,
stats: Stats,
/// The ordered actions this session took, for the ordering assertions.
pub audit: Vec<String>,
@ -203,6 +198,9 @@ pub struct Coordinator {
pub injection_log: Vec<InjectionOutcome>,
/// How many times an exact duplicate met IN_PROGRESS while resolving an uncertain call.
pub in_progress_replies: u64,
started: std::time::Instant,
last_advance_request: Option<DomainRequestId>,
last_commit_requests: Vec<TraceRequest>,
}
impl Coordinator {
@ -248,6 +246,9 @@ impl Coordinator {
injections: Injections::default(),
injection_log: Vec::new(),
in_progress_replies: 0,
started: std::time::Instant::now(),
last_advance_request: None,
last_commit_requests: Vec::new(),
}
}
@ -324,7 +325,7 @@ impl Coordinator {
}
fn scope(&self, step: u64) -> Scope {
Scope::new(&self.session_id, &self.epoch, step)
scope_at(&self.session_id, &self.epoch, step)
}
fn transition(&mut self, next: Phase) -> Outcome<()> {
@ -388,7 +389,7 @@ impl Coordinator {
supported_majors: vec![1],
};
let reply = self
.call(&worker, "Worker.Hello", None, &params, &[], &[])
.call(&worker, "Worker.Hello", None, object(params.to_json()), &[], &[])
.await?;
let result: HelloResult = reply.parse().map_err(|e| self.fail_now(e, "hello"))?;
if result.contract_digest != contract_digest() {
@ -409,7 +410,7 @@ impl Coordinator {
"hello",
));
}
let required = Id::lit(required);
let required = id(required);
if !result.capabilities.contains(&required) {
return Err(self.fail_now(
DomainError::before(
@ -447,7 +448,7 @@ impl Coordinator {
let error = DomainError::new(
ErrorCode::BackendFailure,
format!("declaring {name}: {}", e.message),
Mutation::None,
MutationCertainty::None,
);
self.fail_now(error, "declare-topic")
})?;
@ -471,7 +472,7 @@ impl Coordinator {
"counter-arena-setup-v1",
),
episode_id: self.episode_id.clone(),
port_bindings: bindings,
port_bindings: bindings.iter().map(PortBinding::pair).collect(),
};
let worker = self.environment.clone();
let scope = self.scope(0);
@ -480,7 +481,7 @@ impl Coordinator {
&worker,
"Environment.Initialize",
Some(scope),
&params,
object(params.to_json()),
&[],
&["view.arena".to_owned()],
)
@ -493,9 +494,9 @@ impl Coordinator {
.map_err(|e| self.fail_now(DomainError::invalid(e), "environment-descriptor"))?;
result
.observation
.validate(&result.descriptor)
.validate_against(&result.descriptor)
.map_err(|e| self.fail_now(DomainError::invalid(e), "observation-0"))?;
if result.observation.boundary.0 != 0 || !result.observation.world_time.is_zero() {
if result.observation.boundary != 0 || !result.observation.world_time.is_zero() {
return Err(self.fail_now(
DomainError::invalid("boundary 0 must have world time zero"),
"observation-0",
@ -518,7 +519,7 @@ impl Coordinator {
self.views = reply.artifacts;
self.descriptor = Some(result.descriptor);
self.observation = Some(result.observation);
self.lifecycle_acks.push((worker, reply.request_id.id()));
self.lifecycle_acks.push((worker, reply.request_id.clone()));
self.audit.push("environment.initialize".to_owned());
Ok(())
}
@ -535,7 +536,7 @@ impl Coordinator {
.bootstrap(&observation.inspection, &bindings)
.map_err(|e| self.fail_now(e, "task-bootstrap"))?;
for event in &bootstrap.events {
if event.source_step.0 != 0 {
if event.source_step != 0 {
return Err(self.fail_now(
DomainError::invalid("a bootstrap event has source step 0"),
"task-bootstrap",
@ -577,12 +578,12 @@ impl Coordinator {
let reply = {
let refs: Vec<(&str, &flybus::Artifact)> =
attachments.iter().map(|(n, a)| (n.as_str(), a)).collect();
self.call(&slot_worker, "Agent.Initialize", Some(scope), &params, &refs, &[])
self.call(&slot_worker, "Agent.Initialize", Some(scope), object(params.to_json()), &refs, &[])
.await?
};
let result: AgentInitializeResult =
reply.parse().map_err(|e| self.fail_now(e, "agent-initialize"))?;
if result.committed_step.0 != 0 {
if result.committed_step != 0 {
return Err(self.fail_now(
DomainError::invalid("Agent.Initialize must establish committed step 0"),
"agent-initialize",
@ -613,7 +614,7 @@ impl Coordinator {
self.agents[index].tick_duration = result.tick_duration;
self.agents[index].warmup_ticks = result.warmup_ticks;
self.agents[index].committed_step = 0;
self.lifecycle_acks.push((slot_worker, reply.request_id.id()));
self.lifecycle_acks.push((slot_worker, reply.request_id.clone()));
let agent_id = self.agents[index].agent_id.clone();
self.audit.push(format!("agent.initialize:{agent_id}"));
Ok(())
@ -625,7 +626,7 @@ impl Coordinator {
/// cache. It is not a bus `delivery.consumed`, which the SDK did when we dropped the
/// replies.
async fn acknowledge_lifecycle(&mut self) -> Outcome<()> {
let mut by_worker: BTreeMap<String, (WorkerRef, Vec<Id>)> = BTreeMap::new();
let mut by_worker: BTreeMap<String, (WorkerRef, Vec<DomainRequestId>)> = BTreeMap::new();
for (worker, request_id) in std::mem::take(&mut self.lifecycle_acks) {
by_worker
.entry(worker.service.clone())
@ -634,11 +635,11 @@ impl Coordinator {
.push(request_id);
}
for (worker, ids) in by_worker.into_values() {
let params = crate::types::AcknowledgeParams { request_ids: ids.clone() };
let params = AcknowledgeParams { request_ids: ids.clone() };
let reply = self
.call(&worker, "Worker.Acknowledge", None, &params, &[], &[])
.call(&worker, "Worker.Acknowledge", None, object(params.to_json()), &[], &[])
.await?;
let result: crate::types::AcknowledgeResult =
let result: AcknowledgeResult =
reply.parse().map_err(|e| self.fail_now(e, "acknowledge"))?;
if result.acknowledged.len() != ids.len() {
return Err(self.fail_now(
@ -654,25 +655,20 @@ impl Coordinator {
/// Queries one worker's status without waiting for its current mutation.
pub async fn status(&mut self, worker: &WorkerRef) -> Outcome<StatusResult> {
let reply = self
.call(worker, "Worker.Status", None, &json!({}), &[], &[])
.call(worker, "Worker.Status", None, Map::new(), &[], &[])
.await?;
reply.parse().map_err(|e| self.fail_now(e, "status"))
}
/// Asks one worker to stop. Only the configured supervisor may do this.
pub async fn shutdown(&mut self, worker: &WorkerRef, reason: &str) -> Outcome<()> {
let params = crate::types::ShutdownParams { reason: Id::lit(reason) };
let params = ShutdownParams { reason: id(reason) };
let reply = self
.call(worker, "Worker.Shutdown", None, &params, &[], &[])
.call(worker, "Worker.Shutdown", None, object(params.to_json()), &[], &[])
.await?;
let result: crate::types::ShutdownResult =
reply.parse().map_err(|e| self.fail_now(e, "shutdown"))?;
if !result.stopping {
return Err(self.fail_now(
DomainError::invalid("a responsive worker reports stopping:true"),
"shutdown",
));
}
// A responsive worker reports `stopping: true`, which is the only shape the contract
// type reads at all.
let _: ShutdownResult = reply.parse().map_err(|e| self.fail_now(e, "shutdown"))?;
Ok(())
}
@ -707,7 +703,7 @@ struct Job {
scope: Option<Scope>,
params: Map<String, Value>,
attachments: Vec<(String, flybus::Artifact)>,
request_id: RequestId,
request_id: DomainRequestId,
}
/// Issues one domain call with owned arguments, so it can run in its own task.
@ -719,7 +715,7 @@ async fn call_owned(
scope: Option<Scope>,
params: Map<String, Value>,
attachments: Vec<(String, flybus::Artifact)>,
request_id: RequestId,
request_id: DomainRequestId,
want: Vec<String>,
) -> Result<DomainReply, DomainError> {
let refs: Vec<(&str, &flybus::Artifact)> =
@ -728,21 +724,17 @@ async fn call_owned(
}
impl Coordinator {
/// Serializes `params`, takes the next request serial for this worker, calls, and checks
/// Takes the next request serial for this worker, calls, and checks
/// the reply's identity and echoed scope.
async fn call<P: serde::Serialize>(
async fn call(
&mut self,
worker: &WorkerRef,
method: &'static str,
scope: Option<Scope>,
params: &P,
params: Map<String, Value>,
attachments: &[(&str, &flybus::Artifact)],
want: &[String],
) -> Outcome<DomainReply> {
let params = match serde_json::to_value(params).expect("params serialize") {
Value::Object(m) => m,
_ => Map::new(),
};
let request_id = self.serials.next(&worker.service);
let owned: Vec<(String, flybus::Artifact)> = attachments
.iter()
@ -782,10 +774,10 @@ impl Coordinator {
method: &'static str,
) -> Outcome<()> {
let (worker_id, incarnation, echoed) = match &reply.outcome {
crate::types::SessionRpcOutcome::Success(s) => {
SessionRpcOutcome::Success(s) => {
(&s.worker_id, &s.incarnation_id, &s.scope)
}
crate::types::SessionRpcOutcome::Failure(f) => {
SessionRpcOutcome::Failure(f) => {
(&f.worker_id, &f.incarnation_id, &f.scope)
}
};
@ -812,7 +804,7 @@ impl Coordinator {
method,
));
}
if *reply.outcome.request_id() != reply.request_id.id() {
if *reply.outcome.request_id() != reply.request_id {
return Err(self.fail_now(
DomainError::before(
ErrorCode::IdentityMismatch,
@ -835,7 +827,7 @@ impl Coordinator {
scope: Option<Scope>,
params: Map<String, Value>,
attachments: Vec<(String, flybus::Artifact)>,
request_id: RequestId,
request_id: DomainRequestId,
want: &[String],
) -> Outcome<DomainReply> {
// The original may still be running, which answers IN_PROGRESS for this bus call and
@ -849,7 +841,7 @@ impl Coordinator {
scope.clone(),
params.clone(),
attachments.clone(),
request_id,
request_id.clone(),
want.to_vec(),
)
.await;
@ -872,7 +864,7 @@ impl Coordinator {
DomainError::new(
ErrorCode::BackendFailure,
"the uncertain operation never resolved",
Mutation::Unknown,
MutationCertainty::Unknown,
),
method,
))
@ -888,8 +880,8 @@ impl Coordinator {
scope: Option<Scope>,
params: Map<String, Value>,
attachments: Vec<(String, flybus::Artifact)>,
request_id: RequestId,
expected: Option<&Map<String, Value>>,
request_id: DomainRequestId,
expected: Option<&Value>,
what: &str,
) {
let reply = call_owned(
@ -936,7 +928,7 @@ impl Coordinator {
method: &'static str,
scope: Option<Scope>,
params: Value,
) -> Result<Map<String, Value>, DomainError> {
) -> Result<Value, DomainError> {
let params = match params {
Value::Object(m) => m,
_ => Map::new(),
@ -959,7 +951,7 @@ impl Coordinator {
/// The sensory input one agent is permitted to consume at `boundary`.
fn sensory_input(&self, observation: &WorldObservation, boundary: u64) -> SensoryInput {
SensoryInput {
boundary: U64(boundary),
boundary,
views: observation.sensory_views.clone(),
// This profile senses pixels only, so structured input stays null rather than
// smuggling inspection data into the neural path.
@ -1137,8 +1129,6 @@ impl Coordinator {
self.agents[index].context_digest = context.digest();
self.agents[index].context = context;
self.agents[index].committed_step = k + 1;
self.agents[index].prepared = None;
self.agents[index].prepare_request = None;
}
// The previous boundary's handles are no longer needed; the new ones take over.
self.views = new_views;
@ -1153,7 +1143,6 @@ impl Coordinator {
.collect();
self.record_trace(
k,
&descriptor,
&prepared,
&commits,
&controls,
@ -1162,6 +1151,12 @@ impl Coordinator {
&outcomes,
&event_ids,
);
// The prepared decisions and their request ids are needed by the trace, so they are
// released only after it has been recorded.
for slot in &mut self.agents {
slot.prepared = None;
slot.prepare_request = None;
}
self.publish_events(k + 1, &evaluation.events).await?;
self.publish_snapshot(k + 1, &decisions, &controls, &event_ids).await?;
@ -1195,7 +1190,7 @@ impl Coordinator {
}
fn batch_id(&self, k: u64) -> Id {
Id::parse(&format!("batch-{}-{k}", self.epoch)).expect("epoch and step make an Id")
parse_id(&format!("batch-{}-{k}", self.epoch)).expect("epoch and step make an Id")
}
/// Phase A. Every agent sees the same environment interval and the same world boundary.
@ -1219,14 +1214,19 @@ impl Coordinator {
// No audience input exists in the first synthetic composition.
pre_step_stimulations: Vec::<Stimulus>::new(),
};
let params = match serde_json::to_value(&params).expect("params serialize") {
let params = match params.to_json() {
Value::Object(m) => m,
_ => Map::new(),
};
let request_id = self.serials.next(&slot.worker.service);
self.agents[index].prepare_request = Some(request_id);
self.agents[index].prepare_request = Some(request_id.clone());
let slot = &self.agents[index];
bodies.push((slot.agent_id.clone(), slot.worker.clone(), params.clone(), request_id));
bodies.push((
slot.agent_id.clone(),
slot.worker.clone(),
params.clone(),
request_id.clone(),
));
jobs.push(Job {
agent_id: slot.agent_id.clone(),
worker: slot.worker.clone(),
@ -1298,10 +1298,7 @@ impl Coordinator {
let expected = prepared
.iter()
.find(|(id, _)| *id == target)
.map(|(_, decision)| match serde_json::to_value(decision).expect("serializes") {
Value::Object(m) => m,
_ => Map::new(),
});
.map(|(_, decision)| decision.to_json());
if let Some((agent_id, worker, params, request_id)) =
bodies.into_iter().find(|(id, _, _, _)| *id == target)
{
@ -1412,16 +1409,17 @@ impl Coordinator {
controls: &[PortControl],
) -> Outcome<StepResult> {
let scope = self.scope(k);
let params = crate::types::AdvanceParams {
let params = AdvanceParams {
batch_id: batch_id.clone(),
controls: controls.to_vec(),
};
let params = match serde_json::to_value(&params).expect("params serialize") {
let params = match params.to_json() {
Value::Object(m) => m,
_ => Map::new(),
};
let worker = self.environment.clone();
let request_id = self.serials.next(&worker.service);
self.last_advance_request = Some(request_id.clone());
let want = vec!["view.arena".to_owned()];
self.audit.push(format!("advance:{k}"));
@ -1436,12 +1434,14 @@ impl Coordinator {
&worker.service,
Some(&worker.bus_incarnation),
"Environment.Advance",
crate::types::SessionRpcRequest::new(
&request_id,
Some(scope.clone()),
params.clone(),
)
.to_payload(),
object(
SessionRpcRequest {
request_id: request_id.clone(),
scope: Some(scope.clone()),
params: Value::Object(params.clone()),
}
.to_json(),
),
&[],
)
.await
@ -1449,25 +1449,37 @@ impl Coordinator {
let error = DomainError::new(
ErrorCode::BackendFailure,
format!("Environment.Advance: bus {:?}", e.code),
Mutation::Unknown,
MutationCertainty::Unknown,
);
self.fail_now(error, "advance")
})?;
let state = pending.cancel().await.ok();
drop(pending);
self.injection_log.push(InjectionOutcome {
what: "lost-advance-result".to_owned(),
code: None,
identical: state == Some(flybus::CancelState::ExecutionUnknown)
|| state == Some(flybus::CancelState::Completed),
});
// A status probe first, exactly as the uncertain-call procedure says.
let status = self.status(&worker).await?;
// The loss has to happen *after* the world stepped, so the injection waits for
// the worker to report the operation before abandoning the call. A status probe
// is exactly what the uncertain-call procedure does first anyway.
let mut knows = false;
for _ in 0..500u32 {
let status = self.status(&worker).await?;
knows = status.last_batch_id.as_ref() == Some(batch_id)
|| status.active_request_id.as_ref() == Some(&request_id)
|| status.last_completed_request_id.as_ref() == Some(&request_id);
if knows {
break;
}
tokio::time::sleep(std::time::Duration::from_millis(2)).await;
}
self.injection_log.push(InjectionOutcome {
what: "status-after-loss".to_owned(),
code: None,
identical: status.last_batch_id.as_ref() == Some(batch_id)
|| status.active_request_id.as_ref() == Some(&request_id.id()),
identical: knows,
});
let state = pending.cancel().await.ok();
drop(pending);
// Cancellation after dispatch cannot undo the work: the execution outcome is
// uncertain, which is the case this injects.
self.injection_log.push(InjectionOutcome {
what: "lost-advance-result".to_owned(),
code: None,
identical: state != Some(flybus::CancelState::CancelledBeforeDispatch),
});
self.resolve(
&worker,
@ -1475,7 +1487,7 @@ impl Coordinator {
Some(scope.clone()),
params.clone(),
Vec::new(),
request_id,
request_id.clone(),
&want,
)
.await?
@ -1487,7 +1499,7 @@ impl Coordinator {
Some(scope.clone()),
params.clone(),
Vec::new(),
request_id,
request_id.clone(),
want.clone(),
)
.await;
@ -1517,22 +1529,16 @@ impl Coordinator {
{
button.down = !button.down;
}
let altered_params = match serde_json::to_value(&crate::types::AdvanceParams {
batch_id: batch_id.clone(),
controls: altered,
})
.expect("params serialize")
{
Value::Object(m) => m,
_ => Map::new(),
};
let altered_params = object(
AdvanceParams { batch_id: batch_id.clone(), controls: altered }.to_json(),
);
self.probe_duplicate(
&worker,
"Environment.Advance",
Some(scope.clone()),
altered_params,
Vec::new(),
request_id,
request_id.clone(),
None,
"altered-advance-controls",
)
@ -1554,7 +1560,7 @@ impl Coordinator {
Some(scope.clone()),
params.clone(),
Vec::new(),
request_id,
request_id.clone(),
&want,
)
.await?;
@ -1591,7 +1597,7 @@ impl Coordinator {
"step-result",
));
}
if result.applied_from_step.0 != k || result.next_step.0 != k + 1 {
if result.applied_from_step != k || result.next_step != k + 1 {
return Err(self.fail_now(
DomainError::before(
ErrorCode::IdentityMismatch,
@ -1609,7 +1615,7 @@ impl Coordinator {
"step-result",
));
}
if result.observation.boundary.0 != k + 1 {
if result.observation.boundary != k + 1 {
return Err(self.fail_now(
DomainError::before(ErrorCode::IdentityMismatch, "the observation is not k+1"),
"step-result",
@ -1643,22 +1649,59 @@ impl Coordinator {
));
}
// A missing required sensory input is never silently replaced by an older frame.
if let Err(e) = result.observation.validate(descriptor) {
// The contract's validator checks the views that are present against their
// descriptors; requiring each declared view to be there at all is the coordinator's
// Phase C check, so it is made here.
for view in &descriptor.views {
let want = required_produced_step(view, result.observation.boundary);
let got = result
.observation
.sensory_views
.iter()
.find(|given| given.view_id == view.view_id);
match got {
Some(given) if given.produced_step == want => {}
Some(_) => {
return Err(self.fail_now(
DomainError::new(
ErrorCode::BufferInvalid,
format!(
"view {} did not come from the boundary its declared delay requires",
view.view_id
),
MutationCertainty::Unknown,
),
"step-result",
));
}
None => {
return Err(self.fail_now(
DomainError::new(
ErrorCode::BufferInvalid,
format!("required sensory view {} is missing", view.view_id),
MutationCertainty::Unknown,
),
"step-result",
));
}
}
}
if let Err(e) = result.observation.validate_against(descriptor) {
return Err(self.fail_now(
DomainError::new(ErrorCode::BufferInvalid, e, Mutation::Unknown),
DomainError::new(ErrorCode::BufferInvalid, e, MutationCertainty::Unknown),
"step-result",
));
}
for view in &result.observation.sensory_views {
let name = format!("view.{}", view.view_id);
match self.pending_views.get(&name) {
Some(artifact) if artifact.reference() == &view.pixels.0 => {}
Some(artifact) if artifact.reference() == &view.pixels => {}
_ => {
return Err(self.fail_now(
DomainError::new(
ErrorCode::BufferInvalid,
format!("required view {} arrived without a live owned handle", view.view_id),
Mutation::Unknown,
MutationCertainty::Unknown,
),
"step-result",
));
@ -1686,25 +1729,30 @@ impl Coordinator {
let agent_id = self.agents[index].agent_id.clone();
let prepared_request = self.agents[index]
.prepare_request
.expect("every agent prepared")
.id();
.clone()
.expect("every agent prepared");
let outcome = outcomes.get(&agent_id).cloned().unwrap_or_default();
let next_context = next_contexts.get(&agent_id).cloned().expect("checked");
let params = CommitParams {
agent_id: agent_id.clone(),
prepared_request_id: prepared_request,
prepared_request_id: prepared_request.clone(),
next_input: self.sensory_input(observation, k + 1),
next_decision_context: next_context,
rewards: outcome.rewards.clone(),
task_stimulations: outcome.stimulations.clone(),
};
let params = match serde_json::to_value(&params).expect("params serialize") {
let params = match params.to_json() {
Value::Object(m) => m,
_ => Map::new(),
};
let request_id = self.serials.next(&self.agents[index].worker.service);
let worker = self.agents[index].worker.clone();
bodies.push((agent_id.clone(), worker.clone(), params.clone(), request_id));
bodies.push((
agent_id.clone(),
worker.clone(),
params.clone(),
request_id.clone(),
));
jobs.push(Job {
agent_id,
worker,
@ -1715,6 +1763,13 @@ impl Coordinator {
request_id,
});
}
self.last_commit_requests = bodies
.iter()
.map(|(agent_id, _, _, request_id)| TraceRequest {
agent_id: agent_id.clone(),
request_id: request_id.clone(),
})
.collect();
let results = self.run_jobs(jobs, self.dispatch).await;
let mut commits = Vec::new();
let mut first_failure = None;
@ -1733,7 +1788,7 @@ impl Coordinator {
Ok(result) => result,
Err(e) => return Err(self.fail_now(e, method)),
};
if result.committed_step.0 != k + 1 {
if result.committed_step != k + 1 {
return Err(self.fail_now(
DomainError::before(
ErrorCode::IdentityMismatch,
@ -1786,10 +1841,7 @@ impl Coordinator {
let expected = commits
.iter()
.find(|(id, _)| *id == target)
.map(|(_, result)| match serde_json::to_value(result).expect("serializes") {
Value::Object(m) => m,
_ => Map::new(),
});
.map(|(_, result)| result.to_json());
if let Some((_, worker, params, request_id)) =
bodies.into_iter().find(|(id, _, _, _)| *id == target)
{
@ -1811,11 +1863,15 @@ impl Coordinator {
}
/// Records the `step-v1` section 8 trace for this transition.
/// Records the `step-v1` section 8 trace for this transition.
///
/// Behaviour and operational metadata are separated by the contract type: the behaviour is
/// what a reordered run must reproduce exactly, and the request ids, batch correlation and
/// wall time are recorded beside it rather than inside it.
#[allow(clippy::too_many_arguments)]
fn record_trace(
&mut self,
k: u64,
_descriptor: &EnvironmentDescriptor,
prepared: &[(Id, PreparedDecision)],
commits: &[(Id, AgentCommitResult)],
controls: &[PortControl],
@ -1825,15 +1881,16 @@ impl Coordinator {
event_ids: &[Id],
) {
let mut agents = Vec::new();
let mut outcome_ids = Vec::new();
let mut prepare_request_ids = Vec::new();
for (agent_id, decision) in prepared {
let slot = self.agent(agent_id).expect("configured");
let committed = commits
.iter()
.find(|(id, _)| id == agent_id)
.map(|(_, result)| result.committed_step)
.unwrap_or(U64(0));
let outcome = outcomes.get(agent_id).cloned().unwrap_or_default();
agents.push(AgentTransitionTrace {
.unwrap_or(0);
agents.push(TraceAgent {
agent_id: agent_id.clone(),
profile_digest: slot.profile.digest.clone(),
ticks_advanced: decision.ticks_advanced,
@ -1841,43 +1898,53 @@ impl Coordinator {
remainder: decision.remainder,
decision_digest: decision.decision.digest(),
committed_step: committed,
rewards: outcome.rewards,
stimulations: outcome.stimulations,
prepare_request_id: slot
.prepare_request
.map(|r| r.id())
.unwrap_or_else(|| Id::lit("req-0")),
commit_request_id: Id::parse(&format!(
"req-{}",
self.serials.highest(&slot.worker.service)
))
.expect("a serial is an Id"),
});
if let Some(request_id) = slot.prepare_request.clone() {
prepare_request_ids.push(TraceRequest { agent_id: agent_id.clone(), request_id });
}
// Outcome ids in task order: the reward events this agent was routed.
if let Some(outcome) = outcomes.get(agent_id) {
for reward in &outcome.rewards {
outcome_ids.push(reward.event_id.clone());
}
}
}
let observation_boundaries = result
let mut observation_boundaries: Vec<TraceObservation> = result
.observation
.sensory_views
.iter()
.map(|view| ViewProvenance {
.map(|view| TraceObservation {
view_id: view.view_id.clone(),
produced_step: view.produced_step,
})
.collect();
self.trace.transition(TransitionTrace {
observation_boundaries.sort_by(|a, b| a.view_id.cmp(&b.view_id));
let behaviour = TraceBehaviour {
scope: self.scope(k),
agents,
controls_digest: controls_digest(controls),
batch_id: batch_id.clone(),
control_digest: controls_digest(controls),
acknowledged_boundary: result.next_step,
observation_boundaries,
task_event_ids: event_ids.to_vec(),
published_boundary: U64(k + 1),
batch_id: batch_id.clone(),
advance_request_id: Id::parse(&format!(
"req-{}",
self.serials.highest(&self.environment.service)
))
.expect("a serial is an Id"),
});
outcome_ids,
event_ids: event_ids.to_vec(),
published_boundary: k + 1,
};
let operational = TraceOperational {
// Wall time is for pacing, health and presentation only.
wall_time_ns: u64::try_from(self.started.elapsed().as_nanos()).unwrap_or(u64::MAX),
prepare_request_ids,
advance_request_id: self
.last_advance_request
.clone()
.unwrap_or_else(|| DomainRequestId::from_serial(0)),
commit_request_ids: self.last_commit_requests.clone(),
// SESSION-01 does not record transport correlation ids; a safe retry changes them
// and nothing in the behaviour above.
bus_call_ids: Vec::new(),
delivery_ids: Vec::new(),
};
self.trace.transition(TransitionTrace { behaviour, operational });
}
// -----------------------------------------------------------------------------------
@ -1899,7 +1966,7 @@ impl Coordinator {
let error = DomainError::new(
ErrorCode::BackendFailure,
format!("publishing {topic}: {}", e.message),
Mutation::None,
MutationCertainty::None,
);
Err(self.fail_now(error, "publish"))
}
@ -1916,7 +1983,7 @@ impl Coordinator {
"agentId": slot.agent_id.as_str(),
"portId": slot.port_id.as_str(),
"profileDigest": slot.profile.digest.as_str(),
"tickDuration": serde_json::to_value(slot.tick_duration).expect("rational"),
"tickDuration": slot.tick_duration.to_json(),
"warmupTicks": slot.warmup_ticks.to_string(),
})
})
@ -1926,8 +1993,8 @@ impl Coordinator {
"revision": "1",
"compositionDigest": self.composition_digest().as_str(),
"schedulerId": "lockstep-v1",
"environment": serde_json::to_value(&descriptor).expect("descriptor"),
"taskSchema": serde_json::to_value(self.task.schema()).expect("schema"),
"environment": descriptor.to_json(),
"taskSchema": self.task.schema().to_json(),
"agents": agents,
});
let topic = self.topics.descriptor.clone();
@ -1952,10 +2019,10 @@ impl Coordinator {
slot.agent_id, slot.port_id, slot.profile.digest
));
}
Digest::of(text.as_bytes())
digest_of_bytes(text.as_bytes())
}
async fn publish_events(&mut self, source_step: u64, events: &[crate::types::TaskEvent]) -> Outcome<()> {
async fn publish_events(&mut self, source_step: u64, events: &[TaskEvent]) -> Outcome<()> {
if events.is_empty() {
return Ok(());
}
@ -1963,7 +2030,7 @@ impl Coordinator {
"sessionId": self.session_id.as_str(),
"epoch": self.epoch.as_str(),
"sourceStep": source_step.to_string(),
"events": serde_json::to_value(events).expect("events"),
"events": Value::Array(events.iter().map(DomainType::to_json).collect()),
});
let topic = self.topics.events.clone();
self.publish(&topic, match payload {
@ -1999,12 +2066,12 @@ impl Coordinator {
let control = controls
.iter()
.find(|c| c.port_id == slot.port_id)
.map(|c| serde_json::to_value(c).expect("control"));
.map(|c| c.to_json());
json!({
"agentId": slot.agent_id.as_str(),
"selectedDecision": decisions
.get(&slot.agent_id)
.map(|d| serde_json::to_value(d).expect("decision")),
.map(|d| d.to_json()),
"appliedControls": control,
"committedStep": slot.committed_step.to_string(),
})
@ -2013,14 +2080,14 @@ impl Coordinator {
let payload = json!({
"descriptorRevision": "1",
"publisherIncarnation": self.bus.info().connection_id.clone(),
"scope": serde_json::to_value(self.scope(boundary)).expect("scope"),
"scope": self.scope(boundary).to_json(),
"episodeId": self.episode_id.as_str(),
"sequence": self.stats.publications.to_string(),
"worldTime": serde_json::to_value(observation.world_time).expect("rational"),
"worldTime": observation.world_time.to_json(),
"agents": agents,
"progress": serde_json::to_value(self.task.progress()).expect("progress"),
"progress": self.task.progress().to_json(),
"media": json!({
"views": serde_json::to_value(&observation.broadcast_views).expect("views"),
"views": Value::Array(observation.broadcast_views.iter().map(DomainType::to_json).collect()),
"audio": [],
}),
"eventIds": event_ids.iter().map(Id::as_str).collect::<Vec<_>>(),

130
services/flysim/crates/fly-session/src/dedup.rs
View file

@ -11,7 +11,9 @@
use std::collections::{BTreeMap, VecDeque};
use crate::types::{Digest, DomainError, ErrorCode, Id, RequestId, SessionRpcOutcome};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
// glob keeps the contract's own names in sight instead of restating them.
use crate::types::*;
/// `ipc-v1` section 5: unacknowledged lifecycle replies are bounded at 16, then BUSY.
pub const MAX_UNACKNOWLEDGED: usize = 16;
@ -88,7 +90,7 @@ pub enum Admission {
#[derive(Clone, Debug)]
struct Record {
request_id: RequestId,
request_id: DomainRequestId,
body: Digest,
reply: CachedReply,
}
@ -96,10 +98,10 @@ struct Record {
/// One worker's domain request cache.
pub struct ResultCache {
steps: BTreeMap<OperationKey, Record>,
active: BTreeMap<OperationKey, (RequestId, Digest)>,
lifecycle: BTreeMap<Id, Record>,
lifecycle_order: VecDeque<Id>,
readonly: VecDeque<(Id, CachedReply)>,
active: BTreeMap<OperationKey, (DomainRequestId, Digest)>,
lifecycle: BTreeMap<String, Record>,
lifecycle_order: VecDeque<String>,
readonly: VecDeque<(String, CachedReply)>,
highest_serial: Option<u64>,
step_watermark: Option<u64>,
/// Serials retired by `Worker.Acknowledge`; reuse below this is refused without keeping a
@ -132,7 +134,7 @@ impl ResultCache {
&mut self,
class: OpClass,
key: &OperationKey,
request: RequestId,
request: DomainRequestId,
body: &Digest,
) -> Admission {
match class {
@ -142,7 +144,7 @@ impl ResultCache {
}
}
fn admit_step(&mut self, key: &OperationKey, request: RequestId, body: &Digest) -> Admission {
fn admit_step(&mut self, key: &OperationKey, request: DomainRequestId, body: &Digest) -> Admission {
if let Some(record) = self.steps.get(key) {
if record.request_id == request && record.body == *body {
return Admission::Replay(record.reply.clone());
@ -151,7 +153,7 @@ impl ResultCache {
ErrorCode::Conflict,
"this operation key already holds a different request or body",
// The earlier result stands; refusing the duplicate undoes nothing.
crate::types::Mutation::None,
MutationCertainty::None,
));
}
if !self.active.is_empty() && !self.active.contains_key(key) {
@ -164,16 +166,17 @@ impl ResultCache {
}
if let Some((active_request, active_body)) = self.active.get(key) {
if *active_request == request && *active_body == *body {
return Admission::Refuse(DomainError::new(
// The duplicate bus call started no work at all, so its certainty is none;
// it must not be mistaken for the original operation's failure.
return Admission::Refuse(DomainError::before(
ErrorCode::InProgress,
"the original operation is still executing; this duplicate started no work",
crate::types::Mutation::Unknown,
));
}
return Admission::Refuse(DomainError::new(
ErrorCode::Conflict,
"an operation with a different body is active for this key",
crate::types::Mutation::None,
MutationCertainty::None,
));
}
// No record and nothing active. Eviction must never re-enable execution, so a step
@ -181,12 +184,12 @@ impl ResultCache {
if let Some(watermark) = self.step_watermark
&& key.step + RETAINED_STEPS <= watermark
{
let issued = self.highest_serial.is_some_and(|h| request.0 <= h);
let issued = self.highest_serial.is_some_and(|h| request.serial() <= h);
return Admission::Refuse(if issued {
DomainError::new(
ErrorCode::ResultExpired,
"the retained result for this request is gone; it is never recomputed",
crate::types::Mutation::Unknown,
MutationCertainty::Unknown,
)
} else {
DomainError::before(
@ -196,20 +199,20 @@ impl ResultCache {
});
}
if let Some(watermark) = self.acknowledged_watermark
&& request.0 <= watermark
&& request.serial() <= watermark
{
return Admission::Refuse(DomainError::new(
ErrorCode::ResultExpired,
"this request serial was acknowledged and cannot be reused",
crate::types::Mutation::Unknown,
MutationCertainty::Unknown,
));
}
self.begin(key.clone(), request, body.clone());
Admission::Execute
}
fn admit_lifecycle(&mut self, request: RequestId, body: &Digest) -> Admission {
let id = request.id();
fn admit_lifecycle(&mut self, request: DomainRequestId, body: &Digest) -> Admission {
let id = request.as_str().to_owned();
if let Some(record) = self.lifecycle.get(&id) {
if record.body == *body {
return Admission::Replay(record.reply.clone());
@ -220,12 +223,12 @@ impl ResultCache {
));
}
if let Some(watermark) = self.acknowledged_watermark
&& request.0 <= watermark
&& request.serial() <= watermark
{
return Admission::Refuse(DomainError::new(
ErrorCode::ResultExpired,
"this request serial was acknowledged and cannot be reused",
crate::types::Mutation::Unknown,
MutationCertainty::Unknown,
));
}
if self.lifecycle.len() >= MAX_UNACKNOWLEDGED {
@ -237,8 +240,8 @@ impl ResultCache {
Admission::Execute
}
fn begin(&mut self, key: OperationKey, request: RequestId, body: Digest) {
self.highest_serial = Some(self.highest_serial.map_or(request.0, |h| h.max(request.0)));
fn begin(&mut self, key: OperationKey, request: DomainRequestId, body: Digest) {
self.highest_serial = Some(self.highest_serial.map_or(request.serial(), |h| h.max(request.serial())));
self.step_watermark = Some(self.step_watermark.map_or(key.step, |w| w.max(key.step)));
self.active.insert(key, (request, body));
}
@ -247,7 +250,7 @@ impl ResultCache {
pub fn record(
&mut self,
key: OperationKey,
request: RequestId,
request: DomainRequestId,
body: Digest,
reply: CachedReply,
) {
@ -259,9 +262,9 @@ impl ResultCache {
}
/// Stores a lifecycle reply, retained until `Worker.Acknowledge`.
pub fn record_lifecycle(&mut self, request: RequestId, body: Digest, reply: CachedReply) {
let id = request.id();
self.highest_serial = Some(self.highest_serial.map_or(request.0, |h| h.max(request.0)));
pub fn record_lifecycle(&mut self, request: DomainRequestId, body: Digest, reply: CachedReply) {
let id = request.as_str().to_owned();
self.highest_serial = Some(self.highest_serial.map_or(request.serial(), |h| h.max(request.serial())));
if self.lifecycle.insert(id.clone(), Record { request_id: request, body, reply }).is_none()
{
self.lifecycle_order.push_back(id);
@ -269,8 +272,8 @@ impl ResultCache {
}
/// Stores a read-only reply in the last-16 cache.
pub fn record_readonly(&mut self, request: RequestId, reply: CachedReply) {
let id = request.id();
pub fn record_readonly(&mut self, request: DomainRequestId, reply: CachedReply) {
let id = request.as_str().to_owned();
self.readonly.retain(|(existing, _)| *existing != id);
self.readonly.push_back((id, reply));
while self.readonly.len() > MAX_READONLY_REPLIES {
@ -285,14 +288,14 @@ impl ResultCache {
/// `Worker.Acknowledge`: drops those lifecycle records, ignoring unknown ids, and raises
/// the serial watermark so an acknowledged id cannot be reused.
pub fn acknowledge(&mut self, ids: &[Id]) -> Vec<Id> {
pub fn acknowledge(&mut self, ids: &[DomainRequestId]) -> Vec<DomainRequestId> {
let mut out = Vec::new();
for id in ids {
if let Some(record) = self.lifecycle.remove(id) {
self.lifecycle_order.retain(|existing| existing != id);
if let Some(record) = self.lifecycle.remove(id.as_str()) {
self.lifecycle_order.retain(|existing| existing != id.as_str());
self.acknowledged_watermark = Some(
self.acknowledged_watermark
.map_or(record.request_id.0, |w| w.max(record.request_id.0)),
.map_or(record.request_id.serial(), |w| w.max(record.request_id.serial())),
);
out.push(id.clone());
}
@ -326,16 +329,17 @@ impl ResultCache {
#[cfg(test)]
mod tests {
use super::*;
use crate::types::{Scope, SessionRpcSuccess, SuccessTag};
use serde_json::Value;
use super::*;
fn key(step: u64, method: &str) -> OperationKey {
OperationKey {
session_id: Id::lit("demo"),
epoch: Id::lit("e1"),
session_id: id("demo"),
epoch: id("e1"),
step,
method: method.to_owned(),
worker_id: Id::lit("fly-a"),
worker_id: id("fly-a"),
}
}
@ -343,26 +347,25 @@ mod tests {
let mut result = serde_json::Map::new();
result.insert("tag".into(), tag.into());
CachedReply::new(SessionRpcOutcome::Success(SessionRpcSuccess {
kind: SuccessTag::Result,
request_id: Id::lit("req-1"),
worker_id: Id::lit("fly-a"),
incarnation_id: Id::lit("inc-1"),
scope: Some(Scope::new(&Id::lit("demo"), &Id::lit("e1"), 0)),
result,
request_id: DomainRequestId::from_serial(1),
worker_id: id("fly-a"),
incarnation_id: id("inc-1"),
scope: Some(scope_at("demo", "e1", 0)),
result: Value::Object(result),
}))
}
fn body(s: &str) -> Digest {
Digest::of(s.as_bytes())
digest_of_bytes(s.as_bytes())
}
#[test]
fn the_same_key_request_and_body_replays() {
let mut c = ResultCache::new();
let k = key(0, "Agent.Prepare");
assert!(matches!(c.admit(OpClass::StepMutation, &k, RequestId(1), &body("a")), Admission::Execute));
c.record(k.clone(), RequestId(1), body("a"), reply("first"));
match c.admit(OpClass::StepMutation, &k, RequestId(1), &body("a")) {
assert!(matches!(c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a")), Admission::Execute));
c.record(k.clone(), DomainRequestId::from_serial(1), body("a"), reply("first"));
match c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a")) {
Admission::Replay(r) => {
assert_eq!(r.outcome.result().unwrap()["tag"], "first");
}
@ -374,9 +377,9 @@ mod tests {
fn a_changed_body_for_a_recorded_key_is_a_conflict() {
let mut c = ResultCache::new();
let k = key(0, "Environment.Advance");
c.admit(OpClass::StepMutation, &k, RequestId(1), &body("a"));
c.record(k.clone(), RequestId(1), body("a"), reply("first"));
match c.admit(OpClass::StepMutation, &k, RequestId(1), &body("b")) {
c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a"));
c.record(k.clone(), DomainRequestId::from_serial(1), body("a"), reply("first"));
match c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("b")) {
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::Conflict),
other => panic!("wanted CONFLICT, got {other:?}"),
}
@ -386,8 +389,8 @@ mod tests {
fn a_duplicate_while_the_original_runs_is_in_progress() {
let mut c = ResultCache::new();
let k = key(0, "Agent.Prepare");
c.admit(OpClass::StepMutation, &k, RequestId(1), &body("a"));
match c.admit(OpClass::StepMutation, &k, RequestId(1), &body("a")) {
c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a"));
match c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(1), &body("a")) {
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::InProgress),
other => panic!("wanted IN_PROGRESS, got {other:?}"),
}
@ -398,17 +401,17 @@ mod tests {
let mut c = ResultCache::new();
for step in 0..4u64 {
let k = key(step, "Agent.Prepare");
c.admit(OpClass::StepMutation, &k, RequestId(step + 1), &body("a"));
c.record(k, RequestId(step + 1), body("a"), reply("x"));
c.admit(OpClass::StepMutation, &k, DomainRequestId::from_serial(step + 1), &body("a"));
c.record(k, DomainRequestId::from_serial(step + 1), body("a"), reply("x"));
}
assert_eq!(c.retained_steps(), 2);
// req-1 named step 0, whose record is long gone.
match c.admit(OpClass::StepMutation, &key(0, "Agent.Prepare"), RequestId(1), &body("a")) {
match c.admit(OpClass::StepMutation, &key(0, "Agent.Prepare"), DomainRequestId::from_serial(1), &body("a")) {
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::ResultExpired),
other => panic!("wanted RESULT_EXPIRED, got {other:?}"),
}
// A serial above the highest issued is a new operation naming an old step.
match c.admit(OpClass::StepMutation, &key(0, "Agent.Prepare"), RequestId(99), &body("a")) {
match c.admit(OpClass::StepMutation, &key(0, "Agent.Prepare"), DomainRequestId::from_serial(99), &body("a")) {
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::StaleStep),
other => panic!("wanted STALE_STEP, got {other:?}"),
}
@ -419,20 +422,23 @@ mod tests {
let mut c = ResultCache::new();
for serial in 1..=MAX_UNACKNOWLEDGED as u64 {
assert!(matches!(
c.admit(OpClass::Lifecycle, &key(0, "Agent.Initialize"), RequestId(serial), &body("a")),
c.admit(OpClass::Lifecycle, &key(0, "Agent.Initialize"), DomainRequestId::from_serial(serial), &body("a")),
Admission::Execute
));
c.record_lifecycle(RequestId(serial), body("a"), reply("init"));
c.record_lifecycle(DomainRequestId::from_serial(serial), body("a"), reply("init"));
}
match c.admit(OpClass::Lifecycle, &key(0, "Agent.Initialize"), RequestId(99), &body("a")) {
match c.admit(OpClass::Lifecycle, &key(0, "Agent.Initialize"), DomainRequestId::from_serial(99), &body("a")) {
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::Busy),
other => panic!("wanted BUSY, got {other:?}"),
}
let dropped = c.acknowledge(&[Id::lit("req-1"), Id::lit("req-404")]);
assert_eq!(dropped, vec![Id::lit("req-1")]);
let dropped = c.acknowledge(&[
DomainRequestId::from_serial(1),
DomainRequestId::from_serial(404),
]);
assert_eq!(dropped, vec![DomainRequestId::from_serial(1)]);
assert_eq!(c.unacknowledged(), MAX_UNACKNOWLEDGED - 1);
// The acknowledged serial cannot come back.
match c.admit(OpClass::Lifecycle, &key(0, "Agent.Initialize"), RequestId(1), &body("a")) {
match c.admit(OpClass::Lifecycle, &key(0, "Agent.Initialize"), DomainRequestId::from_serial(1), &body("a")) {
Admission::Refuse(e) => assert_eq!(e.code, ErrorCode::ResultExpired),
other => panic!("wanted RESULT_EXPIRED, got {other:?}"),
}

102
services/flysim/crates/fly-session/src/environment.rs
View file

@ -9,18 +9,14 @@ use std::collections::BTreeSet;
use std::io::Write;
use crate::task::{controller_schema_ref, inspection, inspection_schema};
use crate::types::{
AdvanceParams, AxisRange, AxisSchema, ControllerSchema, Determinism, Digest, DomainError,
DomainResult, EnvironmentDescriptor, EnvironmentInitializeParams, EnvironmentInitializeResult,
ErrorCode, Id, MAX_PORTS, Mutation, PixelAspect, PortBinding, PortControl, PortDescriptor,
RationalNs, Recovery, Role, Scope, StepResult, U64, ViewDescriptor, ViewFormat, ViewRef,
WorkerState, WorldObservation, controls_digest,
};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
// glob keeps the contract's own names in sight instead of restating them.
use crate::types::*;
use crate::worker::{BoxFuture, HandlerCtx, HandlerReply, StatusCell, WorkerEndpoint};
/// The arena's view: a 4x4 RGBA8 tile whose bytes carry the counter.
pub const VIEW_WIDTH: u32 = 4;
pub const VIEW_HEIGHT: u32 = 4;
pub const VIEW_WIDTH: u64 = 4;
pub const VIEW_HEIGHT: u64 = 4;
/// Deliberate faults a test can ask the environment for.
#[derive(Clone, Debug, Default)]
@ -68,7 +64,7 @@ impl CounterEnvironment {
bindings: Vec::new(),
boundary: 0,
counter: 0,
world_time: RationalNs::zero(),
world_time: RationalNs::ZERO,
advances: 0,
batches: BTreeSet::new(),
config,
@ -96,9 +92,9 @@ impl CounterEnvironment {
pub fn controller_schema() -> ControllerSchema {
ControllerSchema {
schema: controller_schema_ref(),
buttons: vec![Id::lit("inc"), Id::lit("dec")],
buttons: vec![id("inc"), id("dec")],
axes: vec![AxisSchema {
id: Id::lit("bias"),
id: id("bias"),
range: AxisRange::Bipolar,
neutral: 0.0,
}],
@ -107,21 +103,21 @@ impl CounterEnvironment {
pub fn view_descriptor() -> ViewDescriptor {
ViewDescriptor {
view_id: Id::lit("arena"),
view_id: id("arena"),
width: VIEW_WIDTH,
height: VIEW_HEIGHT,
format: ViewFormat::Rgba8,
row_stride: VIEW_WIDTH * 4,
pixel_aspect: PixelAspect { numerator: 1, denominator: 1 },
pixel_aspect_numerator: 1,
pixel_aspect_denominator: 1,
observation_delay_steps: 0,
}
}
fn build_descriptor(&self) -> DomainResult<EnvironmentDescriptor> {
let descriptor = EnvironmentDescriptor {
backend_digest: Digest::of(b"counter-arena-backend-v1"),
content_digest: Digest::of(b"counter-arena-content-v1"),
configuration_digest: Digest::of(
backend_digest: digest_of_bytes(b"counter-arena-backend-v1"),
content_digest: digest_of_bytes(b"counter-arena-content-v1"),
configuration_digest: digest_of_bytes(
format!(
"counter-arena-config-v1\nstep={}/{}\nports={}\n",
self.config.step_duration.numerator,
@ -166,7 +162,7 @@ impl CounterEnvironment {
DomainError::new(
ErrorCode::BackendFailure,
format!("frame allocation failed: {}", e.message),
Mutation::Applied,
MutationCertainty::Applied,
)
})?;
// Every pixel carries the counter's low byte, so an agent reading the frame reads the
@ -178,20 +174,20 @@ impl CounterEnvironment {
DomainError::new(
ErrorCode::BackendFailure,
format!("frame write failed: {e}"),
Mutation::Applied,
MutationCertainty::Applied,
)
})?;
let artifact = writer.seal().await.map_err(|e| {
DomainError::new(
ErrorCode::BackendFailure,
format!("frame seal failed: {}", e.message),
Mutation::Applied,
MutationCertainty::Applied,
)
})?;
let view = ViewRef {
view_id: descriptor.view_id.clone(),
produced_step: U64(descriptor.required_produced_step(self.boundary)),
pixels: crate::types::ArtifactRef(artifact.reference().clone()),
produced_step: descriptor.required_produced_step(self.boundary),
pixels: artifact.reference().clone(),
};
Ok((view, artifact))
}
@ -209,7 +205,7 @@ impl CounterEnvironment {
(vec![view], vec![(name, artifact)])
};
let observation = WorldObservation {
boundary: U64(self.boundary),
boundary: self.boundary,
world_time: self.world_time,
engine_frame: Some(self.boundary.to_string()),
sensory_views: views.clone(),
@ -235,38 +231,42 @@ impl CounterEnvironment {
"this environment belongs to another session",
));
}
if scope.step.0 != 0 {
if scope.step != 0 {
return Err(DomainError::before(
ErrorCode::FutureStep,
"Environment.Initialize uses the new epoch at step 0",
));
}
let params: EnvironmentInitializeParams = ctx.params()?;
if params.port_bindings.len() > MAX_PORTS as usize {
if params.port_bindings.len() > MAX_PORTS {
return Err(DomainError::invalid("at most 4 ports in the first composition"));
}
let mut seen = BTreeSet::new();
for binding in &params.port_bindings {
if !self.config.ports.contains(&binding.port_id) {
for (port_id, _agent_id) in &params.port_bindings {
if !self.config.ports.contains(port_id) {
return Err(DomainError::before(
ErrorCode::IdentityMismatch,
format!("port {} is not a port of this arena", binding.port_id),
format!("port {port_id} is not a port of this arena"),
));
}
if !seen.insert(binding.port_id.clone()) {
return Err(DomainError::invalid(format!(
"port {} is bound twice",
binding.port_id
)));
if !seen.insert(port_id.clone()) {
return Err(DomainError::invalid(format!("port {port_id} is bound twice")));
}
}
let descriptor = self.build_descriptor()?;
self.epoch = Some(scope.epoch.clone());
self.episode_id = Some(params.episode_id.clone());
self.bindings = params.port_bindings.clone();
self.bindings = params
.port_bindings
.iter()
.map(|(port_id, agent_id)| PortBinding {
port_id: port_id.clone(),
agent_id: agent_id.clone(),
})
.collect();
self.boundary = 0;
self.counter = 0;
self.world_time = RationalNs::zero();
self.world_time = RationalNs::ZERO;
self.batches.clear();
self.descriptor = Some(descriptor.clone());
// The world is stopped when O[0] goes out and cannot free-run while the brains boot.
@ -304,9 +304,9 @@ impl CounterEnvironment {
));
}
}
if scope.step.0 != self.boundary {
if scope.step != self.boundary {
return Err(DomainError::before(
if scope.step.0 < self.boundary {
if scope.step < self.boundary {
ErrorCode::StaleStep
} else {
ErrorCode::FutureStep
@ -337,11 +337,11 @@ impl CounterEnvironment {
self.world_time = self
.world_time
.checked_add(&descriptor.step_duration)
.map_err(|e| DomainError::new(ErrorCode::Internal, e, Mutation::Applied))?;
.map_err(|e| DomainError::new(ErrorCode::Internal, e, MutationCertainty::Applied))?;
self.advances += 1;
self.batches.insert(params.batch_id.clone());
self.status.set_batch(params.batch_id.clone());
self.status.set_scope(Some(Scope::new(
self.status.set_scope(Some(scope_at(
&scope.session_id,
&scope.epoch,
self.boundary,
@ -359,8 +359,8 @@ impl CounterEnvironment {
// The record of batch id, result and next boundary exists before the acknowledgment.
let result = StepResult {
batch_id: params.batch_id,
applied_from_step: U64(applied_from),
next_step: U64(self.boundary),
applied_from_step: applied_from,
next_step: self.boundary,
applied_controls_digest: digest,
observation,
};
@ -415,9 +415,9 @@ impl WorkerEndpoint for CounterEnvironment {
fn capabilities(&self) -> Vec<Id> {
vec![
Id::lit("world-step-v1"),
Id::lit("pixel-observation-v1"),
Id::lit("checkpoint-v1"),
id("world-step-v1"),
id("pixel-observation-v1"),
id("checkpoint-v1"),
]
}
@ -444,11 +444,11 @@ impl WorkerEndpoint for CounterEnvironment {
}
/// A synthetic backend/task configuration asset for the arena.
pub fn synthetic_asset(id: &str, body: &str) -> crate::types::AssetRef {
crate::types::AssetRef {
id: Id::lit(id),
digest: Digest::of(body.as_bytes()),
byte_length: U64(body.len() as u64),
format: Id::lit("fly-config-v1"),
pub fn synthetic_asset(asset_id: &str, body: &str) -> AssetRef {
AssetRef {
id: id(asset_id),
digest: digest_of_bytes(body.as_bytes()),
byte_length: body.len() as u64,
format: id("fly-config-v1"),
}
}

139
services/flysim/crates/fly-session/src/fly_session_types/canonical.rs
View file

@ -1,139 +0,0 @@
//! Canonical JSON (RFC 8785) and the digests built on it.
//!
//! Two payloads mean the same domain operation when their canonical encodings are equal, so a
//! duplicate `Agent.Prepare` can be told from a changed one without depending on key order,
//! whitespace or float formatting. Object keys sort by UTF-16 code unit; numbers use the
//! ECMAScript `Number::toString` shortest form (`ryu_js`), so `1.0` and `1` are one value.
use serde_json::Value;
use super::scalars::Digest;
/// The canonical JSON encoding of `value`.
///
/// Panics on a non-finite number, which cannot appear in a validated payload and cannot be
/// represented in JSON at all.
pub fn canonical_json(value: &Value) -> String {
let mut out = String::new();
write_value(&mut out, value);
out
}
/// The SHA-256 of the canonical encoding.
pub fn canonical_digest(value: &Value) -> Digest {
Digest::of(canonical_json(value).as_bytes())
}
fn write_value(out: &mut String, value: &Value) {
match value {
Value::Null => out.push_str("null"),
Value::Bool(true) => out.push_str("true"),
Value::Bool(false) => out.push_str("false"),
Value::Number(n) => write_number(out, n),
Value::String(s) => write_string(out, s),
Value::Array(items) => {
out.push('[');
for (i, item) in items.iter().enumerate() {
if i > 0 {
out.push(',');
}
write_value(out, item);
}
out.push(']');
}
Value::Object(map) => {
let mut keys: Vec<&String> = map.keys().collect();
keys.sort_by(|a, b| utf16_cmp(a, b));
out.push('{');
for (i, key) in keys.iter().enumerate() {
if i > 0 {
out.push(',');
}
write_string(out, key);
out.push(':');
write_value(out, &map[*key]);
}
out.push('}');
}
}
}
fn write_number(out: &mut String, n: &serde_json::Number) {
if let Some(u) = n.as_u64() {
out.push_str(&u.to_string());
return;
}
if let Some(i) = n.as_i64() {
out.push_str(&i.to_string());
return;
}
let f = n.as_f64().expect("a JSON number is representable");
assert!(f.is_finite(), "canonical JSON cannot encode a non-finite number");
let mut buf = ryu_js::Buffer::new();
out.push_str(buf.format(f));
}
fn write_string(out: &mut String, s: &str) {
out.push('"');
for c in s.chars() {
match c {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\u{08}' => out.push_str("\\b"),
'\u{0c}' => out.push_str("\\f"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if (c as u32) < 0x20 => {
out.push_str(&format!("\\u{:04x}", c as u32));
}
c => out.push(c),
}
}
out.push('"');
}
/// Compares two strings by UTF-16 code unit, which is what RFC 8785 sorts object keys by.
///
/// Byte order and code-unit order disagree only above the BMP, so the surrogate expansion
/// matters for a key containing an astral character.
fn utf16_cmp(a: &str, b: &str) -> std::cmp::Ordering {
let mut ai = a.encode_utf16();
let mut bi = b.encode_utf16();
loop {
match (ai.next(), bi.next()) {
(None, None) => return std::cmp::Ordering::Equal,
(None, Some(_)) => return std::cmp::Ordering::Less,
(Some(_), None) => return std::cmp::Ordering::Greater,
(Some(x), Some(y)) if x == y => continue,
(Some(x), Some(y)) => return x.cmp(&y),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn keys_sort_and_floats_take_their_shortest_form() {
let v = json!({"b": 1.0, "a": [1, 2.5], "\u{00e9}": "x"});
assert_eq!(canonical_json(&v), "{\"a\":[1,2.5],\"b\":1,\"\u{00e9}\":\"x\"}");
}
#[test]
fn the_same_object_written_two_ways_has_one_digest() {
let a: Value = serde_json::from_str("{\"x\":1,\"y\":{\"p\":2,\"q\":3}}").unwrap();
let b: Value = serde_json::from_str("{\"y\":{\"q\":3,\"p\":2},\"x\":1}").unwrap();
assert_eq!(canonical_digest(&a), canonical_digest(&b));
}
#[test]
fn an_astral_key_sorts_by_code_unit_not_by_byte() {
// U+10000 encodes as the surrogate pair D800 DC00, which is below U+FFFD's E000-range
// code unit in UTF-16 but above it byte-wise.
let v = json!({"\u{10000}": 1, "\u{fffd}": 2});
assert_eq!(canonical_json(&v), "{\"\u{10000}\":1,\"\u{fffd}\":2}");
}
}

1026
services/flysim/crates/fly-session/src/fly_session_types/methods.rs
View file

File diff suppressed because it is too large Load diff

28
services/flysim/crates/fly-session/src/fly_session_types/mod.rs
View file

@ -1,28 +0,0 @@
//! `fly-session-types` as a local module.
//!
//! CONTRACT-01 owns these definitions -- the scalar types, the method payloads, their
//! validation, the canonical digests and the trace format -- as the crate
//! `services/flysim/crates/fly-session-types`. Until that crate exists this module holds the
//! minimum this slice needs, under the same names and field shapes, so the swap is a change of
//! `use` lines in `lib.rs` and nothing else.
pub mod canonical;
pub mod methods;
pub mod scalars;
pub mod trace;
pub use canonical::{canonical_digest, canonical_json};
pub use methods::*;
pub use scalars::*;
pub use trace::*;
/// The identity of the contract revision these types implement.
///
/// A worker reports it in `Worker.Hello`; a coordinator refuses a worker whose value differs,
/// which is what stops a session from mixing two payload revisions.
pub const CONTRACT: &str = "fly-session-types/ipc-v1+step-v1+workers-v1 draft-2 2026-09-18";
/// The SHA-256 of [`CONTRACT`].
pub fn contract_digest() -> Digest {
Digest::of(CONTRACT.as_bytes())
}

433
services/flysim/crates/fly-session/src/fly_session_types/scalars.rs
View file

@ -1,433 +0,0 @@
//! The scalar encodings of `ipc-v1` section 2: `Id`, `U64`, `Digest`, `Scope`, `RationalNs`,
//! `SchemaRef` and `TypedValue`.
//!
//! Every type parses strictly. A `U64` is a canonical decimal string, never a JSON number; a
//! `RationalNs` is reduced with a positive denominator; an `Id` matches
//! `^[a-z0-9][a-z0-9._-]{0,63}$`. Nothing here knows about a method, a phase or a bus.
use std::fmt;
use serde::de::Error as _;
use serde::{Deserialize, Deserializer, Serialize, Serializer};
/// `^[a-z0-9][a-z0-9._-]{0,63}$`.
#[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Id(String);
impl Id {
/// Parses an `Id`, or reports why the string is not one.
pub fn parse(s: &str) -> Result<Id, String> {
if s.is_empty() || s.len() > 64 {
return Err(format!("id must be 1..=64 characters, got {}", s.len()));
}
let first = s.as_bytes()[0];
if !first.is_ascii_lowercase() && !first.is_ascii_digit() {
return Err("id must start with a lowercase letter or a digit".to_owned());
}
for b in s.bytes() {
let ok = b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'.' || b == b'_'
|| b == b'-';
if !ok {
return Err("id may only contain [a-z0-9._-]".to_owned());
}
}
Ok(Id(s.to_owned()))
}
/// Parses an `Id` from a trusted literal; panics on a malformed one.
pub fn lit(s: &str) -> Id {
Id::parse(s).expect("malformed Id literal")
}
pub fn as_str(&self) -> &str {
&self.0
}
}
impl fmt::Debug for Id {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{:?}", self.0)
}
}
impl fmt::Display for Id {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.0)
}
}
impl Serialize for Id {
fn serialize<S: Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
s.serialize_str(&self.0)
}
}
impl<'de> Deserialize<'de> for Id {
fn deserialize<D: Deserializer<'de>>(d: D) -> Result<Id, D::Error> {
let s = String::deserialize(d)?;
Id::parse(&s).map_err(D::Error::custom)
}
}
/// `"0"` or `[1-9][0-9]*`, at most `u64::MAX`, carried as a decimal string on the wire.
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Default)]
pub struct U64(pub u64);
impl U64 {
pub fn get(self) -> u64 {
self.0
}
/// Parses the canonical decimal encoding. Leading zeros and signs are refused.
pub fn parse(s: &str) -> Result<U64, String> {
if s.is_empty() {
return Err("u64 must not be empty".to_owned());
}
if s != "0" && s.starts_with('0') {
return Err(format!("u64 {s:?} has a leading zero"));
}
if !s.bytes().all(|b| b.is_ascii_digit()) {
return Err(format!("u64 {s:?} is not decimal digits"));
}
s.parse::<u64>().map(U64).map_err(|_| format!("u64 {s:?} overflows"))
}
}
impl From<u64> for U64 {
fn from(v: u64) -> U64 {
U64(v)
}
}
impl fmt::Debug for U64 {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
}
}
impl fmt::Display for U64 {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
}
}
impl Serialize for U64 {
fn serialize<S: Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
s.serialize_str(&self.0.to_string())
}
}
impl<'de> Deserialize<'de> for U64 {
fn deserialize<D: Deserializer<'de>>(d: D) -> Result<U64, D::Error> {
let s = String::deserialize(d)?;
U64::parse(&s).map_err(D::Error::custom)
}
}
/// 64 lowercase hexadecimal digits: a SHA-256.
#[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Digest(String);
impl Digest {
pub fn parse(s: &str) -> Result<Digest, String> {
if s.len() != 64 || !s.bytes().all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)) {
return Err("digest must be 64 lowercase hexadecimal digits".to_owned());
}
Ok(Digest(s.to_owned()))
}
/// The SHA-256 of `bytes`, hex-encoded.
pub fn of(bytes: &[u8]) -> Digest {
use sha2::{Digest as _, Sha256};
let mut h = Sha256::new();
h.update(bytes);
let out = h.finalize();
let mut s = String::with_capacity(64);
for b in out {
s.push(char::from_digit((b >> 4) as u32, 16).unwrap());
s.push(char::from_digit((b & 0x0f) as u32, 16).unwrap());
}
Digest(s)
}
pub fn as_str(&self) -> &str {
&self.0
}
}
impl fmt::Debug for Digest {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{:?}", self.0)
}
}
impl fmt::Display for Digest {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.0)
}
}
impl Serialize for Digest {
fn serialize<S: Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
s.serialize_str(&self.0)
}
}
impl<'de> Deserialize<'de> for Digest {
fn deserialize<D: Deserializer<'de>>(d: D) -> Result<Digest, D::Error> {
let s = String::deserialize(d)?;
Digest::parse(&s).map_err(D::Error::custom)
}
}
/// The simulation timeline coordinate: which session, which epoch, which step.
///
/// Epoch protects simulation order. It is not a bus route incarnation or a store id.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields, rename_all = "camelCase")]
pub struct Scope {
pub session_id: Id,
pub epoch: Id,
pub step: U64,
}
impl Scope {
pub fn new(session_id: &Id, epoch: &Id, step: u64) -> Scope {
Scope { session_id: session_id.clone(), epoch: epoch.clone(), step: U64(step) }
}
/// The same scope at `step + 1`.
pub fn next(&self) -> Scope {
Scope { step: U64(self.step.0 + 1), ..self.clone() }
}
}
/// A reduced, positive-denominator duration in nanoseconds. Zero is `0/1`.
#[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields, rename_all = "camelCase")]
pub struct RationalNs {
pub numerator: U64,
pub denominator: U64,
}
impl RationalNs {
/// Reduces `numerator / denominator`; the denominator must be positive.
pub fn new(numerator: u64, denominator: u64) -> Result<RationalNs, String> {
if denominator == 0 {
return Err("rational denominator must be positive".to_owned());
}
let g = gcd(numerator, denominator);
let g = if g == 0 { 1 } else { g };
Ok(RationalNs { numerator: U64(numerator / g), denominator: U64(denominator / g) })
}
pub fn zero() -> RationalNs {
RationalNs { numerator: U64(0), denominator: U64(1) }
}
/// `hz` steps per second as an exact nanosecond duration.
pub fn from_hz(hz: u64) -> Result<RationalNs, String> {
RationalNs::new(1_000_000_000, hz)
}
pub fn from_millis(ms: u64) -> Result<RationalNs, String> {
ms.checked_mul(1_000_000)
.ok_or_else(|| "millisecond duration overflows".to_owned())
.and_then(|ns| RationalNs::new(ns, 1))
}
pub fn is_zero(&self) -> bool {
self.numerator.0 == 0
}
pub fn is_positive(&self) -> bool {
self.numerator.0 > 0
}
/// Rejects an unreduced, zero-denominator or non-canonical value that parsed as JSON.
pub fn validate(&self) -> Result<(), String> {
if self.denominator.0 == 0 {
return Err("rational denominator must be positive".to_owned());
}
if self.numerator.0 == 0 {
if self.denominator.0 != 1 {
return Err("rational zero must be encoded 0/1".to_owned());
}
return Ok(());
}
if gcd(self.numerator.0, self.denominator.0) != 1 {
return Err("rational must be reduced".to_owned());
}
Ok(())
}
/// Checked addition, reduced.
pub fn checked_add(&self, other: &RationalNs) -> Result<RationalNs, String> {
let l = self.numerator.0.checked_mul(other.denominator.0);
let r = other.numerator.0.checked_mul(self.denominator.0);
let d = self.denominator.0.checked_mul(other.denominator.0);
match (l, r, d) {
(Some(l), Some(r), Some(d)) => {
let n = l.checked_add(r).ok_or_else(|| "rational add overflows".to_owned())?;
RationalNs::new(n, d)
}
_ => Err("rational add overflows".to_owned()),
}
}
/// Checked subtraction; refuses a negative result.
pub fn checked_sub(&self, other: &RationalNs) -> Result<RationalNs, String> {
let l = self
.numerator
.0
.checked_mul(other.denominator.0)
.ok_or_else(|| "rational sub overflows".to_owned())?;
let r = other
.numerator
.0
.checked_mul(self.denominator.0)
.ok_or_else(|| "rational sub overflows".to_owned())?;
let d = self
.denominator
.0
.checked_mul(other.denominator.0)
.ok_or_else(|| "rational sub overflows".to_owned())?;
let n = l.checked_sub(r).ok_or_else(|| "rational sub would be negative".to_owned())?;
RationalNs::new(n, d)
}
/// Checked multiplication by a whole count.
pub fn checked_mul_u64(&self, k: u64) -> Result<RationalNs, String> {
let n = self
.numerator
.0
.checked_mul(k)
.ok_or_else(|| "rational scale overflows".to_owned())?;
RationalNs::new(n, self.denominator.0)
}
/// `floor(self / other)`; `other` must be positive.
pub fn checked_div_floor(&self, other: &RationalNs) -> Result<u64, String> {
if other.numerator.0 == 0 {
return Err("cannot divide by a zero duration".to_owned());
}
let l = self
.numerator
.0
.checked_mul(other.denominator.0)
.ok_or_else(|| "rational divide overflows".to_owned())?;
let r = other
.numerator
.0
.checked_mul(self.denominator.0)
.ok_or_else(|| "rational divide overflows".to_owned())?;
Ok(l / r)
}
pub fn cmp_value(&self, other: &RationalNs) -> std::cmp::Ordering {
let l = u128::from(self.numerator.0) * u128::from(other.denominator.0);
let r = u128::from(other.numerator.0) * u128::from(self.denominator.0);
l.cmp(&r)
}
}
impl fmt::Debug for RationalNs {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}/{}ns", self.numerator.0, self.denominator.0)
}
}
fn gcd(mut a: u64, mut b: u64) -> u64 {
while b != 0 {
let t = a % b;
a = b;
b = t;
}
a
}
/// Identity of a registered payload schema: what shape, which revision, which definition.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields, rename_all = "camelCase")]
pub struct SchemaRef {
pub id: Id,
pub version: u16,
pub digest: Digest,
}
impl SchemaRef {
/// A schema reference whose digest is derived from its own name and version, so the
/// synthetic composition has stable identities without a schema registry file.
pub fn synthetic(id: &str, version: u16) -> SchemaRef {
let id = Id::lit(id);
let digest = Digest::of(format!("fly-session-schema-v1\n{id}\n{version}\n").as_bytes());
SchemaRef { id, version, digest }
}
pub fn validate(&self) -> Result<(), String> {
if self.version == 0 {
return Err("schema version must be 1..=65535".to_owned());
}
Ok(())
}
}
/// A schema-tagged JSON object. The canonical encoding of `value` is capped at 32 KiB.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct TypedValue {
pub schema: SchemaRef,
pub value: serde_json::Map<String, serde_json::Value>,
}
/// `ipc-v1` section 2: a `TypedValue`'s canonical JSON is at most 32 KiB.
pub const MAX_TYPED_VALUE_BYTES: usize = 32 * 1024;
impl TypedValue {
pub fn new(schema: SchemaRef, value: serde_json::Map<String, serde_json::Value>) -> TypedValue {
TypedValue { schema, value }
}
pub fn validate(&self) -> Result<(), String> {
self.schema.validate()?;
let canonical = crate::fly_session_types::canonical::canonical_json(
&serde_json::Value::Object(self.value.clone()),
);
if canonical.len() > MAX_TYPED_VALUE_BYTES {
return Err(format!(
"typed value is {} canonical bytes, over the 32 KiB limit",
canonical.len()
));
}
Ok(())
}
/// The canonical digest of the whole typed value, schema identity included.
pub fn digest(&self) -> Digest {
crate::fly_session_types::canonical::canonical_digest(
&serde_json::to_value(self).expect("TypedValue serializes"),
)
}
pub fn number(&self, key: &str) -> Result<f64, String> {
self.value
.get(key)
.and_then(serde_json::Value::as_f64)
.filter(|v| v.is_finite())
.ok_or_else(|| format!("typed value has no finite number {key:?}"))
}
pub fn integer(&self, key: &str) -> Result<i64, String> {
self.value
.get(key)
.and_then(serde_json::Value::as_i64)
.ok_or_else(|| format!("typed value has no integer {key:?}"))
}
pub fn boolean(&self, key: &str) -> Result<bool, String> {
self.value
.get(key)
.and_then(serde_json::Value::as_bool)
.ok_or_else(|| format!("typed value has no boolean {key:?}"))
}
}

121
services/flysim/crates/fly-session/src/fly_session_types/trace.rs
View file

@ -1,121 +0,0 @@
//! The trace format of `step-v1` section 8, split into behaviour and operational metadata.
//!
//! Behaviour is what two runs of the same composition must agree on whatever order their
//! messages took. Operational metadata -- request ids, batch ids, bus call ids, wall time --
//! is recorded but excluded from that comparison, which is exactly what section 8 asks for.
use serde::Serialize;
use super::methods::{Reward, Stimulus};
use super::scalars::{Digest, Id, RationalNs, Scope, U64};
/// One agent's contribution to one transition.
#[derive(Clone, Debug, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AgentTransitionTrace {
pub agent_id: Id,
pub profile_digest: Digest,
pub ticks_advanced: U64,
pub brain_ticks: U64,
pub remainder: RationalNs,
pub decision_digest: Digest,
pub committed_step: U64,
pub rewards: Vec<Reward>,
pub stimulations: Vec<Stimulus>,
/// Operational: the `req-<U64>` of this agent's Prepare.
#[serde(skip_serializing)]
pub prepare_request_id: Id,
/// Operational: the `req-<U64>` of this agent's Commit.
#[serde(skip_serializing)]
pub commit_request_id: Id,
}
/// The producing boundary of one observation view.
#[derive(Clone, Debug, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct ViewProvenance {
pub view_id: Id,
pub produced_step: U64,
}
/// One complete transition `k -> k+1`.
#[derive(Clone, Debug, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct TransitionTrace {
pub scope: Scope,
/// Agents in sorted agent-id order, never completion order.
pub agents: Vec<AgentTransitionTrace>,
pub controls_digest: Digest,
pub acknowledged_boundary: U64,
pub observation_boundaries: Vec<ViewProvenance>,
pub task_event_ids: Vec<Id>,
pub published_boundary: U64,
/// Operational: the complete batch's id.
#[serde(skip_serializing)]
pub batch_id: Id,
/// Operational: the `req-<U64>` of the single Environment.Advance.
#[serde(skip_serializing)]
pub advance_request_id: Id,
}
impl TransitionTrace {
/// The canonical behaviour encoding: everything a reordered run must reproduce exactly.
pub fn behavior(&self) -> String {
let value = serde_json::to_value(self).expect("a trace serializes");
super::canonical::canonical_json(&value)
}
/// Operational identities, for a report rather than a comparison.
pub fn operational(&self) -> Vec<(String, String)> {
let mut out = vec![
("batchId".to_owned(), self.batch_id.to_string()),
("advanceRequestId".to_owned(), self.advance_request_id.to_string()),
];
for agent in &self.agents {
out.push((
format!("prepareRequestId.{}", agent.agent_id),
agent.prepare_request_id.to_string(),
));
out.push((
format!("commitRequestId.{}", agent.agent_id),
agent.commit_request_id.to_string(),
));
}
out
}
}
/// One session phase transition, recorded whether or not it ends a step.
#[derive(Clone, Debug, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct PhaseTransition {
pub from: String,
pub to: String,
}
/// Everything a run recorded: its phase transitions and its completed transitions.
#[derive(Clone, Debug, Default)]
pub struct TraceLog {
pub phases: Vec<PhaseTransition>,
pub transitions: Vec<TransitionTrace>,
}
impl TraceLog {
pub fn phase(&mut self, from: String, to: String) {
self.phases.push(PhaseTransition { from, to });
}
pub fn transition(&mut self, trace: TransitionTrace) {
self.transitions.push(trace);
}
/// The behaviour of every transition, in order, with operational metadata excluded.
pub fn behavior(&self) -> Vec<String> {
self.transitions.iter().map(TransitionTrace::behavior).collect()
}
/// The phase path, as `from -> to` strings.
pub fn phase_path(&self) -> Vec<String> {
self.phases.iter().map(|p| format!("{} -> {}", p.from, p.to)).collect()
}
}

34
services/flysim/crates/fly-session/src/harness.rs
View file

@ -20,7 +20,9 @@ use crate::coordinator::{AgentSlot, Coordinator};
use crate::environment::{CounterEnvironment, EnvironmentConfig, EnvironmentFaults};
use crate::rpc::WorkerRef;
use crate::task::{ActionExecutor, CounterTask, IdentityExecutor, Terminal};
use crate::types::{Id, RationalNs};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
// glob keeps the contract's own names in sight instead of restating them.
use crate::types::*;
use crate::worker::{StatusCell, WorkerHandle, serve};
/// Which transport the session runs over. Both must produce the same behaviour.
@ -59,19 +61,19 @@ pub struct HarnessConfig {
impl Default for HarnessConfig {
fn default() -> HarnessConfig {
HarnessConfig {
session_id: Id::lit("demo"),
epoch: Id::lit("e1"),
episode_id: Id::lit("ep1"),
session_id: id("demo"),
epoch: id("e1"),
episode_id: id("ep1"),
agents: vec![
AgentSpec {
agent_id: Id::lit("fly-a"),
port_id: Id::lit("p1"),
agent_id: id("fly-a"),
port_id: id("p1"),
seed: 7,
faults: AgentFaults::default(),
},
AgentSpec {
agent_id: Id::lit("fly-b"),
port_id: Id::lit("p2"),
agent_id: id("fly-b"),
port_id: id("p2"),
seed: 11,
faults: AgentFaults::default(),
},
@ -203,8 +205,8 @@ impl SessionHarness {
listeners: Mutex::new(Vec::new()),
};
let step_duration = RationalNs::from_hz(config.step_hz).expect("a positive cadence");
let tick_duration = RationalNs::from_millis(config.tick_ms).expect("a positive tick");
let step_duration = hz(config.step_hz).expect("a positive cadence");
let tick_duration = millis(config.tick_ms).expect("a positive tick");
// The environment first: it owns the world and the descriptor.
let env_client = connector.client(ENV_CLIENT).await?;
@ -215,8 +217,8 @@ impl SessionHarness {
env_service,
CounterEnvironment::new(EnvironmentConfig {
session_id: config.session_id.clone(),
worker_id: Id::lit(ENV_WORKER),
incarnation_id: Id::lit("arena-inc-1"),
worker_id: id(ENV_WORKER),
incarnation_id: id("arena-inc-1"),
step_duration,
ports: config.agents.iter().map(|a| a.port_id.clone()).collect(),
faults: config.environment_faults.clone(),
@ -236,7 +238,7 @@ impl SessionHarness {
FakeAgentWorker::new(AgentConfig {
session_id: config.session_id.clone(),
agent_id: spec.agent_id.clone(),
incarnation_id: Id::parse(&format!("{}-inc-1", spec.agent_id))
incarnation_id: parse_id(&format!("{}-inc-1", spec.agent_id))
.expect("an agent id plus a suffix is an Id"),
tick_duration,
warmup_ticks: config.warmup_ticks,
@ -266,7 +268,7 @@ impl SessionHarness {
config.session_id.clone(),
config.epoch.clone(),
config.episode_id.clone(),
WorkerRef::new(ENV_SERVICE, &env_incarnation, &Id::lit(ENV_WORKER)),
WorkerRef::new(ENV_SERVICE, &env_incarnation, &id(ENV_WORKER)),
slots,
Box::new(CounterTask::new(&config.epoch, config.terminal)),
executors,
@ -304,7 +306,7 @@ impl SessionHarness {
/// The coordinator still pins the old registration, so its next call to that agent fails
/// rather than silently reaching another brain.
pub async fn restart_agent(&mut self, agent_id: &Id) -> Result<Restarted, flybus::BusError> {
let tick_duration = RationalNs::from_millis(self.config.tick_ms).expect("a positive tick");
let tick_duration = millis(self.config.tick_ms).expect("a positive tick");
if let Some(old) = self.agents.remove(agent_id) {
old.stop().await;
}
@ -328,7 +330,7 @@ impl SessionHarness {
.expect("a configured agent")
.clone();
let incarnation_id =
Id::parse(&format!("{agent_id}-inc-2")).expect("an agent id plus a suffix is an Id");
parse_id(&format!("{agent_id}-inc-2")).expect("an agent id plus a suffix is an Id");
let restarted = Restarted {
service: service_name,
service_incarnation: service.incarnation().to_owned(),

11
services/flysim/crates/fly-session/src/lib.rs
View file

@ -16,6 +16,9 @@
//! the result caches of `ipc-v1` section 5 in front of every mutation. Nothing here contains a
//! public controller API, an implicit best-effort retry, a real emulator or a real brain.
//!
//! The domain types come from the CONTRACT-01 crate [`fly_session_types`]; [`types`] is a
//! facade over it plus the few session-side additions a coordinator needs.
//!
//! [`step-v1`]: https://example.invalid/step-v1
pub mod agent;
@ -29,10 +32,10 @@ pub mod rpc;
pub mod task;
pub mod worker;
// CONTRACT-01 owns the domain types. Until its crate exists they live in this module under
// the same names; swapping it for the crate is a change to these two lines.
pub mod fly_session_types;
pub use fly_session_types as types;
// CONTRACT-01 owns the domain types; `types` is a facade over its crate plus the few
// session-side additions a coordinator needs.
pub mod types;
pub use fly_session_types;
pub use coordinator::{Coordinator, DispatchOrder, Injections, SessionFailure, StepReport};
pub use phase::{Phase, PhaseMachine};

38
services/flysim/crates/fly-session/src/rpc.rs
View file

@ -1,4 +1,4 @@
//! Domain RPC over Flybus: `req-<U64>` serials, incarnation pinning and the retry rule.
//! Domain RPC over Flybus: `req-<u64>` serials, incarnation pinning and the retry rule.
//!
//! A domain retry keeps its `requestId` and body and takes a fresh bus `callId`. Nothing here
//! retries on its own: an uncertain call is resolved by the caller, which is the coordinator.
@ -7,9 +7,9 @@ use std::collections::BTreeMap;
use serde_json::{Map, Value};
use crate::types::{
DomainError, ErrorCode, Id, Mutation, RequestId, SessionRpcOutcome, SessionRpcRequest, Scope,
};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
// glob keeps the contract's own names in sight instead of restating them.
use crate::types::*;
/// A named worker endpoint, pinned to one bus registration and one domain incarnation.
#[derive(Clone, Debug)]
@ -36,20 +36,20 @@ impl WorkerRef {
/// One terminal domain reply and the artifacts it brought.
pub struct DomainReply {
pub outcome: SessionRpcOutcome,
pub request_id: RequestId,
pub request_id: DomainRequestId,
pub artifacts: BTreeMap<String, flybus::Artifact>,
}
impl DomainReply {
/// The success `result`, or the domain error.
pub fn result(&self) -> Result<&Map<String, Value>, DomainError> {
self.outcome.result()
pub fn result(&self) -> Result<&Value, DomainError> {
outcome_result(&self.outcome)
}
pub fn parse<T: serde::de::DeserializeOwned>(&self) -> Result<T, DomainError> {
let result = self.outcome.result()?;
serde_json::from_value(Value::Object(result.clone()))
.map_err(|e| DomainError::invalid(format!("unreadable result: {e}")))
/// Reads and validates the success `result` as a method payload.
pub fn parse<T: DomainType>(&self) -> Result<T, DomainError> {
let result = outcome_result(&self.outcome)?;
T::from_json(result).map_err(|e| DomainError::invalid(format!("unreadable result: {e}")))
}
}
@ -66,22 +66,22 @@ pub async fn call(
scope: Option<Scope>,
params: Map<String, Value>,
attachments: &[(&str, &flybus::Artifact)],
request_id: RequestId,
request_id: DomainRequestId,
want_artifacts: &[String],
) -> Result<DomainReply, DomainError> {
let request = SessionRpcRequest::new(&request_id, scope, params);
let request = SessionRpcRequest { request_id: request_id.clone(), scope, params: Value::Object(params) };
let mut pending = bus
.call(
&target.service,
Some(&target.bus_incarnation),
method,
request.to_payload(),
object(request.to_json()),
attachments,
)
.await
.map_err(|e| bus_error(method, &e))?;
let result = pending.result().await.map_err(|e| bus_error(method, &e))?;
let outcome = SessionRpcOutcome::from_outcome(result.outcome())
let outcome = SessionRpcOutcome::from_json(&Value::Object(result.outcome().clone()))
.map_err(|e| DomainError::invalid(format!("{method}: {e}")))?;
let mut artifacts = BTreeMap::new();
for name in want_artifacts {
@ -105,8 +105,8 @@ pub async fn call(
/// Maps a bus failure onto a domain error, preserving how certain the mutation is.
fn bus_error(method: &str, e: &flybus::BusError) -> DomainError {
let mutation = match e.dispatch {
flybus::Dispatch::NotDispatched => Mutation::None,
flybus::Dispatch::Dispatched | flybus::Dispatch::Unknown => Mutation::Unknown,
flybus::Dispatch::NotDispatched => MutationCertainty::None,
flybus::Dispatch::Dispatched | flybus::Dispatch::Unknown => MutationCertainty::Unknown,
};
let code = match e.code {
flybus::ErrorCode::TargetChanged | flybus::ErrorCode::NoService => {
@ -127,10 +127,10 @@ fn bus_error(method: &str, e: &flybus::BusError) -> DomainError {
pub struct Serials(BTreeMap<String, u64>);
impl Serials {
pub fn next(&mut self, service: &str) -> RequestId {
pub fn next(&mut self, service: &str) -> DomainRequestId {
let slot = self.0.entry(service.to_owned()).or_insert(0);
*slot += 1;
RequestId(*slot)
DomainRequestId::from_serial(*slot)
}
pub fn highest(&self, service: &str) -> u64 {

109
services/flysim/crates/fly-session/src/task.rs
View file

@ -10,39 +10,37 @@ use std::collections::BTreeMap;
use serde_json::{Map, Value, json};
use crate::types::{
AgentOutcome, ControllerIntent, DomainError, DomainResult, EpisodeKind, EpisodeRequest,
ErrorCode, Id, PortBinding, PortControl, Reward, SchemaRef, Scope, Stimulus, TaskEvent,
TypedValue, U64, event_id,
};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
// glob keeps the contract's own names in sight instead of restating them.
use crate::types::*;
/// The schemas the synthetic arena composition registers.
pub fn inspection_schema() -> SchemaRef {
SchemaRef::synthetic("arena.inspection.v1", 1)
synthetic_schema("arena.inspection.v1", 1)
}
pub fn decision_schema() -> SchemaRef {
SchemaRef::synthetic("arena.decision.v1", 1)
synthetic_schema("arena.decision.v1", 1)
}
pub fn context_schema() -> SchemaRef {
SchemaRef::synthetic("arena.context.v1", 1)
synthetic_schema("arena.context.v1", 1)
}
pub fn progress_schema() -> SchemaRef {
SchemaRef::synthetic("arena.progress.v1", 1)
synthetic_schema("arena.progress.v1", 1)
}
pub fn event_schema() -> SchemaRef {
SchemaRef::synthetic("arena.event.v1", 1)
synthetic_schema("arena.event.v1", 1)
}
pub fn episode_schema() -> SchemaRef {
SchemaRef::synthetic("arena.episode.v1", 1)
synthetic_schema("arena.episode.v1", 1)
}
pub fn controller_schema_ref() -> SchemaRef {
SchemaRef::synthetic("arena.controller.v1", 1)
synthetic_schema("arena.controller.v1", 1)
}
/// What `Task.bootstrap` produced.
@ -98,7 +96,7 @@ pub trait ActionExecutor: Send {
decision: &TypedValue,
current_game_state: &TypedValue,
progress: &TypedValue,
clock: &crate::types::RationalNs,
clock: &RationalNs,
) -> DomainResult<(ControllerIntent, Vec<TaskEvent>)>;
}
@ -113,7 +111,7 @@ impl ActionExecutor for IdentityExecutor {
decision: &TypedValue,
_current_game_state: &TypedValue,
_progress: &TypedValue,
_clock: &crate::types::RationalNs,
_clock: &RationalNs,
) -> DomainResult<(ControllerIntent, Vec<TaskEvent>)> {
if decision.schema != decision_schema() {
return Err(DomainError::before(
@ -121,9 +119,8 @@ impl ActionExecutor for IdentityExecutor {
"the decision does not carry the profile's registered intent schema",
));
}
let intent: ControllerIntent =
serde_json::from_value(Value::Object(decision.value.clone()))
.map_err(|e| DomainError::invalid(format!("decision: {e}")))?;
let intent = ControllerIntent::from_json(&decision.value)
.map_err(|e| DomainError::invalid(format!("decision: {e}")))?;
Ok((intent, Vec::new()))
}
}
@ -167,31 +164,21 @@ impl CounterTask {
}
fn context(&self, step: u64, boot: bool) -> TypedValue {
TypedValue::new(
context_schema(),
match json!({
TypedValue::new(context_schema(), json!({
"available": ["inc", "dec"],
"boot": boot,
"step": step,
}) {
Value::Object(m) => m,
_ => unreachable!(),
},
)
}))
.expect("a synthetic typed value fits the contract")
}
fn progress_value(&self) -> TypedValue {
TypedValue::new(
progress_schema(),
match json!({
TypedValue::new(progress_schema(), json!({
"counter": self.counter,
"transitions": self.transitions,
"totalReward": self.total_reward,
}) {
Value::Object(m) => m,
_ => unreachable!(),
},
)
}))
.expect("a synthetic typed value fits the contract")
}
/// The counter delta one port control asks for: `inc` adds one, `dec` subtracts one.
@ -246,16 +233,11 @@ impl Task for CounterTask {
// gameplay outcome at all.
let events = vec![TaskEvent {
id: event_id(&self.epoch, 0, "bootstrap", 0),
kind_id: Id::lit("arena.bootstrap"),
source_step: U64(0),
kind_id: id("arena.bootstrap"),
source_step: 0,
agent_id: None,
payload: TypedValue::new(
event_schema(),
match json!({"counter": self.counter}) {
Value::Object(m) => m,
_ => unreachable!(),
},
),
payload: TypedValue::new(event_schema(), json!({"counter": self.counter}))
.expect("a synthetic typed value fits the contract"),
}];
Ok(Bootstrap { contexts, progress: self.progress_value(), events })
}
@ -269,7 +251,7 @@ impl Task for CounterTask {
) -> DomainResult<Evaluation> {
let old = old_inspection.integer("counter").map_err(DomainError::invalid)?;
let new = new_inspection.integer("counter").map_err(DomainError::invalid)?;
let source_step = scope.step.0 + 1;
let source_step = scope.step + 1;
self.evaluations += 1;
self.transitions += 1;
self.counter = new;
@ -290,20 +272,15 @@ impl Task for CounterTask {
));
};
let delta = CounterTask::delta_of(control);
let id = event_id(&self.epoch, source_step, "counter-delta", ordinal);
let event = event_id(&self.epoch, source_step, "counter-delta", ordinal);
ordinal += 1;
events.push(TaskEvent {
id: id.clone(),
kind_id: Id::lit("arena.counter-delta"),
source_step: U64(source_step),
id: event.clone(),
kind_id: id("arena.counter-delta"),
source_step,
agent_id: Some(agent.clone()),
payload: TypedValue::new(
event_schema(),
match json!({"delta": delta, "counter": new}) {
Value::Object(m) => m,
_ => unreachable!(),
},
),
payload: TypedValue::new(event_schema(), json!({"delta": delta, "counter": new}))
.expect("a synthetic typed value fits the contract"),
});
let outcome = outcomes.get_mut(&agent).ok_or_else(|| {
DomainError::before(
@ -314,8 +291,8 @@ impl Task for CounterTask {
// A shipped positive-only profile would reject a negative value; this task is
// signed on purpose, so the profile that consumes it declares signed rewards.
outcome.rewards.push(Reward {
event_id: id,
rule_id: Id::lit("counter-delta"),
event_id: event,
rule_id: id("counter-delta"),
value: delta as f64,
});
self.total_reward += delta as f64;
@ -324,7 +301,7 @@ impl Task for CounterTask {
if delta != 0 && new.rem_euclid(5) == 0 {
outcome.stimulations.push(Stimulus {
id: event_id(&self.epoch, source_step, "milestone", ordinal),
kind_id: Id::lit("arena.milestone"),
kind_id: id("arena.milestone"),
duration_ms: 4.0,
});
}
@ -333,7 +310,7 @@ impl Task for CounterTask {
return Err(DomainError::new(
ErrorCode::BackendFailure,
"the world did not move by the batch the task read",
crate::types::Mutation::Unknown,
MutationCertainty::Unknown,
));
}
@ -347,16 +324,11 @@ impl Task for CounterTask {
Terminal::Counter(target) => new >= target,
Terminal::AfterTransitions(n) => self.transitions >= n,
};
// The contract's `EpisodeRequest` is terminal by construction: `kind` is a constant.
let episode = terminal.then(|| EpisodeRequest {
kind: EpisodeKind::Terminal,
reason: Id::lit("counter-target"),
outcome: TypedValue::new(
episode_schema(),
match json!({"counter": new, "transitions": self.transitions}) {
Value::Object(m) => m,
_ => unreachable!(),
},
),
reason: id("counter-target"),
outcome: TypedValue::new(episode_schema(), json!({"counter": new, "transitions": self.transitions}))
.expect("a synthetic typed value fits the contract"),
});
Ok(Evaluation {
outcomes,
@ -381,5 +353,6 @@ pub fn inspection(counter: i64, boundary: u64) -> TypedValue {
let mut value = Map::new();
value.insert("counter".into(), counter.into());
value.insert("boundary".into(), boundary.into());
TypedValue::new(inspection_schema(), value)
TypedValue::new(inspection_schema(), Value::Object(value))
.expect("the arena inspection fits the contract")
}

518
services/flysim/crates/fly-session/src/types.rs Normal file
View file

@ -0,0 +1,518 @@
//! The domain types this crate uses, all from the shared `fly-session-types` crate.
//!
//! CONTRACT-01 owns the scalars, the method payloads, their validation, the canonical digests
//! and the trace format. This module is only a facade over that crate plus the few things a
//! coordinator needs that are not part of the contract: a session-side error value, the
//! synthetic composition's schema and event-id derivations, and a log of recorded traces.
//!
//! `Id` and `Digest` are type aliases, because the shared crate carries both as validated
//! `String`s from `flybus::wire` rather than forking the encodings into newtypes.
use serde_json::{Map, Value};
pub use fly_session_types::ArtifactRef;
pub use fly_session_types::canonical::{
self, OperationKey, body_digest, canonicalize, digest_of, sha256_hex,
};
pub use fly_session_types::media::{AudioDescriptor, AudioRef, ViewDescriptor, ViewRef};
pub use fly_session_types::rpc::{
ErrorCode, MutationCertainty, SessionRpcFailure, SessionRpcOutcome, SessionRpcRequest,
SessionRpcSuccess,
};
pub use fly_session_types::scalar::{
ArtifactIdentity, BusCallId, DomainRequestId, DomainType, MAX_TYPED_VALUE_BYTES, RationalNs,
SchemaRef, Scope, TypedValue, is_digest, is_id,
};
pub use fly_session_types::schema::contract_digest;
pub use fly_session_types::trace::{
TraceAgent, TraceBehaviour, TraceObservation, TraceOperational, TraceRequest, TransitionTrace,
};
pub use fly_session_types::workers::{
AcknowledgeParams, AcknowledgeResult, AdvanceParams, AgentCommitResult, AgentInitializeParams,
AgentInitializeResult, AgentTelemetry, AssetRef, AxisRange, AxisSchema, AxisValue, ButtonState,
CommitParams, ControllerSchema, Determinism, EnvironmentDescriptor,
EnvironmentInitializeParams, EnvironmentInitializeResult, EpisodeRequest, HelloParams,
HelloResult, LearningTelemetry, MAX_ACKNOWLEDGE, MAX_AGENTS, MAX_PORTS, MAX_RATE_ROLES,
MAX_REWARDS, MAX_STIMULI, PortControl, PortDescriptor, PrepareParams, PreparedDecision,
RateSample, Recovery, Reward, Role, SensoryInput, ShutdownParams, ShutdownResult, StatusResult,
StepResult, Stimulus, TaskEvent, WorkerState, WorldObservation,
};
/// A validated `Id`: `^[a-z0-9][a-z0-9._-]{0,63}$`, the bus encoding the contracts reuse.
pub type Id = String;
/// 64 lowercase hexadecimal digits: a SHA-256.
pub type Digest = String;
/// An `Id` from a trusted literal. It panics on a malformed one, which is a bug in the
/// composition rather than a runtime condition.
pub fn id(s: &str) -> Id {
assert!(is_id(s), "{s:?} is not a valid Id");
s.to_owned()
}
/// An `Id` from an untrusted string.
pub fn parse_id(s: &str) -> Result<Id, String> {
if is_id(s) {
Ok(s.to_owned())
} else {
Err(format!("{s:?} is not a valid Id"))
}
}
/// The SHA-256 of some bytes, hex-encoded.
pub fn digest_of_bytes(bytes: &[u8]) -> Digest {
sha256_hex(bytes)
}
/// A `Scope` from trusted composition values.
pub fn scope_at(session_id: &str, epoch: &str, step: u64) -> Scope {
Scope::new(session_id, epoch, step).expect("a composition scope is valid")
}
/// `hz` steps per second as an exact nanosecond duration.
pub fn hz(hz: u64) -> Result<RationalNs, String> {
RationalNs::reduced(1_000_000_000, u128::from(hz)).map_err(|e| e.0)
}
/// A whole number of milliseconds as an exact nanosecond duration.
pub fn millis(ms: u64) -> Result<RationalNs, String> {
RationalNs::reduced(u128::from(ms) * 1_000_000, 1).map_err(|e| e.0)
}
/// A schema reference whose digest is derived from its own name and version, so the synthetic
/// composition has stable identities without a schema registry file.
pub fn synthetic_schema(name: &str, version: u16) -> SchemaRef {
let digest = digest_of_bytes(format!("fly-session-schema-v1\n{name}\n{version}\n").as_bytes());
SchemaRef::new(name, version, &digest).expect("a synthetic schema reference is valid")
}
/// A deterministic task event id from epoch, source step, rule and ordinal.
pub fn event_id(epoch: &str, source_step: u64, rule: &str, ordinal: u32) -> Id {
let digest = digest_of_bytes(format!("{epoch}\n{source_step}\n{rule}\n{ordinal}\n").as_bytes());
id(&format!("ev-{}", &digest[..16]))
}
/// The digest of a validated, canonical control batch, in descriptor port order.
pub fn controls_digest(controls: &[PortControl]) -> Digest {
let array = Value::Array(controls.iter().map(DomainType::to_json).collect());
digest_of(&array).expect("a validated control batch canonicalizes")
}
/// The canonical digest of a typed value, schema identity included.
pub fn typed_digest(value: &TypedValue) -> Digest {
digest_of(&value.to_json()).expect("a validated typed value canonicalizes")
}
/// The byte length one view's artifact must have.
pub fn view_byte_length(descriptor: &ViewDescriptor) -> u64 {
descriptor.row_stride * descriptor.height
}
/// The boundary a required sensory view must have been produced at.
pub fn required_produced_step(descriptor: &ViewDescriptor, boundary: u64) -> u64 {
boundary.saturating_sub(descriptor.observation_delay_steps)
}
/// Reads a finite number out of a typed value.
pub fn typed_number(value: &TypedValue, key: &str) -> Result<f64, String> {
value
.value
.get(key)
.and_then(Value::as_f64)
.filter(|v| v.is_finite())
.ok_or_else(|| format!("typed value has no finite number {key:?}"))
}
/// Reads an integer out of a typed value.
pub fn typed_integer(value: &TypedValue, key: &str) -> Result<i64, String> {
value
.value
.get(key)
.and_then(Value::as_i64)
.ok_or_else(|| format!("typed value has no integer {key:?}"))
}
/// Builds a typed value from a JSON object, refusing one the contract would reject.
pub fn typed(schema: SchemaRef, value: Value) -> Result<TypedValue, String> {
TypedValue::new(schema, value).map_err(|e| e.0)
}
/// The object form of a payload, for a bus `payload` or `outcome` field.
pub fn object(value: Value) -> Map<String, Value> {
match value {
Value::Object(m) => m,
_ => Map::new(),
}
}
/// `ipc-v1` section 7: the domain error a worker returns, with its mutation certainty.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct DomainError {
pub code: ErrorCode,
pub message: String,
pub mutation: MutationCertainty,
}
/// `ipc-v1` section 7: messages are at most 512 code points and carry no raw memory.
pub const MAX_ERROR_MESSAGE_CHARS: usize = 512;
impl DomainError {
pub fn new(
code: ErrorCode,
message: impl std::fmt::Display,
mutation: MutationCertainty,
) -> DomainError {
let message: String = message.to_string();
let message = if message.chars().count() > MAX_ERROR_MESSAGE_CHARS {
message.chars().take(MAX_ERROR_MESSAGE_CHARS).collect()
} else {
message
};
DomainError { code, message, mutation }
}
/// An error raised before anything was mutated.
pub fn before(code: ErrorCode, message: impl std::fmt::Display) -> DomainError {
DomainError::new(code, message, MutationCertainty::None)
}
pub fn invalid(message: impl std::fmt::Display) -> DomainError {
DomainError::before(ErrorCode::InvalidArgument, message)
}
}
impl std::fmt::Display for DomainError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{} ({}): {}", self.code.as_str(), self.mutation.as_str(), self.message)
}
}
impl std::error::Error for DomainError {}
pub type DomainResult<T> = Result<T, DomainError>;
/// The success `result` of a terminal outcome, or its domain error.
pub fn outcome_result(outcome: &SessionRpcOutcome) -> DomainResult<&Value> {
match outcome {
SessionRpcOutcome::Success(s) => Ok(&s.result),
SessionRpcOutcome::Failure(f) => Err(DomainError::new(
f.code,
f.message.clone(),
f.mutation,
)),
}
}
/// The worker id and incarnation a terminal outcome came from, and the scope it echoes.
pub fn outcome_identity(
outcome: &SessionRpcOutcome,
) -> (&Id, &Id, &Option<Scope>) {
match outcome {
SessionRpcOutcome::Success(s) => (&s.worker_id, &s.incarnation_id, &s.scope),
SessionRpcOutcome::Failure(f) => (&f.worker_id, &f.incarnation_id, &f.scope),
}
}
/// One session phase transition, recorded whether or not it ends a step.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct PhaseTransition {
pub from: String,
pub to: String,
}
/// Everything a run recorded: its phase transitions and its completed transitions.
///
/// The transitions are the contract's [`TransitionTrace`]; the phase path is this crate's own,
/// because the state machine lives here and not in the type contract.
#[derive(Clone, Debug, Default)]
pub struct TraceLog {
pub phases: Vec<PhaseTransition>,
pub transitions: Vec<TransitionTrace>,
}
impl TraceLog {
pub fn phase(&mut self, from: String, to: String) {
self.phases.push(PhaseTransition { from, to });
}
pub fn transition(&mut self, trace: TransitionTrace) {
self.transitions.push(trace);
}
/// The behaviour of every transition, in order, with operational metadata excluded.
///
/// This is what `step-v1` section 8 compares across dispatch and completion orders.
pub fn behavior(&self) -> Vec<String> {
self.transitions
.iter()
.map(|t| {
canonicalize(&t.behaviour.to_json())
.expect("a recorded behaviour canonicalizes")
})
.collect()
}
/// The phase path, as `from -> to` strings.
pub fn phase_path(&self) -> Vec<String> {
self.phases.iter().map(|p| format!("{} -> {}", p.from, p.to)).collect()
}
}
// -------------------------------------------------------------------------------------------
// Coordinator-local types
//
// `workers-v1` section 4 calls these library interfaces rather than payloads, so the contract
// crate does not carry them: they never cross the bus.
/// What an executor returns: buttons and axes, with no port assignment.
///
/// The coordinator supplies the port, which is why this is not a [`PortControl`].
#[derive(Clone, Debug, PartialEq)]
pub struct ControllerIntent {
pub buttons: Vec<ButtonState>,
pub axes: Vec<AxisValue>,
}
impl ControllerIntent {
/// The canonical JSON of the intent: exactly a `PortControl` without its port.
pub fn to_json(&self) -> Value {
let buttons = self
.buttons
.iter()
.map(|b| {
let mut m = Map::new();
m.insert("id".into(), b.id.clone().into());
m.insert("down".into(), Value::Bool(b.down));
Value::Object(m)
})
.collect();
let axes = self
.axes
.iter()
.map(|a| {
let mut m = Map::new();
m.insert("id".into(), a.id.clone().into());
m.insert("value".into(), Value::from(a.value));
Value::Object(m)
})
.collect();
let mut m = Map::new();
m.insert("buttons".into(), Value::Array(buttons));
m.insert("axes".into(), Value::Array(axes));
Value::Object(m)
}
/// Reads an intent out of a decision payload.
pub fn from_json(value: &Value) -> Result<ControllerIntent, String> {
let control = Value::Object({
let mut m = object(value.clone());
m.insert("portId".into(), "p0".into());
m
});
let control = PortControl::from_json(&control).map_err(|e| e.0)?;
Ok(ControllerIntent { buttons: control.buttons, axes: control.axes })
}
/// Binds this intent to a port, which only the coordinator may do.
pub fn at_port(self, port_id: &str) -> PortControl {
PortControl {
port_id: port_id.to_owned(),
buttons: self.buttons,
axes: self.axes,
}
}
}
/// One port-to-agent assignment. It crosses the bus as a pair inside
/// [`EnvironmentInitializeParams`]; inside the session it is named.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct PortBinding {
pub port_id: Id,
pub agent_id: Id,
}
impl PortBinding {
pub fn pair(&self) -> (Id, Id) {
(self.port_id.clone(), self.agent_id.clone())
}
}
/// Everything the task routes to one agent for this transition. Always explicit, even empty.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct AgentOutcome {
pub rewards: Vec<Reward>,
pub stimulations: Vec<Stimulus>,
}
/// Every button up and every axis at its declared neutral.
///
/// Uncontrolled ports are configured neutral before the epoch, not supplied ad hoc.
pub fn neutral_control(schema: &ControllerSchema, port_id: &str) -> PortControl {
PortControl {
port_id: port_id.to_owned(),
buttons: schema
.buttons
.iter()
.map(|id| ButtonState { id: id.clone(), down: false })
.collect(),
axes: schema
.axes
.iter()
.map(|a| AxisValue { id: a.id.clone(), value: a.neutral })
.collect(),
}
}
// -------------------------------------------------------------------------------------------
// Session-side conveniences over the contract types
//
// These are extension traits rather than forks: the data and the rules stay in the contract
// crate, and these only spell out the readings a coordinator and its workers keep needing.
/// Reading a typed value the way the synthetic composition writes it.
pub trait TypedValueExt {
/// The canonical digest of the whole typed value, schema identity included.
fn digest(&self) -> Digest;
fn number(&self, key: &str) -> Result<f64, String>;
fn integer(&self, key: &str) -> Result<i64, String>;
}
impl TypedValueExt for TypedValue {
fn digest(&self) -> Digest {
typed_digest(self)
}
fn number(&self, key: &str) -> Result<f64, String> {
typed_number(self, key)
}
fn integer(&self, key: &str) -> Result<i64, String> {
typed_integer(self, key)
}
}
/// Checking and building one port's complete controller state.
pub trait ControllerSchemaExt {
/// Checks that `control` names every declared button and axis, in descriptor order, with
/// every value inside its range.
fn check(&self, control: &PortControl) -> Result<(), String>;
/// Every button up and every axis at its declared neutral.
fn neutral(&self, port_id: &str) -> PortControl;
}
impl ControllerSchemaExt for ControllerSchema {
fn check(&self, control: &PortControl) -> Result<(), String> {
control.validate_against(self).map_err(|e| e.0)
}
fn neutral(&self, port_id: &str) -> PortControl {
neutral_control(self, port_id)
}
}
/// The byte shape and producing boundary one declared view requires.
pub trait ViewDescriptorExt {
fn byte_length(&self) -> u64;
fn required_produced_step(&self, boundary: u64) -> u64;
}
impl ViewDescriptorExt for ViewDescriptor {
fn byte_length(&self) -> u64 {
view_byte_length(self)
}
fn required_produced_step(&self, boundary: u64) -> u64 {
required_produced_step(self, boundary)
}
}
/// `RationalNs` readings the session uses.
pub trait RationalExt {
fn is_positive(&self) -> bool;
}
impl RationalExt for RationalNs {
fn is_positive(&self) -> bool {
!self.is_zero()
}
}
/// The next step of a scope, which is the only arithmetic a coordinator does on one.
pub trait ScopeExt {
fn next(&self) -> Scope;
}
impl ScopeExt for Scope {
fn next(&self) -> Scope {
scope_at(&self.session_id, &self.epoch, self.step + 1)
}
}
/// The `Id` form of a domain request id, for a payload field that carries it as a string.
pub trait DomainRequestIdExt {
fn id(&self) -> Id;
}
impl DomainRequestIdExt for DomainRequestId {
fn id(&self) -> Id {
self.as_str().to_owned()
}
}
/// Carrying a terminal domain outcome in a bus `outcome` object.
pub trait SessionRpcOutcomeExt: Sized {
fn to_outcome(&self) -> Map<String, Value>;
fn from_outcome(outcome: &Map<String, Value>) -> Result<Self, String>;
fn result(&self) -> DomainResult<&Value>;
}
impl SessionRpcOutcomeExt for SessionRpcOutcome {
fn to_outcome(&self) -> Map<String, Value> {
object(self.to_json())
}
fn from_outcome(outcome: &Map<String, Value>) -> Result<SessionRpcOutcome, String> {
SessionRpcOutcome::from_json(&Value::Object(outcome.clone())).map_err(|e| e.0)
}
fn result(&self) -> DomainResult<&Value> {
outcome_result(self)
}
}
/// One domain failure, ready to send.
pub fn failure_outcome(
request_id: &DomainRequestId,
worker_id: &str,
incarnation_id: &str,
scope: Option<Scope>,
error: DomainError,
) -> SessionRpcOutcome {
SessionRpcOutcome::Failure(SessionRpcFailure {
request_id: request_id.clone(),
worker_id: worker_id.to_owned(),
incarnation_id: incarnation_id.to_owned(),
scope,
code: error.code,
message: error.message,
mutation: error.mutation,
})
}
/// One domain success, ready to send.
pub fn success_outcome(
request_id: &DomainRequestId,
worker_id: &str,
incarnation_id: &str,
scope: Option<Scope>,
result: Map<String, Value>,
) -> SessionRpcOutcome {
SessionRpcOutcome::Success(SessionRpcSuccess {
request_id: request_id.clone(),
worker_id: worker_id.to_owned(),
incarnation_id: incarnation_id.to_owned(),
scope,
result: Value::Object(result),
})
}

162
services/flysim/crates/fly-session/src/worker.rs
View file

@ -13,19 +13,16 @@ use std::sync::{Arc, Mutex};
use serde_json::{Map, Value};
use crate::dedup::{Admission, CachedReply, OpClass, OperationKey, ResultCache};
use crate::types::{
self as types, AcknowledgeParams, AcknowledgeResult, Digest, DomainError, DomainResult,
ErrorCode, HelloLimits, HelloParams, HelloResult, Id, Mutation, RequestId, Role,
SessionRpcFailure, SessionRpcOutcome, SessionRpcRequest, SessionRpcSuccess, ShutdownParams,
ShutdownResult, StatusResult, U64, WorkerState,
};
// `crate::types` is this crate's facade over the shared `fly-session-types` crate; the
// glob keeps the contract's own names in sight instead of restating them.
use crate::types::*;
/// `Worker.Acknowledge` accepts 1..=16 request ids.
pub const MAX_ACKNOWLEDGE_IDS: usize = 16;
/// The build identity a worker reports in Hello. It is not a profile digest.
pub fn build_digest() -> Digest {
Digest::of(b"fly-session/synthetic-workers-v1")
digest_of_bytes(b"fly-session/synthetic-workers-v1")
}
/// The status a worker reports, kept outside the endpoint mutex so `Worker.Status` stays
@ -35,9 +32,9 @@ pub struct StatusCell(Arc<Mutex<StatusInner>>);
struct StatusInner {
state: WorkerState,
current_scope: Option<types::Scope>,
active_request_id: Option<Id>,
last_completed_request_id: Option<Id>,
current_scope: Option<Scope>,
active_request_id: Option<DomainRequestId>,
last_completed_request_id: Option<DomainRequestId>,
last_batch_id: Option<Id>,
progress_counter: u64,
}
@ -73,15 +70,15 @@ impl StatusCell {
self.with(|s| s.state)
}
pub fn set_scope(&self, scope: Option<types::Scope>) {
pub fn set_scope(&self, scope: Option<Scope>) {
self.with(|s| s.current_scope = scope);
}
pub fn set_active(&self, request_id: Option<Id>) {
pub fn set_active(&self, request_id: Option<DomainRequestId>) {
self.with(|s| s.active_request_id = request_id);
}
pub fn set_completed(&self, request_id: Id) {
pub fn set_completed(&self, request_id: DomainRequestId) {
self.with(|s| {
s.active_request_id = None;
s.last_completed_request_id = Some(request_id);
@ -114,7 +111,7 @@ impl StatusCell {
active_request_id: s.active_request_id.clone(),
last_completed_request_id: s.last_completed_request_id.clone(),
last_batch_id: s.last_batch_id.clone(),
progress_counter: U64(s.progress_counter),
progress_counter: s.progress_counter,
})
}
}
@ -139,12 +136,9 @@ impl HandlerReply {
HandlerReply { result, artifacts, mutated: true }
}
pub fn from<T: serde::Serialize>(value: &T) -> HandlerReply {
let v = serde_json::to_value(value).expect("a result serializes");
match v {
Value::Object(m) => HandlerReply::new(m),
_ => unreachable!("a struct serializes to an object"),
}
/// The canonical JSON of one method result.
pub fn from<T: DomainType>(value: &T) -> HandlerReply {
HandlerReply::new(object(value.to_json()))
}
}
@ -157,14 +151,17 @@ pub struct HandlerCtx<'a> {
}
impl HandlerCtx<'_> {
/// Deserializes `params` into a method payload, reporting INVALID_ARGUMENT.
pub fn params<T: serde::de::DeserializeOwned>(&self) -> DomainResult<T> {
serde_json::from_value(Value::Object(self.request.params.clone()))
/// Reads and validates `params` as a method payload, reporting INVALID_ARGUMENT.
///
/// The contract crate does the reading, so a misspelled required field fails here rather
/// than silently defaulting.
pub fn params<T: DomainType>(&self) -> DomainResult<T> {
T::from_json(&self.request.params)
.map_err(|e| DomainError::invalid(format!("{}: {e}", self.method)))
}
/// The scope the request must carry.
pub fn scope(&self) -> DomainResult<&types::Scope> {
pub fn scope(&self) -> DomainResult<&Scope> {
self.request
.scope
.as_ref()
@ -281,12 +278,14 @@ async fn run<E: WorkerEndpoint>(
while let Some(incoming) = service.next().await {
let method = incoming.method().to_owned();
let responder = incoming.responder();
let request = match SessionRpcRequest::from_payload(incoming.payload()) {
let request = match SessionRpcRequest::from_json(&Value::Object(
incoming.payload().clone(),
)) {
Ok(request) => request,
Err(e) => {
// A malformed envelope has no usable requestId, so the reply names req-0.
let failure = failure(
&Id::lit("req-0"),
&DomainRequestId::from_serial(0),
&worker_id,
&incarnation_id,
None,
@ -296,20 +295,8 @@ async fn run<E: WorkerEndpoint>(
continue;
}
};
let serial = match RequestId::parse(&request.request_id) {
Ok(serial) => serial,
Err(e) => {
let failure = failure(
&request.request_id,
&worker_id,
&incarnation_id,
request.scope.clone(),
DomainError::invalid(e),
);
let _ = responder.reply(failure.to_outcome(), &[]).await;
continue;
}
};
// The contract type already validated the `req-<U64>` form when it read the envelope.
let serial = request.request_id.clone();
// The common methods never enter the endpoint mutex, so they answer during a mutation.
match method.as_str() {
@ -326,7 +313,7 @@ async fn run<E: WorkerEndpoint>(
continue;
}
"Worker.Status" => {
let result = serde_json::to_value(status.snapshot()).expect("status serializes");
let result = status.snapshot().to_json();
let outcome = success(
&request,
&worker_id,
@ -349,12 +336,10 @@ async fn run<E: WorkerEndpoint>(
"Worker.Shutdown" => {
let outcome = match request.params.get("reason") {
Some(_) => {
match serde_json::from_value::<ShutdownParams>(Value::Object(
request.params.clone(),
)) {
match ShutdownParams::from_json(&request.params) {
Ok(_) => {
status.set_state(WorkerState::Stopping);
Ok(ShutdownResult { stopping: true })
Ok(ShutdownResult)
}
Err(e) => Err(DomainError::invalid(format!("Worker.Shutdown: {e}"))),
}
@ -363,7 +348,7 @@ async fn run<E: WorkerEndpoint>(
};
match outcome {
Ok(result) => {
let value = serde_json::to_value(result).expect("shutdown serializes");
let value = result.to_json();
let outcome =
success(&request, &worker_id, &incarnation_id, object(value));
let _ = responder.reply(outcome.to_outcome(), &[]).await;
@ -402,12 +387,25 @@ async fn run<E: WorkerEndpoint>(
continue;
};
let body = request.body_digest(&method);
let body = match request.body_digest(&method) {
Ok(body) => body,
Err(e) => {
let outcome = failure(
&request.request_id,
&worker_id,
&incarnation_id,
request.scope.clone(),
DomainError::invalid(format!("{method}: {e}")),
);
let _ = responder.reply(outcome.to_outcome(), &[]).await;
continue;
}
};
let key = match (class, &request.scope) {
(OpClass::StepMutation, Some(scope)) => OperationKey {
session_id: scope.session_id.clone(),
epoch: scope.epoch.clone(),
step: scope.step.0,
step: scope.step,
method: method.clone(),
worker_id: worker_id.clone(),
},
@ -424,7 +422,7 @@ async fn run<E: WorkerEndpoint>(
}
_ => OperationKey {
session_id: session_id.clone(),
epoch: Id::lit("lifecycle"),
epoch: id("lifecycle"),
step: 0,
method: method.clone(),
worker_id: worker_id.clone(),
@ -436,7 +434,7 @@ async fn run<E: WorkerEndpoint>(
// consumed and needs only the cached result.
let admission = {
let mut c = cache.lock().await;
c.admit(class, &key, serial, &body)
c.admit(class, &key, serial.clone(), &body)
};
match admission {
Admission::Replay(reply) => {
@ -495,7 +493,7 @@ async fn execute<E: WorkerEndpoint>(
incarnation_id: Id,
class: OpClass,
key: OperationKey,
serial: RequestId,
serial: DomainRequestId,
body: Digest,
method: String,
request: SessionRpcRequest,
@ -541,19 +539,18 @@ async fn execute<E: WorkerEndpoint>(
let _ = responder.reply(outcome.to_outcome(), &attachments).await;
}
Err(e) => {
if e.mutation == Mutation::None {
if e.mutation == MutationCertainty::None {
// Nothing happened, so the key stays free for the corrected request.
let mut c = cache.lock().await;
c.abandon(&key);
} else {
let cached = CachedReply::new(SessionRpcOutcome::Failure(SessionRpcFailure {
kind: types::FailureTag::Error,
request_id: request.request_id.clone(),
worker_id: worker_id.clone(),
incarnation_id: incarnation_id.clone(),
scope: request.scope.clone(),
error: e.clone(),
}));
let cached = CachedReply::new(failure_outcome(
&request.request_id,
&worker_id,
&incarnation_id,
request.scope.clone(),
e.clone(),
));
let mut c = cache.lock().await;
if class == OpClass::StepMutation {
c.record(key, serial, body, cached);
@ -600,31 +597,23 @@ fn success(
incarnation_id: &Id,
result: Map<String, Value>,
) -> SessionRpcOutcome {
SessionRpcOutcome::Success(SessionRpcSuccess {
kind: types::SuccessTag::Result,
request_id: request.request_id.clone(),
worker_id: worker_id.clone(),
incarnation_id: incarnation_id.clone(),
scope: request.scope.clone(),
success_outcome(
&request.request_id,
worker_id,
incarnation_id,
request.scope.clone(),
result,
})
)
}
fn failure(
request_id: &Id,
request_id: &DomainRequestId,
worker_id: &Id,
incarnation_id: &Id,
scope: Option<types::Scope>,
scope: Option<Scope>,
error: DomainError,
) -> SessionRpcOutcome {
SessionRpcOutcome::Failure(SessionRpcFailure {
kind: types::FailureTag::Error,
request_id: request_id.clone(),
worker_id: worker_id.clone(),
incarnation_id: incarnation_id.clone(),
scope,
error,
})
failure_outcome(request_id, worker_id, incarnation_id, scope, error)
}
fn hello(
@ -635,8 +624,7 @@ fn hello(
role: Role,
capabilities: &[Id],
) -> SessionRpcOutcome {
let params: HelloParams =
match serde_json::from_value(Value::Object(request.params.clone())) {
let params: HelloParams = match HelloParams::from_json(&request.params) {
Ok(params) => params,
Err(e) => {
return failure(
@ -682,21 +670,20 @@ fn hello(
);
}
let result = HelloResult {
selected_major: 1,
selected_minor: 0,
worker_id: worker_id.clone(),
incarnation_id: incarnation_id.clone(),
role,
build_digest: build_digest(),
contract_digest: types::contract_digest(),
contract_digest: contract_digest(),
capabilities: capabilities.to_vec(),
limits: HelloLimits { max_agents: types::MAX_AGENTS, max_ports: types::MAX_PORTS },
max_agents: MAX_AGENTS as u64,
max_ports: MAX_PORTS as u64,
};
success(
request,
worker_id,
incarnation_id,
object(serde_json::to_value(result).expect("hello serializes")),
object(result.to_json()),
)
}
@ -704,9 +691,8 @@ async fn acknowledge(
request: &SessionRpcRequest,
cache: &Arc<tokio::sync::Mutex<ResultCache>>,
) -> DomainResult<Map<String, Value>> {
let params: AcknowledgeParams =
serde_json::from_value(Value::Object(request.params.clone()))
.map_err(|e| DomainError::invalid(format!("Worker.Acknowledge: {e}")))?;
let params: AcknowledgeParams = AcknowledgeParams::from_json(&request.params)
.map_err(|e| DomainError::invalid(format!("Worker.Acknowledge: {e}")))?;
if params.request_ids.is_empty() || params.request_ids.len() > MAX_ACKNOWLEDGE_IDS {
return Err(DomainError::invalid("Worker.Acknowledge takes 1..=16 request ids"));
}
@ -715,5 +701,5 @@ async fn acknowledge(
c.acknowledge(&params.request_ids)
};
let result = AcknowledgeResult { acknowledged };
Ok(object(serde_json::to_value(result).expect("acknowledge serializes")))
Ok(object(result.to_json()))
}

6
services/flysim/crates/fly-session/tests/common/mod.rs
View file

@ -5,7 +5,7 @@
use std::time::Duration;
use fly_session::harness::{HarnessConfig, SessionHarness, Via};
use fly_session::types::Id;
use fly_session::types::*;
pub const WAIT: Duration = Duration::from_secs(20);
@ -68,11 +68,11 @@ pub async fn default_fixture(via: Via) -> Fixture {
}
pub fn fly_a() -> Id {
Id::lit("fly-a")
id("fly-a")
}
pub fn fly_b() -> Id {
Id::lit("fly-b")
id("fly-b")
}
/// The index of an audit entry, or a panic naming what was missing.

31
services/flysim/crates/fly-session/tests/failures.rs
View file

@ -14,7 +14,7 @@ use fly_session::coordinator::Injections;
use fly_session::environment::EnvironmentFaults;
use fly_session::harness::{HarnessConfig, SessionHarness, Via};
use fly_session::phase::Phase;
use fly_session::types::{ErrorCode, Id};
use fly_session::types::*;
both_transports!(
a_duplicate_prepare_after_a_lost_reply_repeats_nothing,
@ -156,8 +156,11 @@ async fn a_lost_advance_result_resolves_the_same_operation(via: Via) {
)
.await;
assert!(
injected.injections.iter().any(|o| o.what == "lost-advance-result"),
"the call was abandoned with an uncertain outcome"
injected
.injections
.iter()
.any(|o| o.what == "lost-advance-result" && o.identical),
"the call was abandoned after dispatch, with an uncertain outcome"
);
assert!(
injected.injections.iter().any(|o| o.what == "status-after-loss" && o.identical),
@ -202,7 +205,7 @@ async fn one_commit_failing_after_another_succeeds_fails_the_epoch(via: Via) {
.await
.expect_err("the epoch fails when one commit fails");
assert_eq!(err.error.code, ErrorCode::BackendFailure);
assert_eq!(err.error.mutation, fly_session::types::Mutation::Applied);
assert_eq!(err.error.mutation, MutationCertainty::Applied);
assert_eq!(f.harness.coordinator.phase(), Phase::Failed);
// The world took the step whose commits failed, and it takes no further step.
@ -210,12 +213,12 @@ async fn one_commit_failing_after_another_succeeds_fails_the_epoch(via: Via) {
assert_eq!(advances, 1, "the failed transition never reached a committed boundary");
let env = f.harness.coordinator.environment_ref().clone();
let status = within("status", f.harness.coordinator.status(&env)).await.unwrap();
assert_eq!(status.current_scope.as_ref().unwrap().step.get(), 2);
assert_eq!(status.current_scope.as_ref().unwrap().step, 2);
let again = f.harness.coordinator.step().await.expect_err("no step from Failed");
assert_eq!(again.error.code, ErrorCode::InvalidPhase);
let status = within("status", f.harness.coordinator.status(&env)).await.unwrap();
assert_eq!(
status.current_scope.unwrap().step.get(),
status.current_scope.unwrap().step,
2,
"no world step follows a partial commit"
);
@ -223,10 +226,10 @@ async fn one_commit_failing_after_another_succeeds_fails_the_epoch(via: Via) {
// The agent that succeeded is at the new boundary; the one that failed reports Failed.
let a = f.harness.coordinator.agent_ref(&fly_a()).cloned().unwrap();
let status = within("status", f.harness.coordinator.status(&a)).await.unwrap();
assert_eq!(status.current_scope.unwrap().step.get(), 2);
assert_eq!(status.current_scope.unwrap().step, 2);
assert_eq!(
f.harness.agent_status(&fly_b()).unwrap().state(),
fly_session::types::WorkerState::Failed
WorkerState::Failed
);
// Nothing was published for the boundary that failed to commit.
let audit = f.harness.coordinator.audit.clone();
@ -331,7 +334,7 @@ async fn an_exact_duplicate_of_a_running_operation_is_in_progress(via: Via) {
);
// And the original still completed: exactly one world step, at one boundary.
assert_eq!(f.harness.coordinator.stats().advances, 1);
assert_eq!(f.harness.coordinator.observation().unwrap().boundary.get(), 1);
assert_eq!(f.harness.coordinator.observation().unwrap().boundary, 1);
f.shutdown().await;
}
@ -348,16 +351,12 @@ async fn an_old_epoch_operation_is_refused_with_stale_epoch() {
// The agent is live and initialized under epoch e1. An operation naming another epoch is
// refused as stale rather than applied to this brain.
let worker = harness.coordinator.agent_ref(&fly_a()).cloned().unwrap();
let scope = fly_session::types::Scope::new(
&Id::lit("demo"),
&Id::lit("e0"),
1,
);
let scope = scope_at("demo", "e0", 1);
let params = serde_json::json!({
"agentId": "fly-a",
"profileDigest": fly_session::types::Digest::of(b"whatever").to_string(),
"profileDigest": digest_of_bytes(b"whatever").to_string(),
"interval": {"numerator": "16666667", "denominator": "1"},
"decisionContextDigest": fly_session::types::Digest::of(b"whatever").to_string(),
"decisionContextDigest": digest_of_bytes(b"whatever").to_string(),
"preStepStimulations": [],
});
let bus = harness.client("coordinator2").await;

41
services/flysim/crates/fly-session/tests/session.rs
View file

@ -11,7 +11,7 @@ use common::{Fixture, at, count, default_fixture, fixture, fly_a, fly_b, within}
use fly_session::coordinator::DispatchOrder;
use fly_session::harness::{AgentSpec, HarnessConfig, Via};
use fly_session::phase::Phase;
use fly_session::types::Id;
use fly_session::types::*;
use fly_session::agent::AgentFaults;
both_transports!(
@ -40,7 +40,7 @@ async fn one_world_advance_per_complete_batch(via: Via) {
assert_eq!(f.harness.coordinator.stats().advances, STEPS);
assert_eq!(f.harness.coordinator.phase(), Phase::Ready(STEPS));
assert_eq!(
f.harness.coordinator.observation().unwrap().boundary.get(),
f.harness.coordinator.observation().unwrap().boundary,
STEPS,
"the world is at exactly one boundary per batch"
);
@ -135,12 +135,12 @@ async fn a_60_hz_world_with_a_1_ms_tick_runs_16_17_17(via: Via) {
let ticks: Vec<u64> = transitions
.iter()
.map(|t| {
t.agents
t.behaviour
.agents
.iter()
.find(|a| a.agent_id == agent)
.expect("the agent is in every transition")
.ticks_advanced
.get()
})
.collect();
assert_eq!(ticks, vec![16, 17, 17], "{agent} tick profile");
@ -148,13 +148,14 @@ async fn a_60_hz_world_with_a_1_ms_tick_runs_16_17_17(via: Via) {
let last = transitions
.last()
.unwrap()
.behaviour
.agents
.iter()
.find(|a| a.agent_id == agent)
.unwrap();
assert!(last.remainder.is_zero(), "{agent} remainder after three steps");
// Warm-up ticks are counted too, so brainTicks is warm-up plus the 50 gameplay ticks.
assert_eq!(last.brain_ticks.get(), 50 + f.harness.config.warmup_ticks);
assert_eq!(last.brain_ticks, 50 + f.harness.config.warmup_ticks);
}
f.shutdown().await;
}
@ -195,7 +196,7 @@ async fn a_pause_mid_step_completes_the_step_and_pauses_at_the_boundary(via: Via
// A paused worker retains its state and answers Status; the world does not advance.
let env = f.harness.coordinator.environment_ref().clone();
let status = within("status", f.harness.coordinator.status(&env)).await.unwrap();
assert_eq!(status.current_scope.unwrap().step.get(), 2);
assert_eq!(status.current_scope.unwrap().step, 2);
assert_eq!(f.harness.coordinator.stats().advances, 2);
f.harness.coordinator.resume().unwrap();
@ -213,14 +214,14 @@ async fn bootstrap_cannot_advance_the_world_or_produce_a_reward(via: Via) {
within("bootstrap", f.harness.coordinator.bootstrap()).await.unwrap();
assert_eq!(f.harness.coordinator.phase(), Phase::Ready(0));
let observation = f.harness.coordinator.observation().unwrap();
assert_eq!(observation.boundary.get(), 0);
assert_eq!(observation.boundary, 0);
assert!(observation.world_time.is_zero());
assert_eq!(f.harness.coordinator.stats().advances, 0);
assert_eq!(f.harness.coordinator.evaluations(), 0);
// The environment advanced nothing, so its status is still at boundary 0 with no batch.
let env = f.harness.coordinator.environment_ref().clone();
let status = within("status", f.harness.coordinator.status(&env)).await.unwrap();
assert_eq!(status.current_scope.as_ref().unwrap().step.get(), 0);
assert_eq!(status.current_scope.as_ref().unwrap().step, 0);
assert!(status.last_batch_id.is_none(), "no batch was ever applied");
// Warm-up did run, with learning disabled, so the models did mutate.
for agent in [fly_a(), fly_b()] {
@ -294,7 +295,7 @@ async fn a_terminal_episode_pauses_at_its_own_boundary(via: Via) {
assert!(f.harness.coordinator.episode_request().is_some());
// No worker resets itself, and no further gameplay transition is allowed.
let err = f.harness.coordinator.step().await.expect_err("no transition after terminal");
assert_eq!(err.error.code, fly_session::types::ErrorCode::InvalidPhase);
assert_eq!(err.error.code, ErrorCode::InvalidPhase);
f.shutdown().await;
}
@ -306,12 +307,12 @@ async fn status_answers_with_the_committed_boundary(via: Via) {
for agent in [fly_a(), fly_b()] {
let worker = f.harness.coordinator.agent_ref(&agent).cloned().unwrap();
let status = within("status", f.harness.coordinator.status(&worker)).await.unwrap();
assert_eq!(status.state, fly_session::types::WorkerState::Ready);
assert_eq!(status.current_scope.unwrap().step.get(), 2);
let before = status.progress_counter.get();
assert_eq!(status.state, WorkerState::Ready);
assert_eq!(status.current_scope.unwrap().step, 2);
let before = status.progress_counter;
// A status query is not progress.
let again = within("status", f.harness.coordinator.status(&worker)).await.unwrap();
assert_eq!(again.progress_counter.get(), before);
assert_eq!(again.progress_counter, before);
}
f.shutdown().await;
}
@ -323,7 +324,7 @@ async fn a_worker_refuses_a_second_initialize(via: Via) {
let err = within("second bootstrap", f.harness.coordinator.bootstrap())
.await
.expect_err("the environment is already initialized");
assert_eq!(err.error.code, fly_session::types::ErrorCode::InvalidPhase);
assert_eq!(err.error.code, ErrorCode::InvalidPhase);
f.shutdown().await;
}
@ -356,8 +357,10 @@ async fn sequential_concurrent_and_reversed_orders_agree() {
assert_eq!(behaviour.len(), 4);
behaviours.insert(format!("{via:?}/{order:?}"), behaviour);
// The operational metadata is recorded but is not part of the comparison.
let operational = f.harness.coordinator.trace.transitions[0].operational();
assert!(operational.iter().any(|(k, _)| k == "batchId"));
// The operational metadata is recorded beside the behaviour, not inside it.
let operational = &f.harness.coordinator.trace.transitions[0].operational;
assert_eq!(operational.prepare_request_ids.len(), 2);
assert_eq!(operational.commit_request_ids.len(), 2);
f.shutdown().await;
}
}
@ -377,8 +380,8 @@ async fn sequential_concurrent_and_reversed_orders_agree() {
async fn a_single_agent_composition_runs_the_same_transaction() {
let config = HarnessConfig {
agents: vec![AgentSpec {
agent_id: Id::lit("fly-a"),
port_id: Id::lit("p1"),
agent_id: id("fly-a"),
port_id: id("p1"),
seed: 7,
faults: AgentFaults::default(),
}],
@ -394,7 +397,7 @@ async fn a_single_agent_composition_runs_the_same_transaction() {
.trace
.transitions
.iter()
.map(|t| t.agents[0].ticks_advanced.get())
.map(|t| t.behaviour.agents[0].ticks_advanced)
.collect();
assert_eq!(ticks, vec![16, 17, 17]);
f.shutdown().await;