flybrain/docs/design/malecns-modular-implementation.md

# Implementation backlog: MaleCNS and modular sessions

Status: **planned, not started**. Written 2026-09-18. Companion to the
[design and code analysis](malecns-modular-sessions.md), based on `f7bc13a`.

This is the execution order for that proposal. It does not authorize deployment or a live
stream. Reconcile the baseline with merged macro/shop/recovery work before implementation.
Existing feed/control contracts and TypeScript oracle rules remain binding.

Concrete second-game audit: [Melee framework and emulator plan](melee-framework-audit.md).
Its MELEE-01/02 spikes specialize EMULATOR-01 below and can proceed alongside framework
extraction; they do not depend on importing MaleCNS first.

The [session framework contract set](session-framework/README.md) now specifies the new
multi-process architecture. Its [implementation guide](session-framework/implementation.md)
breaks RUNTIME/WIRE/STATE work into concrete schema, transport, coordinator and worker slices.

**Communications decision:** build [Flybus](session-framework/bus-v1.md), one Rust RPC/pub-sub
router with immutable external artifacts, delivery guards and last-owner GC. It serves workers,
application supervision, presentation and storage. No second direct-RPC system or NATS broker.
Application-specific orchestration and frontend are developed together; native frames leave
the environment, while resizing/compositing/streaming belongs to presentation.

## 1. Delivery strategy

Deliver working vertical slices; keep the existing FAFB/Game Boy composition usable throughout.

1. Establish a behavior baseline and explicit dataset/profile identities.
   Independently build the generic Flybus example; it needs neither dataset nor emulator.
2. In parallel workstreams, characterize MaleCNS and extract the single-agent session runtime.
3. Demonstrate two isolated brains driving one ROM-free shared environment.
4. Expose that session through versioned feed/control contracts and a multi-agent broadcast.
5. Integrate a specifically chosen alternative emulator after its capability spike passes.
6. Finish physical package reorganization once the second consumer proves the boundaries.

**First milestone:** a reproducible headless MaleCNS run and a reusable single-agent session.
**Second milestone:** two flies in a synthetic arena with coherent resume and a local broadcast.
**Third milestone:** two flies in the chosen fighting game, with documented task scaffolding.

No elapsed-time estimate is committed before the import and emulator spikes establish their
unknowns. Split an item further when its contract and implementation cannot be reviewed together.

## 2. Work queue

Every item starts pending. Branch names are suggested implementation branches, not branches
already created. Builders use separate worktrees; the coordinator reviews contracts and results.

### FOUNDATION-01 — Pin existing behavior

- **Branch:** `test/session-baseline`
- **Depends on:** reconciliation with current main and related open work.
- Record effective legacy configuration, fingerprint, version strings and frame ordering.
- Add a ROM-free transition harness around the service's frame orchestration; capture brain
  ticks, decoded actions, reward application and recovery effects with explicit clocks.
- Distinguish exact numerical replay from intentional macro/hold/transient reset on restore.
- **Done:** existing goldens pass; a trace fixture detects reordered vision/reward application;
  legacy feed/API fixtures and compatibility identity are unchanged.

### FOUNDATION-02 — Specify bundle and behavior identities

- **Branch:** `feat/brain-profile-contract`
- **Depends on:** FOUNDATION-01.
- Specify dataset manifests, original-ID mapping, anatomical roles, sensory bindings,
  readout bindings and composite behavior identity in a focused contract.
- Add profile resolution/validation around the existing core in TypeScript and Rust.
- Keep schema-1 fingerprinting and the legacy macro-role exception behind the legacy path.
- Add strict graph validation for new bundles, shared invalid fixtures and profile mismatch tests.
- **Done:** empty required populations, malformed CSR and incorrect profile restores fail;
  legacy FAFB artifacts and default numerical version strings remain unchanged.

**2026-09-23: split by operator decision.** PROF-02a, the legacy profile, is done as a contract:
[legacy Game Boy composition v1](session-framework/legacy-gameboy-v1.md) defines
`gameboy-legacy-fafb-v783-v1` (today's schema-1 fingerprint embedded, `lif-1ms-f64-v2` and
`fly-kc-mbon-rstdp-v2` unchanged, the macro-role exception declared), the readout context
`gameboy-readout-context-v1`, the decision `gameboy-channels-v1`, and a composition declaration
whose digest carries the decoder and macro-channel configuration instead of the legacy
compatibility string. PROF-02b -- the bundle manifests, role mapping, strict graph validation and
profile-mismatch fixtures above -- is later and gates DATA-01, not the session port.

### DATA-01 — Acquire and normalize MaleCNS

- **Branch:** `feat/malecns-import`
- **Depends on:** FOUNDATION-02.
- Build a source-specific importer from official v1.0 tables with checksummed source locks.
- Reconcile the Codex versus neuPrint inventories, or explicitly select and document one.
- Preserve raw contact counts, transmitter evidence, original IDs and missing-data indicators.
- Emit deterministic graph bundles, weight≥1/weight≥5 comparison variants, and an exclusion/
  clipping/coverage report. Decide schema-1 feasibility from measured weight ranges.
- Add attribution and license records with actual artifacts; keep raw downloads out of git.
- **Done:** repeated builds match; endpoint/role/index invariants pass; both loaders agree;
  no service startup or ordinary unit test needs a download.

### DATA-02 — Characterize MaleCNS in the current kernel

- **Branch:** `feat/malecns-baseline-profile`
- **Depends on:** DATA-01.
- Audit L1 geometry, hemisphere handling, KC/MBON/PAM mappings and brain-versus-VNC motor roles.
- Define a versioned fixed readout; keep task action partitions out of anatomical truth.
- Run learning-off first, then learning-on, using fixed sensory traces and multiple seeds.
- Generate TS reference goldens and compare Rust exactly; measure activity, saturation,
  initialization, memory and per-phase latency for both graph thresholds.
- **Done:** publish a reproducible characterization report and profile choice. Stop task
  integration if required mappings are missing or dynamics are unusable; any recalibration
  becomes a named profile rather than an edit to the legacy model.

### RUNTIME-01 — Separate environment execution from task interpretation

- **Branch:** `refactor/environment-task-boundary`
- **Depends on:** FOUNDATION-02.
- Specify controller ports, digital/analog controls, rational cadence, media descriptors,
  observation ownership and backend capabilities.
- Wrap binjgb as the first environment; retain Game Boy FFI/cache/state behavior.
- Keep Pokémon memory inspection, objective routing, macros and reward rules in its task.
- The executor receives coherent current game state, progress/objective view and clock on
  every step. This richer context is not implicitly passed to the neural sensory encoder.
- Preserve existing imports through a facade; avoid simultaneous directory moves.
- **Done:** existing single-agent action/reward traces match and a fake environment can be
  driven through the same boundary without importing binjgb or task-specific addresses.

**2026-09-23: contract written (RT-01a), implementation pending.** The operator decided the
boundary: macros run in the coordinator's action executor over a per-boundary 64-KiB memory
image carried as an inspection artifact plus the ROM as an `AssetRef`; the emulator shim gains
one read-only bulk read and the joypad stays its only write; the Pokémon task and executor are
one object (`pokered-macros-v1`); the ratchet is the `legacy-ratchet-rollback-v1` episode policy
over the environment extension `gameboy-slots-v1`. The dated amendments are in
[workers-v1](session-framework/workers-v1.md), [step-v1](session-framework/step-v1.md) and
[state-media-v1](session-framework/state-media-v1.md); the session implementation guide's
ENV-01 carries the build.

### RUNTIME-02 — Extract the single-agent session

- **Branch:** `refactor/session-runtime`
- **Depends on:** RUNTIME-01 and BUS-01..03 from the contract implementation guide.
- Move deterministic agent/environment/task orchestration out of `Sim` into a library.
- Keep HTTP, WebSocket serialization, wall-clock publication and process supervision in flysim.
- Route internal worker calls and observations through Flybus; keep public compatibility
  adapters at the application edge. Domain caches own artifact handles for replay.
- Give session clock, action executor, task ledger and recovery state explicit owners.
- Wrap the existing composition with legacy ordering, checkpoint and reset semantics.
- **Done:** headless bus client runs a session without Twitch/browser; the legacy composition
  passes its traces and restore tests; a slow snapshot consumer cannot stall simulation.

### RUNTIME-03 — Add synchronized multi-agent sessions

- **Branch:** `feat/multi-agent-arena`
- **Depends on:** RUNTIME-02.
- Implement a ROM-free two-player arena and per-port controller ownership.
- Evaluate both brains against one observation boundary; apply one complete action batch;
  advance the world once. Start sequentially, then verify parallel execution equivalence.
- Isolate RNG, stimulation, decoder holds, gains, traces and rewards per agent; share only
  immutable topology. Enforce a total worker budget and single-dispatcher pool ownership.
- Define participant failure, lateness and episode reset policies.
- **Done:** no cross-agent state leakage; swapping evaluation order leaves results unchanged;
  one failed participant cannot accidentally advance a half-controlled match.

### STATE-01 — Capture and resume whole sessions

- **Branch:** `feat/session-checkpoints`
- **Depends on:** RUNTIME-03; specify the state contract during RUNTIME-02.
- Define the new envelope/manifest and preserve the `FLYSIM01` reader.
- Capture all agents, environment, task/executor/admission state and clock remainders at one
  boundary. Bound off-thread write jobs and retain atomic manifest commit semantics.
- Validate all components before installing any restored state; define external-backend staging.
- **Done:** uninterrupted and resumed synthetic matches agree; corrupting any participant
  refuses the generation without partial restore; crash-injection fallback tests pass.

### WIRE-01 — Introduce session feed/control v2

- **Branch:** `feat/session-protocol-v2`
- **Depends on:** FOUNDATION-02, RUNTIME-02; use RUNTIME-03 fixtures for integration.
- Write binding contracts before consumer implementation: descriptors, scoped agents/events,
  media IDs/timestamps, task progress, targeted stimulation and retry/idempotency behavior.
- Implement Rust/TS codecs, schemas and a fake server; preserve the legacy v1 surface.
- Specify descriptor reconnect behavior, asset/index identity, bounded message sizes and audio gaps.
- **Done:** cross-language fixtures pass for unequal neuron counts and shared/private views;
  duplicate attachment kinds no longer collide; ambiguous targets and incompatible schemas fail.

### PRESENTATION-01 — Compose multi-agent stage and bridge

- **Branch:** `feat/multi-agent-broadcast`
- **Depends on:** WIRE-01, RUNTIME-03.
- Replace stage store/scaler singletons with session/agent instances and one paint scheduler.
- Combine framework descriptors/measurements with application-owned state/cues over the bus.
  Rendering may retain an artifact after message drop; last-use release returns delivery credit.
- Resolve geometry from hashed descriptors, preserve the Game Boy presentation, and add a
  shared-match layout with explicit audio ownership.
- Route bridge commands/redemptions to persistent session/agent identities; test lost responses,
  retries and restart without applying an interaction twice or to a different agent.
- **Done:** local synthetic match broadcast works; two-agent PNGs receive operator review;
  browser/fixture/legibility checks pass; bridge remains template-only and quiet-mode capable.

### DATA-03 — Run MaleCNS through the complete application

- **Branch:** `feat/malecns-session`
- **Depends on:** DATA-02 and descriptor-aware assets from WIRE-01/PRESENTATION-01.
- Expose explicit profile selection and create a fresh MaleCNS state namespace.
- Verify task/controller bindings, stimulation capability and displayed anatomy identity.
- A narrow single-agent descriptor extension may ship earlier only with matching v1 contract
  and consumer updates; do not publish MaleCNS spikes as implicit FAFB indices.
- **Done:** local one-hour soak and restore drill pass; paired learning-off/on observations
  are recorded without claiming improved play; FAFB remains available unchanged.

### EMULATOR-01 — Establish the alternative backend's capabilities

- **Branch:** `spike/fighting-game-backend`
- **Depends on:** RUNTIME-01; can proceed alongside later runtime work.
- Choose the exact game/version and emulator; Melee/Dolphin is a candidate, not a commitment.
- Prove pause/step, simultaneous ports, analog input, frame/audio capture, state inspection,
  save/restore, process lifecycle and achievable cadence with synthetic controller traces.
- Prefer a bus-connected helper if embedding would leak emulator internals into the session
  library. Its native emulator protocol is an implementation detail, not a second framework API.
- **Done:** capability report includes pinned backend/content identity and reproducible results.
  If bounded stepping or coherent restore fails, stop and revise the backend/requirements
  before writing neural game logic. No game content enters repository fixtures.

### EMULATOR-02 — Build the two-fly fighting-game slice

- **Branch:** `feat/two-fly-fighting-game`
- **Depends on:** EMULATOR-01, STATE-01, PRESENTATION-01.
- Implement fixed controller mapping, match/round interpretation, positive attributed rewards,
  observation policy and episode recovery. Display selected actions and actual controls.
- Validate one fly, two flies, round transitions, backend failure and resume in that order.
- Run side swaps and repeated seeds; compare learning-off and simple control baselines before
  interpreting win rates. Separate show settings from controlled evaluation settings.
- **Done:** repeated local matches sustain declared cadence; restoration/failure policies work;
  scaffold, interventions and limits are documented; reviewed match presentation is legible.

### PACKAGE-01 — Finalize reusable packages and release compositions

- **Branch:** `refactor/reusable-package-layout`
- **Depends on:** a useful second backend plus PRESENTATION-01.
- Extract proven crate/package boundaries from the design's module table; preserve facades.
- Move the Rust workspace only in a mechanical follow-up if it makes library consumption clearer.
- Update CI/build/vendor/golden paths, dataset/view asset packaging and compatibility preflight.
- Add minimal external-style Rust/TS consumers and a synthetic example composition.
- Reconcile current docs, stale template explanations and licensing/asset attribution.
- **Done:** both legacy and new compositions package successfully; incompatible state is
  rejected before release selection; libraries run without importing broadcast services.

## 3. Dependency map and first execution batch

```text
FOUNDATION-01 → FOUNDATION-02 ┬→ DATA-01 → DATA-02 ───────────────→ DATA-03
                            └→ RUNTIME-01 → RUNTIME-02 → RUNTIME-03 → STATE-01
                                  │             └→ WIRE-01 ───────┐
                                  └→ EMULATOR-01          PRESENTATION-01
                                                              │
                         STATE-01 + EMULATOR-01 + PRESENTATION-01 → EMULATOR-02
                                         second backend + presentation → PACKAGE-01
```

The item dependency lists are authoritative; the diagram is a reading aid. The detailed
contract guide adds BUS-01 (RPC), BUS-02 (pub/sub) and BUS-03 (artifacts/GC) before the
distributed RUNTIME-02/03 slices. These can proceed independently of MaleCNS import.

Start the preservation track at FOUNDATION-01; the generic bus track can start with the
contract guide's small RPC/pub-sub/artifact example. Then review FOUNDATION-02's contract
before assigning DATA-01 and RUNTIME-01 to independent worktrees.
Contract/schema authorship is serialized to avoid conflicting definitions. Deployment host
work remains serialized under the repository's claim protocol.

## 4. Definition of done for every implementation branch

- Scope and intentional behavior changes are stated; compatibility impact is explicit.
- Meaningful boundary tests cover the changed behavior; the TS oracle is not adjusted to
  accommodate Rust output. Existing committed real-data goldens stay mandatory.
- `npm test`, `npm run typecheck`, `cargo test --workspace` (Rust workspace) and
  `infra/tests/lint.sh` pass before merge. Visual changes also pass applicable Playwright
  checks and PNG review. Optional full-MaleCNS/ROM runs record skips honestly.
- Performance-sensitive changes report representative activity, agent count, thread budget,
  memory and tail latency. New experiments state what is modeled versus handwritten.
- Review the complete diff, merge with `--no-ff` when authorized, and update this queue with
  commit, evidence and unresolved follow-ups. Rollback includes compatible state, not just code.

## 5. Decisions needed before the relevant work starts

| Decision | Deadline | Default recommendation |
| --- | --- | --- |
| MaleCNS inventory/filter policy | DATA-01 completion | Official versioned source; retain both threshold variants until measured |
| MaleCNS sensory/readout profile | DATA-02 | Audited L1 mapping with existing numerical model first |
| Exact fighting game and backend | EMULATOR-01 | Evaluate one concrete title/backend rather than supporting a console family at once |
| Number of flies and target resource budget | RUNTIME-03 performance gate | Two first; characterize four before promising it |
| Learning retention and sugar in matches | EMULATOR-02 task contract | Retention explicit; stimulation disabled in controlled comparisons |
| Package publication versus monorepo reuse | PACKAGE-01 | Monorepo libraries/examples first; public package publishing later |

There is no need to resolve these now to plan another feature. This backlog is ready for
resumption at FOUNDATION-01 and the independent BUS-01..03 track.