flybrain

A simulated fruit-fly brain that plays video games. A connectome-constrained spiking network reads the screen, its population rates become controller inputs, and a scalar reward nudges a bounded set of Kenyon-cell to MBON gains.

The library is @flybrain/brain in packages/brain. It holds the connectome dataset format, the LIF kernel, the plasticity rule, the population-rate readout and the activity-map geometry. It holds no game, no emulator and no reward rules: those belong to whatever embeds it.

Workspace layout

Path	Contents
packages/brain	the library (@flybrain/brain)
data/fafb-v783	FlyWire-derived browser artifacts (CC BY-NC 4.0)
tools/	the Python builder that regenerates data/ from official Codex exports
docs/	overview, dataset format, model, plasticity, readout, integration, limitations, verification
services/flysim	the Rust service: brain, emulator, snapshot feed, control API, checkpoints
apps/	planned: one directory per game demo
infra/	planned: deployment for the 24/7 stream (see docs/streaming-plan.md)

Quick start

npm ci
npm test
npm run typecheck

76 tests, about 7 seconds. There is no build step. npx tsx packages/brain/examples/node-random-frames.ts 60 runs the full 139,255-neuron brain with the Game Boy readout on noise frames in plain Node (about 0.9x Game Boy real time single-threaded on a WSL laptop).

Usage

import { NeuralAgent, gameboyDecoderConfig, toButtonMask } from '@flybrain/brain';
import { loadBrainDatasetFromDir } from '@flybrain/brain/node';

const dataset = await loadBrainDatasetFromDir('data/fafb-v783');
const agent = new NeuralAgent(dataset, { decoder: gameboyDecoderConfig() });
agent.warmup(firstFrame);                        // 2,500 ms with plasticity off, then calibrate

// every emulator frame:
const { active } = agent.tick(framebuffer, {     // RGBA 160x144 by default; any size via config
  rewards: [{ value: 0.5 }],                     // scalar rewards your game adapter detected
  boot: !inGame,                                 // relaxes Start/Select throttling on title screens
});
emulator.setButtons(toButtonMask(active));

const checkpoint = agent.exportState();          // bit-exact resume, validated on import

The lower layers (LifNetwork, RewardModulatedStdp, PopulationDecoder) are exported too for hosts that want to run the loop themselves.

integration.md has the full per-frame loop, the fractional frame timing and the checkpoint contract.

Documentation

• Overview: the pipeline, the layer map and the design principles.
• Dataset format: artifacts, CSR layout, weight encoding, every role and its count, fingerprinting, regeneration, license.
• Model: the 1-ms LIF kernel step by step, every default constant, the retina projection, the RNG, state export and version strings.
• Plasticity: edge selection, the eligibility and reinforcement equations, statistics, topology hash and the explicit non-claims.
• Readout: scores, exclusive groups, pulse channels, the blocked-direction cooldown, the Game Boy preset table and checkpoint versions.
• Integration: what a game must provide, the Pokemon Red integration as a worked example, and a sketch of a platformer adapter.
• Limitations: what is not claimed, what is unproven, measured throughput.
• Verification: the oracle-test strategy and what each test file covers.
• Streaming plan: headless capture, Twitch, VM design and the phased plan for a 24/7 stream.
• Artifact builder: how to regenerate and verify data/fafb-v783.
• Data attribution: source, license and citations.

Provenance

The network, plasticity rule, readout, FlyWire pipeline and activity viewer were extracted from the fly-plays-pokemon prototype so several game demos can share one core. The default configuration reproduces that prototype's kernel bit for bit, and verbatim copies of its modules live in packages/brain/tests/legacy/ as oracles. Built with Astra.

Licensing

The data/fafb-v783 artifacts are derived from the FlyWire FAFB public Codex v783 exports and are licensed CC BY-NC 4.0. That is a non-commercial license, so a commercial demo needs a different data source or separate permission. Citations and the list of modifications are in data/fafb-v783/ATTRIBUTION.md.

No license has been chosen for the code in this repository yet.

ROMs and save states never enter this repository.