84ed0410bf
BACK was 678 macro starts in 47 minutes on rung 10, 189 of them on Pewter City with no text box on screen at all. BACK is on no overworld and no dialog pad, so every one of them was dealt by Scene::Unknown -- which holds two states under one name. One is a screen this crate cannot name, the Pokedex or the trainer card or OPTION, where A and B are what leave it. The other is a frame of the overworld where the cartridge is driving -- a warp in flight, a scripted push-back, the museum guide walking the fly through the door -- which the detector calls Unknown because the buttons are not reaching the player. On the second, NEXT and BACK are an A and a B pressed into somebody else's script: they change nothing and they complete where the fly stands, which is section 12.2's trap with no box to advance. So Unknown's pad is dealt on whether a box is drawn, and a scripted overworld frame is an empty pad the fly waits out -- the cartridge gives the buttons back by itself. |
||
|---|---|---|
| .forgejo/workflows | ||
| .github/workflows | ||
| apps/stage | ||
| data/fafb-v783 | ||
| docs | ||
| infra | ||
| packages | ||
| services | ||
| tools | ||
| .gitignore | ||
| CLAUDE.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| LICENSES.md | ||
| Makefile | ||
| NOTICE | ||
| package-lock.json | ||
| package.json | ||
| README.md | ||
| ROM-POLICY.md | ||
flybrain
A simulated fruit-fly brain that plays video games. A connectome-constrained spiking network reads the screen, its population rates become controller inputs, and a scalar reward nudges a bounded set of Kenyon-cell to MBON gains.
The library is @flybrain/brain in packages/brain. It holds the connectome dataset format, the
LIF kernel, the plasticity rule, the population-rate readout and the activity-map geometry. It
holds no game, no emulator and no reward rules: those belong to whatever embeds it.
Workspace layout
| Path | Contents |
|---|---|
packages/brain |
the library (@flybrain/brain) |
data/fafb-v783 |
FlyWire-derived browser artifacts (CC BY-NC 4.0) |
tools/ |
the Python builder that regenerates data/ from official Codex exports |
docs/ |
overview, dataset format, model, plasticity, readout, integration, limitations, verification |
services/flysim |
the Rust service: brain, emulator, snapshot feed, control API, checkpoints |
apps/ |
planned: one directory per game demo |
infra/ |
planned: deployment for the 24/7 stream (see docs/streaming-plan.md) |
Quick start
npm ci
npm test
npm run typecheck
76 tests, about 7 seconds. There is no build step. npx tsx packages/brain/examples/node-random-frames.ts 60
runs the full 139,255-neuron brain with the Game Boy readout on noise frames in plain Node
(about 0.9x Game Boy real time single-threaded on a WSL laptop).
Usage
import { NeuralAgent, gameboyDecoderConfig, toButtonMask } from '@flybrain/brain';
import { loadBrainDatasetFromDir } from '@flybrain/brain/node';
const dataset = await loadBrainDatasetFromDir('data/fafb-v783');
const agent = new NeuralAgent(dataset, { decoder: gameboyDecoderConfig() });
agent.warmup(firstFrame); // 2,500 ms with plasticity off, then calibrate
// every emulator frame:
const { active } = agent.tick(framebuffer, { // RGBA 160x144 by default; any size via config
rewards: [{ value: 0.5 }], // scalar rewards your game adapter detected
boot: !inGame, // relaxes Start/Select throttling on title screens
});
emulator.setButtons(toButtonMask(active));
const checkpoint = agent.exportState(); // bit-exact resume, validated on import
The lower layers (LifNetwork, RewardModulatedStdp, PopulationDecoder) are exported too for
hosts that want to run the loop themselves.
integration.md has the full per-frame loop, the fractional frame timing and the checkpoint contract.
Documentation
- Overview: the pipeline, the layer map and the design principles.
- Dataset format: artifacts, CSR layout, weight encoding, every role and its count, fingerprinting, regeneration, license.
- Model: the 1-ms LIF kernel step by step, every default constant, the retina projection, the RNG, state export and version strings.
- Plasticity: edge selection, the eligibility and reinforcement equations, statistics, topology hash and the explicit non-claims.
- Readout: scores, exclusive groups, pulse channels, the blocked-direction cooldown, the Game Boy preset table and checkpoint versions.
- Integration: what a game must provide, the Pokemon Red integration as a worked example, and a sketch of a platformer adapter.
- Limitations: what is not claimed, what is unproven, measured throughput.
- Verification: the oracle-test strategy and what each test file covers.
- Streaming plan: headless capture, Twitch, VM design and the phased plan for a 24/7 stream.
- Artifact builder: how to regenerate and verify
data/fafb-v783. - Data attribution: source, license and citations.
Provenance
The network, plasticity rule, readout, FlyWire pipeline and activity viewer were extracted from
the fly-plays-pokemon prototype so several game demos can share one core. The default
configuration reproduces that prototype's kernel bit for bit, and verbatim copies of its modules
live in packages/brain/tests/legacy/ as oracles. Built with Astra.
Licensing
The data/fafb-v783 artifacts are derived from the FlyWire FAFB public Codex v783 exports and are
licensed CC BY-NC 4.0. That is a
non-commercial license, so a commercial demo needs a different data source or separate permission.
Citations and the list of modifications are in
data/fafb-v783/ATTRIBUTION.md.
No license has been chosen for the code in this repository yet.
ROMs and save states never enter this repository.