57d9bdff5a
THROW BALL was 63 starts and 63 blocked on v0.4.3 and SWITCH 15 of them, and the reason is neither macro's own script. The screen reads FIGHT PKMN over ITEM RUN and the game's index does not: wCurrentMenuItem is the row inside the column the cursor is in, and selection adds two for the right column, so the order is FIGHT, ITEM, PKMN, RUN. battle_entry had PKMN 1 and ITEM 2 -- the row-major reading of the picture -- so every macro that meant to open the bag opened the party list and every macro that meant to open the party list opened the bag. Surveyed on the cartridge from the rung-9 checkpoint: a THROW BALL aiming at 2 walked the cursor to wTopMenuItemX 15 / wCurrentMenuItem 0, pressed A, and the party list opened -- wTopMenuItemY 1, wTopMenuItemX 0, wListMenuID $02 -- with the game writing wCurrentMenuItem 2 on the frame after the press. The fake had the same mistake in its own geometry, so it could not have caught it: its two-by-two moved row-major. It is column-major now, like the cartridge. |
||
|---|---|---|
| .forgejo/workflows | ||
| .github/workflows | ||
| apps/stage | ||
| data/fafb-v783 | ||
| docs | ||
| infra | ||
| packages | ||
| services | ||
| tools | ||
| .gitignore | ||
| CLAUDE.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| LICENSES.md | ||
| Makefile | ||
| NOTICE | ||
| package-lock.json | ||
| package.json | ||
| README.md | ||
| ROM-POLICY.md | ||
flybrain
A simulated fruit-fly brain that plays video games. A connectome-constrained spiking network reads the screen, its population rates become controller inputs, and a scalar reward nudges a bounded set of Kenyon-cell to MBON gains.
The library is @flybrain/brain in packages/brain. It holds the connectome dataset format, the
LIF kernel, the plasticity rule, the population-rate readout and the activity-map geometry. It
holds no game, no emulator and no reward rules: those belong to whatever embeds it.
Workspace layout
| Path | Contents |
|---|---|
packages/brain |
the library (@flybrain/brain) |
data/fafb-v783 |
FlyWire-derived browser artifacts (CC BY-NC 4.0) |
tools/ |
the Python builder that regenerates data/ from official Codex exports |
docs/ |
overview, dataset format, model, plasticity, readout, integration, limitations, verification |
services/flysim |
the Rust service: brain, emulator, snapshot feed, control API, checkpoints |
apps/ |
planned: one directory per game demo |
infra/ |
planned: deployment for the 24/7 stream (see docs/streaming-plan.md) |
Quick start
npm ci
npm test
npm run typecheck
76 tests, about 7 seconds. There is no build step. npx tsx packages/brain/examples/node-random-frames.ts 60
runs the full 139,255-neuron brain with the Game Boy readout on noise frames in plain Node
(about 0.9x Game Boy real time single-threaded on a WSL laptop).
Usage
import { NeuralAgent, gameboyDecoderConfig, toButtonMask } from '@flybrain/brain';
import { loadBrainDatasetFromDir } from '@flybrain/brain/node';
const dataset = await loadBrainDatasetFromDir('data/fafb-v783');
const agent = new NeuralAgent(dataset, { decoder: gameboyDecoderConfig() });
agent.warmup(firstFrame); // 2,500 ms with plasticity off, then calibrate
// every emulator frame:
const { active } = agent.tick(framebuffer, { // RGBA 160x144 by default; any size via config
rewards: [{ value: 0.5 }], // scalar rewards your game adapter detected
boot: !inGame, // relaxes Start/Select throttling on title screens
});
emulator.setButtons(toButtonMask(active));
const checkpoint = agent.exportState(); // bit-exact resume, validated on import
The lower layers (LifNetwork, RewardModulatedStdp, PopulationDecoder) are exported too for
hosts that want to run the loop themselves.
integration.md has the full per-frame loop, the fractional frame timing and the checkpoint contract.
Documentation
- Overview: the pipeline, the layer map and the design principles.
- Dataset format: artifacts, CSR layout, weight encoding, every role and its count, fingerprinting, regeneration, license.
- Model: the 1-ms LIF kernel step by step, every default constant, the retina projection, the RNG, state export and version strings.
- Plasticity: edge selection, the eligibility and reinforcement equations, statistics, topology hash and the explicit non-claims.
- Readout: scores, exclusive groups, pulse channels, the blocked-direction cooldown, the Game Boy preset table and checkpoint versions.
- Integration: what a game must provide, the Pokemon Red integration as a worked example, and a sketch of a platformer adapter.
- Limitations: what is not claimed, what is unproven, measured throughput.
- Verification: the oracle-test strategy and what each test file covers.
- Streaming plan: headless capture, Twitch, VM design and the phased plan for a 24/7 stream.
- Artifact builder: how to regenerate and verify
data/fafb-v783. - Data attribution: source, license and citations.
Provenance
The network, plasticity rule, readout, FlyWire pipeline and activity viewer were extracted from
the fly-plays-pokemon prototype so several game demos can share one core. The default
configuration reproduces that prototype's kernel bit for bit, and verbatim copies of its modules
live in packages/brain/tests/legacy/ as oracles. Built with Astra.
Licensing
The data/fafb-v783 artifacts are derived from the FlyWire FAFB public Codex v783 exports and are
licensed CC BY-NC 4.0. That is a
non-commercial license, so a commercial demo needs a different data source or separate permission.
Citations and the list of modifications are in
data/fafb-v783/ATTRIBUTION.md.
No license has been chosen for the code in this repository yet.
ROMs and save states never enter this repository.