So, a fly neural map just dropped, and the internet has been getting up to some delightful, borderline horrifying shenanigans. A team led by HHMI Janelia, working with Google Research and collaborators, mapped a male fruit fly’s brain and nerve cord. Thin slices, electron microscope images, a whole lotta analysis, and the dataset is now on the internet.
I was introduced to this madness with Infinite Sugar, where William Kelly (cnqso) uses the earlier FlyWire connectome, published in 2024, to simulate a fly’s brain. Some neural activity drives movement in a simulated body, and a button “feeds” the fly infinite sugar by triggering sweet-sensing neurons. Most motor output is disconnected, and the fly doesn’t get visual input from its terrarium.
I was on the road when I read about this and began doom-coding (prompting an LLM on my phone instead of binging social media) a prototype. It started with questions about the dataset Infinite Sugar used, and the conversation meandered into project territory.
I wanted to do something with this. People were already hooking these simulations up to games, including Alex Wormuth’s Doom project, which maps game frames to sensory input and neural activity to controls. There’s also a Minecraft fly mod that uses the newer MaleCNS connectome, with simplified integrate-and-fire neurons driving a fly mob.
The first thing that came to mind was watching SethBling’s MarI/O, which evolves neural networks to play Super Mario World. Then I remembered the various Twitch Plays Pokémon streams, specifically Fish Plays Pokémon.
So, fly plays Pokémon. But how? What would this look like?
A Twitch stream, obviously, an emulator, and mapping the fly to the game. Initially, the plan was to map motor neurons to buttons, the screen to visual neurons, and set up some kind of reward-based feedback mechanism. Additionally, some tracking mechanism to prevent regressions and take periodic snapshots of the game.
The tracking/checkpoint code analyzes the save file to determine how much progress was made and gives the fly sugar at milestones.
Additionally, I set up a macro system to connect some neurons to pre-canned actions that are state-dependent, again by watching emulator memory. So, for example, in battle there are neurons hooked up to things like “use an item,” “make an attack,” or “throw a Poké Ball.” I know this is a little against the “fish plays” vibe I was going for, but I think it turns the whole thing into a low-key idle RPG by enabling more focused actions.
I’m still working on it. In my haste, I built it around the wrong connectome: the 2024 female FlyWire brain used by Infinite Sugar, rather than the newer male brain and nerve cord map. I don’t expect swapping it to be too hard, but I haven’t done it yet.
I’ll do a more detailed write-up and a project page at a later point, but if you want to check it out, the stream can be found here.