All hacks

Raspberry Pi Pico 2 W running a tiny world model that dreams up games

A Pico 2 W generates Breakout, Pong and Flappy Bird frame by frame with a small transformer, while a second tiny network plays them.

Raspberry Pi Pico 2 W running a tiny world model that dreams up games
Raspberry Pi Pico 2 W running a tiny world model that dreams up games, as posted by SheepSeb (Sebastian-Ion Nae)
Photo: SheepSeb (Sebastian-Ion Nae), from the original post.

A Raspberry Pi Pico 2 W now makes up its own video games and plays them. There’s no real game running on the chip. A small looped transformer, int8-quantised at 26k to 131k parameters, takes the current frame and an action and predicts the next frame and the reward. Breakout, Pong, Flappy Bird and a Doom-like game all work this way.

A second network, a small MLP with 9k to 16k parameters, picks the actions. It learns one transition at a time with streaming reinforcement learning and no replay buffer. Both run on the RP2350’s second core, and an optional 320x240 ST7789 screen shows the result.

The firmware for each game fits in 361KB to 471KB of flash. The repo also has a browser version and the training commands, if you’d rather start without hardware. Code and write-up are on SheepSeb’s GitHub.

All credit to SheepSeb (Sebastian-Ion Nae). The full story is at the original source.

Cracked something open?

Send us your hack. Clever, useful or gloriously pointless, we want to see it.

Submit a hack