Skip to main content
Back

slvDev Runs a 28.9M-Parameter AI on an $8 Chip Offline

A developer squeezed a 28.9M parameter language model onto an $8 ESP32-S3 chip by moving most weights into flash memory.

  • A 28.9M parameter language model now runs fully offline on an ESP32-S3 microcontroller.
  • Achieves 9.88 tokens per second with 14.9MB weights at 4-bit precision.
  • Uses Google's Per-Layer Embeddings from Gemma 3n to keep 25M params in flash memory.
  • Only ~450 bytes are read from flash per token, roughly 0.7% of memory traffic.
  • Trained on TinyStories, producing coherent short stories but no reasoning or facts.
  • Open-source under MIT license, weights on Hugging Face.

Fitting a modern language model on a microcontroller has always been a memory problem, not a compute problem. A developer known as slvDev just released esp32-ai, an open-source project that runs a 28.9 million parameter transformer on a standard ESP32-S3 chip, generating coherent text at roughly 9.88 tokens per second without ever touching a network.

The trick avoids brute-force compression. It fits because most of the model lives in flash instead of RAM, using Per-Layer Embeddings, an idea borrowed from Google's Gemma 3n. The repo has already pulled in over 4,000 GitHub stars and reframes what an $8 chip can host locally.

28.9M-parameter LLM running on an ESP32-S3 microcontroller

The memory wall, and the workaround

An ESP32-S3 was never built for AI. It gives you 512KB of SRAM, and only the values touched many times per token can live there: activations and norm weights. The dense core and output head, scanned once per position, sit in PSRAM. Everything else, especially the embedding tables, has historically decided whether a given model could even boot.

slvDev's approach flips the usual assumption. Most parameters here sit in an embedding table, which the model reads from rather than computes on. So the 25-million-parameter table stays in slow flash, and only the few rows each token needs get pulled from it, about 450 bytes. That is small enough that the flash read effectively disappears next to the actual matrix math.

The three-tier layout looks like this:

  • SRAM (fast, tiny): activations and normalization weights, touched many times per token
  • PSRAM (medium): the dense transformer core and the output head, read once per position