Running a 28.9M parameter LLM on an $8 microcontroller

What changed

Open to Work · 𝕏 slvDev · LinkedIn This is a 28.9 million parameter language model that generates text on an ESP32-S3, a microcontroller that costs about $8. It runs on the chip itself, with nothing sent to a server, and it writes each word to a small screen wired to the chip at roughly 9 tokens per second. The last language model people ran on a chip like this had 260 thousand parameters, so this one holds about a hundred times more.

Why it matters

A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.

How it compares

Related prior coverage to compare against:

• Open-weight AI is having its Kubernetes moment
• Nvidia, Microsoft, Meta warn against overregulating open-weight models
• Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

• Open-weight AI is having its Kubernetes moment similarity 0.48
• Nvidia, Microsoft, Meta warn against overregulating open-weight models similarity 0.48
• Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models similarity 0.18