New Project Runs Large Language Models on Vintage and Low-Power Hardware

A new engineering project shows that everyday electronics, from obsolete smartphones and routers to modern budget TV boxes, possess enough dormant hardware capacity to execute large language models. Dubbed "smolmaxxing," the approach matches devices with the largest feasible model and applies custom execution kernels, efficient memory management, and weight streaming to bypass hardware limitations. All performance claims are validated directly on physical devices rather than simulated environments.

Testing highlights the versatility of the optimization techniques across vastly different generations of hardware. On a third-generation Xiaomi TV Box running Android 14 with roughly one gigabyte of usable RAM, the setup manages models ranging from 260 thousand to 16 billion parameters by working around strict operating system memory caps and storage read speeds. Similarly, a vintage 2013 iPad Air featuring an Apple A7 chip and one gigabyte of RAM manages to run a 1.7-billion parameter model at speeds between 24 and 26 tokens per second, and larger mixture-of-experts models using flash paging.

The methodology relies on targeted interventions to attack specific hardware bottlenecks, utilizing custom base NEON and 1-bit lookup-table kernels. While dense models are constrained by flash read speeds when exceeding system memory, mixture-of-experts architectures help bypass these limits by loading only a fraction of active weights per token. Detailed experimental logs document dozens of incremental iterations aimed at maximizing token generation speeds while preserving English language quality.

Every device around you is more capable than it looks, provided the software is engineered to match the hardware's actual physical limits.

What to watch next: Watch for further refinements in weight streaming and expert paging techniques that could bring advanced local AI capabilities to even more resource-constrained consumer electronics.