LIVE

Developer Recreates DeepSeek-V3 Architecture in Under 200 Lines of Pure JAX

🤖 AI Models 1 min read 👁 17

A developer has implemented the DeepSeek-V3 technical architecture using roughly 190 lines of pure JAX code, allowing for pretraining from scratch on consumer hardware.

Developer Recreates DeepSeek-V3 Architecture in Under 200 Lines of Pure JAX🤖AI Models

Photo: Hacker News Front Page

Key points

  • DeepSeek-V3 architecture replicated in roughly 190 lines of pure JAX.
  • Trained locally on an Apple Silicon MacBook Air using public-domain detective fiction.
  • Includes configurable options for testing expert balancing and lower-memory devices.

A new open-source project has successfully condensed the architecture outlined in the DeepSeek-V3 technical report into approximately 190 non-blank, non-comment lines of pure JAX. Eschewing heavy frameworks like PyTorch, Flax, or Equinox, the codebase uses plain dictionaries of arrays to make every component readable from top to bottom.

The demonstration was pretrained from scratch on an Apple Silicon MacBook Air utilizing 24 GB of memory. The training corpus comprised public-domain detective and crime fiction—including works by Dashiell Hammett sourced from Standard Ebooks alongside roughly 200 novels from Project Gutenberg—amounting to about 120 MB of text.

According to the release details, a default configuration test evaluated the impact of expert balancing, yielding matching validation losses at step 2,000 with and without the mechanism. The repository also includes a smaller configuration designed to accommodate Macs with 8 to 16 GB of memory, though it is intended as an architectural demonstration rather than a fully capable language model.

What to watch next: Further community experiments leveraging minimal framework implementations for complex AI architectures on local consumer hardware.

Source Hacker News Front Page — github.com/Vaibhaav-Tiwari/deepseek-v3-jax ↗

Contact the newsroom

Got a tip, a proposal or a partnership question? Write to us and we will reply shortly.