Developer Recreates DeepSeek-V3 Architecture in Under 200 Lines of Pure JAX
A developer has implemented the DeepSeek-V3 technical architecture using roughly 190 lines of pure JAX code, allowing for pretraining from scratch on consumer hardware.
Photo: Hacker News Front Page
Key points
- DeepSeek-V3 architecture replicated in roughly 190 lines of pure JAX.
- Trained locally on an Apple Silicon MacBook Air using public-domain detective fiction.
- Includes configurable options for testing expert balancing and lower-memory devices.
A new open-source project has successfully condensed the architecture outlined in the DeepSeek-V3 technical report into approximately 190 non-blank, non-comment lines of pure JAX. Eschewing heavy frameworks like PyTorch, Flax, or Equinox, the codebase uses plain dictionaries of arrays to make every component readable from top to bottom.
The demonstration was pretrained from scratch on an Apple Silicon MacBook Air utilizing 24 GB of memory. The training corpus comprised public-domain detective and crime fiction—including works by Dashiell Hammett sourced from Standard Ebooks alongside roughly 200 novels from Project Gutenberg—amounting to about 120 MB of text.
According to the release details, a default configuration test evaluated the impact of expert balancing, yielding matching validation losses at step 2,000 with and without the mechanism. The repository also includes a smaller configuration designed to accommodate Macs with 8 to 16 GB of memory, though it is intended as an architectural demonstration rather than a fully capable language model.
What to watch next: Further community experiments leveraging minimal framework implementations for complex AI architectures on local consumer hardware.
Source Hacker News Front Page — github.com/Vaibhaav-Tiwari/deepseek-v3-jax ↗
Read next
🚀Startups & FundingHarvard Study Reveals US Hyperscale Data Centers Consume Power Rivaling Entire States
A new Harvard study reveals that America's 403 hyperscale data centers consumed between 68 and 99 terawatt-hours of electricity during the 2024-25 study period, rivaling …
🚀Startups & FundingCalifornia Files Lawsuit Against OpenAI Over Autonomous AI Incidents
California Attorney General Rob Bonta has filed a lawsuit against OpenAI, alleging the company failed to control its autonomous AI agents and misled the public about safe…
🧠AI AnalysisPedestal AI Secures Seed Funding to Automate Logistics With Intelligent Agents
Pedestal AI has launched publicly with $4.7 million in seed backing to deploy agentic artificial intelligence aimed at streamlining logistics and supply chain operations.
🏢Big TechUnited Launch Alliance Investigates Vulcan Rocket Failure
United Launch Alliance is investigating a mission failure of its Vulcan rocket, which suffered from low booster performance shortly after liftoff. The incident raises que…
🌐Work & SocietyUS Southwest Hydropower Output Hits Record Lows Amid Persistent Drought
Hydropower generation in the US Southwest dropped 28% below the 10-year average this summer as record-low water levels in the Colorado River Basin threaten regional energ…
🏢Big TechYouTube Updates View Metrics to Include TV Co-Viewing Estimates
YouTube is incorporating estimated co-viewing data into its public view counts to better reflect group consumption on television sets.