Uploaded March 2025 | Updated September 2026, 1 hour ago
Watch as I build a monster PC to run Deepseek-V3-0324:671b-Q8 locally at 6-8 tokens per second. I'm using dual EPYC 9355 processors and 768Gb of 5600mhz RDIMMs 24x32Gb on a MZ73-LM0 Gigabyte motherboard. I flash the BIOS, install Ubuntu 24.04.2 LTS, ollama, Open WebUI, and more, step by step!
14 tok/s follow up video: youtu.be/fI6uGPcxDbM
0:00 Intro
3:19 PC Build
14:00 Successful BMC Firmware Update
16:43 Failed BIOS Firmware Update
17:04 Successful BIOS Firmware Update
18:07 Explain BIOS Firmware Update Procedure
19:09 Successful Ubuntu Install
28:43 Deepseek V3 Demo and Setup Notes
34:54 5600Mhz RDIMMs and NUMO 0
40:52 Context Lengths and Open WebUI
49:29 Power Usage
50:45 Context Length vs Token / Second Vibe Coding or Agentic
Amazon Affiliate Links
NEMIX RAM 384GB (12X32GB) DDR5 5600MHZ PC5-44800 1Rx4 1.1V CL46 288-PIN ECC RDIMM Registered Server Memory KIT
[amazon affiliate us]: amzn.to/42nzXJn
Corsair HX1000i Fully Modular Ultra-Low Noise ATX Power Supply - ATX 3.0 & PCIe 5.0 Compliant - Fluid Dynamic Bearing Fan - CORSAIR iCUE Software Compatible - 80 Plus Platinum Efficiency - Black
[amazon affiliate us]: amzn.to/4hNuPTn
SAMSUNG 990 PRO Heatsink SSD 4TB, NVMe M.2, Speeds Up to 7,450MB/s Best for PlayStation® 5 (PS5 SSD) Console Expansion MZ-V9P4T0CW
[amazon affiliate us]: amzn.to/4lc60Uc
ARCTIC F12 PWM PST (5 Pack) - 120 mm PWM PST Case Fan with PWM Sharing Technology (PST), Quiet Motor, Computer, Fan Speed: 230-1350 RPM - Black
[amazon affiliate us]: amzn.to/4jgJxDL
Thanks to Matthew Carrigan's X post for the idea: https://x.com/carrigmat/status/1884244369907278106
Special thanks to /dev/full, microsuxx, pat.3:33b-q3_K_L, BaRAG O'Llama and everyone else on the Ollama Discord for help getting this up and running.
Watch as I build a monster PC to run Deepseek-V3-0324:671b-Q8 locally at 6-8 tokens per second. I'm using dual EPYC 9355 processors and 768Gb of 5600mhz RDIMMs 24x32Gb on a MZ73-LM0 Gigabyte motherboard. I flash the BIOS, install Ubuntu 24.04.2 LTS, ollama, Open WebUI, and more, step by step!
14 tok/s follow up video: youtu.be/fI6uGPcxDbM
0:00 Intro
3:19 PC Build
14:00 Successful BMC Firmware Update
16:43 Failed BIOS Firmware Update
17:04 Successful BIOS Firmware Update
18:07 Explain BIOS Firmware Update Procedure
19:09 Successful Ubuntu Install
28:43 Deepseek V3 Demo and Setup Notes
34:54 5600Mhz RDIMMs and NUMO 0
40:52 Context Lengths and Open WebUI
49:29 Power Usage
50:45 Context Length vs Token / Second Vibe Coding or Agentic
Amazon Affiliate Links
NEMIX RAM 384GB (12X32GB) DDR5 5600MHZ PC5-44800 1Rx4 1.1V CL46 288-PIN ECC RDIMM Registered Server Memory KIT
[amazon affiliate us]: amzn.to/42nzXJn
Corsair HX1000i Fully Modular Ultra-Low Noise ATX Power Supply - ATX 3.0 & PCIe 5.0 Compliant - Fluid Dynamic Bearing Fan - CORSAIR iCUE Software Compatible - 80 Plus Platinum Efficiency - Black
[amazon affiliate us]: amzn.to/4hNuPTn
SAMSUNG 990 PRO Heatsink SSD 4TB, NVMe M.2, Speeds Up to 7,450MB/s Best for PlayStation® 5 (PS5 SSD) Console Expansion MZ-V9P4T0CW
[amazon affiliate us]: amzn.to/4lc60Uc
ARCTIC F12 PWM PST (5 Pack) - 120 mm PWM PST Case Fan with PWM Sharing Technology (PST), Quiet Motor, Computer, Fan Speed: 230-1350 RPM - Black
[amazon affiliate us]: amzn.to/4jgJxDL
Thanks to Matthew Carrigan's X post for the idea: https://x.com/carrigmat/status/1884244369907278106
Special thanks to /dev/full, microsuxx, pat.3:33b-q3_K_L, BaRAG O'Llama and everyone else on the Ollama Discord for help getting this up and running.
