Uploaded June 2025 | Updated September 2026, 13 hours ago
This is the third video about my AI inference machine, Larry. I recently installed a Blackwell 6000 Pro 96gb. You should watch to see how well it runs popular LLMs like DeepSeek-V3-0324 Q8_0 and Q4_K_XL, as well as DeepSeek-R1-0528 Q4_K_M, and Qwen3-235B-A22B-128K. I also throw in a crash test of ktransformers and talk about why I've recently stopped using ktransformers in favor of llama.cpp. Finally, I run a memory bandwidth test of my system using Stream Triad to help understand performance bottlenecks.
0:00 Intro and hardware
2:11 llama.cpp CLI options
4:26 NPS0 vs NPS4 and numactl
6:12 llama.cpp CLI options cont.
8:09 R1-0528 Q4_K_M
11:04 V3-0324 Q8_0
14:08 V3-0324 Q4_K_XL
17:46 ktransformers V3-0324 Q4_K_XL
21:54 ktransformers forced crash
24:44 Stream Triad memory performance test
26:37 Qwen3
30:47 Outro
Previous videos:
ktransformers and 3090 GPU youtu.be/fI6uGPcxDbM
PC Build and CPU Only youtu.be/v4810MVGhog
Github issues:
github.com/kvcache-ai/ktransformers/issues/1389
github.com/ggml-org/llama.cpp/issues/13160
Amazon Affiliate Links:
NVD RTX PRO 6000 Blackwell Professional Workstation Edition Graphics Card for AI, Design, Simulation, Engineering - 96GB DDR7 ECC Memory - 4th Gen RT/5th Gen Tensor Core GPU - OEM Packaging
[amazon affiliate us]: amzn.to/4kNYXjU
Gigabyte MZ73-LM0 Motherboard Socket SP5 Extended ATX
[amazon affiliate us]: amzn.to/43YROaH
Corsair HX1500i Fully Modular Ultra-Low Noise ATX Power Supply - ATX 3.0 & PCIe 5.0 Compliant - Fluid Dynamic Bearing Fan - CORSAIR iCUE Software Compatible - 80 Plus Platinum Efficiency - Black
[amazon affiliate us]: amzn.to/42lzPcr
NEMIX RAM 384GB (12X32GB) DDR5 5600MHZ PC5-44800 1Rx4 1.1V CL46 288-PIN ECC RDIMM Registered Server Memory KIT
[amazon affiliate us]: amzn.to/42nzXJn
SAMSUNG 990 PRO Heatsink SSD 4TB, NVMe M.2, Speeds Up to 7,450MB/s Best for PlayStation® 5 (PS5 SSD) Console Expansion MZ-V9P4T0CW
[amazon affiliate us]: amzn.to/4lc60Uc
I have six of these on the case:
ARCTIC P14 PWM PST (5 Pack) - 140 mm Case Fan with PWM Sharing Technology (PST), Pressure-optimised, Computer, Fan Speed: 200-1700 RPM - Black
[amazon affiliate us]: amzn.to/3T6bVgQ
This is the one small fan at the back of the case:
ARCTIC F12 PWM PST (5 Pack) - 120 mm PWM PST Case Fan with PWM Sharing Technology (PST), Quiet Motor, Computer, Fan Speed: 230-1350 RPM - Black
[amazon affiliate us]: amzn.to/4jgJxDL
This is the third video about my AI inference machine, Larry. I recently installed a Blackwell 6000 Pro 96gb. You should watch to see how well it runs popular LLMs like DeepSeek-V3-0324 Q8_0 and Q4_K_XL, as well as DeepSeek-R1-0528 Q4_K_M, and Qwen3-235B-A22B-128K. I also throw in a crash test of ktransformers and talk about why I've recently stopped using ktransformers in favor of llama.cpp. Finally, I run a memory bandwidth test of my system using Stream Triad to help understand performance bottlenecks.
0:00 Intro and hardware
2:11 llama.cpp CLI options
4:26 NPS0 vs NPS4 and numactl
6:12 llama.cpp CLI options cont.
8:09 R1-0528 Q4_K_M
11:04 V3-0324 Q8_0
14:08 V3-0324 Q4_K_XL
17:46 ktransformers V3-0324 Q4_K_XL
21:54 ktransformers forced crash
24:44 Stream Triad memory performance test
26:37 Qwen3
30:47 Outro
Previous videos:
ktransformers and 3090 GPU youtu.be/fI6uGPcxDbM
PC Build and CPU Only youtu.be/v4810MVGhog
Github issues:
github.com/kvcache-ai/ktransformers/issues/1389
github.com/ggml-org/llama.cpp/issues/13160
Amazon Affiliate Links:
NVD RTX PRO 6000 Blackwell Professional Workstation Edition Graphics Card for AI, Design, Simulation, Engineering - 96GB DDR7 ECC Memory - 4th Gen RT/5th Gen Tensor Core GPU - OEM Packaging
[amazon affiliate us]: amzn.to/4kNYXjU
Gigabyte MZ73-LM0 Motherboard Socket SP5 Extended ATX
[amazon affiliate us]: amzn.to/43YROaH
Corsair HX1500i Fully Modular Ultra-Low Noise ATX Power Supply - ATX 3.0 & PCIe 5.0 Compliant - Fluid Dynamic Bearing Fan - CORSAIR iCUE Software Compatible - 80 Plus Platinum Efficiency - Black
[amazon affiliate us]: amzn.to/42lzPcr
NEMIX RAM 384GB (12X32GB) DDR5 5600MHZ PC5-44800 1Rx4 1.1V CL46 288-PIN ECC RDIMM Registered Server Memory KIT
[amazon affiliate us]: amzn.to/42nzXJn
SAMSUNG 990 PRO Heatsink SSD 4TB, NVMe M.2, Speeds Up to 7,450MB/s Best for PlayStation® 5 (PS5 SSD) Console Expansion MZ-V9P4T0CW
[amazon affiliate us]: amzn.to/4lc60Uc
I have six of these on the case:
ARCTIC P14 PWM PST (5 Pack) - 140 mm Case Fan with PWM Sharing Technology (PST), Pressure-optimised, Computer, Fan Speed: 200-1700 RPM - Black
[amazon affiliate us]: amzn.to/3T6bVgQ
This is the one small fan at the back of the case:
ARCTIC F12 PWM PST (5 Pack) - 120 mm PWM PST Case Fan with PWM Sharing Technology (PST), Quiet Motor, Computer, Fan Speed: 230-1350 RPM - Black
[amazon affiliate us]: amzn.to/4jgJxDL
