Uploaded August 2026 | Updated November 2026, 3 days ago
Qwen 3.8 Flash Next is a 180B model and I ran the whole thing on CPU only, then measured exactly how much speed the RTX 5090 buys back.
This is a full benchmark of the Qwen 4 preview weights on a normal desktop. I run the 1-bit and 2-bit builds on an Intel Core Ultra 9 285K with 96GB of DDR5 and the GPU switched off completely, which gets 10 to 11 tokens per second. Then I bring the RTX 5090 back in, offload the attention blocks first and the expert layers after, and finish at 51.97 tokens per second. Along the way you will see where all 72.5GB of memory actually goes, why adding CPU cores stops helping past 8 threads, why the 2-bit build falls off a cliff on Windows, and whether a 1-bit 180B model still answers better than a 4-bit 27B. You do not need this exact machine to follow it. The reasons the numbers behave this way are the same on any hardware.
๐บ Full playlist: youtube.com/playlist?list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD
โฑ Chapters:
0:00 Intro: Qwen 3.8 Flash Next on CPU only
0:23 180B total, 6B active: what those numbers mean
1:53 Speed preview: 10 tok/s on CPU, 52 tok/s with offload
3:40 My test rig: Core Ultra 9 285K, 96GB DDR5, RTX 5090
5:11 Why I benchmark on this hardware
7:05 Building llama.cpp for CPU only (avoiding the CUDA fallback)
7:44 Why 4-bit will not run: the 111GB wall
8:52 Where 72.5GB goes at 1-bit
12:05 The single tensor eating 40% of the memory
13:50 CPU-only results on 96GB of DDR5
14:37 Do more CPU cores help? Thread scaling test
16:22 Prompt processing vs token generation
17:28 Why generation is capped by RAM bandwidth
18:35 Adding the RTX 5090: what VRAM changes
20:16 Inside the model: 48 blocks, 512 experts per layer
22:47 Offload attention only: 3x speed jump
23:29 Layer by layer up to 51.97 tokens per second
24:17 Why 2-bit collapses at 32 layers on Windows
25:39 Speed drop as the context fills up
27:23 Perplexity: 1-bit 180B vs 4-bit 27B
29:03 Hallucination test: who invented the transistor
31:14 Recap and where to get the full write-up
๐ Resources:
Full written benchmark with every table: kgptalkie.com/tutorials/llm-benchmarking/qwen-3-8-flash-next-on-cpu-only
Unsloth guide and the 1-bit / 2-bit quants I used: unsloth.ai/docs/models/qwen3.8-next
llama.cpp (build it without CUDA for a true CPU-only run): github.com/ggml-org/llama.cpp
Test machine: Intel Core Ultra 9 285K (8 P-cores + 16 E-cores, 24 threads), 96GB DDR5, NVIDIA RTX 5090 32GB, Windows 11 Pro
๐บ Watch next:
Qwen 3.8 Flash Next (Qwen 4) vs 27B: Qwen 4 Architecture Teardown: youtube.com/watch?v=VvV3QRHfzYU&list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD
๐ Go deeper, my Udemy course:
Master Langchain v1 and Ollama - Chatbot, RAG and AI Agents: kgptalkie.com/langchain
If this saved you a weekend of trial and error, hit like, and tell me in the comments how much RAM you have and which quantization you are running so I know what to benchmark next. Subscribe and turn on the bell for the rest of this local LLM series.
#Qwen4 #LocalLLM #llamacpp #Qwen38FlashNext
Qwen 3.8 Flash Next is a 180B model and I ran the whole thing on CPU only, then measured exactly how much speed the RTX 5090 buys back.
This is a full benchmark of the Qwen 4 preview weights on a normal desktop. I run the 1-bit and 2-bit builds on an Intel Core Ultra 9 285K with 96GB of DDR5 and the GPU switched off completely, which gets 10 to 11 tokens per second. Then I bring the RTX 5090 back in, offload the attention blocks first and the expert layers after, and finish at 51.97 tokens per second. Along the way you will see where all 72.5GB of memory actually goes, why adding CPU cores stops helping past 8 threads, why the 2-bit build falls off a cliff on Windows, and whether a 1-bit 180B model still answers better than a 4-bit 27B. You do not need this exact machine to follow it. The reasons the numbers behave this way are the same on any hardware.
๐บ Full playlist: youtube.com/playlist?list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD
โฑ Chapters:
0:00 Intro: Qwen 3.8 Flash Next on CPU only
0:23 180B total, 6B active: what those numbers mean
1:53 Speed preview: 10 tok/s on CPU, 52 tok/s with offload
3:40 My test rig: Core Ultra 9 285K, 96GB DDR5, RTX 5090
5:11 Why I benchmark on this hardware
7:05 Building llama.cpp for CPU only (avoiding the CUDA fallback)
7:44 Why 4-bit will not run: the 111GB wall
8:52 Where 72.5GB goes at 1-bit
12:05 The single tensor eating 40% of the memory
13:50 CPU-only results on 96GB of DDR5
14:37 Do more CPU cores help? Thread scaling test
16:22 Prompt processing vs token generation
17:28 Why generation is capped by RAM bandwidth
18:35 Adding the RTX 5090: what VRAM changes
20:16 Inside the model: 48 blocks, 512 experts per layer
22:47 Offload attention only: 3x speed jump
23:29 Layer by layer up to 51.97 tokens per second
24:17 Why 2-bit collapses at 32 layers on Windows
25:39 Speed drop as the context fills up
27:23 Perplexity: 1-bit 180B vs 4-bit 27B
29:03 Hallucination test: who invented the transistor
31:14 Recap and where to get the full write-up
๐ Resources:
Full written benchmark with every table: kgptalkie.com/tutorials/llm-benchmarking/qwen-3-8-flash-next-on-cpu-only
Unsloth guide and the 1-bit / 2-bit quants I used: unsloth.ai/docs/models/qwen3.8-next
llama.cpp (build it without CUDA for a true CPU-only run): github.com/ggml-org/llama.cpp
Test machine: Intel Core Ultra 9 285K (8 P-cores + 16 E-cores, 24 threads), 96GB DDR5, NVIDIA RTX 5090 32GB, Windows 11 Pro
๐บ Watch next:
Qwen 3.8 Flash Next (Qwen 4) vs 27B: Qwen 4 Architecture Teardown: youtube.com/watch?v=VvV3QRHfzYU&list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD
๐ Go deeper, my Udemy course:
Master Langchain v1 and Ollama - Chatbot, RAG and AI Agents: kgptalkie.com/langchain
If this saved you a weekend of trial and error, hit like, and tell me in the comments how much RAM you have and which quantization you are running so I know what to benchmark next. Subscribe and turn on the bell for the rest of this local LLM series.
#Qwen4 #LocalLLM #llamacpp #Qwen38FlashNext










