I Ran Qwen 3.8 Flash Next (Qwen 4) on CPU Only @KGPTalkie
I Ran Qwen 3.8 Flash Next (Qwen 4) on CPU Only  @KGPTalkie
Uploaded August 2026 | Updated November 2026, 3 days ago
Qwen 3.8 Flash Next is a 180B model and I ran the whole thing on CPU only, then measured exactly how much speed the RTX 5090 buys back.

This is a full benchmark of the Qwen 4 preview weights on a normal desktop. I run the 1-bit and 2-bit builds on an Intel Core Ultra 9 285K with 96GB of DDR5 and the GPU switched off completely, which gets 10 to 11 tokens per second. Then I bring the RTX 5090 back in, offload the attention blocks first and the expert layers after, and finish at 51.97 tokens per second. Along the way you will see where all 72.5GB of memory actually goes, why adding CPU cores stops helping past 8 threads, why the 2-bit build falls off a cliff on Windows, and whether a 1-bit 180B model still answers better than a 4-bit 27B. You do not need this exact machine to follow it. The reasons the numbers behave this way are the same on any hardware.

๐Ÿ“บ Full playlist: youtube.com/playlist?list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD

โฑ Chapters:
0:00 Intro: Qwen 3.8 Flash Next on CPU only
0:23 180B total, 6B active: what those numbers mean
1:53 Speed preview: 10 tok/s on CPU, 52 tok/s with offload
3:40 My test rig: Core Ultra 9 285K, 96GB DDR5, RTX 5090
5:11 Why I benchmark on this hardware
7:05 Building llama.cpp for CPU only (avoiding the CUDA fallback)
7:44 Why 4-bit will not run: the 111GB wall
8:52 Where 72.5GB goes at 1-bit
12:05 The single tensor eating 40% of the memory
13:50 CPU-only results on 96GB of DDR5
14:37 Do more CPU cores help? Thread scaling test
16:22 Prompt processing vs token generation
17:28 Why generation is capped by RAM bandwidth
18:35 Adding the RTX 5090: what VRAM changes
20:16 Inside the model: 48 blocks, 512 experts per layer
22:47 Offload attention only: 3x speed jump
23:29 Layer by layer up to 51.97 tokens per second
24:17 Why 2-bit collapses at 32 layers on Windows
25:39 Speed drop as the context fills up
27:23 Perplexity: 1-bit 180B vs 4-bit 27B
29:03 Hallucination test: who invented the transistor
31:14 Recap and where to get the full write-up

๐Ÿ”— Resources:
Full written benchmark with every table: kgptalkie.com/tutorials/llm-benchmarking/qwen-3-8-flash-next-on-cpu-only
Unsloth guide and the 1-bit / 2-bit quants I used: unsloth.ai/docs/models/qwen3.8-next
llama.cpp (build it without CUDA for a true CPU-only run): github.com/ggml-org/llama.cpp
Test machine: Intel Core Ultra 9 285K (8 P-cores + 16 E-cores, 24 threads), 96GB DDR5, NVIDIA RTX 5090 32GB, Windows 11 Pro

๐Ÿ“บ Watch next:
Qwen 3.8 Flash Next (Qwen 4) vs 27B: Qwen 4 Architecture Teardown: youtube.com/watch?v=VvV3QRHfzYU&list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD

๐ŸŽ“ Go deeper, my Udemy course:
Master Langchain v1 and Ollama - Chatbot, RAG and AI Agents: kgptalkie.com/langchain

If this saved you a weekend of trial and error, hit like, and tell me in the comments how much RAM you have and which quantization you are running so I know what to benchmark next. Subscribe and turn on the bell for the rest of this local LLM series.

#Qwen4 #LocalLLM #llamacpp #Qwen38FlashNext
I Ran Qwen 3.8 Flash Next (Qwen 4) on CPU OnlyInstagram Feed System Design: Interview Question ExplainedProduction RAG for Finance with Qwen 3.8 27B: Fully Local Ollama + Qdrant (Live)AI Chatbot Hallucination: The Bot Promised the Wrong Refund | AI System Design InterviewSentencePiece Tokenizer Explained | Tokenization Without SpacesProduction RAG for Finance with NVIDIA Nemotron 3.5 Lightning: Fully Local Ollama + Qdrant (Live)Ryzen AI Halo vs DGX Spark: $700 Apart, Almost the Same Speed #shortsFine-Tune Gemma 3 270M on MacBook with Unsloth Studio (No Code)Hindi vs English Tokens: Why Hindi Needs 3x MoreMCP Tutorial 3 -  Create PPT Presentation Using PowerPoint MCP ServerWhy Transformers Replaced RNNs: 499 Steps vs Just One | LLM InterviewRun a Coding AI on Your Own Laptop (8B, No GPU) #shorts
Laxmi Kant | KGP Talkie |

I Ran Qwen 3.8 Flash Next (Qwen 4) on CPU Only

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER