Uploaded August 2026 | Updated September 2026, 2 weeks ago
Qwen 3.8 27B on llama.cpp: I tested 45 configurations on a single RTX 5090 to find the settings that actually make it fast, and here is everything that worked.
This is a settings deep dive, not a benchmark montage. I go through speculative decoding (the MTP layer built into Qwen 3.8), how far you can push the draft depth before it backfires, KV cache quantization, flash attention, reasoning effort, and what happens to speed and VRAM as you raise the context window. Every number here came from my own runs on my own prompts, including reverse-order runs so a hot GPU does not fake a result. If you run Qwen 3.8 27B locally on a 24GB or 32GB card, you can copy the final settings straight from this video.
📺 Full playlist: youtube.com/playlist?list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD
⏱ Chapters:
0:00 Intro
1:12 Speculative decoding and the MTP layer inside Qwen 3.8
3:10 How multi-token prediction actually works
6:32 How many tokens should you draft ahead?
12:15 Why n=3 is the sweet spot and n=5 is not
13:41 GPU heat drift and how to benchmark honestly
15:56 Reasoning effort is the biggest speed lever
17:31 KV cache quantization: faster and smaller
19:16 Should you keep flash attention on?
20:50 Raising the context is cheap, filling it is not
22:38 Why time to first token explodes on a full context
24:21 VRAM you need at 32K, 128K and 256K context
26:46 Is Ollama faster than llama.cpp?
28:14 Reasoning effort you can only control on llama.cpp
28:45 What I am not claiming
29:19 Wrap up
🔗 Resources:
Full write up with all 45 configurations and charts: kgptalkie.com/tutorials/generative-ai/qwen-3-8-27b-llama-cpp-speed-settings
Settings I run for interactive chat and coding (136.7 tok/s, 17.0GB VRAM):
llama-server -m Qwen3.8-27B-Q4_K_M.gguf -ngl 999 -fa on --jinja -np 1 -c 32768 --cache-type-k q4_0 --cache-type-v q4_0 --spec-type draft-mtp --spec-draft-n-max 2
Settings I switch to for long documents (121.8 tok/s, 21.7GB VRAM):
llama-server -m Qwen3.8-27B-Q4_K_M.gguf -ngl 999 -fa on --jinja -np 1 -c 131072 --cache-type-k q8_0 --cache-type-v q8_0 --spec-type draft-mtp --spec-draft-n-max 3
📺 Watch next:
Qwen 3.8 27B vs Nemotron 3.5 vs Muse Glimmer on RTX 5090: Full Benchmark
youtube.com/watch?v=XtcoCRu1V2s
🎓 Go deeper, my Udemy course:
Master Langchain v1 and Ollama - Chatbot, RAG and AI Agents
kgptalkie.com/langchain
If this saved you a few hours of trial and error, hit like so more local AI folks find it. Tell me in the comments which GPU you are running Qwen 3.8 on and what token speed you get, and subscribe with the bell if you want the rest of this local LLM series.
#Qwen3 #LlamaCpp #LocalLLM #RTX5090
Qwen 3.8 27B on llama.cpp: I tested 45 configurations on a single RTX 5090 to find the settings that actually make it fast, and here is everything that worked.
This is a settings deep dive, not a benchmark montage. I go through speculative decoding (the MTP layer built into Qwen 3.8), how far you can push the draft depth before it backfires, KV cache quantization, flash attention, reasoning effort, and what happens to speed and VRAM as you raise the context window. Every number here came from my own runs on my own prompts, including reverse-order runs so a hot GPU does not fake a result. If you run Qwen 3.8 27B locally on a 24GB or 32GB card, you can copy the final settings straight from this video.
📺 Full playlist: youtube.com/playlist?list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD
⏱ Chapters:
0:00 Intro
1:12 Speculative decoding and the MTP layer inside Qwen 3.8
3:10 How multi-token prediction actually works
6:32 How many tokens should you draft ahead?
12:15 Why n=3 is the sweet spot and n=5 is not
13:41 GPU heat drift and how to benchmark honestly
15:56 Reasoning effort is the biggest speed lever
17:31 KV cache quantization: faster and smaller
19:16 Should you keep flash attention on?
20:50 Raising the context is cheap, filling it is not
22:38 Why time to first token explodes on a full context
24:21 VRAM you need at 32K, 128K and 256K context
26:46 Is Ollama faster than llama.cpp?
28:14 Reasoning effort you can only control on llama.cpp
28:45 What I am not claiming
29:19 Wrap up
🔗 Resources:
Full write up with all 45 configurations and charts: kgptalkie.com/tutorials/generative-ai/qwen-3-8-27b-llama-cpp-speed-settings
Settings I run for interactive chat and coding (136.7 tok/s, 17.0GB VRAM):
llama-server -m Qwen3.8-27B-Q4_K_M.gguf -ngl 999 -fa on --jinja -np 1 -c 32768 --cache-type-k q4_0 --cache-type-v q4_0 --spec-type draft-mtp --spec-draft-n-max 2
Settings I switch to for long documents (121.8 tok/s, 21.7GB VRAM):
llama-server -m Qwen3.8-27B-Q4_K_M.gguf -ngl 999 -fa on --jinja -np 1 -c 131072 --cache-type-k q8_0 --cache-type-v q8_0 --spec-type draft-mtp --spec-draft-n-max 3
📺 Watch next:
Qwen 3.8 27B vs Nemotron 3.5 vs Muse Glimmer on RTX 5090: Full Benchmark
youtube.com/watch?v=XtcoCRu1V2s
🎓 Go deeper, my Udemy course:
Master Langchain v1 and Ollama - Chatbot, RAG and AI Agents
kgptalkie.com/langchain
If this saved you a few hours of trial and error, hit like so more local AI folks find it. Tell me in the comments which GPU you are running Qwen 3.8 on and what token speed you get, and subscribe with the bell if you want the rest of this local LLM series.
#Qwen3 #LlamaCpp #LocalLLM #RTX5090
