Uploaded June 2026 | Updated November 2026, 22 hours ago
Google’s Diffusion Gemma, its first open-weight diffusion-based language model released under Apache 2.0. I explain how diffusion decoding differs from autoregressive generation (parallel fixed-window generation that can revise earlier tokens), walk through the step mechanics (256-token patches, entropy/uncertainty locking with a budget, temperature cooling, early stopping), and why it becomes a hybrid: diffusion within blocks and autoregressive across blocks. I cover the MoE network details (26B total, ~4B active, 128 experts, sliding-window attention with periodic global layers, up to 256K context, small vision encoder), hardware/VRAM needs across BF16/FP8/NVFP4/GGUF, and day-one support in Transformers, vLLM, MLX, and llama.cpp. I also compare speed vs accuracy, show a local MLX demo UI, and generate a simple Pokémon website example.
https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
huggingface.co/google/diffusiongemma-26B-A4B-it
https://ai.google.dev/gemma/docs/diffusiongemma
My voice to text App: whryte.com
Website: engineerprompt.ai
RAG Beyond Basics Course:
prompt-s-site.thinkific.com/courses/rag
Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0
Let's Connect:
🦾 Discord: discord.com/invite/t4eYQRUcXB
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Patreon: patreon.com/PromptEngineering
💼Consulting: calendly.com/engineerprompt/consulting-call
📧 Business Contact: engineerprompt@gmail.com
Become Member: tinyurl.com/y5h28s6h
💻 Pre-configured localGPT VM: bit.ly/localGPT (use Code: PromptEngineering for 50% off).
Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0
Diffusion Gemma Explained: Google’s First Open-Weight Diffusion LLM (26B MoE) + Local Demo
00:00 Diffusion Gemma
01:02 Diffusion vs Autoregressive
02:03 How Diffusion Works
02:50 Inside a Denoising Step
04:08 Blocks and Hybrid Decoding
04:50 MoE Network Breakdown
05:37 Hardware and Quantization
07:06 Speed vs Accuracy Tradeoffs
08:14 Serving Options and Demo Setup
08:47 Parallel Generation Examples
09:53 Local UI and Coding Demo
Google’s Diffusion Gemma, its first open-weight diffusion-based language model released under Apache 2.0. I explain how diffusion decoding differs from autoregressive generation (parallel fixed-window generation that can revise earlier tokens), walk through the step mechanics (256-token patches, entropy/uncertainty locking with a budget, temperature cooling, early stopping), and why it becomes a hybrid: diffusion within blocks and autoregressive across blocks. I cover the MoE network details (26B total, ~4B active, 128 experts, sliding-window attention with periodic global layers, up to 256K context, small vision encoder), hardware/VRAM needs across BF16/FP8/NVFP4/GGUF, and day-one support in Transformers, vLLM, MLX, and llama.cpp. I also compare speed vs accuracy, show a local MLX demo UI, and generate a simple Pokémon website example.
https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
huggingface.co/google/diffusiongemma-26B-A4B-it
https://ai.google.dev/gemma/docs/diffusiongemma
My voice to text App: whryte.com
Website: engineerprompt.ai
RAG Beyond Basics Course:
prompt-s-site.thinkific.com/courses/rag
Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0
Let's Connect:
🦾 Discord: discord.com/invite/t4eYQRUcXB
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Patreon: patreon.com/PromptEngineering
💼Consulting: calendly.com/engineerprompt/consulting-call
📧 Business Contact: engineerprompt@gmail.com
Become Member: tinyurl.com/y5h28s6h
💻 Pre-configured localGPT VM: bit.ly/localGPT (use Code: PromptEngineering for 50% off).
Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0
Diffusion Gemma Explained: Google’s First Open-Weight Diffusion LLM (26B MoE) + Local Demo
00:00 Diffusion Gemma
01:02 Diffusion vs Autoregressive
02:03 How Diffusion Works
02:50 Inside a Denoising Step
04:08 Blocks and Hybrid Decoding
04:50 MoE Network Breakdown
05:37 Hardware and Quantization
07:06 Speed vs Accuracy Tradeoffs
08:14 Serving Options and Demo Setup
08:47 Parallel Generation Examples
09:53 Local UI and Coding Demo






![CLM-8B: Contrastive Language Models
Jev made System 1 models hot. CLM from Stanford and NVIDIA Research takes a completely different approach: it never generates an answer.
Instead, it embeds the situation and every possible action into the same space and picks whichever action lands closest. Because actions are embedded once and cached, 1,000 options cost about the same as one: 44 ms vs 579 ms for Jev.
The catch: each option is encoded on its own, so it never compares them side by side.
Full breakdown + demos on my DGX Spark: [LINK TO FULL VIDEO]
📄 Blog: https://contrastive-lm.notion.site
💻 Code: https://github.com/Contrastive-LM/CLM
#AI #LLM #MachineLearning #SystemOne #Jev #CLM #NVIDIA CLM-8B: Contrastive Language Models](https://i.ytimg.com/vi/MK1zLrR9gpM/mqdefault.jpg)



