New Google DiffusionGemma. Google’s DiffusionGemma 26B A4B-IT: A High-Speed, Open-Weight MoE Powerhouse
Google just pulled back the curtain on its newest large language model: DiffusionGemma 26B A4B-IT. Built from the ground up using a mixture-of-experts (MoE) architecture, this 25.2-billion-parameter system is designed to deliver mind-blowing generation speeds while keeping things open-weight for the community. We’ve put together a deep dive into the technical details below so you can see if it is a good fit for your production workloads.
New Google DiffusionGemma – Overview of the Release
Google DeepMind announced DiffusionGemma 26B A4B-IT on June 10, 2026, rolling it out jointly across Hugging Face, NVIDIA’s NIM service, and Vertex AI. Unlike your typical autoregressive (AR) models that spit out tokens one by one, this is a text-diffusion language model. It generates text by gradually denoising random noise. The model readily handles multimodal inputs—like plain text, images, or short video clips—and can output coherent paragraphs up to a massive 256K tokens.
The real secret sauce here is parallel generation. Instead of the slow, one-token-at-a-time approach, the model commits 15 to 20 tokens per forward pass across 48 denoising steps. When you run this on an NVIDIA Hopper H100 with FP8 precision, it clocks in at over 1,100 tokens per second. That is roughly four times faster than its standard autoregressive sibling, Gemma 4 26B.
What Is Google DiffusionGemma?
Think of diffusion language models like a sculptor starting with a messy block of marble. They start with random noise and iteratively “diffusing” or refining it until it becomes clear, natural text. By reversing this noisy process, the model creates highly fluent output. Google’s implementation, DiffusionGemma, leverages the same 26B MoE backbone as Gemma 4 but pairs it with a unique diffusion front-end. This design effectively gets rid of the sequential bottleneck that slows down traditional autoregressive generation.
By keeping a massive portion of its experts inactive during standard generation, the model significantly cuts down on overall FLOPs. This means you get the power of a massive parameter budget without burning through computational resources.
MoE Design (Mixture-of-Experts)
– Massive Sparse Network:** Each MoE block features a sparse feed-forward network with 4,096 experts.
– High Efficiency during Inference:** Only about 15% of these experts light up per token. This dynamic sparsity pattern keeps throughput incredibly high.
– Smart Scheduling:** The activation schedule learns to adapt to the expected length of the input (for example, a 256-token prompt will activate around 48 experts).
– Scalability:** This setup allows the system to scale and perform far beyond the typical limits of a dense 30B-parameter system.
-Encoder-Decoder Layout
– The Encoder:** A classic Transformer encoder featuring 24 layers, equipped with a large attention head set and position embeddings.
– Diffusion Front-End:** This adds noise conditioning vectors that are processed in parallel across all 48 denoising steps.
– The Decoder:** A separate MoE decoder handles token generation, reusing a subset of the encoder’s residual connections to keep things coherent.
Model Architecture and Parameter Counts
| Metric | Value |
|---|---|
| Total parameters (all experts) | 25.2 B |
| Active (computational) parameters | 3.8 B |
| Expert count per layer | 4096 |
| MoE sparsity ratio | ~15 % active at inference time |
Because the architecture is custom-built for parallel decoding, all 256-token blocks are processed at the exact same time rather than waiting in line.
Context Window and Token Blocking
– Massive Context:** It boasts a maximum context length of 256K tokens—the largest window of any diffusion model Google has ever released.
– Block Generation:** Generation happens in blocks of 15 to 20 tokens, with each block representing a single denoising step (totaling 48 steps).
– Granular Controls:** You can fine-tune generation using standard token-level control variables like `max_new_tokens`, `temperature`, and `top_p`.
With a 256K window, you can throw massive tasks at it—like auditing legal contracts, doing deep code reviews, or writing long-form stories—without worrying about the text getting cut off.
Keep in mind that Google is open about the quality trade-offs here. Diffusion models do tend to be slightly less precise on benchmark tasks compared to traditional autoregressive models. But for many, the massive boost in raw speed is well worth it.
Multimodal Capabilities
DiffusionGemma isn’t just a text play. It easily accepts three different types of input:
1. **Plain text:** Tokenized just like you would expect.
2. **Images (JPEG/PNG):** Processed through a CLIP-style encoder, where the diffusion front-end adds noise conditioned directly on the visual elements.
3. **Short video clips:** You can feed it clips up to 5 seconds long, which are rendered into a fixed-size image and processed by the MoE backbone.
Performance Benchmarks
Generation Speed on NVIDIA GPUs
| GPU | Precision | Tokens/s (steady) |
|---|---|---|
| H100 | FP8 | >1,100 |
| RTX 5090 | FP32 | 700+ |
| A10G | BF16 | 450 |
The output is always pure text. This multimodal flexibility is a game-changer for building chatbots that need to interpret screenshots or images without spinning up a completely separate vision pipeline.
All API endpoints accept straightforward JSON payloads where you can specify your prompt, `max_new_tokens`, and optional parameters like `temperature`.
Quality Trade‑offs vs Autoregressive Models
| Benchmark | AR Gemma 4 26B (90 %) | DiffusionGemma A4B‑IT |
|---|---|---|
| MMLU Pro | 82.6 | 77.6 |
| GPQA | 82.3 | 73.2 |
| MMMU Pro | 73.8 | 54.3 |
For production deployment, consider NVIDIA NIM for automatic FP8 handling or Vertex AI to manage scaling automatically.
Future Outlook
Looking down the road, Google’s roadmap points to a massive DiffusionGemma 70B release slated for late 2026. This upcoming model will scale up the MoE architecture even further, bringing the active expert count up to 8.5B. We can also expect even lower costs per token and better native compatibility with newer hardware setups like the H100-Hopper-Fused-Fabric.
If you are a developer, the big takeaway here is clear: switching to diffusion-based generation can give you a massive four-fold speed bump for a very minor hit to absolute accuracy. This is incredibly appealing for real-time applications where latency is the ultimate dealbreaker.
How to Get Started
When you are ready to deploy to production, running it via NVIDIA NIM is your best bet for painless FP8 handling, while Vertex AI is perfect for scaling things automatically.
Wrapping Up
New Google DiffusionGemma 26B A4B-IT is a major milestone that changes what we can expect from open-weight diffusion language models. Offering 25.2 billion parameters, speeds topping 1,100 tokens per second on H100 GPUs, and a huge 256K context window, it runs circles around autoregressive alternatives when it comes to sheer throughput. If you need to optimize for low-latency interactive chat or fast document analysis, DiffusionGemma is a highly compelling option to add to your stack.
