DiffusionGemma: fast local AI inference

With DiffusionGemma, Google has introduced an experimental open model that generates text differently from classic autoregressive LLMs. Instead of producing tokens one after another, the model works with text diffusion and generates whole blocks in parallel, according to Google.

For companies, the model name is less interesting than the operational question: what happens when local AI no longer fails primarily on latency?

What is technically new

According to Google, DiffusionGemma is an open 26B mixture-of-experts model under the Apache 2.0 license. During inference, only 3.8 billion parameters are activated. Google cites up to four times faster text generation on dedicated GPUs, including more than 1,000 tokens per second on an NVIDIA H100 and more than 700 tokens per second on a GeForce RTX 5090.

The important difference lies in decoding. According to Google, DiffusionGemma processes up to 256 tokens in parallel per step. NVIDIA describes the same effect from a hardware perspective: the bottleneck shifts more from memory bandwidth toward compute, which can make better use of GPU architectures.

Why this matters

Many enterprise applications don’t need the largest model, but short response times: assistance systems in specialist applications, code and text completion, local agent loops or interactive document editing. When a model responds noticeably faster on local hardware, that changes the cost and architecture decision.

At the same time, DiffusionGemma remains explicitly experimental. Google itself writes that standard Gemma 4 models are still the recommendation for maximum output quality. For production workloads, that means: don’t replace, but test.

What DACH companies should review

For organizations in the DACH region, local operation is especially interesting when data protection, latency or cost must remain predictable. You should evaluate DiffusionGemma above all for bounded, internal workflows: fast suggestions, drafts, corrections, technical assistance.

The guiding question is not “Is the best model now local?” but: which tasks need quality, which need speed — and which data should never leave your own environment in the first place?

Anyone planning AI platforms today should seriously put local inference back into the target architecture. Not as a dogma. As an additional operating option.