Gemma 4 GPU requirements: sizes, VRAM and what to run it on

Gemma 4 comes in five sizes, and the largest, the 31B, needs about 70 GB of GPU memory to load at full precision and about 17.5 GB at 4-bit, by Google's own table. The small sizes, E2B and E4B, are built for phones and laptops, and the 12B runs in 16 GB. A cloud GPU matters for the 26B A4B and the 31B: one 48 GB card runs either at 4-bit with a long context, and one 80 GB or 96 GB card runs them at full precision.

The five Gemma 4 sizes#

Gemma 4 is Google DeepMind's open model family, released on 2026-04-02 in four sizes and joined by the 12B on 2026-06-03. These are the figures on Google's model card, with the BF16 file size of each instruction-tuned repository on Hugging Face:

The E in E2B and E4B stands for effective parameters: these two carry large per-layer embedding tables that are only looked up, so their files are bigger than the name suggests. The A in 26B A4B stands for active. It is a mixture-of-experts model that uses 8 of its 128 experts per token plus one shared expert, and Google's documentation is clear that "all 26 billion parameters must be loaded into memory", so size it as a 26B model. Every size also reads video as a sequence of frames, up to 60 seconds at one frame per second, and the three audio-capable sizes take up to 30 seconds of audio.

The context window is 128K tokens for E2B and E4B and 256K for the other three. The configs set it to 131,072 and 262,144 tokens.

Gemma 4 is released under the Apache 2.0 licence, which Google calls "commercially permissive" in its launch post. The Hugging Face repositories are tagged apache-2.0 and are not gated, so you need no token and no click-through to download them.

Memory to load each size#

Google publishes a memory table for Gemma 4 in its documentation. It covers loading the weights only, with 20% added, and Google notes that it does not include the context window:

Those figures follow from the files. The 31B's BF16 weights are 62.55 GB, which is 58.25 GiB, and 58.25 x 1.2 is 69.9 (our calculation). The 8-bit column is half of that and the 4-bit column a quarter.

For 4-bit, Google ships its own quantization-aware-trained builds, which it says keep quality close to BF16. Two of the four QAT formats Google lists matter on a GPU server. The Q4_0 GGUF files run on llama.cpp and Ollama, and the w4a16 compressed-tensors files run on vLLM:

Each GGUF repository also holds a separate projector file for multimodal input, between 0.18 GB and 1.20 GB. The GGUF guide explains the format.

KV cache at long context#

Gemma 4 keeps its cache small with two kinds of layer. Most layers use a sliding window of 512 or 1,024 tokens, so their cache stops growing after the window. The rest are global layers that see the whole context, and in the 12B, 26B A4B and 31B those global layers use unified keys and values, which lets an engine store one tensor where other models store two.

Only the global layers grow with the context. For the 31B that is 10 layers x 4 KV heads x 512 x 2 bytes = 40,960 bytes per token when keys and values are stored once, and twice that if an engine stores them separately (our calculation, 16-bit cache):

E2B and E4B have seven global layers each and store at most 14 KiB and 28 KiB per token there, 1.75 GiB and 3.5 GiB at their full 131,072 tokens, and less in practice because they also share the cache across layers (our calculation, an upper bound). So a full 262,144-token conversation on the 31B costs between 10.8 and 20.8 GiB, and the low end applies only if the engine uses the unified layout. Check the GPU KV cache size line in vLLM's startup log for the real figure. The generic script in the KV cache guide overstates Gemma 4, because it reads the sliding-layer head settings.

Which QuantaCloud GPU holds which size#

The table shows the memory left for the cache after the weights. The budget is 92% of what each card reports in nvidia-smi, vLLM's default, in GiB (our calculation):

The 48 GB column covers the RTX A6000, RTX 6000 Ada, L40 and L40S with ECC on. These are upper bounds, before activations and CUDA graphs. Two 48 GB cards hold the BF16 31B as well, on size, with 24.5 GiB left across the pair, and the vLLM multi-GPU guide covers that setup.

Google's own sizing agrees with the 80 GB column: its launch post says the unquantized bfloat16 weights of the 26B and 31B "fit efficiently on a single 80GB NVIDIA H100 GPU". On an 80 GB A100 the 31B has 15.3 GiB left, enough for one 131,072-token conversation under either cache layout. The 96 GB RTX PRO 6000 leaves 29.7 GiB, which covers the full 256K context.

Prices checked 14 Nov 2026, 08:32 UTC

Laptop or cloud GPU#

The honest answer is that most of Gemma 4 does not need a rented GPU. E2B and E4B are made for phones and edge boards, and Google describes the 12B as "small enough to run locally with just 16GB of VRAM or unified memory". At 4-bit even the 31B is a 17.65 GB file, which a 24 GB consumer card holds with a short context.

A 48 GB card earns its price in four cases. The first is long context on the large sizes: Google's QAT 31B for vLLM leaves 19.7 GiB on a 48 GB card, where a 24 GB card has under 1 GiB left. The second is many users at once: the 12B at full precision leaves 19.1 GiB, enough for more than twenty conversations of 32,768 tokens (our calculation). The third is precision: the 12B, E4B and E2B run in BF16 on 48 GB, and the 26B A4B and 31B run in BF16 on an 80 GB or 96 GB card or on two 48 GB cards. The fourth is fine-tuning, which Google names as a purpose of the 31B, and the LoRA, QLoRA and full fine-tuning VRAM guide sizes that separately.

Run Gemma 4 with vLLM#

vLLM serves every Gemma 4 size. Its recipes give 0.19.1 as the minimum version for the 31B, E4B and E2B, 0.23.0 for the 12B and 0.25.0 for the 26B A4B. The setup is the one in the vLLM Docker guide: the Bare Metal template, the driver check that picks the image tag, and an API key in VLLM_API_KEY. On an A100 80 GB or an RTX PRO 6000:

docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  "vllm/vllm-openai:$VLLM_TAG" \
  google/gemma-4-31B-it \
  --max-model-len 32768 \
  --reasoning-parser gemma4 \
  --api-key "$VLLM_API_KEY"

On a 48 GB card, change the model to google/gemma-4-31B-it-qat-w4a16-ct, Google's 4-bit build for vLLM. The 26B A4B takes the same command with google/gemma-4-26B-A4B-it on an 80 GB card, and since Google publishes no vLLM 4-bit build of it, run it through Ollama on a 48 GB card. vLLM's recipe lists the H100 as verified for the 31B. The A100, the RTX PRO 6000 and the 48 GB cards are not on that list, so on them the fit is on size only: read the first startup log closely. --max-model-len 32768 is the value in vLLM's own example, and you can raise it as far as the table above allows. --reasoning-parser gemma4 returns the model's thinking separately when you switch thinking on. For tool calling, the recipe adds --enable-auto-tool-choice --tool-call-parser gemma4 and a chat template file from the vLLM repository. Add --language-model-only when you only send text. Stopping the instance deletes its disk, so each launch downloads the weights again.

Gemma 4 on Ollama#

The Ollama library carries all five sizes under gemma4. The library shows a size range for the short tags and one size for each explicit tag:

gemma4:latest is the E4B, so pull gemma4:31b by name if you want the large model. The explicit tags spell out the size and the quantization, for example gemma4:31b-it-q8_0, gemma4:31b-it-qat and gemma4:26b-a4b-it-q8_0. Skip gemma4:cloud and gemma4:31b-cloud, which run on Ollama's servers and never download to your GPU. Ollama lists text and image as the inputs for every tag.

Every 4-bit and 8-bit tag fits one 48 GB card. Ollama picks its default context from the GPU's memory, and Ollama context length shows how to set it. The Open WebUI + Ollama template gives you a chat interface with no terminal work, and Open WebUI with Ollama covers it step by step.

Gemma 4 hardware FAQ#

What are the Gemma 4 sizes?

Five: E2B, E4B, 12B, 26B A4B and 31B. E2B and E4B have 2.3B and 4.5B effective parameters, the 12B has 11.95B, the 26B A4B has 25.2B with 3.8B active, and the 31B has 30.7B.

What is the Gemma 4 context window?

128K tokens for E2B and E4B, and 256K tokens for the 12B, 26B A4B and 31B.

How much VRAM does Gemma 4 31B need?

Google's table gives 69.9 GB at BF16, 34.9 GB at 8-bit and 17.5 GB at 4-bit to load the weights. Add the cache for your context: between 10.8 and 20.8 GiB for one full 256K conversation, and far less for shorter ones (our calculation).

Can I run Gemma 4 on a laptop?

Yes for E2B, E4B and the 12B, which Google says runs in 16 GB. The 26B A4B and 31B take 14.4 GB and 17.5 GB to load at 4-bit by Google's table, and an 80 GB GPU at full precision.

Is Gemma 4 free for commercial use?

Yes. Gemma 4 is under the Apache 2.0 licence.

The rule I follow for Gemma 4 is to match the precision to the card. On 48 GB, run Google's QAT build of the 31B or the 12B at full precision. On 80 GB or 96 GB, run the 31B or the 26B A4B in BF16 and spend what is left on context. The LLM VRAM calculator checks other models against every GPU configuration, and how much VRAM you need covers the general rule.