Gemma 4 12B VRAM requirements

Can I run Gemma 4 12B on…

• RTX 4090: Yes, up to Q8_0
• RTX 3090: Yes, up to Q8_0
• RTX 3060 12GB: Yes, up to Q5_K_M
• RTX 4060 Ti 16GB: Yes, up to Q8_0
• RTX 5060 Ti 16GB: Yes, up to Q8_0
• RTX 4070 12GB: Yes, up to Q5_K_M
• RTX 5080 16GB: Yes, up to Q8_0
• RTX 4080 Super 16GB: Yes, up to Q8_0
• RX 7900 XTX: Yes, up to Q8_0

How much VRAM does Gemma 4 12B need?

Gemma 4 12B needs 8.57 GB at Q4_K_M, 14.2 GB at Q8_0 and 25.7 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.

Fine-tuning Gemma 4 12B? Gemma 4 12B VRAM for LoRA, QLoRA and full training.

Gemma 4 12B GGUF files on Hugging Face

The 8 GGUF files llama.cpp picks for Gemma 4 12B with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.

GB here is GiB (1024³ bytes), as everywhere on this site; Hugging Face shows the Q4_K_M file as 7.1 GB, in 10⁹ bytes.

Gemma 4 12B Heretic (decensored) variants

Heretic removes refusals from a model automatically: it ablates the "refusal direction" in the weights and tunes the ablation to keep the output close to the original. igorls/gemma-4-12B-it-heretic-v1 reports 0 of 100 refusals on harmful prompts (Gemma 4 12B: 99 of 100) at a KL divergence of 0.0284 from the original on harmless prompts. Gemma's license and usage policy still apply.

It is the same model with edited weights, not a new architecture: its safetensors take 23,919,548,424 bytes against 23,919,549,408 for google/gemma-4-12B-it, so every KV cache, context and GPU figure on this page applies to it. Only the GGUF files differ. The most downloaded ones, with the totals this page's method gives:

The Heretic Q4_K_M files are 259,520,320 bytes (0.24 GB) larger than the unsloth Q4_K_M above; the Q8_0 files are within 2 KB of ggml-org's. For images, both culturerevolt and SC117 ship the 175,115,840-byte (0.16 GB) vision projector (mmproj) as a separate file, which adds that much to the totals. File sizes from the Hugging Face file lists, read on 2026-09-29; totals as in the table above, one request, FP16 KV cache.

Which Gemma 4 12B GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac

From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for Q4_K_M (6.63 GB). One request, FP16 KV cache, -np 1. The Q4_K_M file is 109 MB under its bits-per-weight estimate; the estimate tables and the llama-server command on this page count the estimate, so they show less context.

Run Gemma 4 12B with llama.cpp or Ollama

llama-server -hf unsloth/gemma-4-12b-it-GGUF:Q4_K_M -c 262144 -np 1

gemma-4-12b-it-Q4_K_M.gguf, 7.1 GB, from unsloth/gemma-4-12b-it-GGUF (checked 2026-09-29). On a 24 GB RTX 3090 / 4090, Q4_K_M with -c 262144 takes 12.8 GB and leaves 11.2 GB. -np 1: one slot, so one 1,024-token window.

ollama run gemma4:12b

Ollama library tag gemma4:12b; its quant is Ollama's choice. Same context: /set parameter num_ctx 262144.

Gemma 4 12B at 8K, 32K, 128K and 256K (full) tokens of context

8 of its 48 layers use full attention and 40 keep a sliding window of 1,024 tokens. Each extra token of context adds 16 KB of FP16 cache per request once the sliding windows are full. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.

How fast Gemma 4 12B writes

Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth. A dash means it does not fit on one card. Try other settings in the speed calculator.

Longest context on one GPU

How many tokens of context Gemma 4 12B fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.

Model details

Parameters

12.0B (11,959,730,224)

Layers

8 of its 48 layers use full attention and 40 keep a sliding window of 1,024 tokens

Attention cache

1 KV heads × 512 (values come from the key projection but are cached separately)

Sliding-window layers

8 KV heads × 256

Context length

262,144 tokens

Published weights

22.3 GB (BF16)

On Hugging Face

google/gemma-4-12B-it

Reported benchmarks

GPQA Diamond 78.8 · MMLU-Pro 77.2 · LiveCodeBench v6 72.0. Vendor-reported: from the maker’s own model card, not rerun here; read 2026-09-30.

Gemma 4 12B next to similar-sized models

The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.