Can I run Gemma 4 12B on an RX 7900 XTX?

Yes: Gemma 4 12B needs about 8.98 GB at Q4_K_M with 32K tokens of context, which fits the 24 GB RX 7900 XTX with 15.0 GB to spare. At 32K the RX 7900 XTX holds up to Q8_0 (14.6 GB), and Q4_K_M runs up to 256K (full) tokens.

Yes Q4_K_M with 32K tokens of context

Q4_K_M, 32K: 8.98 GB
RX 7900 XTX: 24 GB, 960 GB/s
To spare: 15.0 GB
Tokens/s: 58–83 tokens/s

At Q4_K_M with 32K tokens of context it writes about 58–83 tokens/s for one request on an RX 7900 XTX.

Best precision for Gemma 4 12B on an RX 7900 XTX

The most precise setting that leaves at least 0.5 GB free; one that fits with less is marked tight.

Context	Best fit	Memory	Free	Tokens/s
8K	Q8_0	14.2 GB	9.83 GB	37–53
32K	Q8_0	14.6 GB	9.42 GB	36–51
128K	Q8_0	16.2 GB	7.77 GB	33–46
256K (full)	Q8_0	18.4 GB	5.57 GB	29–40

Gemma 4 12B on the RX 7900 XTX as the context fills

One request, FP16 KV cache, 0.5 GB plus 10% overhead; a minus sign is memory missing, and tight is less than 0.5 GB free.

Context	Q4_K_M	Free	Tokens/s	Q8_0	Free	Tokens/s
4K	8.50 GB	15.5 GB	61–88	14.1 GB	9.90 GB	38–53
8K	8.57 GB	15.4 GB	61–87	14.2 GB	9.83 GB	37–53
16K	8.70 GB	15.3 GB	60–86	14.3 GB	9.69 GB	37–52
32K	8.98 GB	15.0 GB	58–83	14.6 GB	9.42 GB	36–51
64K	9.53 GB	14.5 GB	55–78	15.1 GB	8.87 GB	35–49
128K	10.6 GB	13.4 GB	49–70	16.2 GB	7.77 GB	33–46
256K (full)	12.8 GB	11.2 GB	41–58	18.4 GB	5.57 GB	29–40

Gemma 4 12B on more than one RX 7900 XTX

Cards	Best at 32K	Tokens/s	Q4_K_M longest	Q4_K_M tokens/s
2× (48 GB)	BF16	37–54	256K (full)	88–142
4× (96 GB)	BF16	65–100	256K (full)	135–240

Tensor parallel with the combined bandwidth, as in vLLM; llama.cpp splits layers by default and runs at about one card's speed.

Other options

The next smaller setting, Q3_K_M, takes 7.55 GB at 32K, 16.4 GB under the RX 7900 XTX; it fits with 0.5 GB to spare up to 256K (full) tokens.
Smallest setup for Q4_K_M at 32K: RTX 3060 12GB.

Run Gemma 4 12B on the RX 7900 XTX with llama-server

llama-server -hf unsloth/gemma-4-12b-it-GGUF:Q4_K_M -c 262144 -ngl 99 -np 1

gemma-4-12b-it-Q4_K_M.gguf, 7.1 GB, from unsloth/gemma-4-12b-it-GGUF (checked 2026-09-29). At -c 262144 on the RX 7900 XTX: 12.8 GB of 24 GB, 11.2 GB free. -np 1: one slot, one sliding window.

Questions

Can I run Gemma 4 12B on an RX 7900 XTX?

How fast is Gemma 4 12B on an RX 7900 XTX?

At Q4_K_M with 32K tokens of context it writes about 58–83 tokens/s for one request on an RX 7900 XTX.

What does a second RX 7900 XTX change for Gemma 4 12B?

Two RX 7900 XTX cards (48 GB in one tensor-parallel group) hold Gemma 4 12B at BF16 with 32K, and Q4_K_M up to 256K (full) tokens, at about 88–142 tokens/s.

Try other settings in the VRAM calculator or the speed calculator. See also Gemma 4 12B VRAM requirements, what LLMs an RX 7900 XTX can run and every pair, or detect your own GPU. Model data checked 2026-09-26.