Can I run Gemma 4 12B on an RTX 3090?
Yes: Gemma 4 12B needs about 8.98 GB at Q4_K_M with 32K tokens of context, which fits the 24 GB RTX 3090 with 15.0 GB to spare. At 32K the RTX 3090 holds up to Q8_0 (14.6 GB), and Q4_K_M runs up to 256K (full) tokens.
Yes Q4_K_M with 32K tokens of context
- Q4_K_M, 32K
- 8.98 GB
- RTX 3090
- 24 GB, 936 GB/s
- To spare
- 15.0 GB
- Tokens/s
- 57–81 tokens/s
At Q4_K_M with 32K tokens of context it writes about 57–81 tokens/s for one request on an RTX 3090.
Best precision for Gemma 4 12B on an RTX 3090
The most precise setting that leaves at least 0.5 GB free; one that fits with less is marked tight.
| Context | Best fit | Memory | Free | Tokens/s |
|---|---|---|---|---|
| 8K | Q8_0 | 14.2 GB | 9.83 GB | 36–51 |
| 32K | Q8_0 | 14.6 GB | 9.42 GB | 35–50 |
| 128K | Q8_0 | 16.2 GB | 7.77 GB | 32–45 |
| 256K (full) | Q8_0 | 18.4 GB | 5.57 GB | 28–39 |
Gemma 4 12B on the RTX 3090 as the context fills
One request, FP16 KV cache, 0.5 GB plus 10% overhead; a minus sign is memory missing, and tight is less than 0.5 GB free.
| Context | Q4_K_M | Free | Tokens/s | Q8_0 | Free | Tokens/s |
|---|---|---|---|---|---|---|
| 4K | 8.50 GB | 15.5 GB | 60–86 | 14.1 GB | 9.90 GB | 37–52 |
| 8K | 8.57 GB | 15.4 GB | 60–85 | 14.2 GB | 9.83 GB | 36–51 |
| 16K | 8.70 GB | 15.3 GB | 59–84 | 14.3 GB | 9.69 GB | 36–51 |
| 32K | 8.98 GB | 15.0 GB | 57–81 | 14.6 GB | 9.42 GB | 35–50 |
| 64K | 9.53 GB | 14.5 GB | 54–77 | 15.1 GB | 8.87 GB | 34–48 |
| 128K | 10.6 GB | 13.4 GB | 48–69 | 16.2 GB | 7.77 GB | 32–45 |
| 256K (full) | 12.8 GB | 11.2 GB | 40–57 | 18.4 GB | 5.57 GB | 28–39 |
Gemma 4 12B on more than one RTX 3090
| Cards | Best at 32K | Tokens/s | Q4_K_M longest | Q4_K_M tokens/s | Rent per hour |
|---|---|---|---|---|---|
| 2× (48 GB) | BF16 | 36–52 | 256K (full) | 87–139 | $0.54 |
| 4× (96 GB) | BF16 | 64–98 | 256K (full) | 133–235 | $1.08 |
Tensor parallel with the combined bandwidth, as in vLLM; llama.cpp splits layers by default and runs at about one card's speed.
Other options
- The next smaller setting, Q3_K_M, takes 7.55 GB at 32K, 16.4 GB under the RTX 3090; it fits with 0.5 GB to spare up to 256K (full) tokens.
- Renting an RTX 3090 costs about $0.27 an hour (median on getdeploying.com, 2026-09-29): $0.92–1.3 per million tokens at 57–81 tokens/s.
- Smallest setup for Q4_K_M at 32K: RTX 3060 12GB.
Run Gemma 4 12B on the RTX 3090 with llama-server
llama-server -hf unsloth/gemma-4-12b-it-GGUF:Q4_K_M -c 262144 -np 1gemma-4-12b-it-Q4_K_M.gguf, 7.1 GB, from unsloth/
Questions
Can I run Gemma 4 12B on an RTX 3090?
Yes: Gemma 4 12B needs about 8.98 GB at Q4_K_M with 32K tokens of context, which fits the 24 GB RTX 3090 with 15.0 GB to spare. At 32K the RTX 3090 holds up to Q8_0 (14.6 GB), and Q4_K_M runs up to 256K (full) tokens.
How fast is Gemma 4 12B on an RTX 3090?
At Q4_K_M with 32K tokens of context it writes about 57–81 tokens/s for one request on an RTX 3090.
What does a second RTX 3090 change for Gemma 4 12B?
Two RTX 3090 cards (48 GB in one tensor-parallel group) hold Gemma 4 12B at BF16 with 32K, and Q4_K_M up to 256K (full) tokens, at about 87–139 tokens/s.
Try other settings in the VRAM calculator or the speed calculator. See also Gemma 4 12B VRAM requirements, what LLMs an RTX 3090 can run and every pair, or detect your own GPU. Model data checked .