Can I run Qwen3.8 27B on an RTX 3060 12GB?
No: Qwen3.8 27B needs about 19.9 GB at Q4_K_M with 32K tokens of context, 7.92 GB more than the RTX 3060 12GB holds, so it takes 2 of them (24 GB) in one tensor-parallel group. The smallest setup here that holds Qwen3.8 27B at Q4_K_M with 32K is RTX 3090 (24 GB).
No Q4_K_M with 32K tokens of context on 2 cards
- Q4_K_M, 32K
- 19.9 GB
- RTX 3060
- 12 GB, 360 GB/s
- Short by
- 7.92 GB
- Tokens/s
- 19–27 tokens/s
At Q4_K_M with 32K tokens of context on 2 cards it writes about 19–27 tokens/s for one request on 2 RTX 3060 12GB cards.
Qwen3.8 27B on the RTX 3060 as the context fills
One request, FP16 KV cache, 0.5 GB plus 10% overhead; a minus sign is memory missing, and tight is less than 0.5 GB free.
| Context | Q4_K_M | Free | Tokens/s | Q8_0 | Free | Tokens/s |
|---|---|---|---|---|---|---|
| 4K | 18.0 GB | −5.99 GB | — | 31.0 GB | −19.0 GB | — |
| 8K | 18.3 GB | −6.27 GB | — | 31.3 GB | −19.3 GB | — |
| 16K | 18.8 GB | −6.82 GB | — | 31.8 GB | −19.8 GB | — |
| 32K | 19.9 GB | −7.92 GB | — | 32.9 GB | −20.9 GB | — |
| 64K | 22.1 GB | −10.1 GB | — | 35.1 GB | −23.1 GB | — |
| 128K | 26.5 GB | −14.5 GB | — | 39.5 GB | −27.5 GB | — |
| 256K (full) | 35.3 GB | −23.3 GB | — | 48.3 GB | −36.3 GB | — |
Qwen3.8 27B on more than one RTX 3060
| Cards | Best at 32K | Tokens/s | Q4_K_M longest | Q4_K_M tokens/s | Rent per hour |
|---|---|---|---|---|---|
| 2× (24 GB) | Q5_K_M | 17–24 | 76K | 19–27 | $0.16 |
| 4× (48 GB) | Q8_0 | 23–33 | 256K (full) | 36–53 | $0.32 |
Tensor parallel with the combined bandwidth, as in vLLM; llama.cpp splits layers by default and runs at about one card's speed.
Other options
- The next smaller setting, Q3_K_M, takes 16.6 GB at 32K, 4.61 GB over the RTX 3060; it does not fit with 0.5 GB to spare even with 1K tokens.
- Renting two RTX 3060 12GB GPUs costs about $0.16 an hour (median on getdeploying.com, 2026-09-29): $1.6–2.3 per million tokens at 19–27 tokens/s.
- Smallest setup for Q4_K_M at 32K: RTX 3090 (24 GB).
Run Qwen3.8 27B on the RTX 3060 with llama-server
llama-server -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M -c 45056Qwen3.8-27B-Q4_K_M.gguf, 19.0 GB, from ggml-org/
Questions
Can I run Qwen3.8 27B on an RTX 3060 12GB?
No: Qwen3.8 27B needs about 19.9 GB at Q4_K_M with 32K tokens of context, 7.92 GB more than the RTX 3060 12GB holds, so it takes 2 of them (24 GB) in one tensor-parallel group. The smallest setup here that holds Qwen3.8 27B at Q4_K_M with 32K is RTX 3090 (24 GB).
How fast is Qwen3.8 27B on an RTX 3060 12GB?
At Q4_K_M with 32K tokens of context on 2 cards it writes about 19–27 tokens/s for one request on 2 RTX 3060 12GB cards.
What does a second RTX 3060 12GB change for Qwen3.8 27B?
Two RTX 3060 12GB cards (24 GB in one tensor-parallel group) hold Qwen3.8 27B at Q5_K_M with 32K, and Q4_K_M up to 76K tokens, at about 19–27 tokens/s.
Try other settings in the VRAM calculator or the speed calculator. See also Qwen3.8 27B VRAM requirements, what LLMs an RTX 3060 12GB can run and every pair, or detect your own GPU. Model data checked .