Can I run Qwen3.6 27B on an RTX 5090?
Yes: Qwen3.6 27B needs about 19.9 GB at Q4_K_M with 32K tokens of context, which fits the 32 GB RTX 5090 with 12.1 GB to spare. At 32K the RTX 5090 holds up to FP8 (31.2 GB), and Q4_K_M runs up to 200K tokens.
Yes Q4_K_M with 32K tokens of context
- Q4_K_M, 32K
- 19.9 GB
- RTX 5090
- 32 GB, 1,792 GB/s
- To spare
- 12.1 GB
- Tokens/s
- 48–68 tokens/s
At Q4_K_M with 32K tokens of context it writes about 48–68 tokens/s for one request on an RTX 5090.
Best precision for Qwen3.6 27B on an RTX 5090
The most precise setting that leaves at least 0.5 GB free; one that fits with less is marked tight.
| Context | Best fit | Memory | Free | Tokens/s |
|---|---|---|---|---|
| 8K | Q8_0 | 31.3 GB | 727 MB | 31–44 |
| 32K | FP8 | 31.2 GB | 859 MB | 31–44 |
| 128K | Q5_K_M | 29.5 GB | 2.53 GB | 33–46 |
| 256K (full) | Q2_K | 30.0 GB | 1.98 GB | 33–46 |
Qwen3.6 27B on the RTX 5090 as the context fills
One request, FP16 KV cache, 0.5 GB plus 10% overhead; a minus sign is memory missing, and tight is less than 0.5 GB free.
| Context | Q4_K_M | Free | Tokens/s | Q8_0 | Free | Tokens/s |
|---|---|---|---|---|---|---|
| 4K | 18.0 GB | 14.0 GB | 53–76 | 31.0 GB | 1,009 MB | 32–44 |
| 8K | 18.3 GB | 13.7 GB | 52–75 | 31.3 GB | 727 MB | 31–44 |
| 16K | 18.8 GB | 13.2 GB | 51–72 | 31.8 GB | 164 MB (tight) | 31–43 |
| 32K | 19.9 GB | 12.1 GB | 48–68 | 32.9 GB | −962 MB | — |
| 64K | 22.1 GB | 9.88 GB | 44–62 | 35.1 GB | −3.14 GB | — |
| 128K | 26.5 GB | 5.48 GB | 37–52 | 39.5 GB | −7.54 GB | — |
| 256K (full) | 35.3 GB | −3.32 GB | — | 48.3 GB | −16.3 GB | — |
Qwen3.6 27B on more than one RTX 5090
| Cards | Best at 32K | Tokens/s | Q4_K_M longest | Q4_K_M tokens/s | Rent per hour |
|---|---|---|---|---|---|
| 2× (64 GB) | BF16 | 31–44 | 256K (full) | 76–120 | $1.38 |
| 4× (128 GB) | BF16 | 55–83 | 256K (full) | 120–207 | $2.76 |
Tensor parallel with the combined bandwidth, as in vLLM; llama.cpp splits layers by default and runs at about one card's speed.
Other options
- The next smaller setting, Q3_K_M, takes 16.6 GB at 32K, 15.4 GB under the RTX 5090; it fits with 0.5 GB to spare up to 248K tokens.
- Renting an RTX 5090 costs about $0.69 an hour (median on getdeploying.com, 2026-09-29): $2.8–4.0 per million tokens at 48–68 tokens/s.
- Smallest setup for Q4_K_M at 32K: RTX 3090 (24 GB).
Run Qwen3.6 27B on the RTX 5090 with llama-server
llama-server -hf ggml-org/Qwen3.6-27B-GGUF:Q4_K_M -c 169984Qwen3.6-27B-Q4_K_M.gguf, 19.1 GB, from ggml-org/
Questions
Can I run Qwen3.6 27B on an RTX 5090?
Yes: Qwen3.6 27B needs about 19.9 GB at Q4_K_M with 32K tokens of context, which fits the 32 GB RTX 5090 with 12.1 GB to spare. At 32K the RTX 5090 holds up to FP8 (31.2 GB), and Q4_K_M runs up to 200K tokens.
How fast is Qwen3.6 27B on an RTX 5090?
At Q4_K_M with 32K tokens of context it writes about 48–68 tokens/s for one request on an RTX 5090.
What does a second RTX 5090 change for Qwen3.6 27B?
Two RTX 5090 cards (64 GB in one tensor-parallel group) hold Qwen3.6 27B at BF16 with 32K, and Q4_K_M up to 256K (full) tokens, at about 76–120 tokens/s.
Try other settings in the VRAM calculator or the speed calculator. See also Qwen3.6 27B VRAM requirements, what LLMs an RTX 5090 can run and every pair, or detect your own GPU. Model data checked .