Can I run Qwen3.6 27B on an RTX 5080 16GB?
Only at Q2_K: Qwen3.6 27B needs about 19.9 GB at Q4_K_M with 32K tokens of context, 3.92 GB more than the RTX 5080 16GB holds, but 14.6 GB at Q2_K, which fits with 1.38 GB to spare. The smallest setup here that holds Qwen3.6 27B at Q4_K_M with 32K is RTX 3090 (24 GB).
Partly Q2_K with 32K tokens of context
- Q4_K_M, 32K
- 19.9 GB
- RTX 5080
- 16 GB, 960 GB/s
- Short by
- 3.92 GB
- Tokens/s
- 36–51 tokens/s
At Q2_K with 32K tokens of context it writes about 36–51 tokens/s for one request on an RTX 5080 16GB.
Best precision for Qwen3.6 27B on an RTX 5080 16GB
The most precise setting that leaves at least 0.5 GB free; one that fits with less is marked tight.
| Context | Best fit | Memory | Free | Tokens/s |
|---|---|---|---|---|
| 8K | Q3_K_M | 15.0 GB | 1.04 GB | 35–50 |
| 32K | Q2_K | 14.6 GB | 1.38 GB | 36–51 |
| 128K | Nothing fits | — | — | — |
| 256K (full) | Nothing fits | — | — | — |
Qwen3.6 27B on the RTX 5080 as the context fills
One request, FP16 KV cache, 0.5 GB plus 10% overhead; a minus sign is memory missing, and tight is less than 0.5 GB free.
| Context | Q4_K_M | Free | Tokens/s | Q8_0 | Free | Tokens/s |
|---|---|---|---|---|---|---|
| 4K | 18.0 GB | −1.99 GB | — | 31.0 GB | −15.0 GB | — |
| 8K | 18.3 GB | −2.27 GB | — | 31.3 GB | −15.3 GB | — |
| 16K | 18.8 GB | −2.82 GB | — | 31.8 GB | −15.8 GB | — |
| 32K | 19.9 GB | −3.92 GB | — | 32.9 GB | −16.9 GB | — |
| 64K | 22.1 GB | −6.12 GB | — | 35.1 GB | −19.1 GB | — |
| 128K | 26.5 GB | −10.5 GB | — | 39.5 GB | −23.5 GB | — |
| 256K (full) | 35.3 GB | −19.3 GB | — | 48.3 GB | −32.3 GB | — |
Qwen3.6 27B on more than one RTX 5080
| Cards | Best at 32K | Tokens/s | Q4_K_M longest | Q4_K_M tokens/s |
|---|---|---|---|---|
| 2× (32 GB) | Q6_K | 37–54 | 192K | 47–69 |
| 4× (64 GB) | BF16 | 32–47 | 256K (full) | 80–127 |
Tensor parallel with the combined bandwidth, as in vLLM; llama.cpp splits layers by default and runs at about one card's speed.
Other options
- The next smaller setting, IQ3_XXS, takes 14.4 GB at 32K, 1.56 GB under the RTX 5080; it fits with 0.5 GB to spare up to 47K tokens.
- Smallest setup for Q4_K_M at 32K: RTX 3090 (24 GB).
Run Qwen3.6 27B on the RTX 5080 with llama-server
llama-server -hf bartowski/Qwen_Qwen3.6-27B-GGUF:Q2_K -c 38912Qwen_Qwen3.6-27B-Q2_K.gguf, 12.1 GB, from bartowski/
Questions
Can I run Qwen3.6 27B on an RTX 5080 16GB?
Only at Q2_K: Qwen3.6 27B needs about 19.9 GB at Q4_K_M with 32K tokens of context, 3.92 GB more than the RTX 5080 16GB holds, but 14.6 GB at Q2_K, which fits with 1.38 GB to spare. The smallest setup here that holds Qwen3.6 27B at Q4_K_M with 32K is RTX 3090 (24 GB).
How fast is Qwen3.6 27B on an RTX 5080 16GB?
At Q2_K with 32K tokens of context it writes about 36–51 tokens/s for one request on an RTX 5080 16GB.
What does a second RTX 5080 16GB change for Qwen3.6 27B?
Two RTX 5080 16GB cards (32 GB in one tensor-parallel group) hold Qwen3.6 27B at Q6_K with 32K, and Q4_K_M up to 192K tokens, at about 47–69 tokens/s.
Try other settings in the VRAM calculator or the speed calculator. See also Qwen3.6 27B VRAM requirements, what LLMs an RTX 5080 16GB can run and every pair, or detect your own GPU. Model data checked .