Can I run Qwen3.6 27B on an RTX 4080 Super 16GB?
Only at Q2_K: Qwen3.6 27B needs about 19.9 GB at Q4_K_M with 32K tokens of context, 3.92 GB more than the RTX 4080 Super 16GB holds, but 14.6 GB at Q2_K, which fits with 1.38 GB to spare. The smallest setup here that holds Qwen3.6 27B at Q4_K_M with 32K is RTX 3090 (24 GB).
Partly Q2_K with 32K tokens of context
- Q4_K_M, 32K
- 19.9 GB
- RTX 4080 Super
- 16 GB, 736 GB/s
- Short by
- 3.92 GB
- Tokens/s
- 28–39 tokens/s
At Q2_K with 32K tokens of context it writes about 28–39 tokens/s for one request on an RTX 4080 Super 16GB.
Best precision for Qwen3.6 27B on an RTX 4080 Super 16GB
The most precise setting that leaves at least 0.5 GB free; one that fits with less is marked tight.
| Context | Best fit | Memory | Free | Tokens/s |
|---|---|---|---|---|
| 8K | Q3_K_M | 15.0 GB | 1.04 GB | 27–38 |
| 32K | Q2_K | 14.6 GB | 1.38 GB | 28–39 |
| 128K | Nothing fits | — | — | — |
| 256K (full) | Nothing fits | — | — | — |
Qwen3.6 27B on the RTX 4080 Super as the context fills
One request, FP16 KV cache, 0.5 GB plus 10% overhead; a minus sign is memory missing, and tight is less than 0.5 GB free.
| Context | Q4_K_M | Free | Tokens/s | Q8_0 | Free | Tokens/s |
|---|---|---|---|---|---|---|
| 4K | 18.0 GB | −1.99 GB | — | 31.0 GB | −15.0 GB | — |
| 8K | 18.3 GB | −2.27 GB | — | 31.3 GB | −15.3 GB | — |
| 16K | 18.8 GB | −2.82 GB | — | 31.8 GB | −15.8 GB | — |
| 32K | 19.9 GB | −3.92 GB | — | 32.9 GB | −16.9 GB | — |
| 64K | 22.1 GB | −6.12 GB | — | 35.1 GB | −19.1 GB | — |
| 128K | 26.5 GB | −10.5 GB | — | 39.5 GB | −23.5 GB | — |
| 256K (full) | 35.3 GB | −19.3 GB | — | 48.3 GB | −32.3 GB | — |
Qwen3.6 27B on more than one RTX 4080 Super
| Cards | Best at 32K | Tokens/s | Q4_K_M longest | Q4_K_M tokens/s |
|---|---|---|---|---|
| 2× (32 GB) | Q6_K | 29–42 | 192K | 37–54 |
| 4× (64 GB) | BF16 | 26–36 | 256K (full) | 66–101 |
Tensor parallel with the combined bandwidth, as in vLLM; llama.cpp splits layers by default and runs at about one card's speed.
Other options
- The next smaller setting, IQ3_XXS, takes 14.4 GB at 32K, 1.56 GB under the RTX 4080 Super; it fits with 0.5 GB to spare up to 47K tokens.
- Smallest setup for Q4_K_M at 32K: RTX 3090 (24 GB).
Run Qwen3.6 27B on the RTX 4080 Super with llama-server
llama-server -hf bartowski/Qwen_Qwen3.6-27B-GGUF:Q2_K -c 38912Qwen_Qwen3.6-27B-Q2_K.gguf, 12.1 GB, from bartowski/
Questions
Can I run Qwen3.6 27B on an RTX 4080 Super 16GB?
Only at Q2_K: Qwen3.6 27B needs about 19.9 GB at Q4_K_M with 32K tokens of context, 3.92 GB more than the RTX 4080 Super 16GB holds, but 14.6 GB at Q2_K, which fits with 1.38 GB to spare. The smallest setup here that holds Qwen3.6 27B at Q4_K_M with 32K is RTX 3090 (24 GB).
How fast is Qwen3.6 27B on an RTX 4080 Super 16GB?
At Q2_K with 32K tokens of context it writes about 28–39 tokens/s for one request on an RTX 4080 Super 16GB.
What does a second RTX 4080 Super 16GB change for Qwen3.6 27B?
Two RTX 4080 Super 16GB cards (32 GB in one tensor-parallel group) hold Qwen3.6 27B at Q6_K with 32K, and Q4_K_M up to 192K tokens, at about 37–54 tokens/s.
Try other settings in the VRAM calculator or the speed calculator. See also Qwen3.6 27B VRAM requirements, what LLMs an RTX 4080 Super 16GB can run and every pair, or detect your own GPU. Model data checked .