Can I run Qwen3.8 Flash Next on an RTX 3090?
With --n-cpu-moe 37: the UD-Q4_K_XL GGUF keeps 22.8 GB on the RTX 3090 and 55.8 GB in system RAM, with the experts of 37 of its 48 layers moved there. Whole, Qwen3.8 Flash Next needs about 113 GB at Q4_K_M with 32K tokens of context, 88.9 GB more than the 24 GB RTX 3090 holds. The smallest setup here that holds Qwen3.8 Flash Next at Q4_K_M with 32K is DGX Spark (128 GB, 120 GB usable).
With offload the UD-Q4_K_XL GGUF with --n-cpu-moe 37 and 32K tokens of context
- On the card
- 22.8 GB
- RTX 3090
- 24 GB, 936 GB/s
- In RAM
- 55.8 GB
- Tokens/s
- 15–26 tokens/s
With the UD-Q4_K_XL GGUF, --n-cpu-moe 37 and 32K tokens of context it writes about 15–26 tokens/s for one request on an RTX 3090 and dual-channel DDR5-5600.
Qwen3.8 Flash Next on the RTX 3090 as the context fills
One request, FP16 KV cache, 0.5 GB plus 10% overhead; a minus sign is memory missing, and tight is less than 0.5 GB free.
| Context | Q4_K_M | Free | Tokens/s | Q8_0 | Free | Tokens/s |
|---|---|---|---|---|---|---|
| 4K | 112 GB | −88.2 GB | — | 197 GB | −173 GB | — |
| 8K | 112 GB | −88.3 GB | — | 197 GB | −173 GB | — |
| 16K | 112 GB | −88.5 GB | — | 197 GB | −173 GB | — |
| 32K | 113 GB | −88.9 GB | — | 197 GB | −173 GB | — |
| 64K | 114 GB | −89.7 GB | — | 198 GB | −174 GB | — |
| 128K | 115 GB | −91.4 GB | — | 200 GB | −176 GB | — |
| 256K (full) | 119 GB | −94.7 GB | — | 203 GB | −179 GB | — |
--n-cpu-moe for Qwen3.8 Flash Next on the RTX 3090
With --n-cpu-moe 37 the UD-Q4_K_XL GGUF keeps 22.8 GB on the RTX 3090 and 55.8 GB in RAM, about 15–26 tokens/s with DDR5-5600. Measured UD-Q4_K_XL file, 1 GB of buffers; tokens/s by system RAM speed.
Qwen3.8 Flash Next on more than one RTX 3090
| Cards | Best at 32K | Tokens/s | Q4_K_M longest | Q4_K_M tokens/s | Rent per hour |
|---|---|---|---|---|---|
| 2× (48 GB) | Nothing fits | — | Does not fit | — | $0.54 |
| 4× (96 GB) | Q3_K_M | 146–303 | Does not fit | — | $1.08 |
Tensor parallel with the combined bandwidth, as in vLLM; llama.cpp splits layers by default and runs at about one card's speed.
Other options
- The next smaller setting, Q3_K_M, takes 91.5 GB at 32K, 67.5 GB over the RTX 3090; it does not fit with 0.5 GB to spare even with 1K tokens.
- Smallest setup for Q4_K_M at 32K: DGX Spark (128 GB, 120 GB usable).
Run Qwen3.8 Flash Next on the RTX 3090 with llama-server
llama-server -hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q4_K_XL -c 32768 -ngl 99 --n-cpu-moe 37UD-Q4_K_XL/
Questions
Can I run Qwen3.8 Flash Next on an RTX 3090?
With --n-cpu-moe 37: the UD-Q4_K_XL GGUF keeps 22.8 GB on the RTX 3090 and 55.8 GB in system RAM, with the experts of 37 of its 48 layers moved there. Whole, Qwen3.8 Flash Next needs about 113 GB at Q4_K_M with 32K tokens of context, 88.9 GB more than the 24 GB RTX 3090 holds. The smallest setup here that holds Qwen3.8 Flash Next at Q4_K_M with 32K is DGX Spark (128 GB, 120 GB usable).
How fast is Qwen3.8 Flash Next on an RTX 3090?
With the UD-Q4_K_XL GGUF, --n-cpu-moe 37 and 32K tokens of context it writes about 15–26 tokens/s for one request on an RTX 3090 and dual-channel DDR5-5600.
Does Qwen3.8 Flash Next need --n-cpu-moe on an RTX 3090?
With --n-cpu-moe 37 the UD-Q4_K_XL GGUF keeps 22.8 GB on the RTX 3090 and 55.8 GB in RAM, about 15–26 tokens/s with DDR5-5600.
What does a second RTX 3090 change for Qwen3.8 Flash Next?
Even two RTX 3090 cards (48 GB) do not hold Qwen3.8 Flash Next at 32K.
Try other settings in the VRAM calculator, the speed calculator or the MoE offload planner. See also Qwen3.8 Flash Next VRAM requirements, what LLMs an RTX 3090 can run and every pair, or detect your own GPU. Model data checked .