Qwen3.8 Flash Next VRAM requirements
Qwen3.8 Flash Next needs about 112 GB of VRAM at Q4_K_M with 8K context, so it needs a 141 GB card such as the H200; a 24 GB card is too small. Qwen3.8 Flash Next has 180.0B parameters, of which about 6B are used per token; all 512 experts still have to be in memory. With the same 8K context and one request it needs 185 GB at FP8 and 370 GB at FP16/BF16. The published weights take 335 GB (BF16). It does not fit on a 24 GB card at Q4_K_M; the smallest setup here that holds it with an 8K context is the DGX Spark (128 GB, 120 GB usable).
112 GB at Q4_K_M, 8K context, one request
- FP8
- 185 GB
- FP16 / BF16
- 370 GB
- Published weights
- 335 GB
- Smallest setup, Q4_K_M
- DGX Spark (128 GB, 120 GB usable)
Worked example: Qwen3.8 Flash Next with 32K tokens
Inputs: Qwen3.8 Flash Next · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (180.0B at Q4_K_M)
- 101 GB
- KV cache (24 KB per token)
- 768 MB
- Buffers and runtime (0.5 GB + 10%)
- 10.7 GB
- Total
- 113 GB
The smallest setup here that holds it is the DGX Spark (128 GB, 120 GB usable), with about 7.11 GB to spare. Change the inputs in the calculator
Can I run Qwen3.8 Flash Next on…
- RTX 4090: --n-cpu-moe 37
- RTX 3090: --n-cpu-moe 37
- RTX 5090: --n-cpu-moe 31
- RTX 3060 12GB: --n-cpu-moe 45
- RTX 4060 Ti 16GB: --n-cpu-moe 42
- RTX 5060 Ti 16GB: --n-cpu-moe 42
- RTX 4070 12GB: --n-cpu-moe 45
- RTX 5080 16GB: --n-cpu-moe 42
- RTX 4080 Super 16GB: --n-cpu-moe 42
- RX 7900 XTX: --n-cpu-moe 37
- M4 Max Mac (128 GB): Only Q3_K_M
- H100 SXM: Only Q2_K
What makes Qwen3.8 Flash Next's memory use different
36 of its 48 layers keep a fixed-size linear-attention state, so the cache grows by 24 KB per token where caching every layer would take 96 KB.
Per token of context it adds 24 KB of FP16 cache; Mistral Medium 3.5 128B (127.7B), the nearest-sized model here with plain full attention, adds 352 KB, so Qwen3.8 Flash Next needs 7% as much.
As a mixture-of-experts model it reads about 3.38 GB of its 101 GB Q4_K_M weights per generated token (3%), so it writes like a much smaller model while needing memory for all of them.
Running Qwen3.8 Flash Next with experts in system RAM
llama.cpp's --n-cpu-moe N keeps the experts of the first N layers in RAM. For the measured
UD-Q4_K_XL GGUF with 32K tokens of context, the smallest N
that fits each card, and the speed with 89.6 GB/s of system RAM; its 26.8 GB n-gram table is read from the mapped file.
Plan the offload for another GPU.
The VRAM here can sit below the 113 GB of the worked example because it counts the file's measured
bytes, keeps the input embedding (token_embd) in system RAM as llama.cpp does and adds a flat 1 GB of buffers, where the
estimate above adds 0.5 GB plus 10% to the preset's weights.
| GPU | --n-cpu-moe | VRAM used | In system RAM | Tokens/s |
|---|---|---|---|---|
| RTX 5060 Ti 16GB 16 GB | 42 | 15.5 GB | 63.1 GB | 11–18 |
| RTX 4090 24 GB | 37 | 22.8 GB | 55.8 GB | 16–26 |
| RTX 5090 32 GB | 31 | 31.6 GB | 47.0 GB | 20–34 |
In system RAM counts the offloaded experts plus the input embedding (token_embd), which stays in RAM even with --n-cpu-moe 0.
How much VRAM does Qwen3.8 Flash Next need?
Qwen3.8 Flash Next needs 112 GB at Q4_K_M, 197 GB at Q8_0 and 370 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (BF16) | 335 GB | 370 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| FP16 / BF16 | 335 GB | 370 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| FP8 / INT8 | 168 GB | 185 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| INT4 (AWQ / GPTQ) | 89.1 GB | 98.7 GB | DGX Spark (128 GB, 120 GB usable) |
| GGUF Q8_0 | 178 GB | 197 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| GGUF Q6_K | 137 GB | 152 GB | B200 (180 GB) |
| GGUF Q5_K_M | 119 GB | 131 GB | H200 (141 GB) |
| GGUF Q4_K_M | 101 GB | 112 GB | DGX Spark (128 GB, 120 GB usable) |
| GGUF IQ4_XS | 91.2 GB | 101 GB | DGX Spark (128 GB, 120 GB usable) |
| GGUF Q3_K_M | 81.9 GB | 90.8 GB | Ryzen AI Max+ 395 (128 GB, 96 GB usable) |
| GGUF IQ3_XXS | 69.2 GB | 76.8 GB | A100 80GB |
| GGUF Q2_K | 70.2 GB | 77.9 GB | A100 80GB |
Fine-tuning Qwen3.8 Flash Next? Qwen3.8 Flash Next VRAM for LoRA, QLoRA and full training.
Qwen3.8 Flash Next GGUF files on Hugging Face
The 8 GGUF files llama.cpp picks for Qwen3.8 Flash Next with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.
| Quant | File | Size | vs estimate | 8K | 32K | 128K |
|---|---|---|---|---|---|---|
| Q2_K | bartowski/ | 75.4 GB80,926,779,072 bytes | +7% | 83.6 GB | 84.2 GB | 86.7 GB |
| Q3_K_M | bartowski/ | 85.6 GB91,957,778,112 bytes | +5% | 94.9 GB | 95.5 GB | 98.0 GB |
| IQ4_XS | unsloth/ | 87.2 GB93,682,584,224 bytes | −4% | 96.7 GB | 97.3 GB | 99.8 GB |
| Q4_K_M | bartowski/ | 111 GB119,599,030,080 bytes | +10% | 123 GB | 124 GB | 126 GB |
| Q5_K_M | bartowski/ | 125 GB134,668,639,040 bytes | +6% | 139 GB | 139 GB | 142 GB |
| Q8_0 | ggml-org/ | 151 GB162,624,826,656 bytes | −15% | 167 GB | 168 GB | 170 GB |
| Q6_K | bartowski/ | 157 GB168,113,805,248 bytes | +14% | 173 GB | 174 GB | 176 GB |
| BF16 | unsloth/ | 330 GB354,029,930,496 bytes | −2% | 363 GB | 364 GB | 366 GB |
GB here is GiB (1024³ bytes), as everywhere on this site; Hugging Face shows the Q4_K_M file as 119.6 GB, in 10⁹ bytes.
Which Qwen3.8 Flash Next GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac
From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for Q4_K_M (111 GB). One request, FP16 KV cache. Where nothing fits, the --n-cpu-moe table above keeps experts in system RAM.
| Hardware | Largest GGUF, 8K | Its longest context | Q4_K_M, longest context |
|---|---|---|---|
| RTX 5060 Ti 16GB | None fits | — | No |
| RTX 4090 (24 GB) | None fits | — | No |
| RTX 5090 (32 GB) | None fits | — | No |
| L40S (48 GB) | None fits | — | No |
| H100 SXM (80 GB) | None fits | — | No |
| M4 Pro Mac (64 GB, 48 GB usable) | None fits | — | No |
| M4 Max Mac (128 GB, 96 GB usable) | Q3_K_M, 85.6 GB | 30K | No |
Run Qwen3.8 Flash Next with llama.cpp
llama-server -hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q4_K_XL -c 32768 --n-cpu-moe 37UD-Q4_K_XL/
Qwen3.8 Flash Next at 8K, 32K, 128K and 256K (full) tokens of context
12 of its 48 layers use full attention and 36 are linear-attention layers with no growing cache. Each extra token of context adds 24 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.
| Context | KV cache, FP16 | Total at Q4_K_M | Total at Q8_0 | Smallest setup, Q4_K_M |
|---|---|---|---|---|
| 8K tokens | 192 MB | 112 GB | 197 GB | DGX Spark (128 GB, 120 GB usable) |
| 32K tokens | 768 MB | 113 GB | 197 GB | DGX Spark (128 GB, 120 GB usable) |
| 128K tokens | 3.00 GB | 115 GB | 200 GB | DGX Spark (128 GB, 120 GB usable) |
| 256K tokens | 6.00 GB | 119 GB | 203 GB | DGX Spark (128 GB, 120 GB usable) |
How fast Qwen3.8 Flash Next writes
Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.
| Hardware | Bandwidth | Q4_K_M | FP8 |
|---|---|---|---|
| RTX 3060 12GB | 360 GB/s | — | — |
| RTX 4090 | 1,008 GB/s | — | — |
| RTX 5090 | 1,792 GB/s | — | — |
| M4 Max Mac (128 GB) | 546 GB/s | — | — |
| M3 Ultra Mac Studio (512 GB) | 819 GB/s | 59–101 | 37–64 |
| H100 SXM | 3,350 GB/s | — | — |
| H200 | 4,800 GB/s | 240–477 | — |
Longest context on one GPU
How many tokens of context Qwen3.8 Flash Next fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.
| GPU | Memory | Q4_K_M | Q8_0 | FP8 |
|---|---|---|---|---|
| RTX 3060 | 12 GB | No | No | No |
| RTX 4060 Ti 16GB | 16 GB | No | No | No |
| RTX 3090 / 4090 | 24 GB | No | No | No |
| RTX 5090 | 32 GB | No | No | No |
| A100 40GB | 40 GB | No | No | No |
| Mac, 64 GB unified memory | 48 GB | No | No | No |
| L40S / RTX 6000 Ada | 48 GB | No | No | No |
| A100 / H100 80GB | 80 GB | No | No | No |
| Mac, 128 GB unified memory | 96 GB | No | No | No |
| H200 | 141 GB | 256K (full) | No | No |
| B200 | 180 GB | 256K (full) | No | No |
Model details
- Parameters
- 180.0B (179,999,981,459)
- Experts
- 512 routed experts, all loaded
- Active per token
- 6B
- Layers
- 12 of its 48 layers use full attention and 36 are linear-attention layers with no growing cache
- Attention cache
- 2 KV heads × 256
- Context length
- 262,144 tokens
- Published weights
- 335 GB (BF16)
- On Hugging Face
- Qwen/
Qwen3.8-Flash-Next
Why this estimate looks this way
At Q4_K_M and an 8K-token context, Qwen3.8 Flash Next uses 101 GB for weights, 192 MB for its FP16 KV cache and 10.7 GB for estimated runtime overhead, totaling 112 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
This is a mixture-of-experts model: all 512 experts contribute to the 180.0B parameters held in memory, even though only about 6B parameters run per token. Using only active parameters would understate VRAM.
Its attention layout matters for long context: 12 of its 48 layers use full attention and 36 are linear-attention layers with no growing cache. The FP16 cache grows by about 24 KB per additional token per request.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
Qwen3.8 Flash Next next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| Qwen3.8 Flash Next | 180.0B, 6B active | 112 GB | 24 KB | DGX Spark (128 GB, 120 GB usable) |
| Step 3.7 Flash 196B-A11B | 201.4B, 11B active | 126 GB | 48 KB | H200 (141 GB) |
| MiniMax M2.7 | 228.7B, 10B active | 144 GB | 248 KB | B200 (180 GB) |
| Mistral Medium 3.5 128B | 127.7B | 82.7 GB | 352 KB | Ryzen AI Max+ 395 (128 GB, 96 GB usable) |
Qwen3.8 Flash Next VRAM questions
How much VRAM do I need to run Qwen3.8 Flash Next locally?
About 112 GB at Q4_K_M with an 8K context and one request, by the bits-per-weight estimate, and 370 GB at BF16. The Q4_K_M GGUF Qwen3.8-Flash-Next-Q4_
Can I run Qwen3.8 Flash Next on a 24 GB GPU?
Not entirely in VRAM. The RTX 4090 (24 GB) alone cannot hold it: the smallest listed GGUF, Q2_K at 75.4 GB, needs 83.6 GB with an 8K context. With llama.cpp's --n-cpu-moe 37 and the UD-Q4_K_XL GGUF it uses 22.8 GB on the card and 55.8 GB of system RAM at 32K tokens.
Which quantization of Qwen3.8 Flash Next fits in 16 GB of VRAM?
The RTX 5060 Ti 16GB alone cannot hold it: the smallest listed GGUF, Q2_K at 75.4 GB, needs 83.6 GB with an 8K context. With llama.cpp's --n-cpu-moe 42 and the UD-Q4_K_XL GGUF it uses 15.5 GB on the card and 63.1 GB of system RAM at 32K tokens.
Can I run Qwen3.8 Flash Next on a Mac?
The M4 Pro Mac (64 GB, 48 GB usable) alone cannot hold it: the smallest listed GGUF, Q2_K at 75.4 GB, needs 83.6 GB with an 8K context. On the M4 Max Mac (128 GB, 96 GB usable), the largest listed GGUF that fits with an 8K context and 0.5 GB free is Q3_K_M (85.6 GB), with up to 30K tokens of context. By default macOS lets the GPU use about 75% of unified memory.
How much more VRAM does Qwen3.8 Flash Next need for 32K or 128K tokens of context?
Its FP16 KV cache is 192 MB at 8K, 768 MB at 32K, 3.00 GB at 128K for one request, so 128K adds 2.81 GB over 8K, plus 10% overhead. A q8_0 cache
Why does Qwen3.8 Flash Next use more VRAM than its GGUF file size?
The file holds only the weights. At Q4_K_M with an 8K context: 111 GB of weights (the file's 119,599,030,080 bytes) + 192 MB of FP16 KV cache + 11.7 GB of compute buffers and runtime (0.5 GB + 10%) = 123 GB.
Does Qwen3.8 Flash Next need less VRAM because only 6B parameters are active?
No. All 512 experts stay loaded, so the Q4_K_M file takes 111 GB. The active parameters decide speed: each generated token reads about 3.38 GB of them. With llama.cpp's --n-cpu-moe 37 and the UD-Q4_K_XL GGUF it uses 22.8 GB on the card and 55.8 GB of system RAM at 32K tokens.
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.