Step 3.7 Flash 196B-A11B VRAM requirements
Step 3.7 Flash 196B-A11B has 201.4B parameters (the 196B in the name is the language model; the checkpoint's parameter count includes weights outside it, such as MTP layers), of which about 11B are used per token; all 288 experts still have to be in memory. With an 8K-token context and one request it needs about 126 GB of GPU memory at Q4_K_M, 207 GB at FP8 and 414 GB at FP16/BF16. The published weights take 375 GB (BF16). It does not fit on a 24 GB card at Q4_K_M; the smallest setup here that holds it with an 8K context is the H200 (141 GB).
126 GB at Q4_K_M, 8K context, one request
- FP8
- 207 GB
- FP16 / BF16
- 414 GB
- Published weights
- 375 GB
- Smallest setup, Q4_K_M
- H200 (141 GB)
Worked example: Step 3.7 Flash 196B-A11B with 32K tokens
Inputs: Step 3.7 Flash 196B-A11B · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (201.4B at Q4_K_M)
- 113 GB
- KV cache (48 KB per token)
- 1.63 GB
- Buffers and runtime (0.5 GB + 10%)
- 12.0 GB
- Total
- 127 GB
The smallest setup here that holds it is the H200 (141 GB), with about 13.9 GB to spare. Change the inputs in the calculator
What makes Step 3.7 Flash 196B-A11B's memory use different
Its 33 sliding-window layers keep only the last 512 tokens (llama.cpp gives them 1,024 cells: the window plus a 512-token batch, rounded up to 256): at its 256K limit they hold 132 MB of cache instead of the 33.0 GB they would need with full attention.
Per token of context it adds 48 KB of FP16 cache; Mistral Medium 3.5 128B (127.7B), the nearest-sized model here with plain full attention, adds 352 KB, so Step 3.7 Flash 196B-A11B needs 14% as much.
As a mixture-of-experts model it reads about 6.20 GB of its 113 GB Q4_K_M weights per generated token (5%), so it writes like a much smaller model while needing memory for all of them.
How much VRAM does Step 3.7 Flash 196B-A11B need?
Step 3.7 Flash 196B-A11B needs 126 GB at Q4_K_M, 220 GB at Q8_0 and 414 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (BF16) | 375 GB | 414 GB | 8× H100 SXM (640 GB) |
| FP16 / BF16 | 375 GB | 414 GB | 8× H100 SXM (640 GB) |
| FP8 / INT8 | 188 GB | 207 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| INT4 (AWQ / GPTQ) | 99.6 GB | 111 GB | DGX Spark (128 GB, 120 GB usable) |
| GGUF Q8_0 | 199 GB | 220 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| GGUF Q6_K | 154 GB | 170 GB | B200 (180 GB) |
| GGUF Q5_K_M | 133 GB | 147 GB | B200 (180 GB) |
| GGUF Q4_K_M | 113 GB | 126 GB | H200 (141 GB) |
| GGUF IQ4_XS | 102 GB | 113 GB | DGX Spark (128 GB, 120 GB usable) |
| GGUF Q3_K_M | 91.7 GB | 102 GB | DGX Spark (128 GB, 120 GB usable) |
| GGUF IQ3_XXS | 77.4 GB | 86.1 GB | Ryzen AI Max+ 395 (128 GB, 96 GB usable) |
| GGUF Q2_K | 78.5 GB | 87.4 GB | Ryzen AI Max+ 395 (128 GB, 96 GB usable) |
Fine-tuning Step 3.7 Flash 196B-A11B? Step 3.7 Flash 196B-A11B VRAM for LoRA, QLoRA and full training.
Run Step 3.7 Flash 196B-A11B with llama.cpp
llama-server -hf unsloth/Step-3.7-Flash-GGUF:Q4_K_M -c 262144 -ngl 99 -np 1UD-Q4_K_M/
Step 3.7 Flash 196B-A11B at 8K, 32K, 128K and 256K (full) tokens of context
12 of its 45 layers use full attention and 33 keep a sliding window of 512 tokens. Each extra token of context adds 48 KB of FP16 cache per request once the sliding windows are full. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.
| Context | KV cache, FP16 | Total at Q4_K_M | Total at Q8_0 | Smallest setup, Q4_K_M |
|---|---|---|---|---|
| 8K tokens | 516 MB | 126 GB | 220 GB | H200 (141 GB) |
| 32K tokens | 1.63 GB | 127 GB | 221 GB | H200 (141 GB) |
| 128K tokens | 6.13 GB | 132 GB | 226 GB | H200 (141 GB) |
| 256K tokens | 12.1 GB | 139 GB | 233 GB | H200 (141 GB) |
How fast Step 3.7 Flash 196B-A11B writes
Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.
| Hardware | Bandwidth | Q4_K_M | FP8 |
|---|---|---|---|
| RTX 3060 12GB | 360 GB/s | — | — |
| RTX 4090 | 1,008 GB/s | — | — |
| RTX 5090 | 1,792 GB/s | — | — |
| M4 Max Mac (128 GB) | 546 GB/s | — | — |
| M3 Ultra Mac Studio (512 GB) | 819 GB/s | 32–55 | 21–35 |
| H100 SXM | 3,350 GB/s | — | — |
| H200 | 4,800 GB/s | 154–286 | — |
Longest context on one GPU
How many tokens of context Step 3.7 Flash 196B-A11B fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.
| GPU | Memory | Q4_K_M | Q8_0 | FP8 |
|---|---|---|---|---|
| RTX 3060 | 12 GB | No | No | No |
| RTX 4060 Ti 16GB | 16 GB | No | No | No |
| RTX 3090 / 4090 | 24 GB | No | No | No |
| RTX 5090 | 32 GB | No | No | No |
| A100 40GB | 40 GB | No | No | No |
| Mac, 64 GB unified memory | 48 GB | No | No | No |
| L40S / RTX 6000 Ada | 48 GB | No | No | No |
| A100 / H100 80GB | 80 GB | No | No | No |
| Mac, 128 GB unified memory | 96 GB | No | No | No |
| H200 | 141 GB | 256K (full) | No | No |
| B200 | 180 GB | 256K (full) | No | No |
Model details
- Parameters
- 201.4B (201,365,316,160); the 196B in the name is the language model; the checkpoint's parameter count includes weights outside it, such as MTP layers
- Experts
- 288 routed experts, all loaded
- Active per token
- 11B
- Layers
- 12 of its 45 layers use full attention and 33 keep a sliding window of 512 tokens
- Attention cache
- 8 KV heads × 128
- Context length
- 262,144 tokens
- Published weights
- 375 GB (BF16)
- On Hugging Face
- stepfun-ai/
Step-3.7-Flash
Why this estimate looks this way
At Q4_K_M and an 8K-token context, Step 3.7 Flash 196B-A11B uses 113 GB for weights, 516 MB for its FP16 KV cache and 11.9 GB for estimated runtime overhead, totaling 126 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
This is a mixture-of-experts model: all 288 experts contribute to the 201.4B parameters held in memory, even though only about 11B parameters run per token. Using only active parameters would understate VRAM.
Its attention layout matters for long context: 12 of its 45 layers use full attention and 33 keep a sliding window of 512 tokens. The FP16 cache grows by about 48 KB per additional token per request once the sliding windows are full.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
Step 3.7 Flash 196B-A11B next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| Step 3.7 Flash 196B-A11B | 201.4B, 11B active | 126 GB | 48 KB | H200 (141 GB) |
| Qwen3.8 Flash Next | 180.0B, 6B active | 112 GB | 24 KB | DGX Spark (128 GB, 120 GB usable) |
| MiniMax M2.7 | 228.7B, 10B active | 144 GB | 248 KB | B200 (180 GB) |
| DeepSeek V4 Flash | 290.9B, 13B active | 182 GB | 86 KB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.