MiMo V2.6 Pro VRAM requirements
MiMo V2.6 Pro needs about 636 GB of VRAM at Q4_K_M with 8K context, so it needs a 640 GB setup such as 8× H100 SXM; a 24 GB card is too small. MiMo V2.6 Pro has 1.02T parameters, of which about 42B are used per token; all 384 experts still have to be in memory. With the same 8K context and one request it needs 1,050 GB at FP8 and 2,100 GB at FP16/BF16. The published weights take 527 GB (4.4 bits/weight). It does not fit on a 24 GB card at Q4_K_M; the smallest setup here that holds it with an 8K context is 8× H100 SXM (640 GB).
636 GB at Q4_K_M, 8K context, one request
- FP8
- 1,050 GB
- FP16 / BF16
- 2,100 GB
- Published weights
- 527 GB
- Smallest setup, Q4_K_M
- 8× H100 SXM (640 GB)
Worked example: MiMo V2.6 Pro with 32K tokens
Inputs: MiMo V2.6 Pro · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (1.02T at Q4_K_M)
- 577 GB
- KV cache (50 KB per token)
- 1.78 GB
- Buffers and runtime (0.5 GB + 10%)
- 58.4 GB
- Total
- 637 GB
The smallest setup here that holds it is 8× H200 (1,128 GB): split across the 8 cards, with the 0.5 GB runtime base on each of them, it takes 641 GB and leaves about 487 GB to spare. Change the inputs in the calculator
What makes MiMo V2.6 Pro's memory use different
Its 60 sliding-window layers keep only the last 128 tokens (llama.cpp gives them 768 cells: the window plus a 512-token batch, rounded up to 256): at its 1M limit they hold 225 MB of cache instead of the 300 GB they would need with full attention.
As a mixture-of-experts model it reads about 23.7 GB of its 577 GB Q4_K_M weights per generated token (4%), so it writes like a much smaller model while needing memory for all of them.
How much VRAM does MiMo V2.6 Pro need?
MiMo V2.6 Pro needs 636 GB at Q4_K_M, 1,116 GB at Q8_0 and 2,100 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (4.4 bits/weight) | 527 GB | 581 GB | 8× H100 SXM (640 GB) |
| FP16 / BF16 | 1,908 GB | 2,100 GB | None listed |
| FP8 / INT8 | 954 GB | 1,050 GB | 8× H200 (1,128 GB) |
| INT4 (AWQ / GPTQ) | 507 GB | 559 GB | 8× H100 SXM (640 GB) |
| GGUF Q8_0 | 1,013 GB | 1,116 GB | 8× H200 (1,128 GB) |
| GGUF Q6_K | 782 GB | 862 GB | 8× H200 (1,128 GB) |
| GGUF Q5_K_M | 676 GB | 745 GB | 8× H200 (1,128 GB) |
| GGUF Q4_K_M | 577 GB | 636 GB | 8× H100 SXM (640 GB) |
| GGUF IQ4_XS | 519 GB | 572 GB | 8× H100 SXM (640 GB) |
| GGUF Q3_K_M | 466 GB | 514 GB | 8× H100 SXM (640 GB) |
| GGUF IQ3_XXS | 393 GB | 434 GB | 8× H100 SXM (640 GB) |
| GGUF Q2_K | 399 GB | 441 GB | 8× H100 SXM (640 GB) |
Fine-tuning MiMo V2.6 Pro? MiMo V2.6 Pro VRAM for LoRA, QLoRA and full training.
MiMo V2.6 Pro at 8K, 32K, 128K and 1M (full) tokens of context
10 of its 70 layers use full attention and 60 keep a sliding window of 128 tokens. Each extra token of context adds 50 KB of FP16 cache per request once the sliding windows are full. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.
| Context | KV cache, FP16 | Total at Q4_K_M | Total at Q8_0 | Smallest setup, Q4_K_M |
|---|---|---|---|---|
| 8K tokens | 625 MB | 636 GB | 1,116 GB | 8× H100 SXM (640 GB) |
| 32K tokens | 1.78 GB | 637 GB | 1,117 GB | 8× H200 (1,128 GB) |
| 128K tokens | 6.47 GB | 642 GB | 1,122 GB | 8× H200 (1,128 GB) |
| 1M tokens | 50.2 GB | 691 GB | 1,171 GB | 8× H200 (1,128 GB) |
How fast MiMo V2.6 Pro writes on a multi-GPU group
Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. No single card or machine holds it at Q4_K_M or FP8 (the precisions below), so these rows are the GPU groups here that do, in tensor parallel with their combined bandwidth. A dash means it does not fit at that precision. Try other settings in the speed calculator.
| Hardware | Bandwidth | Q4_K_M | FP8 |
|---|---|---|---|
| 8× H100 SXM | 26,800 GB/s | 148–308 | — |
| 8× H200 | 38,400 GB/s | 174–376 | 139–284 |
Longest context on a multi-GPU group
No single card or machine holds it at Q4_K_M, Q8_0 or FP8 (the precisions below), so these rows use GPU groups: how many tokens of context MiMo V2.6 Pro fits on each, in one tensor-parallel group with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.
| Group | Memory | Q4_K_M | Q8_0 | FP8 |
|---|---|---|---|---|
| 2× RTX 3060 12GB | 24 GB | No | No | No |
| 2× RTX 3090 | 48 GB | No | No | No |
| 2× RTX 4090 | 48 GB | No | No | No |
| 2× RTX 5090 | 64 GB | No | No | No |
| 4× RTX 3090 | 96 GB | No | No | No |
| 8× H100 SXM | 640 GB | 8K | No | No |
| 8× H200 | 1,128 GB | 1M (full) | 156K | 1M (full) |
Model details
- Parameters
- 1.02T (1,024,216,603,392)
- Experts
- 384 routed experts, all loaded
- Active per token
- 42B
- Layers
- 10 of its 70 layers use full attention and 60 keep a sliding window of 128 tokens
- Attention cache
- 8 KV heads × 192 keys, 128 values
- Sliding-window layers
- 8 KV heads × 192
- Context length
- 1,048,576 tokens
- Published weights
- 527 GB (4.4 bits/weight)
- On Hugging Face
- XiaomiMiMo/
MiMo-V2.6-Pro-RL - Reported benchmarks
- None published in the model card for GPQA, MMLU-Pro or LiveCodeBench.
Why this estimate looks this way
At Q4_K_M and an 8K-token context, MiMo V2.6 Pro uses 577 GB for weights, 625 MB for its FP16 KV cache and 58.3 GB for estimated runtime overhead, totaling 636 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
This is a mixture-of-experts model: all 384 experts contribute to the 1.02T parameters held in memory, even though only about 42B parameters run per token. Using only active parameters would understate VRAM.
Its attention layout matters for long context: 10 of its 70 layers use full attention and 60 keep a sliding window of 128 tokens. The FP16 cache grows by about 50 KB per additional token per request once the sliding windows are full.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
MiMo V2.6 Pro next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| MiMo V2.6 Pro | 1.02T, 42B active | 636 GB | 50 KB | 8× H100 SXM (640 GB) |
| Hy4 Preview 770B-A49B | 780.0B, 49B active | 485 GB | 90 KB | 8× H100 SXM (640 GB) |
| DeepSeek V4.1 Flash 1017 | 763.2B, 16B active | 474 GB | 80 KB | 8× H100 SXM (640 GB) |
| DeepSeek V4.1 Flash | 763.2B, 16B active | 474 GB | 80 KB | 8× H100 SXM (640 GB) |
MiMo V2.6 Pro VRAM questions
How much VRAM do I need to run MiMo V2.6 Pro locally?
About 636 GB at Q4_K_M with an 8K context and one request, by the bits-per-weight estimate, and 2,100 GB at BF16. The smallest setup listed here that holds Q4_K_M is the 8× H100 SXM (640 GB).
How much more VRAM does MiMo V2.6 Pro need for 32K or 128K tokens of context?
Its FP16 KV cache is 625 MB at 8K, 1.78 GB at 32K, 6.47 GB at 128K for one request, so 128K adds 5.86 GB over 8K, plus 10% overhead. A q8_0 cache
Does MiMo V2.6 Pro need less VRAM because only 42B parameters are active?
No. All 384 experts stay loaded, so the Q4_K_M weights take 577 GB. The active parameters decide speed: each generated token reads about 23.7 GB of them.
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.