Kimi K3 VRAM requirements
Kimi K3 needs about 1,724 GB of VRAM at Q4_K_M with 8K context, more than any GPU setup listed here holds. Kimi K3 has 2.78T parameters, of which about 104B are used per token; all 896 experts still have to be in memory. With the same 8K context and one request it needs 2,849 GB at FP8 and 5,697 GB at FP16/BF16. The published weights take 1,454 GB (4.5 bits/weight). No GPU or multi-GPU group listed here holds it at Q4_K_M with an 8K context.
1,724 GB at Q4_K_M, 8K context, one request
- FP8
- 2,849 GB
- FP16 / BF16
- 5,697 GB
- Published weights
- 1,454 GB
- Smallest setup, Q4_K_M
- None listed
Worked example: Kimi K3 with 32K tokens
Inputs: Kimi K3 · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (2.78T at Q4_K_M)
- 1,566 GB
- KV cache (27 KB per token)
- 864 MB
- Buffers and runtime (0.5 GB + 10%)
- 157 GB
- Total
- 1,724 GB
No GPU or group listed here holds it at Q4_K_M with 32K tokens of context. Change the inputs in the calculator
What makes Kimi K3's memory use different
It uses multi-head latent attention: each MLA layer caches a latent of 576 values per token instead of 30,720 for 96 full key and value heads, 53.3× less.
69 of its 93 layers keep a fixed-size linear-attention state, so the cache grows by 27 KB per token where caching every layer would take 105 KB.
As a mixture-of-experts model it reads about 58.6 GB of its 1,566 GB Q4_K_M weights per generated token (4%), so it writes like a much smaller model while needing memory for all of them.
How much VRAM does Kimi K3 need?
Kimi K3 needs 1,724 GB at Q4_K_M, 3,027 GB at Q8_0 and 5,697 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (4.5 bits/weight) | 1,454 GB | 1,600 GB | None listed |
| FP16 / BF16 | 5,178 GB | 5,697 GB | None listed |
| FP8 / INT8 | 2,589 GB | 2,849 GB | None listed |
| INT4 (AWQ / GPTQ) | 1,375 GB | 1,514 GB | None listed |
| GGUF Q8_0 | 2,751 GB | 3,027 GB | None listed |
| GGUF Q6_K | 2,123 GB | 2,336 GB | None listed |
| GGUF Q5_K_M | 1,835 GB | 2,019 GB | None listed |
| GGUF Q4_K_M | 1,566 GB | 1,724 GB | None listed |
| GGUF IQ4_XS | 1,408 GB | 1,549 GB | None listed |
| GGUF Q3_K_M | 1,265 GB | 1,393 GB | None listed |
| GGUF IQ3_XXS | 1,068 GB | 1,175 GB | None listed |
| GGUF Q2_K | 1,084 GB | 1,193 GB | None listed |
Fine-tuning Kimi K3? Kimi K3 VRAM for LoRA, QLoRA and full training.
Kimi K3 at 8K, 32K, 128K and 1M (full) tokens of context
24 of its 93 layers use multi-head latent attention and 69 are linear-attention layers with no growing cache. Each extra token of context adds 27 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.
| Context | KV cache, FP16 | Total at Q4_K_M | Total at Q8_0 | Smallest setup, Q4_K_M |
|---|---|---|---|---|
| 8K tokens | 216 MB | 1,724 GB | 3,027 GB | None listed |
| 32K tokens | 864 MB | 1,724 GB | 3,027 GB | None listed |
| 128K tokens | 3.38 GB | 1,727 GB | 3,030 GB | None listed |
| 1M tokens | 27.0 GB | 1,753 GB | 3,056 GB | None listed |
Model details
- Parameters
- 2.78T (2,779,931,837,184)
- Experts
- 896 routed experts, all loaded
- Active per token
- 104B
- Layers
- 24 of its 93 layers use multi-head latent attention and 69 are linear-attention layers with no growing cache
- Attention cache
- 576 values per token (compressed latent)
- Context length
- 1,048,576 tokens
- Published weights
- 1,454 GB (4.5 bits/weight)
- On Hugging Face
- moonshotai/Kimi-K3
- Reported benchmarks
- GPQA Diamond 93.5 (max reasoning effort). Vendor-reported: from the maker’s own model card, not rerun here; read 2026-09-30.
Why this estimate looks this way
At Q4_K_M and an 8K-token context, Kimi K3 uses 1,566 GB for weights, 216 MB for its FP16 KV cache and 157 GB for estimated runtime overhead, totaling 1,724 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
This is a mixture-of-experts model: all 896 experts contribute to the 2.78T parameters held in memory, even though only about 104B parameters run per token. Using only active parameters would understate VRAM.
Its attention layout matters for long context: 24 of its 93 layers use multi-head latent attention and 69 are linear-attention layers with no growing cache. The FP16 cache grows by about 27 KB per additional token per request.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
Kimi K3 next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| Kimi K3 | 2.78T, 104B active | 1,724 GB | 27 KB | None listed |
| Qwen3.8 2.4T-A95B | 2.45T, 95B active | 1,517 GB | 92 KB | None listed |
| DeepSeek V4 Pro | 1.60T, 49B active | 993 GB | 122 KB | 8× H200 (1,128 GB) |
| MiMo V2.6 Pro | 1.02T, 42B active | 636 GB | 50 KB | 8× H100 SXM (640 GB) |
Kimi K3 VRAM questions
How much VRAM do I need to run Kimi K3 locally?
About 1,724 GB at Q4_K_M with an 8K context and one request, by the bits-per-weight estimate, and 5,697 GB at BF16. No single GPU or group listed here holds it at Q4_K_M.
How much more VRAM does Kimi K3 need for 32K or 128K tokens of context?
Its FP16 KV cache is 216 MB at 8K, 864 MB at 32K, 3.38 GB at 128K for one request, so 128K adds 3.16 GB over 8K, plus 10% overhead. A q8_0 cache
Does Kimi K3 need less VRAM because only 104B parameters are active?
No. All 896 experts stay loaded, so the Q4_K_M weights take 1,566 GB. The active parameters decide speed: each generated token reads about 58.6 GB of them.
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.