LLM VRAM Calculator: how much GPU memory does your model need?
Estimate VRAM, tokens per second and MoE offload splits for any model on Hugging Face, from its real config and weight files.
Advanced: the model numbers and overhead used
Editing any of these switches the model to Custom.
– estimated GPU memory
- Weights
- –
- KV cache
- –
- Runtime overhead
- –
- KV cache per token
- –
Displayed GB, MB and KB denote GiB, MiB and KiB (powers of 1,024).
Which GPUs fit the memory estimate
| GPU | Memory | Result |
|---|---|---|
| RTX 3060 | 12 GB | – |
| RTX 4060 Ti 16GB | 16 GB | – |
| RTX 3090 / 4090 | 24 GB | – |
| RTX 5090 | 32 GB | – |
| A100 40GB | 40 GB | – |
| Mac, 64 GB unified memory about 75% of it is usable by the GPU by default | 48 GB | – |
| L40S / RTX 6000 Ada | 48 GB | – |
| A100 / H100 80GB | 80 GB | – |
| Mac, 128 GB unified memory about 75% of it is usable by the GPU by default | 96 GB | – |
| H200 | 141 GB | – |
| B200 | 180 GB | – |
Badge for your model card
A static VRAM badge for the selected model at 8K context, for a Hugging Face or GitHub README. It links to the model’s page here. How the badges work.
Badges exist for the preset models in the list; pick one of them.
Fits? Then see how many tokens per second it will write.
Training instead of running it? Estimate VRAM for full fine-tuning, LoRA and QLoRA.
VRAM for popular models
Total GPU memory with 8,192 tokens of context, one request and an FP16 KV cache. Each model name leads to its own page, with every quantization, the cache at long context and the GPUs that fit.
| Model | Parameters | As published | Q8_0 | Q4_K_M | Q4_K_M fits on |
|---|---|---|---|---|---|
| Gemma 4 26B-A4B (MoE) | 25.8B | 53.9 GB (BF16) | 29.1 GB | 17.0 GB | RTX 3090 |
| Gemma 4 31B | 31.3B | 66.6 GB (BF16) | 36.5 GB | 21.9 GB | RTX 3090 |
| Qwen3.8 27B | 27.8B | 58.0 GB (BF16) | 31.3 GB | 18.3 GB | RTX 3090 |
| gpt-oss-20b | 20.9B | 14.8 GB (MXFP4) | 23.5 GB | 13.7 GB | RTX 4060 Ti 16GB |
| Llama 3.1 8B | 8.0B | 18.1 GB (BF16) | 10.3 GB | 6.58 GB | RTX 4060 8GB |
| gpt-oss-120b | 116.8B | 67.7 GB (MXFP4) | 128 GB | 73.2 GB | A100 80GB |
| Qwen3.6 35B-A3B (MoE) | 36.0B | 74.3 GB (BF16) | 39.8 GB | 23.0 GB | RTX 3090 |
| Qwen3.6 27B | 27.8B | 58.0 GB (BF16) | 31.3 GB | 18.3 GB | RTX 3090 |
| Gemma 4 12B | 12.0B | 25.7 GB (BF16) | 14.2 GB | 8.57 GB | RTX 3060 12GB |
| GLM-4.7 Flash | 31.2B | 64.9 GB (BF16) | 34.9 GB | 20.3 GB | RTX 3090 |
| Qwen3.8 Flash Next (180B MoE) | 180.0B | 370 GB (BF16) | 197 GB | 112 GB | DGX Spark (128 GB) |
| Nemotron 3 Nano 30B-A3B (MoE) | 31.6B | 65.3 GB (BF16) | 34.9 GB | 20.1 GB | RTX 3090 |
Announced, weights not out yet: Mistral Large 4 (1.05T), estimated from Mistral's figures until the weights land on October 31.
Can I run a given model on a given GPU? One answer per popular model and card, with the quant, offload or card count it takes.
Serving with vLLM? The vLLM KV cache calculator works out the pool vLLM allocates at startup, in tokens, how many requests fit at once, and the vllm serve command.
See the best local LLMs for 8 to 96 GB of VRAM (for example 24 GB), or compare models and GPU memory in the reproducible report. The numbers are an open dataset (CSV and JSON, CC BY 4.0). Every precision is also in a CSV file on GitHub, free to reuse. Not sure which GGUF type to pick? See GGUF quantization explained, or, for Qwen3.8 27B on 8–16 GB, Ternary Bonsai 2 vs GSQ-RCO vs UD quants. How these estimates compare with real runs: predicted vs measured; measured one yourself? Submit a measurement with the log lines.
Free · No sign-up · Runs in your browser
- models with their own page
- 69
- GPU and Mac setups checked
- 73
- weight precisions
- 13
Let the browser detect your GPU and list what your PC can run. See which local LLMs fit 8 to 96 GB of VRAM, whether a model runs on your GPU, LLM VRAM statistics, or download the open VRAM dataset (CSV and JSON). How close the estimates come to real runs: predicted vs measured. Explainers on usable VRAM, Macs, several GPUs, the KV cache and MoE offload: all guides.
Best local LLMs by VRAM: 8 GB, 12 GB, 16 GB, 24 GB, 32 GB, 48 GB, 96 GB
Calculators
Focused tools that share the same formulas and model data.
- LLM Speed Calculator Tokens per second on your GPU or Mac, from memory bandwidth and active parameters. Open →
- Fine-Tuning VRAM Calculator GPU memory for full fine-tuning, LoRA and QLoRA in Transformers or Unsloth, checked against published runs. Open →
- Qwen-Image-2.1 VRAM Calculator Peak VRAM for every DiT, text encoder and VAE combination, with real measurements. Open →
- Image & Video Model VRAM Calculator Peak VRAM and system RAM for FLUX.2, Wan 2.2, LTX-2 and Qwen-Image in ComfyUI, from exact file sizes, checked against public runs. Open →
- MiniMax H3 VRAM Calculator
(ComfyUI) VRAM and system RAM for MiniMax H3 video in ComfyUI: pruned, INT8, NVFP4 and GGUF files on 8–96 GB GPUs. Open → - Speculative Decoding VRAM Calculator (MTP & draft models) Extra VRAM of MTP heads, DFlash drafters and draft models in llama.cpp, for Qwen3.8 27B and Gemma 4 31B. Open →
- MoE Offload Calculator
(--n-cpu-moe) The smallest llama.cpp --n-cpu-moe that fits your GPU, from real GGUF tensor sizes. Open → - Qwen3.8 27B GGUF Quants: Bonsai 2 vs GSQ-RCO vs UD Ternary Bonsai 2, GSQ-RCO and Unsloth UD files of Qwen3.8 27B: VRAM, quality, speed and the engine each needs. Open →
- vLLM KV Cache & Concurrency Calculator The KV cache pool vLLM allocates, in tokens, and how many requests fit at once, with the vllm serve command. Open →
- FLUX 3 Image Klein VRAM Calculator Estimate FLUX 3 Image Klein VRAM by build, CPU offload, prompt upsampler and working-memory budget. Compare INT8, FP8, NVFP4 and BF16 files. Open →
- d1-omni VRAM LiquidAI d1-omni weights: 1.09 GiB in FP16 on GPU, 2.19 GiB in FP32 on CPU. Text, image and audio decisions, input limits, license and official Python usage. Open →
- pplx-decider v1.1 VRAM pplx-decider v1.1 needs 48.6 GiB for its BF16 checkpoint before working memory. GPU fit, decision head, noncausal attention and the official Python loader. Open →
- Bonsai 2 27B VRAM Exact file sizes of Ternary Bonsai 2 27B, the longest context on 8–24 GB GPUs, and why sites say 5.95 GB or 8.60 GB. Open →
- Jev Alternatives You Can Run Locally Open models that replace the API-only Jev on your own machine: Laya, zero-shot encoders and small LLMs, with sizes and VRAM. Open →
- Laya VRAM Requirements Real file sizes and run-time memory of the three Laya decision-encoder checkpoints, and three ways to run them. Open →
- What Can My PC Run? Detects your GPU in the browser and lists the local LLMs it runs, with the best quantization and speed. Open →