LLM VRAM by GPU: every card, Mac and CPU-only PC
73 setups, each with its own page: 24 graphics cards and unified-memory machines, 7 multi-GPU groups, 34 Macs and 8 CPU-only PCs. “Fit at Q4_K_M” counts how many of the 64 models on this site fit at Q4_K_M (MXFP4 for gpt-oss) with 8,192 tokens of context and 0.5 GB to spare; “Largest that fits” is the biggest of them by parameter count, at the best precision that fits. Open a setup for every model’s precision, longest context and speed.
Jump to: Graphics cards (24) · Multi-GPU setups (7) · Mac (34) · CPU-only PCs (8)
Graphics cards
| Setup | Memory | Bandwidth | Fit at Q4_K_M (of 64) | Largest that fits |
|---|---|---|---|---|
| RTX 4060 8GB | 8 GB | 272 GB/s | 18 | Gemma 4 12B at GGUF Q3_K_M |
| RTX 3060 12GB | 12 GB | 360 GB/s | 19 | Gemma 4 12B at GGUF Q6_K |
| Arc B580 12GB | 12 GB | 456 GB/s | 19 | Gemma 4 12B at GGUF Q6_K |
| RTX 4070 12GB | 12 GB | 504 GB/s | 19 | Gemma 4 12B at GGUF Q6_K |
| RTX 5070 12GB | 12 GB | 672 GB/s | 19 | Gemma 4 12B at GGUF Q6_K |
| RTX 4060 Ti 16GB | 16 GB | 288 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| RTX 5060 Ti 16GB | 16 GB | 448 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| RX 9070 XT 16GB | 16 GB | 644 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| RTX 4070 Ti Super 16GB | 16 GB | 672 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| RTX 4080 Super 16GB | 16 GB | 736 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| RTX 5070 Ti 16GB | 16 GB | 896 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| RTX 5080 16GB | 16 GB | 960 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| RTX 3090 | 24 GB | 936 GB/s | 37 | K2-Horizon MoVA 36B-A4B at GGUF Q3_K_M |
| RX 7900 XTX | 24 GB | 960 GB/s | 37 | K2-Horizon MoVA 36B-A4B at GGUF Q3_K_M |
| RTX 4090 | 24 GB | 1,008 GB/s | 37 | K2-Horizon MoVA 36B-A4B at GGUF Q3_K_M |
| RTX 5090 | 32 GB | 1,792 GB/s | 38 | K2-Horizon MoVA 36B-A4B at GGUF Q5_K_M |
| L40S | 48 GB | 864 GB/s | 39 | AliceAI Foundation 80B-A3B at GGUF Q3_K_M |
| A100 80GB | 80 GB | 2,039 GB/s | 44 | Qwen3.8 Flash Next at GGUF Q2_K |
| H100 SXM | 80 GB | 3,350 GB/s | 44 | Qwen3.8 Flash Next at GGUF Q2_K |
| Ryzen AI Max+ 395 (128 GB) | 96 GB usable | 256 GB/s | 45 | Step 3.7 Flash 196B-A11B at GGUF Q2_K |
| RTX PRO 6000 Blackwell | 96 GB | 1,792 GB/s | 45 | Step 3.7 Flash 196B-A11B at GGUF Q2_K |
| DGX Spark (128 GB) | 120 GB usable | 273 GB/s | 46 | MiniMax M2.7 at GGUF Q3_K_M |
| H200 | 141 GB | 4,800 GB/s | 47 | GLM-5.3 Flash at GGUF Q2_K |
| B200 | 180 GB | 8,000 GB/s | 48 | GLM-5.3 Flash at GGUF Q3_K_M |
Multi-GPU setups
| Setup | Memory | Bandwidth | Fit at Q4_K_M (of 64) | Largest that fits |
|---|---|---|---|---|
| 2× RTX 3060 12GB | 24 GB | 720 GB/s | 37 | K2-Horizon MoVA 36B-A4B at GGUF Q3_K_M |
| 2× RTX 3090 | 48 GB | 1,872 GB/s | 39 | AliceAI Foundation 80B-A3B at GGUF Q3_K_M |
| 2× RTX 4090 | 48 GB | 2,016 GB/s | 39 | AliceAI Foundation 80B-A3B at GGUF Q3_K_M |
| 2× RTX 5090 | 64 GB | 3,584 GB/s | 41 | Mistral Medium 3.5 128B at GGUF Q2_K |
| 4× RTX 3090 | 96 GB | 3,744 GB/s | 45 | Step 3.7 Flash 196B-A11B at GGUF Q2_K |
| 8× H100 SXM | 640 GB | 26,800 GB/s | 61 | MiMo V2.6 Pro at GGUF Q4_K_M |
| 8× H200 | 1,128 GB | 38,400 GB/s | 62 | Qwen3.8 2.4T-A95B at GGUF Q2_K |
Mac
| Setup | Memory | Bandwidth | Fit at Q4_K_M (of 64) | Largest that fits |
|---|---|---|---|---|
| M1 Mac (8 GB) | 5.3 GB usable | 68.25 GB/s | 5 | LensVLM 9B at GGUF IQ3_XXS |
| M2 or M3 Mac (8 GB) | 5.3 GB usable | 100 GB/s | 5 | LensVLM 9B at GGUF IQ3_XXS |
| M1 Mac (16 GB) | 10.7 GB usable | 68.25 GB/s | 19 | Gemma 4 12B at GGUF Q5_K_M |
| M2 or M3 Mac (16 GB) | 10.7 GB usable | 100 GB/s | 19 | Gemma 4 12B at GGUF Q5_K_M |
| M4 Mac (16 GB) | 10.7 GB usable | 120 GB/s | 19 | Gemma 4 12B at GGUF Q5_K_M |
| M5 Mac (16 GB) | 10.7 GB usable | 153 GB/s | 19 | Gemma 4 12B at GGUF Q5_K_M |
| M1 Pro or M2 Pro Mac (16 GB) | 10.7 GB usable | 200 GB/s | 19 | Gemma 4 12B at GGUF Q5_K_M |
| M3 Pro Mac (18 GB) | 12 GB usable | 150 GB/s | 19 | Gemma 4 12B at GGUF Q6_K |
| M2 or M3 Mac (24 GB) | 16 GB usable | 100 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| M4 Mac (24 GB) | 16 GB usable | 120 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| M5 Mac (24 GB) | 16 GB usable | 153 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| M4 Pro Mac (24 GB) | 16 GB usable | 273 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| M5 Pro Mac (24 GB) | 16 GB usable | 307 GB/s | 20 | LLM-jp-4.1 32B-A3B Thinking at GGUF Q2_K |
| M4 Mac (32 GB) | 21.3 GB usable | 120 GB/s | 31 | K2-Horizon MoVA 36B-A4B at GGUF Q2_K |
| M5 Mac (32 GB) | 21.3 GB usable | 153 GB/s | 31 | K2-Horizon MoVA 36B-A4B at GGUF Q2_K |
| M1 Pro or M2 Pro Mac (32 GB) | 21.3 GB usable | 200 GB/s | 31 | K2-Horizon MoVA 36B-A4B at GGUF Q2_K |
| M1 Max or M2 Max Mac (32 GB) | 21.3 GB usable | 400 GB/s | 31 | K2-Horizon MoVA 36B-A4B at GGUF Q2_K |
| M3 Pro Mac (36 GB) | 27 GB usable | 150 GB/s | 38 | K2-Horizon MoVA 36B-A4B at GGUF Q4_K_M |
| M3 Max Mac (36 GB) | 27 GB usable | 300 GB/s | 38 | K2-Horizon MoVA 36B-A4B at GGUF Q4_K_M |
| M4 Max Mac (36 GB) | 27 GB usable | 410 GB/s | 38 | K2-Horizon MoVA 36B-A4B at GGUF Q4_K_M |
| M5 Max Mac (36 GB) | 27 GB usable | 460 GB/s | 38 | K2-Horizon MoVA 36B-A4B at GGUF Q4_K_M |
| M4 Pro Mac (48 GB) | 36 GB usable | 273 GB/s | 38 | AliceAI Foundation 80B-A3B at GGUF IQ3_XXS |
| M5 Pro Mac (48 GB) | 36 GB usable | 307 GB/s | 38 | AliceAI Foundation 80B-A3B at GGUF IQ3_XXS |
| M4 Max Mac (48 GB) | 36 GB usable | 546 GB/s | 38 | AliceAI Foundation 80B-A3B at GGUF IQ3_XXS |
| M5 Max Mac (48 GB) | 36 GB usable | 614 GB/s | 38 | AliceAI Foundation 80B-A3B at GGUF IQ3_XXS |
| M4 Pro Mac (64 GB) | 48 GB usable | 273 GB/s | 39 | AliceAI Foundation 80B-A3B at GGUF Q3_K_M |
| M5 Pro Mac (64 GB) | 48 GB usable | 307 GB/s | 39 | AliceAI Foundation 80B-A3B at GGUF Q3_K_M |
| M1–M3 Max Mac (64 GB) | 48 GB usable | 400 GB/s | 39 | AliceAI Foundation 80B-A3B at GGUF Q3_K_M |
| M4 Max Mac (64 GB) | 48 GB usable | 546 GB/s | 39 | AliceAI Foundation 80B-A3B at GGUF Q3_K_M |
| M5 Max Mac (64 GB) | 48 GB usable | 614 GB/s | 39 | AliceAI Foundation 80B-A3B at GGUF Q3_K_M |
| M3 Max Mac (128 GB) | 96 GB usable | 400 GB/s | 45 | Step 3.7 Flash 196B-A11B at GGUF Q2_K |
| M4 Max Mac (128 GB) | 96 GB usable | 546 GB/s | 45 | Step 3.7 Flash 196B-A11B at GGUF Q2_K |
| M5 Max Mac (128 GB) | 96 GB usable | 614 GB/s | 45 | Step 3.7 Flash 196B-A11B at GGUF Q2_K |
| M3 Ultra Mac Studio (512 GB) | 384 GB usable | 819 GB/s | 54 | Hy4 Preview 770B-A49B at GGUF Q2_K |
CPU-only PCs
| Setup | Memory | Bandwidth | Fit at Q4_K_M (of 64) | Largest that fits |
|---|---|---|---|---|
| CPU only, DDR4-3200 dual channel (16 GB) | 10 GB usable | 51.2 GB/s | 19 | Gemma 4 12B at GGUF Q4_K_M |
| CPU only, DDR4-3200 dual channel (32 GB) | 26 GB usable | 51.2 GB/s | 38 | K2-Horizon MoVA 36B-A4B at GGUF Q4_K_M |
| CPU only, DDR5-5600 dual channel (32 GB) | 26 GB usable | 89.6 GB/s | 38 | K2-Horizon MoVA 36B-A4B at GGUF Q4_K_M |
| CPU only, DDR4-3200 dual channel (64 GB) | 58 GB usable | 51.2 GB/s | 41 | Mistral Medium 3.5 128B at GGUF IQ3_XXS |
| CPU only, DDR5-5600 dual channel (64 GB) | 58 GB usable | 89.6 GB/s | 41 | Mistral Medium 3.5 128B at GGUF IQ3_XXS |
| CPU only, DDR5-5600 dual channel (128 GB) | 122 GB usable | 89.6 GB/s | 46 | MiniMax M2.7 at GGUF Q3_K_M |
| CPU only, DDR4-3200 quad channel (128 GB) | 122 GB usable | 102.4 GB/s | 46 | MiniMax M2.7 at GGUF Q3_K_M |
| CPU only, DDR4-3200 quad channel (256 GB) | 250 GB usable | 102.4 GB/s | 53 | MiniMax M3 at GGUF Q3_K_M |
Another model or setting? Use the LLM VRAM Calculator. Starting from your own machine? Detect your GPU and see what your PC can run.