Best local LLMs for 8 to 96 GB of VRAM

On one 24 GB GPU at Q4_K_M with 32K context, the largest dense model that fits is Muse Glimmer 30B (29.8B, 19.5 GB, about 29–40 tokens/s on one RTX 4090), the largest MoE is a tie between Ornith 1.5 35B-A3B and Qwen3.6 35B-A3B (36.0B total, 3.0B active, 23.5 GB, about 103–184 tokens/s), among models that take 12 GB or more, the longest context goes to Nemotron 3 Nano 30B-A3B (its full 256K in 21.7 GB), and the largest at Q8_0 is Gemma 4 12B (12.0B, 14.6 GB).

The table lists, for each VRAM size, the largest models that fit one GPU with 32,768 tokens of context, one request and an FP16 KV cache; each size links to its full list, newest models and questions. This compares memory, not quality. Total = weights + KV cache + 0.5 GB + 10% overhead; GB = GiB. Speeds are bandwidth estimates on one typical card of that size. Updated .

Not sure how much VRAM you have? Let the browser detect your GPU and see what your PC can run.

Every model by the VRAM it needs

y axis
69 open models: VRAM needed vs parameters x: VRAM needed at Q4_K_M with 8K context (log scale); y: total parameters (log scale); vertical lines at 8, 12, 16, 24, 32, 48, 80 GB cards. 8 GB 12 GB 16 GB 24 GB 32 GB 48 GB 80 GB 12481632641282565121,0242,048 1B3B10B30B100B300B1T3T 20406080100 VRAM needed (GB, Q4_K_M, 8K context) Total parameters GPQA (%, vendor-reported) Kimi K3: 1,724 GB at Q4_K_M, 2.78T total, 104.0B active Qwen3.8 2.4T-A95B: 1,517 GB at Q4_K_M, 2.45T total, 95.0B active DeepSeek V4 Pro: 993 GB at Q4_K_M, 1.60T total, 49.0B active MiMo V2.6 Pro: 636 GB at Q4_K_M, 1.02T total, 42.0B active Hy4 Preview 770B-A49B: 485 GB at Q4_K_M, 780.0B total, 49.0B active DeepSeek V4.1 Flash 1017: 474 GB at Q4_K_M, 763.2B total, 16.0B active DeepSeek V4.1 Flash: 474 GB at Q4_K_M, 763.2B total, 16.0B active GLM-5.3: 468 GB at Q4_K_M, 753.3B total, 40.0B active GLM-5.2: 468 GB at Q4_K_M, 753.3B total, 40.0B active DeepSeek V3.2: 426 GB at Q4_K_M, 685.4B total, 37.0B active DeepSeek V3 / R1: 425 GB at Q4_K_M, 684.5B total, 37.0B active Ling 3.1 Flash 560B-A25B: 355 GB at Q4_K_M, 572.1B total, 25.0B active MiniMax M3: 266 GB at Q4_K_M, 427.0B total, 23.0B active GLM-5.3 Flash: 200 GB at Q4_K_M, 321.3B total, 18.0B active IQuest-Q1 320B-A15B: 201 GB at Q4_K_M, 320.3B total, 15.0B active MiMo V2.6 Flash: 193 GB at Q4_K_M, 310.8B total, 15.0B active DeepSeek V4 Flash 0731: 190 GB at Q4_K_M, 304.2B total, 13.0B active DeepSeek V4 Flash: 182 GB at Q4_K_M, 290.9B total, 13.0B active MiniMax M2.7: 144 GB at Q4_K_M, 228.7B total, 10.0B active Step 3.7 Flash 196B-A11B: 126 GB at Q4_K_M, 201.4B total, 11.0B active Qwen3.8 Flash Next: 112 GB at Q4_K_M, 180.0B total, 6.0B active Mistral Medium 3.5 128B: 82.7 GB at Q4_K_M, 127.7B parameters Qwen3.5 122B-A10B: 78.2 GB at Q4_K_M, 125.1B total, 10.0B active Nemotron 3.5 Super VL 120B-A12B: 77.7 GB at Q4_K_M, 124.5B total, 12.0B active Nemotron 3 Super 120B-A12B: 77.2 GB at Q4_K_M, 123.6B total, 12.0B active gpt-oss-120b: 67.7 GB at MXFP4, 116.8B total, 5.1B active AliceAI Foundation 80B-A3B: 51.1 GB at Q4_K_M, 81.3B total, 3.0B active Qwen3-Coder-Next: 50.1 GB at Q4_K_M, 79.7B total, 3.0B active Kolibri-1: 49.2 GB at Q4_K_M, 78.1B total, 3.5B active Llama 3.1 70B: 47.0 GB at Q4_K_M, 70.6B parameters K2-Horizon MoVA 36B-A4B: 25.4 GB at Q4_K_M, 37.4B total, 4.0B active Ornith 1.5 35B-A3B: 23.0 GB at Q4_K_M, 36.0B total, 3.0B active Qwen3.6 35B-A3B: 23.0 GB at Q4_K_M, 36.0B total, 3.0B active Ornith 1.0 35B: 22.4 GB at Q4_K_M, 35.1B total, 3.0B active LLM-jp-4.1 32B-A3B Thinking: 21.0 GB at Q4_K_M, 32.1B total, 3.8B active Nemotron 3 Nano 30B-A3B: 20.1 GB at Q4_K_M, 31.6B total, 3.5B active Gemma 4 31B: 21.9 GB at Q4_K_M, 31.3B parameters GLM-4.7 Flash: 20.3 GB at Q4_K_M, 31.2B total, 3.0B active Xing 4.0 29B-A4B: 20.2 GB at Q4_K_M, 31.2B total, 4.0B active Qwen3-Coder 30B-A3B: 20.2 GB at Q4_K_M, 30.5B total, 3.3B active Qwen3 30B-A3B: 20.2 GB at Q4_K_M, 30.5B total, 3.3B active Muse Glimmer 30B: 19.2 GB at Q4_K_M, 29.8B parameters Granite 4.2 30B: 20.8 GB at Q4_K_M, 29.3B parameters ThinkingCap Qwen3.8 27B: 18.3 GB at Q4_K_M, 27.8B parameters Qwen3.6 27B: 18.3 GB at Q4_K_M, 27.8B parameters Qwen3.8 27B: 18.3 GB at Q4_K_M, 27.8B parameters Hemmingway-1 27B: 18.0 GB at Q4_K_M, 27.3B parameters Gemma 4 26B-A4B: 17.0 GB at Q4_K_M, 25.8B total, 4.0B active gpt-oss-20b: 14.8 GB at MXFP4, 20.9B total, 3.6B active Gemma 4 12B: 8.6 GB at Q4_K_M, 12.0B parameters ZDTaichu 5.0 9B: 6.8 GB at Q4_K_M, 9.8B parameters Ornith 1.5 9B: 6.8 GB at Q4_K_M, 9.7B parameters Qwen3.5 9B: 6.8 GB at Q4_K_M, 9.7B parameters LensVLM 9B: 6.6 GB at Q4_K_M, 9.4B parameters Ornith 1.0 9B: 6.6 GB at Q4_K_M, 9.4B parameters MiMo V2.6 Distill Qwen 9B: 6.6 GB at Q4_K_M, 9.4B parameters Bespoke Nimble 9B: 6.3 GB at Q4_K_M, 9.0B parameters Granite 4.2 8B: 7.3 GB at Q4_K_M, 8.8B parameters LFM2.5 8B-A1B: 5.9 GB at Q4_K_M, 8.5B total, 1.5B active Qwen3 8B: 6.8 GB at Q4_K_M, 8.2B parameters Llama 3.1 8B: 6.6 GB at Q4_K_M, 8.0B parameters Gemma 4 E4B: 5.6 GB at Q4_K_M, 8.0B parameters Ling 3.0 Tiny 7.9B-A1.3B: 5.5 GB at Q4_K_M, 7.9B total, 1.3B active LightOnOCR-3 4B: 3.6 GB at Q4_K_M, 4.5B parameters Spark-X2.5 4B: 3.5 GB at Q4_K_M, 4.1B parameters Nemotron 3 Nano 4B: 3.1 GB at Q4_K_M, 4.0B parameters Granite 4.2 3B: 3.5 GB at Q4_K_M, 3.7B parameters MiniCPM5 2B: 2.4 GB at Q4_K_M, 2.5B parameters Limite 1B Violetto: 1.3 GB at Q4_K_M, 1.0B parameters Kimi K3: GPQA Diamond 93.5, max reasoning effort (vendor-reported); 1,724 GB at Q4_K_M DeepSeek V4 Pro: GPQA Diamond 90.1, V4-Pro Max reasoning (vendor-reported); 993 GB at Q4_K_M DeepSeek V4.1 Flash: GPQA Diamond 90.9 (vendor-reported); 474 GB at Q4_K_M GLM-5.2: GPQA Diamond 91.2 (vendor-reported); 468 GB at Q4_K_M DeepSeek V3 / R1: GPQA Diamond 59.1, chat model (vendor-reported); 425 GB at Q4_K_M DeepSeek V4 Flash: GPQA Diamond 88.1, V4-Flash Max reasoning (vendor-reported); 182 GB at Q4_K_M Qwen3.8 Flash Next: GPQA Diamond 91.7 (vendor-reported); 112 GB at Q4_K_M Qwen3.5 122B-A10B: GPQA Diamond 86.6 (vendor-reported); 78.2 GB at Q4_K_M Nemotron 3 Super 120B-A12B: GPQA (no tools) 79.23 (vendor-reported); 77.2 GB at Q4_K_M gpt-oss-120b: GPQA Diamond (no tools) 80.1, reasoning high (vendor-reported); 67.7 GB at Q4_K_M Llama 3.1 70B: GPQA, 0-shot 46.7 (vendor-reported); 47.0 GB at Q4_K_M K2-Horizon MoVA 36B-A4B: GPQA Diamond 80.8 (vendor-reported); 25.4 GB at Q4_K_M Ornith 1.5 35B-A3B: GPQA Diamond 89.2 (vendor-reported); 23.0 GB at Q4_K_M Qwen3.6 35B-A3B: GPQA 86.0 (vendor-reported); 23.0 GB at Q4_K_M Nemotron 3 Nano 30B-A3B: GPQA (no tools) 73.0, reasoning on (vendor-reported); 20.1 GB at Q4_K_M Gemma 4 31B: GPQA Diamond 84.3 (vendor-reported); 21.9 GB at Q4_K_M GLM-4.7 Flash: GPQA 75.2 (vendor-reported); 20.3 GB at Q4_K_M Granite 4.2 30B: GPQA 66.41 (vendor-reported); 20.8 GB at Q4_K_M Qwen3.6 27B: GPQA Diamond 87.8 (vendor-reported); 18.3 GB at Q4_K_M Qwen3.8 27B: GPQA Diamond 89.2 (vendor-reported); 18.3 GB at Q4_K_M Gemma 4 26B-A4B: GPQA Diamond 82.3 (vendor-reported); 17.0 GB at Q4_K_M gpt-oss-20b: GPQA Diamond (no tools) 71.5, reasoning high (vendor-reported); 14.8 GB at Q4_K_M Gemma 4 12B: GPQA Diamond 78.8 (vendor-reported); 8.6 GB at Q4_K_M Ornith 1.5 9B: GPQA Diamond 86.4 (vendor-reported); 6.8 GB at Q4_K_M Qwen3.5 9B: GPQA Diamond 81.7 (vendor-reported); 6.8 GB at Q4_K_M Granite 4.2 8B: GPQA 64.14 (vendor-reported); 7.3 GB at Q4_K_M Llama 3.1 8B: GPQA, 0-shot 30.4 (vendor-reported); 6.6 GB at Q4_K_M Gemma 4 E4B: GPQA Diamond 58.6 (vendor-reported); 5.6 GB at Q4_K_M Spark-X2.5 4B: GPQA 67.4 (vendor-reported); 3.5 GB at Q4_K_M Nemotron 3 Nano 4B: GPQA 53.2, reasoning on (vendor-reported); 3.1 GB at Q4_K_M Granite 4.2 3B: GPQA 54.8 (vendor-reported); 3.5 GB at Q4_K_M
Solid dots are dense models, rings are mixture-of-experts (hover for active parameters); gpt-oss is at MXFP4 as published. Click a dot to open that model’s page. Named dots are the largest model that fits each line with 0.5 GB to spare. By default the y axis is parameter count from each model’s safetensors metadata on Hugging Face, not a quality score: more parameters does not mean better. Switched to GPQA, it is the score each maker reported in its own model card or report (read 2026-09-30; not rerun here, and reasoning settings differ between makers). The 38 models that published none are left off, and named dots are the highest reported score that fits each line. Sources are linked under Model details on each model’s page.

Top picks by VRAM size (Q4_K_M, 32K context)

VRAM Largest dense Largest MoE Longest context (models using half the VRAM or more) Largest at Q8_0 Models that fit
8 GB
RTX 4060 8GB
LensVLM 9B
7.4 GB, 21–30 tok/s
LFM2.5 8B-A1B (MoE)
6.2 GB, 57–99 tok/s
Ling 3.0 Tiny 7.9B-A1.3B (MoE)
128K, 6.3 GB
LightOnOCR-3 4B
6.5 GB
12 / 69
12 GB
RTX 4070 12GB
Gemma 4 12B
9.0 GB, 32–45 tok/s
LFM2.5 8B-A1B (MoE)
6.2 GB, 98–175 tok/s
Gemma 4 12B
178K, 11.5 GB
LFM2.5 8B-A1B (MoE)
10.1 GB
19 / 69
16 GB
RTX 5060 Ti 16GB
Gemma 4 12B
9.0 GB, 29–40 tok/s
gpt-oss-20b
MXFP4, 15.4 GB, 40–68 tok/s
Gemma 4 12B
256K, 12.8 GB
Gemma 4 12B
14.6 GB
20 / 69
24 GB
RTX 4090
Muse Glimmer 30B
19.5 GB, 29–40 tok/s
Ornith 1.5 35B-A3B (MoE)
23.5 GB, 103–184 tok/s
Nemotron 3 Nano 30B-A3B (MoE)
256K, 21.7 GB
Gemma 4 12B
14.6 GB
35 / 69
32 GB
RTX 5090
Gemma 4 31B
23.9 GB, 40–57 tok/s
K2-Horizon MoVA 36B-A4B (MoE)
30.3 GB, 56–96 tok/s
Ornith 1.5 35B-A3B (MoE)
256K, 28.3 GB
Gemma 4 26B-A4B (MoE)
29.6 GB
38 / 69
48 GB
L40S
Gemma 4 31B
23.9 GB, 20–28 tok/s
K2-Horizon MoVA 36B-A4B (MoE)
30.3 GB, 28–48 tok/s
K2-Horizon MoVA 36B-A4B (MoE)
115K, 47.5 GB
Ornith 1.5 35B-A3B (MoE)
40.3 GB
38 / 69
96 GB
RTX PRO 6000 Blackwell
Mistral Medium 3.5 128B
91.8 GB, 11–15 tok/s
Qwen3.5 122B-A10B (MoE)
78.8 GB, 70–123 tok/s
Nemotron 3.5 Super VL 120B-A12B (MoE)
1M, 86.4 GB
AliceAI Foundation 80B-A3B (MoE)
89.8 GB
47 / 69

The short answer for each size

All numbers, including 8K and 128K context, FP16 and 80 GB cards, are in the open dataset and the model and GPU report; summary figures across all models are on the LLM VRAM statistics page, and the per-tier findings and the KV cache ranking are in the 2026 Local LLM VRAM Report. Each model page shows every step of its calculation.