Best local LLMs for 16 GB of VRAM
On one 16 GB GPU at Q4_K_M with 32K context, the largest dense model that fits is Gemma 4 12B (12.0B, 9.0 GB, about 29–40 tokens/s on one RTX 5060 Ti 16GB), the largest MoE is gpt-oss-20b (20.9B total, 3.6B active, MXFP4, 15.4 GB, about 40–68 tokens/s), among models that take 8 GB or more, the longest context goes to Gemma 4 12B (its full 256K in 12.8 GB), and the largest at Q8_0 is Gemma 4 12B (12.0B, 14.6 GB).
Updated , the latest date the data of a model listed here was checked; 55 models in total. Ordered by memory and size, not quality. Total = weights + FP16 KV cache for one request + 0.5 GB + 10% overhead; GB = GiB. A model counts as fitting when it leaves at least 0.5 GB free, as on the Can I run it? pages. Speeds are bandwidth estimates for one request with the 32K context full, on one RTX 5060 Ti 16GB (448 GB/s), not measurements. Not sure how much VRAM you have? Detect your GPU.
gpt-oss-20b is listed at MXFP4, the format it is published in, in the Q4_K_M lists and not in the Q8_0 list: its GGUF quants keep the experts in MXFP4, so every type is about the same size.
Top picks for 16 GB
| Pick | Model | Parameters | Active | VRAM | Tokens/s (RTX 5060 Ti 16GB) |
|---|---|---|---|---|---|
| Largest dense (Q4_K_M) | Gemma 4 12B | 12.0B | dense | 9.0 GB | 29–40 |
| Largest MoE (MXFP4) | gpt-oss-20b | 20.9B | 3.6B | 15.4 GB | 40–68 |
| Longest context, 8 GB+ models (Q4_K_M) | Gemma 4 12B | 12.0B | dense | 256K (full), 12.8 GB | 29–40 |
| Largest at Q8_0 | Gemma 4 12B | 12.0B | dense | 14.6 GB | 17–24 |
What is the best local LLM for 16 GB VRAM?
By size, the largest model that fits one 16 GB GPU at Q4_K_M with 32K context is gpt-oss-20b (MoE, 20.9B total, 3.6B active, MXFP4, 15.4 GB, 0.6 GB spare); 15 of the 55 models we track fit. This page ranks by memory and size, not by benchmark scores, which our data does not include. gpt-oss-20b is counted at MXFP4, the format it is published in: its GGUF quants keep the experts in MXFP4 and are about the same size.
Can 16 GB run a 70B model?
Not on one card: Llama 3.1 70B (70.6B) needs 55.2 GB at Q4_K_M with 32K context, or 4 × 16 GB cards, and still 41.3 GB at IQ3_XXS.
Every step is on the Llama 3.1 70B VRAM page.
Is Q8_0 or a bigger model better on 16 GB?
On one 16 GB GPU with 32K context, the largest model at Q8_0 is Gemma 4 12B (12.0B, 14.6 GB, about 17–24 tokens/s) and at Q4_K_M it is gpt-oss-20b (MoE, 20.9B total, 3.6B active, MXFP4, 15.4 GB, about 40–68 tokens/s), 1.7× the total parameters. Q8_0 stores about 8.5 bits per weight and Q4_K_M 4.84; our data covers memory and speed, not output quality, so this page does not score the trade-off.
Bits per weight for every GGUF type are in GGUF quantization explained.
Newest models that fit 16 GB
Sorted by the date each model's data was added or checked, newest first (Q4_K_M, 32K context). New models appear here as they are added.
| Model | Added / checked | Parameters | Active | VRAM (GB) | Tokens/s |
|---|---|---|---|---|---|
| Spark-X2.5 4B | 4.1B | dense | 4.40 | 59–84 | |
| Ling 3.0 Tiny 7.9B-A1.3B (MoE) | 7.9B | 1.3B | 5.62 | 111–199 | |
| LFM2.5 8B-A1B (MoE) | 8.5B | 1.5B | 6.16 | 89–158 | |
| Limite 1B Violetto | 1.0B | dense | 1.62 | 168–266 | |
| ZDTaichu 5.0 9B | 9.8B | dense | 7.67 | 33–47 | |
| MiMo V2.6 Distill Qwen 9B | 9.4B | dense | 7.43 | 35–48 | |
| Ornith 1.5 9B | 9.7B | dense | 7.58 | 34–47 | |
| Qwen3.5 9B | 9.7B | dense | 7.58 | 34–47 |
All 15 models that fit 16 GB at Q4_K_M
| Model | Parameters | Active | VRAM (GB) | Spare (GB) | Tokens/s | Calculator |
|---|---|---|---|---|---|---|
| gpt-oss-20b MXFP4 | 20.9B | 3.6B | 15.44 | 0.56 | 40–68 | Open |
| Gemma 4 12B | 12.0B | dense | 8.98 | 7.02 | 29–40 | Open |
| ZDTaichu 5.0 9B | 9.8B | dense | 7.67 | 8.33 | 33–47 | Open |
| Ornith 1.5 9B | 9.7B | dense | 7.58 | 8.42 | 34–47 | Open |
| Qwen3.5 9B | 9.7B | dense | 7.58 | 8.42 | 34–47 | Open |
| MiMo V2.6 Distill Qwen 9B | 9.4B | dense | 7.43 | 8.57 | 35–48 | Open |
| LFM2.5 8B-A1B (MoE) | 8.5B | 1.5B | 6.16 | 9.84 | 89–158 | Open |
| Qwen3 8B | 8.2B | dense | 10.53 | 5.47 | 24–34 | Open |
| Llama 3.1 8B | 8.0B | dense | 9.88 | 6.12 | 26–36 | Open |
| Gemma 4 E4B | 8.0B | dense | 6.05 | 9.95 | 43–60 | Open |
| Ling 3.0 Tiny 7.9B-A1.3B (MoE) | 7.9B | 1.3B | 5.62 | 10.38 | 111–199 | Open |
| Spark-X2.5 4B | 4.1B | dense | 4.40 | 11.60 | 59–84 | Open |
| Nemotron 3 Nano 4B | 4.0B | dense | 3.51 | 12.49 | 74–108 | Open |
| MiniCPM5 2B | 2.5B | dense | 3.50 | 12.50 | 75–108 | Open |
| Limite 1B Violetto | 1.0B | dense | 1.62 | 14.38 | 168–266 | Open |
All 14 models that fit 16 GB at Q8_0
| Model | Parameters | Active | VRAM (GB) | Spare (GB) | Tokens/s | Calculator |
|---|---|---|---|---|---|---|
| Gemma 4 12B | 12.0B | dense | 14.58 | 1.42 | 17–24 | Open |
| ZDTaichu 5.0 9B | 9.8B | dense | 12.26 | 3.74 | 21–29 | Open |
| Ornith 1.5 9B | 9.7B | dense | 12.11 | 3.89 | 21–29 | Open |
| Qwen3.5 9B | 9.7B | dense | 12.11 | 3.89 | 21–29 | Open |
| MiMo V2.6 Distill Qwen 9B | 9.4B | dense | 11.84 | 4.16 | 22–30 | Open |
| LFM2.5 8B-A1B (MoE) | 8.5B | 1.5B | 10.13 | 5.87 | 61–106 | Open |
| Qwen3 8B | 8.2B | dense | 14.37 | 1.63 | 18–25 | Open |
| Llama 3.1 8B | 8.0B | dense | 13.64 | 2.36 | 19–26 | Open |
| Gemma 4 E4B | 8.0B | dense | 9.80 | 6.20 | 26–36 | Open |
| Ling 3.0 Tiny 7.9B-A1.3B (MoE) | 7.9B | 1.3B | 9.32 | 6.68 | 74–130 | Open |
| Spark-X2.5 4B | 4.1B | dense | 6.33 | 9.67 | 41–57 | Open |
| Nemotron 3 Nano 4B | 4.0B | dense | 5.38 | 10.62 | 48–68 | Open |
| MiniCPM5 2B | 2.5B | dense | 4.68 | 11.32 | 55–79 | Open |
| Limite 1B Violetto | 1.0B | dense | 2.11 | 13.89 | 127–193 | Open |
Macs and CPU-only PCs with about 16 GB to use
These machines leave a model between 16 GB and the next size up: on a Mac, the unified memory macOS lets the GPU use by default (about two thirds up to 32 GB, 75% above; sudo sysctl iogpu.wired_limit_mb raises it), on a CPU-only PC the RAM minus about 6 GB for the system. The lists above apply to them too; speed follows each one's bandwidth, and a CPU is also much slower at reading long prompts.
| Machine | Usable memory | Bandwidth | Models at Q4_K_M (8K) | Biggest at 32K | Tokens/s |
|---|---|---|---|---|---|
| M2 or M3 Mac (24 GB) | 16 GB | 100 GB/s | 15 | gpt-oss-20b | 9.3–16 |
| M4 Mac (24 GB) | 16 GB | 120 GB/s | 15 | gpt-oss-20b | 11–19 |
| M5 Mac (24 GB) | 16 GB | 153 GB/s | 15 | gpt-oss-20b | 14–24 |
| M4 Pro Mac (24 GB) | 16 GB | 273 GB/s | 15 | gpt-oss-20b | 25–42 |
| M5 Pro Mac (24 GB) | 16 GB | 307 GB/s | 15 | gpt-oss-20b | 28–47 |
| M4 Mac (32 GB) | 21.3 GB | 120 GB/s | 24 | Nemotron 3 Nano 30B-A3B | 15–26 |
| M5 Mac (32 GB) | 21.3 GB | 153 GB/s | 24 | Nemotron 3 Nano 30B-A3B | 19–32 |
| M1 Pro or M2 Pro Mac (32 GB) | 21.3 GB | 200 GB/s | 24 | Nemotron 3 Nano 30B-A3B | 25–42 |
| M1 Max or M2 Max Mac (32 GB) | 21.3 GB | 400 GB/s | 24 | Nemotron 3 Nano 30B-A3B | 48–83 |
Other VRAM sizes
All numbers, including 8K and 128K context, are in the open dataset, with summary figures on LLM VRAM statistics; any other setting can be worked out in the LLM VRAM Calculator.