What LLMs can two RTX 3060 12GB GPUs run?
Two RTX 3060 12GB GPUs (24 GB in all) run 38 of the 69 open models tracked here at Q4_K_M with 8K context; the largest is Ornith 1.5 35B-A3B (36.0B parameters, 23.5 GB). They read memory at 720 GB/s in total. 1 more fits at a lower precision. Each fit leaves at least 0.5 GB free on 2× RTX 3060 12GB (why a card does not give a model all its memory). The speeds assume tensor parallelism, as in vLLM or SGLang; llama.cpp splits layers across cards by default and then writes at about the speed of one card. How the model is divided between the cards, and what each one holds.
Models that fit on 2× RTX 3060 12GB
The most precise weights that still fit with 8K tokens of context and one request, the memory that takes, the longest context at that precision and the writing speed. Bigger models come first. Memory is for the whole group: weights, cache and 10% split across the 2 cards, plus 0.5 GB of runtime on each card (1 GB in all). gpt-oss-20b (15.3 GB, 65–118 tokens/s on 2× RTX 3060 12GB) is listed at MXFP4 only, as published.
| Model | Best precision | Memory | Longest context | Tokens/s |
|---|---|---|---|---|
| K2-Horizon MoVA 36B-A4B 37.4B, 4B active | GGUF Q3_K_M | 21.4 GB | 17K | 50–89 |
| Ornith 1.5 35B-A3B 36.0B, 3B active | GGUF Q4_K_M | 23.5 GB | 9K | 79–147 |
| Qwen3.6 35B-A3B 36.0B, 3B active | GGUF Q4_K_M | 23.5 GB | 9K | 79–147 |
| Ornith 1.0 35B 35.1B, 3B active | GGUF Q4_K_M | 22.9 GB | 34K | 79–147 |
| LLM-jp-4.1 32B-A3B Thinking 32.1B, 3.8B active | GGUF Q4_K_M | 21.5 GB | 37K | 60–108 |
| Nemotron 3 Nano 30B-A3B 31.6B, 3.5B active | GGUF Q4_K_M | 20.6 GB | 256K (full) | 74–137 |
| Gemma 4 31B 31.3B | GGUF Q4_K_M | 22.4 GB | 21K | 18–25 |
| GLM-4.7 Flash 31.2B, 3B active | GGUF Q4_K_M | 20.8 GB | 55K | 72–132 |
| Xing 4.0 29B-A4B 31.2B, 4B active | GGUF Q4_K_M | 20.7 GB | 64K | 61–110 |
| Qwen3-Coder 30B-A3B 30.5B, 3.3B active | GGUF Q4_K_M | 20.7 GB | 34K | 61–110 |
| Qwen3 30B-A3B 30.5B, 3.3B active | GGUF Q4_K_M | 20.7 GB | 34K | 61–110 |
| Muse Glimmer 30B 29.8B | GGUF Q5_K_M | 22.8 GB | 55K | 17–25 |
| Granite 4.2 30B 29.3B | GGUF Q4_K_M | 21.3 GB | 15K | 19–26 |
| ThinkingCap Qwen3.8 27B 27.8B | GGUF Q5_K_M | 21.7 GB | 33K | 18–26 |
| Qwen3.6 27B 27.8B | GGUF Q5_K_M | 21.7 GB | 33K | 18–26 |
| Qwen3.8 27B 27.8B | GGUF Q5_K_M | 21.7 GB | 33K | 18–26 |
| Hemmingway-1 27B 27.3B | GGUF Q5_K_M | 21.4 GB | 38K | 19–26 |
| Gemma 4 26B-A4B 25.8B, 4B active | GGUF Q6_K | 23.2 GB | 23K | 48–85 |
| gpt-oss-20b 20.9B, 3.6B active | As published (MXFP4) | 15.3 GB | 128K (full) | 65–118 |
| Gemma 4 12B 12.0B | GGUF Q8_0 | 14.7 GB | 256K (full) | 27–38 |
| ZDTaichu 5.0 9B 9.8B | FP16 / BF16 | 21.3 GB | 70K | 19–26 |
| Ornith 1.5 9B 9.7B | FP16 / BF16 | 21.1 GB | 78K | 19–27 |
| Qwen3.5 9B 9.7B | FP16 / BF16 | 21.1 GB | 78K | 19–27 |
| LensVLM 9B 9.4B | FP16 / BF16 | 20.6 GB | 93K | 19–27 |
| Ornith 1.0 9B 9.4B | FP16 / BF16 | 20.6 GB | 93K | 19–27 |
| MiMo V2.6 Distill Qwen 9B 9.4B | FP16 / BF16 | 20.6 GB | 93K | 19–27 |
| Bespoke Nimble 9B 9.0B | FP16 / BF16 | 19.6 GB | 8K (full) | 20–29 |
| Granite 4.2 8B 8.8B | FP16 / BF16 | 20.4 GB | 26K | 19–28 |
| LFM2.5 8B-A1B 8.5B, 1.5B active | FP16 / BF16 | 18.5 GB | 128,000 (full) | 56–101 |
| Qwen3 8B 8.2B | FP16 / BF16 | 19.0 GB | 36K | 21–30 |
| Llama 3.1 8B 8.0B | FP16 / BF16 | 18.6 GB | 43K | 21–30 |
| Gemma 4 E4B 8.0B | FP16 / BF16 | 17.6 GB | 128K (full) | 23–32 |
| Ling 3.0 Tiny 7.9B-A1.3B 7.9B, 1.3B active | FP16 / BF16 | 17.2 GB | 128K (full) | 63–115 |
| LightOnOCR-3 4B 4.5B | FP16 / BF16 | 10.6 GB | 256K (full) | 37–54 |
| Spark-X2.5 4B 4.1B | FP16 / BF16 | 9.85 GB | 360K | 39–58 |
| Nemotron 3 Nano 4B 4.0B | FP16 / BF16 | 9.28 GB | 256K (full) | 42–61 |
| Granite 4.2 3B 3.7B | FP16 / BF16 | 9.19 GB | 128K (full) | 42–62 |
| MiniCPM5 2B 2.5B | FP16 / BF16 | 6.52 GB | 128K (full) | 58–89 |
| Limite 1B Violetto 1.0B | FP16 / BF16 | 3.29 GB | 128K (full) | 109–184 |
Too big for 2× RTX 3060 12GB at 8K context
How many RTX 3060 12GB cards these need at Q4_K_M (gpt-oss-120b at MXFP4) with 8K context and 0.5 GB to spare, in one tensor-parallel group; some need more than 8.
- Llama 3.1 70B: 8 cards
- Kolibri-1: 8 cards
- Qwen3-Coder-Next: 8 cards
- AliceAI Foundation 80B-A3B: 8 cards
- gpt-oss-120b: 8 cards
- Nemotron 3 Super 120B-A12B: 8 cards
- Nemotron 3.5 Super VL 120B-A12B: 8 cards
- Qwen3.5 122B-A10B: 8 cards
- Mistral Medium 3.5 128B: 8 cards
- Qwen3.8 Flash Next: more than 8 cards
- Step 3.7 Flash 196B-A11B: more than 8 cards
- MiniMax M2.7: more than 8 cards
- DeepSeek V4 Flash: more than 8 cards
- DeepSeek V4 Flash 0731: more than 8 cards
- MiMo V2.6 Flash: more than 8 cards
- IQuest-Q1 320B-A15B: more than 8 cards
- GLM-5.3 Flash: more than 8 cards
- MiniMax M3: more than 8 cards
- Ling 3.1 Flash 560B-A25B: more than 8 cards
- DeepSeek V3 / R1: more than 8 cards
- DeepSeek V3.2: more than 8 cards
- GLM-5.3: more than 8 cards
- GLM-5.2: more than 8 cards
- DeepSeek V4.1 Flash 1017: more than 8 cards
- DeepSeek V4.1 Flash: more than 8 cards
- Hy4 Preview 770B-A49B: more than 8 cards
- MiMo V2.6 Pro: more than 8 cards
- DeepSeek V4 Pro: more than 8 cards
- Qwen3.8 2.4T-A95B: more than 8 cards
- Kimi K3: more than 8 cards
Best models for 2× RTX 3060 12GB with room for long context
The biggest models that still fit at Q4_K_M with 32,768 tokens of context (or their whole window, if shorter), the memory left over, the longest context the group takes at that precision and the writing speed with that context in the cache, from 720 GB/s of bandwidth. Memory is for the whole group: weights, cache and 10% split across the 2 cards, plus 0.5 GB of runtime on each card (1 GB in all). Memory and left over add up to the group's 24 GB.
| Model | Context | Memory | Left over | Longest context | Tokens/s |
|---|---|---|---|---|---|
| Ornith 1.0 35B 35.1B | 32K | 23.4 GB | 567 MB | 34K | 67–122 |
| LLM-jp-4.1 32B-A3B Thinking 32.1B | 32K | 23.1 GB | 902 MB | 37K | 41–73 |
| Nemotron 3 Nano 30B-A3B 31.6B | 32K | 20.8 GB | 3.22 GB | 256K (full) | 70–129 |
| GLM-4.7 Flash 31.2B | 32K | 22.2 GB | 1.83 GB | 55K | 50–89 |
| Xing 4.0 29B-A4B 31.2B | 32K | 21.9 GB | 2.11 GB | 64K | 46–82 |
llama-server commands for 2× RTX 3060 12GB
GGUF repos checked 2026-09-29; -c is the longest context with 0.5 GB free on 2× RTX 3060 12GB. No -ngl: llama.cpp’s -fit, on by default since b7440, places the layers.
Ornith 1.0 35B, Q4_K_M with 27K tokens: 69–126 tokens/s on 2× RTX 3060 12GB
llama-server -hf bartowski/deepreinforce-ai_Ornith-1.0-35B-GGUF:Q4_K_M -c 27648deepreinforce-ai_Ornith-1.0-35B-Q4_K_M.gguf, 21.4 GB: 23.5 GB used and 517 MB free of 2× RTX 3060 12GB's 24 GB, 69–126 tokens/s once the 27K cache is full.
GLM-4.7 Flash, Q4_K_M with 55K tokens: 38–68 tokens/s on 2× RTX 3060 12GB
llama-server -hf unsloth/GLM-4.7-Flash-GGUF:Q4_K_M -c 56320GLM-4.7-Flash-Q4_K_M.gguf, 18.3 GB: 23.5 GB used and 538 MB free of 2× RTX 3060 12GB's 24 GB, 38–68 tokens/s once the 55K cache is full.
How fast 2× RTX 3060 12GB writes as the context fills
Tokens per second at Q4_K_M for the biggest models that fit with 32K tokens, with 8K and 32K tokens in the cache and at the longest context the group holds, and at Q8_0 with 8K where that fits. Each token reads the active weights and the whole cache once, so the speed follows the 720 GB/s of bandwidth and falls as the cache grows. The ceiling is that bandwidth divided by the bytes read per token, which no runtime reaches; the reply time is for 1,000 new tokens with 32K (or the longest context) in the cache, prompt processing aside. With the same memory, RTX 3090, 936 GB/s, writes 30% faster by bandwidth; RX 7900 XTX, 960 GB/s, writes 33% faster by bandwidth; RTX 4090, 1,008 GB/s, writes 40% faster by bandwidth.
| Model | 8K | 32K | Longest | At the longest | Q8_0 at 8K | Bandwidth ceiling at 8K | 1,000-token reply at 32K |
|---|---|---|---|---|---|---|---|
| Ornith 1.0 35B | 79–147 | 67–122 | 34K | 66–120 | Does not fit | 363 | 8–15 s |
| LLM-jp-4.1 32B-A3B Thinking | 60–108 | 41–73 | 37K | 39–68 | Does not fit | 252 | 14–24 s |
| Nemotron 3 Nano 30B-A3B | 74–137 | 70–129 | 256K | 48–86 | Does not fit | 332 | 8–14 s |
| GLM-4.7 Flash | 72–132 | 50–89 | 55K | 38–68 | Does not fit | 319 | 11–20 s |
| Xing 4.0 29B-A4B | 61–110 | 46–82 | 64K | 35–61 | Does not fit | 257 | 12–22 s |
| Qwen3-Coder 30B-A3B | 61–110 | 36–63 | 34K | 35–61 | Does not fit | 257 | 16–28 s |
| Qwen3 30B-A3B | 61–110 | 36–63 | 34K | 35–61 | Does not fit | 257 | 16–28 s |
| Muse Glimmer 30B | 20–29 | 20–28 | 128K | 19–26 | Does not fit | 40 | 36–50 s |
| ThinkingCap Qwen3.8 27B | 21–30 | 19–27 | 76K | 17–24 | Does not fit | 42 | 36–51 s |
| Qwen3.6 27B | 21–30 | 19–27 | 76K | 17–24 | Does not fit | 42 | 36–51 s |
Near misses on 2× RTX 3060 12GB
Models that miss at Q4_K_M with 32,768 tokens of context (or their whole window), or leave under 0.5 GB free there, but fit with a shorter context or 3-bit weights, or are short by at most a quarter of the memory and fit with MoE offload. Smallest shortfall first.
| Model | Short by at Q4_K_M | Fits instead |
|---|---|---|
| Ornith 1.5 35B-A3B 36.0B | Tight, 30 MB free | Q4_K_M with 9K context |
| Qwen3.6 35B-A3B 36.0B | Tight, 30 MB free | Q4_K_M with 9K context |
| Gemma 4 31B 31.3B | 432 MB | Q4_K_M with 21K context |
| Granite 4.2 30B 29.3B | 3.95 GB | Q4_K_M with 15K context |
| K2-Horizon MoVA 36B-A4B 37.4B | 6.81 GB | GGUF IQ3_XXS with 32K, 23.4 GB |
What more cards or MoE offload unlock
4× RTX 3060 12GB (48 GB) would add 1 model at Q4_K_M with 8K context: K2-Horizon MoVA 36B-A4B.
Run locally or rent two RTX 3060 12GB GPUs?
Renting two RTX 3060 12GB GPUs costs a median of $0.16 an hour on demand ($0.08 per card), across the providers getdeploying.com tracks (checked ), so every $1,000 of purchase price equals about 6,250 hours of rent, 260 days around the clock. At that rate Ornith 1.0 35B at Q4_K_M with 32K tokens, the biggest model here that fits with a long context, writes 67–122 tokens/s for one request: about $0.36–0.67 per million generated tokens. Electricity and resale value are left out, and no card prices are listed because 2026 street prices vary widely between sellers; compare current ones before you decide.
How these numbers are worked out
Memory is the weights at each precision plus the KV cache for 8,192 tokens in FP16 and runtime overhead (0.5 GB on each card plus 10%), computed from each model's files on Hugging Face with the LLM VRAM Calculator. Speed is estimated from memory bandwidth and the parameters read per token with the LLM Speed Calculator. Both tools take any other model from Hugging Face.
Nearby GPUs and setups
How many of the tracked models each one holds at Q4_K_M with 8K context, against 38 here.
| Setup | How it relates | Memory | Bandwidth | Models at Q4_K_M |
|---|---|---|---|---|
| RTX 3090 | Same memory | 24 GB | 936 GB/s | 38 (same) |
| RX 7900 XTX | Same memory | 24 GB | 960 GB/s | 38 (same) |
| RTX 4090 | Same memory | 24 GB | 1,008 GB/s | 38 (same) |
| RTX 5080 16GB | Next size down | 16 GB | 960 GB/s | 21 (−17) |
| RTX 5090 | Next size up | 32 GB | 1,792 GB/s | 39 (+1) |
| RTX 3060 12GB | Fewer of this card | 12 GB | 360 GB/s | 20 (−18) |
Model numbers read from Hugging Face on .