ModelVRAM
EN 中文

Can I run this LLM on my GPU?

One answer per model and card: whether it fits at Q4_K_M with 32K tokens of context and one request, and if not, the shorter context, smaller quant, --n-cpu-moe setting or number of cards that makes it run. 148 pairs have their own page; the rest fit with room to spare at BF16 and link to the model's page.

Model RTX 4090RTX 3090RTX 5090RTX 3060RTX 4060 TiRTX 5060 TiRTX 4070RTX 5080RTX 4080 SuperRX 7900 XTXM4 Max 128GBH100
Gemma 4 26B-A4B Yes, up to Q6_KYes, up to Q6_KYes, up to Q8_0--n-cpu-moe 12Only Q3_K_MOnly Q3_K_M--n-cpu-moe 12Only Q3_K_MOnly Q3_K_MYes, up to Q6_KYes, up to BF16Yes, up to BF16
Gemma 4 31B Q4_K_M, 26K contextQ4_K_M, 26K contextYes, up to Q6_KNo, 4 cardsNo, 2 cardsNo, 2 cardsNo, 4 cardsNo, 2 cardsNo, 2 cardsQ4_K_M, 26K contextYes, up to BF16Yes, up to BF16
Qwen3.8 27B Yes, up to Q5_K_MYes, up to Q5_K_MYes, up to FP8No, 2 cardsOnly Q2_KOnly Q2_KNo, 2 cardsOnly Q2_KOnly Q2_KYes, up to Q5_K_MYes, up to BF16Yes, up to BF16
gpt-oss-20b Yes, MXFP4Yes, MXFP4Yes, MXFP4--n-cpu-moe 2Yes, MXFP4Yes, MXFP4--n-cpu-moe 2Yes, MXFP4Yes, MXFP4Yes, MXFP4Yes, MXFP4Yes, MXFP4
Llama 3.1 8B Yes, up to BF16Yes, up to BF16Yes, up to BF16Yes, up to Q5_K_MYes, up to Q8_0Yes, up to Q8_0Yes, up to Q5_K_MYes, up to Q8_0Yes, up to Q8_0Yes, up to BF16Yes, up to BF16Yes, up to BF16
gpt-oss-120b --n-cpu-moe 24--n-cpu-moe 24--n-cpu-moe 19--n-cpu-moe 31--n-cpu-moe 29--n-cpu-moe 29--n-cpu-moe 31--n-cpu-moe 29--n-cpu-moe 29--n-cpu-moe 24Yes, MXFP4Yes, MXFP4
Qwen3.6 35B-A3B Yes, Q4_K_MYes, Q4_K_MYes, up to Q6_K--n-cpu-moe 22--n-cpu-moe 13--n-cpu-moe 13--n-cpu-moe 22--n-cpu-moe 13--n-cpu-moe 13Yes, Q4_K_MYes, up to BF16Yes, up to BF16
Qwen3.6 27B Yes, up to Q5_K_MYes, up to Q5_K_MYes, up to FP8No, 2 cardsOnly Q2_KOnly Q2_KNo, 2 cardsOnly Q2_KOnly Q2_KYes, up to Q5_K_MYes, up to BF16Yes, up to BF16
Gemma 4 12B Yes, up to Q8_0Yes, up to Q8_0Yes, up to BF16Yes, up to Q5_K_MYes, up to Q8_0Yes, up to Q8_0Yes, up to Q5_K_MYes, up to Q8_0Yes, up to Q8_0Yes, up to Q8_0Yes, up to BF16Yes, up to BF16
GLM-4.7 Flash Yes, Q4_K_MYes, Q4_K_MYes, up to Q6_K--n-cpu-moe 24--n-cpu-moe 12--n-cpu-moe 12--n-cpu-moe 24--n-cpu-moe 12--n-cpu-moe 12Yes, Q4_K_MYes, up to BF16Yes, up to BF16
Qwen3.8 Flash Next --n-cpu-moe 37--n-cpu-moe 37--n-cpu-moe 31--n-cpu-moe 45--n-cpu-moe 42--n-cpu-moe 42--n-cpu-moe 45--n-cpu-moe 42--n-cpu-moe 42--n-cpu-moe 37Only Q3_K_MOnly Q2_K
Nemotron 3 Nano 30B-A3B Yes, Q4_K_MYes, Q4_K_MYes, up to Q6_K--n-cpu-moe 30Only Q2_KOnly Q2_K--n-cpu-moe 30Only Q2_KOnly Q2_KYes, Q4_K_MYes, up to BF16Yes, up to BF16
Qwen3-Coder-Next --n-cpu-moe 26--n-cpu-moe 26--n-cpu-moe 17--n-cpu-moe 39--n-cpu-moe 35--n-cpu-moe 35--n-cpu-moe 39--n-cpu-moe 35--n-cpu-moe 35--n-cpu-moe 26Yes, up to Q8_0Yes, up to Q6_K
Llama 3.1 70B No, 4 cardsNo, 4 cardsNo, 2 cardsNo, 8 cardsNo, 4 cardsNo, 4 cardsNo, 8 cardsNo, 4 cardsNo, 4 cardsNo, 4 cardsYes, up to Q8_0Yes, up to Q6_K

"Yes" means Q4_K_M (MXFP4 for gpt-oss) fits at 32K with at least 0.5 GB free; "up to" names the most precise setting that does. The figures use the same estimate as the VRAM calculator: weights, FP16 KV cache and 0.5 GB plus 10% overhead. Card not in the table? Detect your GPU in the browser and see every model it runs.