What LLMs can an RTX PRO 6000 Blackwell run?

An RTX PRO 6000 Blackwell (96 GB) runs 48 of the 69 open models tracked here at Q4_K_M with 8K context; the largest is Mistral Medium 3.5 128B (127.7B parameters, 82.7 GB). It reads memory at 1,792 GB/s. 2 more fit at a lower precision. Each fit leaves at least 0.5 GB free on the RTX PRO 6000 Blackwell (why a card does not give a model all its memory).

Not sure your PC has an RTX PRO 6000 Blackwell? Detect your GPU in the browser.

Models that fit on one RTX PRO 6000 Blackwell

The most precise weights that still fit with 8K tokens of context and one request, the memory that takes, the longest context at that precision and the writing speed. Bigger models come first. gpt-oss-120b (67.7 GB, 135–247 tokens/s on the RTX PRO 6000 Blackwell) and gpt-oss-20b (14.8 GB, 158–295 tokens/s on the RTX PRO 6000 Blackwell) are listed at MXFP4 only, as published.

Model	Best precision	Memory	Longest context	Tokens/s
Step 3.7 Flash 196B-A11B 201.4B, 11B active	GGUF Q2_K	87.4 GB	164K	90–160
Qwen3.8 Flash Next 180.0B, 6B active	GGUF Q3_K_M	90.8 GB	189K	136–250
Mistral Medium 3.5 128B 127.7B	GGUF Q4_K_M	82.7 GB	41K	12–17
Qwen3.5 122B-A10B 125.1B, 10B active	GGUF Q5_K_M	91.5 GB	161K	66–116
Nemotron 3.5 Super VL 120B-A12B 124.5B, 12B active	GGUF Q5_K_M	90.9 GB	539K	57–99
Nemotron 3 Super 120B-A12B 123.6B, 12B active	GGUF Q5_K_M	90.3 GB	256K (full)	57–99
gpt-oss-120b 116.8B, 5.1B active	As published (MXFP4)	67.7 GB	128K (full)	135–247
AliceAI Foundation 80B-A3B 81.3B, 3B active	GGUF Q8_0	89.2 GB	252K	128–234
Qwen3-Coder-Next 79.7B, 3B active	GGUF Q8_0	87.4 GB	256K (full)	128–234
Kolibri-1 78.1B, 3.5B active	GGUF Q8_0	85.8 GB	256K (full)	113–204
Llama 3.1 70B 70.6B	GGUF Q8_0	80.0 GB	52K	12–17
K2-Horizon MoVA 36B-A4B 37.4B, 4B active	FP16 / BF16	78.9 GB	88K	52–89
Ornith 1.5 35B-A3B 36.0B, 3B active	FP16 / BF16	74.3 GB	256K (full)	77–135
Qwen3.6 35B-A3B 36.0B, 3B active	FP16 / BF16	74.3 GB	256K (full)	77–135
Ornith 1.0 35B 35.1B, 3B active	FP16 / BF16	72.6 GB	256K (full)	77–135
LLM-jp-4.1 32B-A3B Thinking 32.1B, 3.8B active	FP16 / BF16	66.9 GB	64K (full)	60–104
Nemotron 3 Nano 30B-A3B 31.6B, 3.5B active	FP16 / BF16	65.3 GB	256K (full)	68–119
Gemma 4 31B 31.3B	FP16 / BF16	66.6 GB	256K (full)	15–21
GLM-4.7 Flash 31.2B, 3B active	FP16 / BF16	64.9 GB	198K (full)	74–130
Xing 4.0 29B-A4B 31.2B, 4B active	FP16 / BF16	64.8 GB	256K (full)	59–102
Qwen3-Coder 30B-A3B 30.5B, 3.3B active	FP16 / BF16	63.9 GB	256K (full)	65–114
Qwen3 30B-A3B 30.5B, 3.3B active	FP16 / BF16	63.9 GB	40K (full)	65–114
Muse Glimmer 30B 29.8B	FP16 / BF16	61.7 GB	128K (full)	16–22
Granite 4.2 30B 29.3B	FP16 / BF16	62.7 GB	127K	16–22
ThinkingCap Qwen3.8 27B 27.8B	FP16 / BF16	58.0 GB	256K (full)	17–24
Qwen3.6 27B 27.8B	FP16 / BF16	58.0 GB	256K (full)	17–24
Qwen3.8 27B 27.8B	FP16 / BF16	58.0 GB	256K (full)	17–24
Hemmingway-1 27B 27.3B	FP16 / BF16	57.0 GB	256K (full)	17–24
Gemma 4 26B-A4B 25.8B, 4B active	FP16 / BF16	53.9 GB	256K (full)	58–100
gpt-oss-20b 20.9B, 3.6B active	As published (MXFP4)	14.8 GB	128K (full)	158–295
Gemma 4 12B 12.0B	FP16 / BF16	25.7 GB	256K (full)	38–53
ZDTaichu 5.0 9B 9.8B	FP16 / BF16	20.8 GB	128K (full)	46–65
Ornith 1.5 9B 9.7B	FP16 / BF16	20.6 GB	256K (full)	47–66
Qwen3.5 9B 9.7B	FP16 / BF16	20.6 GB	256K (full)	47–66
LensVLM 9B 9.4B	FP16 / BF16	20.1 GB	256K (full)	48–68
Ornith 1.0 9B 9.4B	FP16 / BF16	20.1 GB	256K (full)	48–68
MiMo V2.6 Distill Qwen 9B 9.4B	FP16 / BF16	20.1 GB	256K (full)	48–68
Bespoke Nimble 9B 9.0B	FP16 / BF16	19.1 GB	8K (full)	50–71
Granite 4.2 8B 8.8B	FP16 / BF16	19.9 GB	128K (full)	48–69
LFM2.5 8B-A1B 8.5B, 1.5B active	FP16 / BF16	18.0 GB	128,000 (full)	138–252
Qwen3 8B 8.2B	FP16 / BF16	18.5 GB	40K (full)	52–74
Llama 3.1 8B 8.0B	FP16 / BF16	18.1 GB	128K (full)	53–75
Gemma 4 E4B 8.0B	FP16 / BF16	17.1 GB	128K (full)	56–80
Ling 3.0 Tiny 7.9B-A1.3B 7.9B, 1.3B active	FP16 / BF16	16.7 GB	128K (full)	155–289
LightOnOCR-3 4B 4.5B	FP16 / BF16	10.1 GB	256K (full)	91–134
Spark-X2.5 4B 4.1B	FP16 / BF16	9.35 GB	1M (full)	97–144
Nemotron 3 Nano 4B 4.0B	FP16 / BF16	8.78 GB	256K (full)	103–154
Granite 4.2 3B 3.7B	FP16 / BF16	8.69 GB	128K (full)	104–155
MiniCPM5 2B 2.5B	FP16 / BF16	6.02 GB	128K (full)	144–222
Limite 1B Violetto 1.0B	FP16 / BF16	2.79 GB	128K (full)	265–462

Too big for one RTX PRO 6000 Blackwell at 8K context

How many RTX PRO 6000 Blackwell cards these need at Q4_K_M with 8K context and 0.5 GB to spare, in one tensor-parallel group; some need more than 8.

MiniMax M2.7: 2 cards
DeepSeek V4 Flash: 2 cards
DeepSeek V4 Flash 0731: 2 cards
MiMo V2.6 Flash: 4 cards
IQuest-Q1 320B-A15B: 4 cards
GLM-5.3 Flash: 4 cards
MiniMax M3: 4 cards
Ling 3.1 Flash 560B-A25B: 4 cards
DeepSeek V3 / R1: 8 cards
DeepSeek V3.2: 8 cards
GLM-5.3: 8 cards
GLM-5.2: 8 cards
DeepSeek V4.1 Flash 1017: 8 cards
DeepSeek V4.1 Flash: 8 cards
Hy4 Preview 770B-A49B: 8 cards
MiMo V2.6 Pro: 8 cards
DeepSeek V4 Pro: more than 8 cards
Qwen3.8 2.4T-A95B: more than 8 cards
Kimi K3: more than 8 cards

Best models for RTX PRO 6000 Blackwell with room for long context

The biggest models that still fit at Q4_K_M with 32,768 tokens of context (or their whole window, if shorter), the memory left over, the longest context the card takes at that precision and the writing speed with that context in the cache, from 1,792 GB/s of bandwidth. gpt-oss-120b is at MXFP4, as published.

Model	Context	Memory	Left over	Longest context	Tokens/s
Mistral Medium 3.5 128B 127.7B	32K	91.8 GB	4.25 GB	41K	11–15
Qwen3.5 122B-A10B 125.1B	32K	78.9 GB	17.1 GB	256K (full)	70–123
Nemotron 3.5 Super VL 120B-A12B 124.5B	32K	77.9 GB	18.1 GB	1M (full)	65–112
Nemotron 3 Super 120B-A12B 123.6B	32K	77.4 GB	18.6 GB	256K (full)	65–112
gpt-oss-120b 116.8B, MXFP4	32K	68.6 GB	27.4 GB	128K (full)	110–198

llama-server commands for one RTX PRO 6000 Blackwell

GGUF repos checked 2026-09-29; -c is the longest context with 0.5 GB free on the RTX PRO 6000 Blackwell. No -ngl: llama.cpp’s -fit, on by default since b7440, places the layers.

Mistral Medium 3.5 128B, Q4_K_M with 41K tokens: 10–14 tokens/s on the RTX PRO 6000 Blackwell

llama-server -hf unsloth/Mistral-Medium-3.5-128B-GGUF:Q4_K_M -c 41984

Q4_K_M/Mistral-Medium-3.5-128B-Q4_K_M-00001-of-00003.gguf, 74.9 GB: 95.2 GB used and 867 MB free of the RTX PRO 6000 Blackwell's 96 GB, 10–14 tokens/s once the 41K cache is full.

Qwen3.5 122B-A10B, Q4_K_M with 256K tokens: 40–69 tokens/s on the RTX PRO 6000 Blackwell

llama-server -hf unsloth/Qwen3.5-122B-A10B-GGUF:Q4_K_M -c 262144

Q4_K_M/Qwen3.5-122B-A10B-Q4_K_M-00001-of-00003.gguf, 76.5 GB: 85.5 GB used and 10.5 GB free of the RTX PRO 6000 Blackwell's 96 GB, 40–69 tokens/s once the 256K cache is full.

How fast the RTX PRO 6000 Blackwell writes as the context fills

Tokens per second at Q4_K_M (gpt-oss-120b at MXFP4) for the biggest models that fit with 32K tokens, with 8K and 32K tokens in the cache and at the longest context the card holds, and at Q8_0 with 8K where that fits. Each token reads the active weights and the whole cache once, so the speed follows the 1,792 GB/s of bandwidth and falls as the cache grows. The ceiling is that bandwidth divided by the bytes read per token, which no runtime reaches; the reply time is for 1,000 new tokens with 32K (or the longest context) in the cache, prompt processing aside. With the same memory, Ryzen AI Max+ 395 (128 GB), 256 GB/s, writes 86% slower by bandwidth; M4 Max Mac (128 GB), 546 GB/s, writes 70% slower by bandwidth; 4× RTX 3090, 3,744 GB/s, writes 109% faster by bandwidth.

Model	8K	32K	Longest	At the longest	Q8_0 at 8K	Bandwidth ceiling at 8K	1,000-token reply at 32K
Mistral Medium 3.5 128B	12–17	11–15	41K	10–14	Does not fit	22	67–92 s
Qwen3.5 122B-A10B	76–134	70–123	256K	40–69	Does not fit	287	8–14 s
Nemotron 3.5 Super VL 120B-A12B	66–115	65–112	1M	32–55	Does not fit	245	9–16 s
Nemotron 3 Super 120B-A12B	66–115	65–112	256K	53–91	Does not fit	245	9–16 s
gpt-oss-120b	135–247	110–198	128K	63–110	MXFP4 only	564	5–9 s
AliceAI Foundation 80B-A3B	190–364	157–292	256K	59–103	128–234	889	3–6 s
Qwen3-Coder-Next	190–364	157–292	256K	59–103	128–234	889	3–6 s
Kolibri-1	170–319	146–270	256K	64–112	113–204	758	4–7 s
Llama 3.1 70B	21–29	18–25	128K	11–16	12–17	39	40–56 s
K2-Horizon MoVA 36B-A4B	111–200	56–96	348K	7.3–12	81–142	445	10–18 s

Near misses on RTX PRO 6000 Blackwell

Models that miss at Q4_K_M with 32,768 tokens of context (or their whole window), or leave under 0.5 GB free there, but fit with a shorter context or 3-bit weights, or are short by at most a quarter of the memory and fit with MoE offload. Smallest shortfall first.

Model	Short by at Q4_K_M	Fits instead
Qwen3.8 Flash Next 180.0B	16.9 GB	GGUF Q3_K_M with 32K, 91.5 GB
Step 3.7 Flash 196B-A11B 201.4B	31.1 GB	GGUF IQ3_XXS with 32K, 87.4 GB

What more cards or MoE offload unlock

One, two and four RTX PRO 6000 Blackwell cards in one tensor-parallel group (vLLM or SGLang), with the models each step adds at Q4_K_M and 8K context and their tokens per second on that group, and the median on-demand rent for that many cards (getdeploying.com, 2026-09-29).

Setup	Memory	Bandwidth	Models at Q4_K_M	Rent per hour	Adds (tokens/s)
RTX PRO 6000 Blackwell	96 GB	1,792 GB/s	48	$2.19	This page
2× RTX PRO 6000 Blackwell	192 GB	3,584 GB/s	53	$4.38	Qwen3.8 Flash Next 142–291, Step 3.7 Flash 196B-A11B 98–188, MiniMax M2.7 90–171, DeepSeek V4 Flash 87–164, DeepSeek V4 Flash 0731 87–164
4× RTX PRO 6000 Blackwell	384 GB	7,168 GB/s	58	$8.76	MiMo V2.6 Flash 127–255, IQuest-Q1 320B-A15B 115–227, GLM-5.3 Flash 116–228, MiniMax M3 96–183, Ling 3.1 Flash 560B-A25B 95–180

Run locally or rent an RTX PRO 6000 Blackwell?

Renting an RTX PRO 6000 Blackwell costs a median of $2.19 an hour on demand, across the providers getdeploying.com tracks (checked 2026-09-29), so every $1,000 of purchase price equals about 457 hours of rent, 19 days around the clock. At that rate Mistral Medium 3.5 128B at Q4_K_M with 32K tokens, the biggest model here that fits with a long context, writes 11–15 tokens/s for one request: about $41–56 per million generated tokens. Electricity and resale value are left out, and no card prices are listed because 2026 street prices vary widely between sellers; compare current ones before you decide.

Serving with vLLM on the RTX PRO 6000 Blackwell

The KV cache pool vLLM builds at its default --gpu-memory-utilization 0.92 and --max-model-len 32768, weights as published, from the vLLM KV cache calculator. The pool is shared by every request; the last column assumes 4,096 tokens each.

Model	KV cache per GPU	Pool (tokens)	At 32K	Requests at 4K
gpt-oss-120b	19.6 GB	379,772	11.59x	69
Qwen3 30B-A3B	27.2 GB	296,640	9.05x	72
gpt-oss-20b	70.4 GB	2,045,825	62.43x	374

How these numbers are worked out

Memory is the weights at each precision plus the KV cache for 8,192 tokens in FP16 and runtime overhead (0.5 GB plus 10%), computed from each model's files on Hugging Face with the LLM VRAM Calculator. Speed is estimated from memory bandwidth and the parameters read per token with the LLM Speed Calculator. Both tools take any other model from Hugging Face.

Nearby GPUs and setups

How many of the tracked models each one holds at Q4_K_M with 8K context, against 48 here.

Setup	How it relates	Memory	Bandwidth	Models at Q4_K_M
Ryzen AI Max+ 395 (128 GB)	Same memory	96 GB usable	256 GB/s	48 (same)
M4 Max Mac (128 GB)	Same memory	96 GB usable	546 GB/s	48 (same)
4× RTX 3090	Same memory	96 GB	3,744 GB/s	48 (same)
H100 SXM	Next size down	80 GB	3,350 GB/s	47 (−1)
DGX Spark (128 GB)	Next size up	120 GB usable	273 GB/s	49 (+1)

Model numbers read from Hugging Face on 2026-10-20.