ToolSite

What LLMs can a B200 run?

A B200 gives a model 180 GB of memory and 8,000 GB/s of bandwidth. Of the 40 open models tracked here, 26 fit at Q4_K_M with an 8,192-token context, and 4 more at a lower precision.

Models that fit on one B200

The most precise weights that still fit with 8K tokens of context and one request, the memory that takes, the longest context at that precision and the writing speed. Bigger models come first.

Model Best precision Memory Longest context Tokens/s
GLM-5.3 Flash 321.3B, 18B active GGUF Q3_K_M 161 GB 1M (full) 192–367
MiMo V2.6 Flash 310.8B, 15B active As published (4.5 bits/weight) 178 GB 95K 197–379
DeepSeek V4 Flash 0731 304.2B, 13B active As published (4.4 bits/weight) 172 GB 92K 210–406
DeepSeek V4 Flash 290.9B, 13B active As published (4.4 bits/weight) 165 GB 172K 210–406
MiniMax M2.7 228.7B, 10B active GGUF Q5_K_M 169 GB 50K 188–358
Qwen3.8 Flash Next 180.0B, 6B active GGUF Q6_K 152 GB 256K (full) 275–562
Mistral Medium 3.5 128B 127.7B GGUF Q8_0 143 GB 107K 30–42
Qwen3.5 122B-A10B 125.1B, 10B active GGUF Q8_0 137 GB 256K (full) 166–312
Nemotron 3 Super 120B-A12B 123.6B, 12B active GGUF Q8_0 135 GB 256K (full) 146–270
gpt-oss-120b 116.8B, 5.1B active GGUF Q8_0 128 GB 128K (full) 257–518
Qwen3-Coder-Next 79.7B, 3B active FP16 / BF16 164 GB 256K (full) 245–488
Llama 3.1 70B 70.6B FP16 / BF16 148 GB 101K 29–41
Qwen3.6 35B-A3B 36.0B, 3B active FP16 / BF16 74.3 GB 256K (full) 246–490
Nemotron 3 Nano 30B-A3B 31.6B, 3.5B active FP16 / BF16 65.3 GB 256K (full) 225–442
Gemma 4 31B 31.3B FP16 / BF16 65.8 GB 256K (full) 63–90
GLM-4.7 Flash 31.2B, 3B active FP16 / BF16 64.9 GB 198K (full) 239–474
Xing 4.0 29B-A4B 31.2B, 4B active FP16 / BF16 64.8 GB 256K (full) 200–385
Qwen3 30B-A3B 30.5B, 3.3B active FP16 / BF16 63.9 GB 40K (full) 218–425
Muse Glimmer 30B 29.8B FP16 / BF16 61.7 GB 128K (full) 66–96
Qwen3.6 27B 27.8B FP16 / BF16 58.0 GB 256K (full) 70–102
Qwen3.8 27B 27.8B FP16 / BF16 58.0 GB 256K (full) 70–102
Gemma 4 26B-A4B 25.8B, 4B active FP16 / BF16 53.7 GB 256K (full) 202–389
gpt-oss-20b 20.9B, 3.6B active FP16 / BF16 43.6 GB 128K (full) 218–425
Gemma 4 12B 12.0B FP16 / BF16 25.4 GB 256K (full) 142–220
Qwen3.5 9B 9.7B FP16 / BF16 20.6 GB 256K (full) 168–266
Qwen3 8B 8.2B FP16 / BF16 18.5 GB 40K (full) 182–291
Llama 3.1 8B 8.0B FP16 / BF16 18.1 GB 128K (full) 185–298
Gemma 4 E4B 8.0B FP16 / BF16 17.0 GB 128K (full) 193–313
Nemotron 3 Nano 4B 4.0B FP16 / BF16 8.78 GB 256K (full) 300–541
MiniCPM5 2B 2.5B FP16 / BF16 6.02 GB 128K (full) 367–715

Too big for one card

How many B200 cards these need at Q4_K_M in one tensor-parallel group; some need more than 8.

How these numbers are worked out

Memory is the weights at each precision plus the KV cache for 8,192 tokens in FP16 and runtime overhead (0.5 GB plus 10%), computed from each model's files on Hugging Face with the LLM VRAM Calculator. Speed is estimated from memory bandwidth and the parameters read per token with the LLM Speed Calculator. Both tools take any other model from Hugging Face.

Other GPUs and Macs

Model numbers read from Hugging Face on .