Nemotron 3 Nano 4B VRAM requirements

VRAM by quantization

Weights plus the KV cache for 8,192 tokens in FP16 and the runtime overhead (0.5 GB plus 10%). Each row opens the calculator with that setting. What the GGUF names mean.

KV cache at long context

4 of its 42 layers use full attention and 38 are Mamba or feed-forward layers with no growing cache. Each extra token of context adds 16 KB of FP16 cache per request. How the KV cache works.

Which GPUs can run Nemotron 3 Nano 4B

With 8,192 tokens of context. Several GPUs means one tensor-parallel group of 2, 4 or 8 cards.

How fast Nemotron 3 Nano 4B writes

Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth. A dash means it does not fit on one card. Try other settings in the speed calculator.

Longest context on one GPU

How many tokens of context fit on a single card with one request and an FP16 KV cache. "Full" means the model's whole context window fits.

Model details

Parameters

4.0B (3,973,556,832)

Layers

4 of its 42 layers use full attention and 38 are Mamba or feed-forward layers with no growing cache

Attention cache

8 KV heads × 128

Context length

262,144 tokens

Published weights

7.40 GB (BF16)

On Hugging Face

nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16

Other models

• DeepSeek V4 Flash 0731
• Qwen3.6 35B-A3B
• Qwen3.6 27B
• Qwen3.8 27B
• Qwen3.8 Flash Next
• Qwen3.8 2.4T-A95B
• Qwen3.5 9B
• Qwen3.5 122B-A10B
• Qwen3-Coder-Next
• DeepSeek V4.1 Flash
• DeepSeek V4 Flash
• DeepSeek V4 Pro
• DeepSeek V3.2
• GLM-5.3
• GLM-5.3 Flash
• GLM-5.2
• GLM-4.7 Flash
• Gemma 4 31B
• Gemma 4 26B-A4B
• Gemma 4 12B
• Gemma 4 E4B
• Kimi K3
• MiniMax M3
• MiniMax M2.7
• MiMo V2.6 Flash
• MiMo V2.6 Pro
• Mistral Medium 3.5 128B
• Nemotron 3 Nano 30B-A3B
• Nemotron 3 Super 120B-A12B
• Xing 4.0 29B-A4B
• Muse Glimmer 30B
• MiniCPM5 2B
• Llama 3.1 8B
• Llama 3.1 70B
• Qwen3 8B
• Qwen3 30B-A3B
• gpt-oss-20b
• gpt-oss-120b
• DeepSeek V3 / R1