Bespoke Nimble 9B VRAM requirements

How much VRAM does Bespoke Nimble 9B need?

Bespoke Nimble 9B needs 6.32 GB at Q4_K_M, 10.5 GB at Q8_0 and 19.1 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.

Fine-tuning Bespoke Nimble 9B? Bespoke Nimble 9B VRAM for LoRA, QLoRA and full training.

Bespoke Nimble 9B Ollama files: how much VRAM each needs

Ollama’s nimble:9b-q4_K_M file is 5,629,108,736 bytes (5.24 GB); at the model’s 8,192-token limit it needs 6.54 GB of VRAM, so it runs on an 8 GB card. The default nimble tag is the Q8_0 file, 8.87 GB, which needs 10.5 GB (12 GB card), and BF16 needs 19.1 GB (24 GB card). The KV cache is small: only 8 of the 32 layers keep one, 256 MB at 8,192 tokens.

Nimble is not a chat model. It reads a text and a schema of questions (choices, booleans, rubric scores) and scores the allowed answer tokens directly, one short forward pass per field, so memory is set by the file and the 8K limit, not by long generations. Ollama 0.35 or later runs it through its decision API; Bespoke Labs’ own nimble code runs the adapter on top of Qwen3.5-9B in PyTorch, which needs the BF16 base (18.0 GB, vision tower included) on a CUDA GPU or an Apple Silicon Mac.

Bespoke Labs publishes the weights as a LoRA adapter on Hugging Face (adapter_model.safetensors, 173,188,512 bytes). The merged GGUF files below, the only full copies of the model, are Ollama’s:

The Q4_K_M file is 5,629,108,736 bytes, larger than the 8,953,803,264 parameters × 4.84 bits would give, because Ollama’s Q4_K_M keeps 2,241,855,488 of the weights, the embedding and output layers among them, in Q6_K. The Q4_K_M estimates higher on this page use the parameter count; this table uses the file. File sizes from the Ollama registry manifests, read on 2026-09-29; totals as in the table above, one request, FP16 KV cache.

Bespoke Nimble 9B at 8K (full) tokens of context

8 of its 32 layers use full attention and 24 are linear-attention layers with no growing cache. Each extra token of context adds 32 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.

How fast Bespoke Nimble 9B writes

Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth. A dash means it does not fit on one card. Try other settings in the speed calculator.

Longest context on one GPU

How many tokens of context Bespoke Nimble 9B fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.

Model details

Parameters

9.0B (8,953,803,264); Bespoke Labs publishes a LoRA adapter for Qwen3.5-9B; the count is the text model in Ollama’s merged nimble GGUF, without Qwen3.5’s vision tower

Layers

8 of its 32 layers use full attention and 24 are linear-attention layers with no growing cache

Attention cache

4 KV heads × 256

Context length

8,192 tokens

Published weights

16.7 GB (BF16)

On Hugging Face

bespokelabs/Bespoke-Nimble-9B

Reported benchmarks

None published in the model card for GPQA, MMLU-Pro or LiveCodeBench.

Bespoke Nimble 9B next to similar-sized models

The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.