Ternary Bonsai 2 27B VRAM Requirements on 8, 12 and 16 GB GPUs

Which GPUs run Bonsai 2 27B, and at what context

Each cell is the longest context that fits the card with at least 0.5 GB to spare, and the VRAM it takes: the whole file on the GPU (-ngl 99), the KV cache of the 16 attention layers (the other 48 layers keep a fixed-size state), and 1 GB of compute buffers. Text only, no MTP.

With the vision projector loaded (the 0.59 GB Q8_0 file, for image input) the same cards hold: 8 GB, PTQ1_0 at 16K (7.41 GB); 12 GB, PTQ1_0 or PQ2_0 at 128K (9.37 GB and 10.5 GB); 16 GB, both at 256K (11.6 GB and 12.8 GB), all with a q4_0 KV cache.

Between "fits" and "does not fit": PTQ1_0 at 64K with q4_0 takes 7.66 GB, which loads on an 8 GB card only when nothing else uses it (no desktop on that GPU). A 24 GB card runs every file at the full 256K, even with an f16 cache for PTQ1_0 (22.5 GB).

Every Bonsai 2 27B file, in bytes

Bytes as the Hugging Face API reports them. "Decimal GB" divides by 1,000,000,000; the site's GB (like nvidia-smi) divides by 1024³, which is why the same file reads 5.95 GB in one place and 5.54 GB here.

Why you see 5.95 GB, 7.2 GB and 8.60 GB

5.95 GB is the PTQ1_0 GGUF, 5,946,648,928 bytes, in decimal gigabytes, which is how the model card's own table lists it (and PQ2_0 as 7.21 GB). In GiB, the unit GPU memory is sold and reported in, it is 5.54 GB. "Under 6 GB" and "5.54 GB" describe the same file.

7.2 GB is the PQ2_0 GGUF (7,206,168,928 bytes, 6.71 GB), the 2.13-bit file; the first community MTP grafts were built on it.

8.60 GB is a different download: the MLX 2-bit pack for Apple Silicon, one model.safetensors of 8,595,477,990 bytes (8.01 GB). Its README splits it into a 7.67 GB language model and a 0.92 GB vision tower, and explains the larger size: MLX stores a scale and a bias per group of 128 weights, so the same ternary weights cost 2.25 bits each instead of 1.75 in the GGUF PTQ1_0 packing. Articles that quote 8.60 GB for "Bonsai 2 27B" are describing the Mac pack, not the file a PC with an NVIDIA card downloads.

How to run it

PTQ1_0 and PQ2_0 are new GGML tensor types that only PrismML's llama.cpp fork implements (release prism-b10743, 2026-09-25). The pull request that would add them to mainline, #29077, was closed unmerged on 2026-09-22, so Ollama, LM Studio and stock llama.cpp builds cannot load these files. With the fork, the commands are the usual llama-server ones:

huggingface-cli download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PTQ1_0.gguf --local-dir .

8 GB card, 32K context:

llama-server -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 -c 32768 -fa on -ctk q4_0 -ctv q4_0

12 GB card, full 256K context:

llama-server -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 -c 262144 -fa on -ctk q4_0 -ctv q4_0

16 GB card, PQ2_0 at 256K:

llama-server -m Ternary-Bonsai-2-27B-PQ2_0.gguf -ngl 99 -c 262144 -fa on -ctk q4_0 -ctv q4_0

The community file with the MTP layer grafted back (sudoingX/Ternary-Bonsai-2-27B-PTQ1_0-MTP-GGUF, 6.53 GB) adds 0.99 GB and lets the fork run --spec-type draft-mtp. Its author measured 25.0 tokens/s without and 27.1 with one draft token on an RTX 3060 12GB at 128K.

On a Mac, llama.cpp from the fork runs the same GGUF files through Metal. The MLX pack is 8.01 GB of weights; with this site's method and an f16 cache that is about 11.0 GB at 32K and 17.0 GB at 128K. Its PACK-RUNTIME.md says ordinary MLX loaders do not apply the required Hadamard transforms, so use the runtime bundled in the repo, and that vision and MTP are not included, although the README counts a vision tower in the 8.60 GB. See What can my PC run for how much of a Mac's memory the GPU may use.

What people measured

Units as each source wrote them. Filled context means the prompt actually used that many tokens.

The estimate is close for short prompts and low at long ones: at 131K filled context the RTX 4070 run used 9,918 MiB, against 8.79 GB here, so leave about 1 GB more headroom for very long prompts.

File sizes read from the Hugging Face API on 2026-09-29: prism-ml/Ternary-Bonsai-2-27B-gguf (3,581,027 downloads, 2,263 likes, Apache 2.0), prism-ml/Ternary-Bonsai-2-27B-mlx-2bit and sudoingX/Ternary-Bonsai-2-27B-PTQ1_0-MTP-GGUF. The architecture is Qwen3.8 27B's. To compare Bonsai 2 with GSQ-RCO and Unsloth UD quants of the same model, or to try any context, KV type and projector setting, open the Qwen3.8 27B quant calculator.