ThinkingCap Qwen3.8 27B VRAM requirements

How much VRAM does ThinkingCap Qwen3.8 27B need?

ThinkingCap Qwen3.8 27B needs 18.3 GB at Q4_K_M, 31.3 GB at Q8_0 and 58.0 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.

Fine-tuning ThinkingCap Qwen3.8 27B? ThinkingCap Qwen3.8 27B VRAM for LoRA, QLoRA and full training.

ThinkingCap Qwen3.8 27B GGUF files on Hugging Face

The 5 GGUF files llama.cpp picks for ThinkingCap Qwen3.8 27B with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.

GB here is GiB (1024³ bytes), as everywhere on this site; Hugging Face shows the Q4_K_M file as 17.4 GB, in 10⁹ bytes.

ThinkingCap Qwen3.8 27B on a 16 or 24 GB GPU

ThinkingCap is BottleCap AI’s fine-tune of Qwen3.8-27B that, on its model card, cuts reasoning tokens by 37% on average while holding 85.8% average accuracy against the base model’s 86.6%. Its config and its 55,563,006,776 bytes of safetensors are Qwen3.8 27B’s, so the memory math is too: only 16 of the 64 layers keep a KV cache, 2.00 GB at 32K and 8.00 GB at 128K for one request in FP16. Fewer thinking tokens fill less of that cache per answer; the weights are the same size.

On a 24 GB card such as the RTX 4090, BottleCap’s own Q4_K_M file (17,442,400,352 bytes, 16.2 GB) needs 18.9 GB at 8K and fits up to about 74K tokens with 0.5 GB free; its IQ4_XS (14.4 GB) goes to about 103K. Neither fits a 16 GB card: IQ4_XS already needs 16.9 GB at 8K. On 16 GB the community builds below do: mradermacher’s i1-Q3_K_M (12.6 GB) holds about 16K tokens, i1-IQ3_M about 27K and i1-Q2_K (10.1 GB) about 56K, at a quality cost BottleCap has not measured.

The BottleCap files carry the MTP head, which llama.cpp v0.4.1 or newer uses for self-speculative decoding with --spec-type draft-mtp; for images, load the 931,146,272-byte (888 MB) mmproj file from the same repo, which adds that much to every total. The GGUF repo is open; the safetensors repo asks you to request access, and both are under the PolyForm Small Business 1.0.0 license. Ollama’s library has no ThinkingCap tag. The smaller GGUF builds, with the totals this page’s method gives:

BottleCap’s Q4_K_M is 1,531,470,176 bytes (1.43 GB) smaller than ggml-org’s Qwen3.8-27B Q4_K_M: it is an importance-matrix build with its own per-tensor precision layout. mradermacher’s i1-IQ3_M and vmarcelo’s IQ4_XS-MIX are estimated here with the nearest bits-per-weight row (Q3_K_M, IQ4_XS); the file sizes are exact. File sizes from the Hugging Face file lists, read on 2026-09-30; totals as in the table above, one request, FP16 KV cache.

Which ThinkingCap Qwen3.8 27B GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac

From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for Q4_K_M (16.2 GB). One request, FP16 KV cache.

Run ThinkingCap Qwen3.8 27B with llama.cpp

llama-server -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M -c 75776

ThinkingCap-Qwen3.8-27B-Q4_K_M.gguf, 17.4 GB, from bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF (checked 2026-09-29). On a 24 GB RTX 3090 / 4090, Q4_K_M with -c 75776 takes 23.5 GB and leaves 557 MB. The file is 605 MB over the Q4_K_M estimate, so -c counts the file.

ThinkingCap Qwen3.8 27B at 8K, 32K, 128K and 256K (full) tokens of context

16 of its 64 layers use full attention and 48 are linear-attention layers with no growing cache. Each extra token of context adds 64 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.

How fast ThinkingCap Qwen3.8 27B writes

Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth. A dash means it does not fit on one card. Try other settings in the speed calculator.

Longest context on one GPU

How many tokens of context ThinkingCap Qwen3.8 27B fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.

Model details

Parameters

27.8B (27,781,427,952)

Layers

16 of its 64 layers use full attention and 48 are linear-attention layers with no growing cache

Attention cache

4 KV heads × 256

Context length

262,144 tokens

Published weights

51.7 GB (BF16)

On Hugging Face

bottlecapai/ThinkingCap-Qwen3.8-27B

Reported benchmarks

None published in the model card for GPQA, MMLU-Pro or LiveCodeBench.

ThinkingCap Qwen3.8 27B next to similar-sized models

The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.