Qwen3.8 27B VRAM requirements

Qwen3.8 27B has 27.8B parameters. With an 8K-token context and one request it needs about 18.3 GB of GPU memory at Q4_K_M, 29.5 GB at FP8 and 58.0 GB at FP16/BF16. The published weights take 51.7 GB (BF16). On one 24 GB RTX 3090 / 4090 it runs at Q4_K_M with up to 91K tokens of context.

Open Qwen3.8 27B in the calculator

VRAM by quantization

Weights plus the KV cache for 8,192 tokens in FP16 and the runtime overhead (0.5 GB plus 10%). Each row opens the calculator with that setting. What the GGUF names mean.

PrecisionWeightsTotalSmallest setup
As published (BF16) 51.7 GB 58.0 GB A100 / H100 80GB
FP16 / BF16 51.7 GB 58.0 GB A100 / H100 80GB
FP8 / INT8 25.9 GB 29.5 GB RTX 5090
INT4 (AWQ / GPTQ) 13.7 GB 16.2 GB RTX 3090 / 4090
GGUF Q8_0 27.5 GB 31.3 GB RTX 5090
GGUF Q6_K 21.2 GB 24.4 GB RTX 5090
GGUF Q5_K_M 18.3 GB 21.2 GB RTX 3090 / 4090
GGUF Q4_K_M 15.7 GB 18.3 GB RTX 3090 / 4090
GGUF Q3_K_M 12.6 GB 15.0 GB RTX 4060 Ti 16GB
GGUF Q2_K 10.8 GB 13.0 GB RTX 4060 Ti 16GB

KV cache at long context

16 of its 64 layers use full attention and 48 are linear-attention layers with no growing cache. Each extra token of context adds 64 KB of FP16 cache per request. How the KV cache works.

ContextKV cache, FP16KV cache, FP8Total at Q4_K_M
4K tokens 256 MB 128 MB 18.0 GB
32K tokens 2.00 GB 1.00 GB 19.9 GB
128K tokens 8.00 GB 4.00 GB 26.5 GB
256K tokens 16.0 GB 8.00 GB 35.3 GB

Which GPUs can run Qwen3.8 27B

With 8,192 tokens of context. Several GPUs means one tensor-parallel group of 2, 4 or 8 cards.

GPUMemoryQ4_K_MFP8
RTX 3060 12 GB Needs 2 Needs 4
RTX 4060 Ti 16GB 16 GB Needs 2 Needs 2
RTX 3090 / 4090 24 GB Fits on one Needs 2
RTX 5090 32 GB Fits on one Fits on one
A100 40GB 40 GB Fits on one Fits on one
Mac, 64 GB unified memory about 75% of it is usable by the GPU by default 48 GB Fits on one Fits on one
L40S / RTX 6000 Ada 48 GB Fits on one Fits on one
A100 / H100 80GB 80 GB Fits on one Fits on one
Mac, 128 GB unified memory about 75% of it is usable by the GPU by default 96 GB Fits on one Fits on one
H200 141 GB Fits on one Fits on one
B200 180 GB Fits on one Fits on one

How fast Qwen3.8 27B writes

Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth. A dash means it does not fit on one card. Try other settings in the speed calculator.

HardwareBandwidthQ4_K_MFP8
RTX 3060 12GB 360 GB/s — —
RTX 4090 1,008 GB/s 31–43 —
RTX 5090 1,792 GB/s 52–75 33–46
M4 Max Mac (128 GB) 546 GB/s 17–23 10–14
M3 Ultra Mac Studio (512 GB) 819 GB/s 25–35 16–21
H100 SXM 3,350 GB/s 92–135 59–85
H200 4,800 GB/s 124–188 82–120

Longest context on one GPU

How many tokens of context fit on a single card with one request and an FP16 KV cache. "Full" means the model's whole context window fits.

GPUMemoryQ4_K_MQ8_0FP8
RTX 3060 12 GB No No No
RTX 4060 Ti 16GB 16 GB No No No
RTX 3090 / 4090 24 GB 91K No No
RTX 5090 32 GB 207K 18K 44K
A100 40GB 40 GB 256K (full) 134K 160K
Mac, 64 GB unified memory 48 GB 256K (full) 251K 256K (full)
L40S / RTX 6000 Ada 48 GB 256K (full) 251K 256K (full)
A100 / H100 80GB 80 GB 256K (full) 256K (full) 256K (full)
Mac, 128 GB unified memory 96 GB 256K (full) 256K (full) 256K (full)
H200 141 GB 256K (full) 256K (full) 256K (full)
B200 180 GB 256K (full) 256K (full) 256K (full)

Model details

Parameters
27.8B (27,781,427,952)
Layers
16 of its 64 layers use full attention and 48 are linear-attention layers with no growing cache
Attention cache
4 KV heads × 256
Context length
262,144 tokens
Published weights
51.7 GB (BF16)
On Hugging Face
Qwen/Qwen3.8-27B

Other models

Numbers read from the model files on Hugging Face on .