Qwen3.5 122B-A10B VRAM requirements

Running Qwen3.5 122B-A10B with experts in system RAM

llama.cpp's --n-cpu-moe N keeps the experts of the first N layers in RAM. For the measured Q4_K_M GGUF with 32K tokens of context, the smallest N that fits each card, and the speed with 89.6 GB/s of system RAM. Plan the offload for another GPU. The VRAM here can sit below the 78.9 GB of the worked example because it counts the file's measured bytes, keeps the input embedding (token_embd) in system RAM as llama.cpp does and adds a flat 1 GB of buffers, where the estimate above adds 0.5 GB plus 10% to the preset's weights.

In system RAM counts the offloaded experts plus the input embedding (token_embd), which stays in RAM even with --n-cpu-moe 0.

How much VRAM does Qwen3.5 122B-A10B need?

Qwen3.5 122B-A10B needs 78.2 GB at Q4_K_M, 137 GB at Q8_0 and 257 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.

Fine-tuning Qwen3.5 122B-A10B? Qwen3.5 122B-A10B VRAM for LoRA, QLoRA and full training.

Qwen3.5 122B-A10B GGUF files on Hugging Face

The 8 GGUF files llama.cpp picks for Qwen3.5 122B-A10B with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.

GB here is GiB (1024³ bytes), as everywhere on this site; Hugging Face shows the Q4_K_M file as 76.5 GB, in 10⁹ bytes.

Which Qwen3.5 122B-A10B GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac

From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for Q4_K_M (71.3 GB). One request, FP16 KV cache. Where nothing fits, the --n-cpu-moe table above keeps experts in system RAM.

Run Qwen3.5 122B-A10B with llama.cpp or Ollama

llama-server -hf unsloth/Qwen3.5-122B-A10B-GGUF:Q4_K_M -c 32768 --n-cpu-moe 36

Q4_K_M/Qwen3.5-122B-A10B-Q4_K_M-00001-of-00003.gguf, 76.5 GB, from unsloth/Qwen3.5-122B-A10B-GGUF (checked 2026-09-29). On the RTX 4090: 23.3 GB on the card, 49.7 GB in RAM, the --n-cpu-moe 36 row above.

ollama run qwen3.5:122b-a10b

Ollama library tag qwen3.5:122b-a10b; its quant is Ollama's choice. Same context: /set parameter num_ctx 32768.

Qwen3.5 122B-A10B at 8K, 32K, 128K and 256K (full) tokens of context

12 of its 48 layers use full attention and 36 are linear-attention layers with no growing cache. Each extra token of context adds 24 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.

How fast Qwen3.5 122B-A10B writes

Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.

Longest context on one GPU

How many tokens of context Qwen3.5 122B-A10B fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.

Model details

Parameters

125.1B (125,086,497,008)

Experts

256 routed experts, all loaded

Active per token

10B

Layers

12 of its 48 layers use full attention and 36 are linear-attention layers with no growing cache

Attention cache

2 KV heads × 256

Context length

262,144 tokens

Published weights

233 GB (BF16)

On Hugging Face

Qwen/Qwen3.5-122B-A10B

Reported benchmarks

GPQA Diamond 86.6 · MMLU-Pro 86.7 · LiveCodeBench v6 78.9. Vendor-reported: from the maker’s own model card, not rerun here; read 2026-09-30.

Qwen3.5 122B-A10B next to similar-sized models

The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.