GLM-4.7 Flash VRAM requirements
GLM-4.7 Flash needs about 20.3 GB of VRAM at Q4_K_M with 8K context, so it fits a 24 GB card such as the RTX 3090 or the RTX 4090. GLM-4.7 Flash has 31.2B parameters, of which about 3B are used per token; all 64 experts still have to be in memory. With the same 8K context and one request it needs 32.9 GB at FP8 and 64.9 GB at FP16/BF16. The published weights take 58.2 GB (BF16). The smallest setup here that holds it at Q4_K_M with an 8K context is the RTX 3090 (24 GB), which runs it with up to 64K tokens of context.
20.3 GB at Q4_K_M, 8K context, one request
- FP8
- 32.9 GB
- FP16 / BF16
- 64.9 GB
- Published weights
- 58.2 GB
- Smallest setup, Q4_K_M
- RTX 3090 (24 GB)
Worked example: GLM-4.7 Flash with 32K tokens
Inputs: GLM-4.7 Flash · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (31.2B at Q4_K_M)
- 17.6 GB
- KV cache (53 KB per token)
- 1.65 GB
- Buffers and runtime (0.5 GB + 10%)
- 2.42 GB
- Total
- 21.7 GB
The smallest setup here that holds it is the RTX 3090 (24 GB), with about 2.33 GB to spare. Change the inputs in the calculator
Can I run GLM-4.7 Flash on…
- RTX 4090: Yes, Q4_K_M
- RTX 3090: Yes, Q4_K_M
- RTX 5090: Yes, up to Q6_K
- RTX 3060 12GB: --n-cpu-moe 24
- RTX 4060 Ti 16GB: --n-cpu-moe 12
- RTX 5060 Ti 16GB: --n-cpu-moe 12
- RTX 4070 12GB: --n-cpu-moe 24
- RTX 5080 16GB: --n-cpu-moe 12
- RTX 4080 Super 16GB: --n-cpu-moe 12
- RX 7900 XTX: Yes, Q4_K_M
What makes GLM-4.7 Flash's memory use different
It uses multi-head latent attention: each layer caches a latent of 576 values per token instead of 10,240 for 20 full key and value heads, 17.8× less.
Per token of context it adds 53 KB of FP16 cache; Granite 4.2 30B (29.3B), the nearest-sized model here with plain full attention, adds 256 KB, so GLM-4.7 Flash needs 21% as much.
As a mixture-of-experts model it reads about 1.69 GB of its 17.6 GB Q4_K_M weights per generated token (10%), so it writes like a much smaller model while needing memory for all of them.
Running GLM-4.7 Flash with experts in system RAM
llama.cpp's --n-cpu-moe N keeps the experts of the first N layers in RAM. For the measured
Q4_K_M GGUF with 32K tokens of context, the smallest N
that fits each card, and the speed with 89.6 GB/s of system RAM.
Plan the offload for another GPU.
The VRAM here can sit below the 21.7 GB of the worked example because it counts the file's measured
bytes, keeps the input embedding (token_embd) in system RAM as llama.cpp does and adds a flat 1 GB of buffers, where the
estimate above adds 0.5 GB plus 10% to the preset's weights.
| GPU | --n-cpu-moe | VRAM used | In system RAM | Tokens/s |
|---|---|---|---|---|
| RTX 5060 Ti 16GB 16 GB | 12 | 15.8 GB | 3.94 GB | 25–42 |
| RTX 4090 24 GB | 0 (all on the GPU) | 19.5 GB | 170 MB | 65–114 |
| RTX 5090 32 GB | 0 (all on the GPU) | 19.5 GB | 170 MB | 108–194 |
In system RAM counts the offloaded experts plus the input embedding (token_embd), which stays in RAM even with --n-cpu-moe 0.
How much VRAM does GLM-4.7 Flash need?
GLM-4.7 Flash needs 20.3 GB at Q4_K_M, 34.9 GB at Q8_0 and 64.9 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (BF16) | 58.2 GB | 64.9 GB | A100 80GB |
| FP16 / BF16 | 58.2 GB | 64.9 GB | A100 80GB |
| FP8 / INT8 | 29.1 GB | 32.9 GB | M4 Pro Mac (64 GB, 48 GB usable) |
| INT4 (AWQ / GPTQ) | 15.4 GB | 17.9 GB | RTX 3090 (24 GB) |
| GGUF Q8_0 | 30.9 GB | 34.9 GB | M4 Pro Mac (64 GB, 48 GB usable) |
| GGUF Q6_K | 23.8 GB | 27.2 GB | RTX 5090 (32 GB) |
| GGUF Q5_K_M | 20.6 GB | 23.6 GB | RTX 3090 (24 GB) |
| GGUF Q4_K_M | 17.6 GB | 20.3 GB | RTX 3090 (24 GB) |
| GGUF IQ4_XS | 15.8 GB | 18.3 GB | RTX 3090 (24 GB) |
| GGUF Q3_K_M | 14.2 GB | 16.6 GB | RTX 3090 (24 GB) |
| GGUF IQ3_XXS | 12.0 GB | 14.1 GB | RTX 4060 Ti 16GB |
| GGUF Q2_K | 12.2 GB | 14.3 GB | RTX 4060 Ti 16GB |
Fine-tuning GLM-4.7 Flash? GLM-4.7 Flash VRAM for LoRA, QLoRA and full training.
GLM-4.7 Flash GGUF files on Hugging Face
The 8 GGUF files llama.cpp picks for GLM-4.7 Flash with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.
| Quant | File | Size | vs estimate | 8K | 32K | 128K |
|---|---|---|---|---|---|---|
| Q2_K | unsloth/ | 10.6 GB11,344,720,224 bytes | −13% | 12.6 GB | 13.9 GB | 19.4 GB |
| Q3_K_M | unsloth/ | 13.6 GB14,614,624,608 bytes | −4% | 15.9 GB | 17.3 GB | 22.7 GB |
| IQ4_XS | unsloth/ | 15.2 GB16,271,360,352 bytes | −4% | 17.6 GB | 19.0 GB | 24.4 GB |
| Q4_K_M | unsloth/ | 17.1 GB18,312,339,808 bytes | −3% | 19.7 GB | 21.1 GB | 26.5 GB |
| Q5_K_M | unsloth/ | 19.9 GB21,408,850,272 bytes | −3% | 22.9 GB | 24.2 GB | 29.7 GB |
| Q6_K | unsloth/ | 23.0 GB24,693,098,848 bytes | −4% | 26.3 GB | 27.6 GB | 33.1 GB |
| Q8_0 | ggml-org/ | 29.7 GB31,842,799,232 bytes | −4% | 33.6 GB | 34.9 GB | 40.4 GB |
| BF16 | unsloth/ | 55.8 GB59,908,837,696 bytes | −4% | 62.3 GB | 63.7 GB | 69.1 GB |
GB here is GiB (1024³ bytes), as everywhere on this site; Hugging Face shows the Q4_K_M file as 18.3 GB, in 10⁹ bytes.
Which GLM-4.7 Flash GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac
From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for Q4_K_M (17.1 GB). One request, FP16 KV cache. Where nothing fits, the --n-cpu-moe table above keeps experts in system RAM. The Q4_K_M file is 550 MB under its bits-per-weight estimate; the estimate tables and the llama-server command on this page count the estimate, so they show less context.
| Hardware | Largest GGUF, 8K | Its longest context | Q4_K_M, longest context |
|---|---|---|---|
| RTX 5060 Ti 16GB | Q2_K, 10.6 GB | 59K | No |
| RTX 4090 (24 GB) | Q5_K_M, 19.9 GB | 18K | 74K |
| RTX 5090 (32 GB) | Q6_K, 23.0 GB | 100K | 198K (full) |
| L40S (48 GB) | Q8_0, 29.7 GB | 198K (full) | 198K (full) |
| H100 SXM (80 GB) | BF16, 55.8 GB | 198K (full) | 198K (full) |
| M4 Pro Mac (64 GB, 48 GB usable) | Q8_0, 29.7 GB | 198K (full) | 198K (full) |
| M4 Max Mac (128 GB, 96 GB usable) | BF16, 55.8 GB | 198K (full) | 198K (full) |
Run GLM-4.7 Flash with llama.cpp or Ollama
llama-server -hf unsloth/GLM-4.7-Flash-GGUF:Q4_K_M -c 65536GLM-4.7-Flash-Q4_K_M.gguf, 18.3 GB, from unsloth/
ollama run glm-4.7-flash
Ollama library tag glm-4.7-flash; its quant is Ollama's choice. Same context: /set parameter num_ctx 65536.
GLM-4.7 Flash at 8K, 32K, 128K and 198K (full) tokens of context
All 47 layers use multi-head latent attention. Each extra token of context adds 53 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.
| Context | KV cache, FP16 | Total at Q4_K_M | Total at Q8_0 | Smallest setup, Q4_K_M |
|---|---|---|---|---|
| 8K tokens | 423 MB | 20.3 GB | 34.9 GB | RTX 3090 (24 GB) |
| 32K tokens | 1.65 GB | 21.7 GB | 36.3 GB | RTX 3090 (24 GB) |
| 128K tokens | 6.61 GB | 27.1 GB | 41.8 GB | RTX 5090 (32 GB) |
| 198K tokens | 10.2 GB | 31.1 GB | 45.7 GB | RTX 5090 (32 GB) |
How fast GLM-4.7 Flash writes
Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.
| Hardware | Bandwidth | Q4_K_M | FP8 |
|---|---|---|---|
| RTX 3060 12GB | 360 GB/s | — | — |
| RTX 4090 | 1,008 GB/s | 111–201 | — |
| RTX 5090 | 1,792 GB/s | 175–331 | — |
| M4 Max Mac (128 GB) | 546 GB/s | 65–114 | 44–76 |
| M3 Ultra Mac Studio (512 GB) | 819 GB/s | 94–166 | 64–112 |
| H100 SXM | 3,350 GB/s | 267–541 | 203–391 |
| H200 | 4,800 GB/s | 326–694 | 257–517 |
Longest context on one GPU
How many tokens of context GLM-4.7 Flash fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.
| GPU | Memory | Q4_K_M | Q8_0 | FP8 |
|---|---|---|---|---|
| RTX 3060 | 12 GB | No | No | No |
| RTX 4060 Ti 16GB | 16 GB | No | No | No |
| RTX 3090 / 4090 | 24 GB | 64K | No | No |
| RTX 5090 | 32 GB | 198K (full) | No | No |
| A100 40GB | 40 GB | 198K (full) | 88K | 123K |
| Mac, 64 GB unified memory | 48 GB | 198K (full) | 198K (full) | 198K (full) |
| L40S / RTX 6000 Ada | 48 GB | 198K (full) | 198K (full) | 198K (full) |
| A100 / H100 80GB | 80 GB | 198K (full) | 198K (full) | 198K (full) |
| Mac, 128 GB unified memory | 96 GB | 198K (full) | 198K (full) | 198K (full) |
| H200 | 141 GB | 198K (full) | 198K (full) | 198K (full) |
| B200 | 180 GB | 198K (full) | 198K (full) | 198K (full) |
Model details
- Parameters
- 31.2B (31,221,488,576)
- Experts
- 64 routed experts, all loaded
- Active per token
- 3B
- Layers
- All 47 layers use multi-head latent attention
- Attention cache
- 576 values per token (compressed latent)
- Context length
- 202,752 tokens
- Published weights
- 58.2 GB (BF16)
- On Hugging Face
- zai-org/
GLM-4.7-Flash
Why this estimate looks this way
At Q4_K_M and an 8K-token context, GLM-4.7 Flash uses 17.6 GB for weights, 423 MB for its FP16 KV cache and 2.30 GB for estimated runtime overhead, totaling 20.3 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
This is a mixture-of-experts model: all 64 experts contribute to the 31.2B parameters held in memory, even though only about 3B parameters run per token. Using only active parameters would understate VRAM.
Its attention layout matters for long context: all 47 layers use multi-head latent attention. The FP16 cache grows by about 53 KB per additional token per request.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
GLM-4.7 Flash next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| GLM-4.7 Flash | 31.2B, 3B active | 20.3 GB | 53 KB | RTX 3090 (24 GB) |
| Xing 4.0 29B-A4B | 31.2B, 4B active | 20.2 GB | 45 KB | RTX 3090 (24 GB) |
| Gemma 4 31B | 31.3B | 21.9 GB | 80 KB | RTX 3090 (24 GB) |
| Nemotron 3 Nano 30B-A3B | 31.6B, 3.5B active | 20.1 GB | 6 KB | RTX 3090 (24 GB) |
GLM-4.7 Flash VRAM questions
How much VRAM do I need to run GLM-4.7 Flash locally?
About 20.3 GB at Q4_K_M with an 8K context and one request, by the bits-per-weight estimate, and 64.9 GB at BF16. The Q4_K_M GGUF GLM-4.7-Flash-Q4_K_M.gguf in unsloth/
Can I run GLM-4.7 Flash on a 24 GB GPU?
Yes. On the RTX 4090 (24 GB), the largest listed GGUF that fits with an 8K context and 0.5 GB free is Q5_K_M (19.9 GB), with up to 18K tokens of context. Q4_K_M (17.1 GB) runs with up to 74K tokens of context.
Which quantization of GLM-4.7 Flash fits in 16 GB of VRAM?
On the RTX 5060 Ti 16GB, the largest listed GGUF that fits with an 8K context and 0.5 GB free is Q2_K (10.6 GB), with up to 59K tokens of context.
Can I run GLM-4.7 Flash on a Mac?
On the M4 Pro Mac (64 GB, 48 GB usable), the largest listed GGUF that fits with an 8K context and 0.5 GB free is Q8_0 (29.7 GB), with its full 198K-token context. On the M4 Max Mac (128 GB, 96 GB usable), the largest listed GGUF that fits with an 8K context and 0.5 GB free is BF16 (55.8 GB), with its full 198K-token context. By default macOS lets the GPU use about 75% of unified memory.
How much more VRAM does GLM-4.7 Flash need for 32K or 128K tokens of context?
Its FP16 KV cache is 423 MB at 8K, 1.65 GB at 32K, 6.61 GB at 128K for one request, so 128K adds 6.20 GB over 8K, plus 10% overhead. A q8_0 cache
Why does GLM-4.7 Flash use more VRAM than its GGUF file size?
The file holds only the weights. At Q4_K_M with an 8K context: 17.1 GB of weights (the file's 18,312,339,808 bytes) + 423 MB of FP16 KV cache + 2.25 GB of compute buffers and runtime (0.5 GB + 10%) = 19.7 GB.
Does GLM-4.7 Flash need less VRAM because only 3B parameters are active?
No. All 64 experts stay loaded, so the Q4_K_M file takes 17.1 GB. The active parameters decide speed: each generated token reads about 1.69 GB of them.
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.