GLM-5.3 Flash VRAM requirements
GLM-5.3 Flash needs about 200 GB of VRAM at Q4_K_M with 8K context, so it needs a 640 GB setup such as 8× H100 SXM; a 24 GB card is too small. GLM-5.3 Flash has 321.3B parameters, of which about 18B are used per token; all 288 experts still have to be in memory. With the same 8K context and one request it needs 330 GB at FP8 and 659 GB at FP16/BF16. The published weights take 306 GB (FP8). It does not fit on a 24 GB card at Q4_K_M; the smallest setup here that holds it with an 8K context is the M3 Ultra Mac Studio (512 GB, 384 GB usable).
200 GB at Q4_K_M, 8K context, one request
- FP8
- 330 GB
- FP16 / BF16
- 659 GB
- Published weights
- 306 GB
- Smallest setup, Q4_K_M
- M3 Ultra Mac Studio (512 GB, 384 GB usable)
Worked example: GLM-5.3 Flash with 32K tokens
Inputs: GLM-5.3 Flash · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (321.3B at Q4_K_M)
- 181 GB
- KV cache (19 KB per token)
- 616 MB
- Buffers and runtime (0.5 GB + 10%)
- 18.7 GB
- Total
- 200 GB
The smallest setup here that holds it is the M3 Ultra Mac Studio (512 GB, 384 GB usable), with about 184 GB to spare. Change the inputs in the calculator
What makes GLM-5.3 Flash's memory use different
It uses multi-head latent attention: each MLA layer caches a latent of 512 values per token instead of 32,768 for 64 full key and value heads, 64.0× less.
34 of its 45 layers keep a fixed-size linear-attention state, so the cache grows by 19 KB per token where caching every layer would take 45 KB.
Its sparse-attention indexer adds 8 KB per token on top of the main cache.
As a mixture-of-experts model it reads about 10.1 GB of its 181 GB Q4_K_M weights per generated token (6%), so it writes like a much smaller model while needing memory for all of them.
How much VRAM does GLM-5.3 Flash need?
GLM-5.3 Flash needs 200 GB at Q4_K_M, 350 GB at Q8_0 and 659 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (FP8) | 306 GB | 337 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| FP16 / BF16 | 599 GB | 659 GB | 8× H200 (1,128 GB) |
| FP8 / INT8 | 299 GB | 330 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| INT4 (AWQ / GPTQ) | 159 GB | 176 GB | B200 (180 GB) |
| GGUF Q8_0 | 318 GB | 350 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| GGUF Q6_K | 245 GB | 271 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| GGUF Q5_K_M | 212 GB | 234 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| GGUF Q4_K_M | 181 GB | 200 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| GGUF IQ4_XS | 163 GB | 180 GB | B200 (180 GB) |
| GGUF Q3_K_M | 146 GB | 162 GB | B200 (180 GB) |
| GGUF IQ3_XXS | 123 GB | 136 GB | H200 (141 GB) |
| GGUF Q2_K | 125 GB | 139 GB | H200 (141 GB) |
Fine-tuning GLM-5.3 Flash? GLM-5.3 Flash VRAM for LoRA, QLoRA and full training.
GLM-5.3 Flash GGUF files on Hugging Face
The 3 GGUF files llama.cpp picks for GLM-5.3 Flash with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.
| Quant | File | Size | vs estimate | 8K | 32K | 128K |
|---|---|---|---|---|---|---|
| IQ4_XS | unsloth/ | 146 GB156,822,111,075 bytes | −10% | 161 GB | 162 GB | 164 GB |
| Q8_0 | unsloth/ | 318 GB340,981,966,112 bytes | ±0% | 350 GB | 350 GB | 352 GB |
| BF16 | unsloth/ | 598 GB641,641,064,192 bytes | ±0% | 658 GB | 658 GB | 660 GB |
GB here is GiB (1024³ bytes), as everywhere on this site.
Which GLM-5.3 Flash GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac
From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for Q4_K_M. One request, FP16 KV cache.
| Hardware | Largest GGUF, 8K | Its longest context | Q4_K_M, longest context |
|---|---|---|---|
| RTX 5060 Ti 16GB | None fits | — | — |
| RTX 4090 (24 GB) | None fits | — | — |
| RTX 5090 (32 GB) | None fits | — | — |
| L40S (48 GB) | None fits | — | — |
| H100 SXM (80 GB) | None fits | — | — |
| M4 Pro Mac (64 GB, 48 GB usable) | None fits | — | — |
| M4 Max Mac (128 GB, 96 GB usable) | None fits | — | — |
GLM-5.3 Flash at 8K, 32K, 128K and 1M (full) tokens of context
11 of its 45 layers use multi-head latent attention and 34 are linear-attention layers with no growing cache. Each extra token of context adds 19 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.
| Context | KV cache, FP16 | Total at Q4_K_M | Total at Q8_0 | Smallest setup, Q4_K_M |
|---|---|---|---|---|
| 8K tokens | 154 MB | 200 GB | 350 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| 32K tokens | 616 MB | 200 GB | 351 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| 128K tokens | 2.41 GB | 202 GB | 353 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| 1M tokens | 19.3 GB | 221 GB | 371 GB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
How fast GLM-5.3 Flash writes
Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.
| Hardware | Bandwidth | Q4_K_M | FP8 |
|---|---|---|---|
| RTX 3060 12GB | 360 GB/s | — | — |
| RTX 4090 | 1,008 GB/s | — | — |
| RTX 5090 | 1,792 GB/s | — | — |
| M4 Max Mac (128 GB) | 546 GB/s | — | — |
| M3 Ultra Mac Studio (512 GB) | 819 GB/s | 22–36 | 13–22 |
| H100 SXM | 3,350 GB/s | — | — |
| H200 | 4,800 GB/s | — | — |
Longest context on a multi-GPU group
No single card in this table holds it at Q4_K_M, Q8_0 or FP8, so these rows use GPU groups: how many tokens of context GLM-5.3 Flash fits on each, in one tensor-parallel group with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.
| Group | Memory | Q4_K_M | Q8_0 | FP8 |
|---|---|---|---|---|
| 2× RTX 3060 12GB | 24 GB | No | No | No |
| 2× RTX 3090 | 48 GB | No | No | No |
| 2× RTX 4090 | 48 GB | No | No | No |
| 2× RTX 5090 | 64 GB | No | No | No |
| 4× RTX 3090 | 96 GB | No | No | No |
| 8× H100 SXM | 640 GB | 1M (full) | 1M (full) | 1M (full) |
| 8× H200 | 1,128 GB | 1M (full) | 1M (full) | 1M (full) |
Model details
- Parameters
- 321.3B (321,323,031,390)
- Experts
- 288 routed experts, all loaded
- Active per token
- 18B
- Layers
- 11 of its 45 layers use multi-head latent attention and 34 are linear-attention layers with no growing cache
- Attention cache
- 512 values per token (compressed latent)
- Context length
- 1,048,576 tokens
- Published weights
- 306 GB (FP8)
- On Hugging Face
- zai-org/
GLM-5.3-Flash - Reported benchmarks
- None published in the model card for GPQA, MMLU-Pro or LiveCodeBench.
Why this estimate looks this way
At Q4_K_M and an 8K-token context, GLM-5.3 Flash uses 181 GB for weights, 154 MB for its FP16 KV cache and 18.6 GB for estimated runtime overhead, totaling 200 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
This is a mixture-of-experts model: all 288 experts contribute to the 321.3B parameters held in memory, even though only about 18B parameters run per token. Using only active parameters would understate VRAM.
Its attention layout matters for long context: 11 of its 45 layers use multi-head latent attention and 34 are linear-attention layers with no growing cache. The FP16 cache grows by about 19 KB per additional token per request.
GLM-5-Next cache uses llama.cpp 5fc4f3c: the indexer stores key, gate and pooled values at the selected KV precision for every context row. Fixed recurrent state, MTP and scratch buffers are excluded. Quantized sizes do not establish kernel support or measured GPU memory.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
GLM-5.3 Flash next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| GLM-5.3 Flash | 321.3B, 18B active | 200 GB | 19 KB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| IQuest-Q1 320B-A15B | 320.3B, 15B active | 201 GB | 100 KB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| MiMo V2.6 Flash | 310.8B, 15B active | 193 GB | 23 KB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
| DeepSeek V4 Flash 0731 | 304.2B, 13B active | 190 GB | 86 KB | M3 Ultra Mac Studio (512 GB, 384 GB usable) |
GLM-5.3 Flash VRAM questions
How much VRAM do I need to run GLM-5.3 Flash locally?
About 200 GB at Q4_K_M with an 8K context and one request, by the bits-per-weight estimate, and 659 GB at BF16. The smallest setup listed here that holds Q4_K_M is the M3 Ultra Mac Studio (512 GB, 384 GB usable).
Can I run GLM-5.3 Flash on a 24 GB GPU?
No. The RTX 4090 (24 GB) alone cannot hold it: the smallest listed GGUF, IQ4_XS at 146 GB, needs 161 GB with an 8K context.
Which quantization of GLM-5.3 Flash fits in 16 GB of VRAM?
The RTX 5060 Ti 16GB alone cannot hold it: the smallest listed GGUF, IQ4_XS at 146 GB, needs 161 GB with an 8K context.
Can I run GLM-5.3 Flash on a Mac?
The M4 Pro Mac (64 GB, 48 GB usable) alone cannot hold it: the smallest listed GGUF, IQ4_XS at 146 GB, needs 161 GB with an 8K context. The M4 Max Mac (128 GB, 96 GB usable) alone cannot hold it: the smallest listed GGUF, IQ4_XS at 146 GB, needs 161 GB with an 8K context. By default macOS lets the GPU use about 75% of unified memory.
How much more VRAM does GLM-5.3 Flash need for 32K or 128K tokens of context?
Its FP16 KV cache is 154 MB at 8K, 616 MB at 32K, 2.41 GB at 128K for one request, so 128K adds 2.26 GB over 8K, plus 10% overhead. For this llama.cpp layout, Q8_0 storage is estimated at 1.28 GB at 128K; kernel support has not been verified. Its 34 linear-attention layers keep a fixed-size state, so only 11 of 45 layers add cache.
Does GLM-5.3 Flash need less VRAM because only 18B parameters are active?
No. All 288 experts stay loaded, so the Q4_K_M weights take 181 GB. The active parameters decide speed: each generated token reads about 10.1 GB of them.
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.