Xing 4.0 29B-A4B VRAM requirements
Xing 4.0 29B-A4B needs about 20.2 GB of VRAM at Q4_K_M with 8K context, so it fits a 24 GB card such as the RTX 3090 or the RTX 4090. Xing 4.0 29B-A4B has 31.2B parameters, of which about 4B are used per token; all 64 experts still have to be in memory. With the same 8K context and one request it needs 32.9 GB at FP8 and 64.8 GB at FP16/BF16. The published weights take 58.1 GB (BF16). The smallest setup here that holds it at Q4_K_M with an 8K context is the RTX 3090 (24 GB), which runs it with up to 75K tokens of context.
20.2 GB at Q4_K_M, 8K context, one request
- FP8
- 32.9 GB
- FP16 / BF16
- 64.8 GB
- Published weights
- 58.1 GB
- Smallest setup, Q4_K_M
- RTX 3090 (24 GB)
Worked example: Xing 4.0 29B-A4B with 32K tokens
Inputs: Xing 4.0 29B-A4B · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (31.2B at Q4_K_M)
- 17.6 GB
- KV cache (45 KB per token)
- 1.41 GB
- Buffers and runtime (0.5 GB + 10%)
- 2.40 GB
- Total
- 21.4 GB
The smallest setup here that holds it is the RTX 3090 (24 GB), with about 2.61 GB to spare. Change the inputs in the calculator
What makes Xing 4.0 29B-A4B's memory use different
It uses multi-head latent attention: each layer caches a latent of 576 values per token instead of 10,240 for 32 full key and value heads, 17.8× less.
Per token of context it adds 45 KB of FP16 cache; Granite 4.2 30B (29.3B), the nearest-sized model here with plain full attention, adds 256 KB, so Xing 4.0 29B-A4B needs 18% as much.
As a mixture-of-experts model it reads about 2.25 GB of its 17.6 GB Q4_K_M weights per generated token (13%), so it writes like a much smaller model while needing memory for all of them.
How much VRAM does Xing 4.0 29B-A4B need?
Xing 4.0 29B-A4B needs 20.2 GB at Q4_K_M, 34.9 GB at Q8_0 and 64.8 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (BF16) | 58.1 GB | 64.8 GB | A100 80GB |
| FP16 / BF16 | 58.1 GB | 64.8 GB | A100 80GB |
| FP8 / INT8 | 29.1 GB | 32.9 GB | M4 Pro Mac (64 GB, 48 GB usable) |
| INT4 (AWQ / GPTQ) | 15.4 GB | 17.9 GB | RTX 3090 (24 GB) |
| GGUF Q8_0 | 30.9 GB | 34.9 GB | M4 Pro Mac (64 GB, 48 GB usable) |
| GGUF Q6_K | 23.8 GB | 27.1 GB | RTX 5090 (32 GB) |
| GGUF Q5_K_M | 20.6 GB | 23.6 GB | RTX 3090 (24 GB) |
| GGUF Q4_K_M | 17.6 GB | 20.2 GB | RTX 3090 (24 GB) |
| GGUF IQ4_XS | 15.8 GB | 18.3 GB | RTX 3090 (24 GB) |
| GGUF Q3_K_M | 14.2 GB | 16.5 GB | RTX 3090 (24 GB) |
| GGUF IQ3_XXS | 12.0 GB | 14.1 GB | RTX 4060 Ti 16GB |
| GGUF Q2_K | 12.2 GB | 14.3 GB | RTX 4060 Ti 16GB |
Fine-tuning Xing 4.0 29B-A4B? Xing 4.0 29B-A4B VRAM for LoRA, QLoRA and full training.
Xing 4.0 29B-A4B at 8K, 32K, 128K and 256K (full) tokens of context
All 40 layers use multi-head latent attention. Each extra token of context adds 45 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.
| Context | KV cache, FP16 | Total at Q4_K_M | Total at Q8_0 | Smallest setup, Q4_K_M |
|---|---|---|---|---|
| 8K tokens | 360 MB | 20.2 GB | 34.9 GB | RTX 3090 (24 GB) |
| 32K tokens | 1.41 GB | 21.4 GB | 36.0 GB | RTX 3090 (24 GB) |
| 128K tokens | 5.63 GB | 26.0 GB | 40.7 GB | RTX 5090 (32 GB) |
| 256K tokens | 11.3 GB | 32.2 GB | 46.9 GB | M4 Pro Mac (64 GB, 48 GB usable) |
How fast Xing 4.0 29B-A4B writes
Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.
| Hardware | Bandwidth | Q4_K_M | FP8 |
|---|---|---|---|
| RTX 3060 12GB | 360 GB/s | — | — |
| RTX 4090 | 1,008 GB/s | 93–165 | — |
| RTX 5090 | 1,792 GB/s | 149–276 | — |
| M4 Max Mac (128 GB) | 546 GB/s | 54–93 | 35–60 |
| M3 Ultra Mac Studio (512 GB) | 819 GB/s | 78–136 | 52–89 |
| H100 SXM | 3,350 GB/s | 233–461 | 171–321 |
| H200 | 4,800 GB/s | 290–600 | 220–430 |
Longest context on one GPU
How many tokens of context Xing 4.0 29B-A4B fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.
| GPU | Memory | Q4_K_M | Q8_0 | FP8 |
|---|---|---|---|---|
| RTX 3060 | 12 GB | No | No | No |
| RTX 4060 Ti 16GB | 16 GB | No | No | No |
| RTX 3090 / 4090 | 24 GB | 75K | No | No |
| RTX 5090 | 32 GB | 241K | No | No |
| A100 40GB | 40 GB | 256K (full) | 103K | 145K |
| Mac, 64 GB unified memory | 48 GB | 256K (full) | 256K (full) | 256K (full) |
| L40S / RTX 6000 Ada | 48 GB | 256K (full) | 256K (full) | 256K (full) |
| A100 / H100 80GB | 80 GB | 256K (full) | 256K (full) | 256K (full) |
| Mac, 128 GB unified memory | 96 GB | 256K (full) | 256K (full) | 256K (full) |
| H200 | 141 GB | 256K (full) | 256K (full) | 256K (full) |
| B200 | 180 GB | 256K (full) | 256K (full) | 256K (full) |
Model details
- Parameters
- 31.2B (31,215,031,088)
- Experts
- 64 routed experts, all loaded
- Active per token
- 4B
- Layers
- All 40 layers use multi-head latent attention
- Attention cache
- 576 values per token (compressed latent)
- Context length
- 262,144 tokens
- Published weights
- 58.1 GB (BF16)
- On Hugging Face
- XingChen-AGI/
Xing4.0-29B-A4B - Reported benchmarks
- None published in the model card for GPQA, MMLU-Pro or LiveCodeBench.
Why this estimate looks this way
At Q4_K_M and an 8K-token context, Xing 4.0 29B-A4B uses 17.6 GB for weights, 360 MB for its FP16 KV cache and 2.29 GB for estimated runtime overhead, totaling 20.2 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
This is a mixture-of-experts model: all 64 experts contribute to the 31.2B parameters held in memory, even though only about 4B parameters run per token. Using only active parameters would understate VRAM.
Its attention layout matters for long context: all 40 layers use multi-head latent attention. The FP16 cache grows by about 45 KB per additional token per request.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
Xing 4.0 29B-A4B next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| Xing 4.0 29B-A4B | 31.2B, 4B active | 20.2 GB | 45 KB | RTX 3090 (24 GB) |
| GLM-4.7 Flash | 31.2B, 3B active | 20.3 GB | 53 KB | RTX 3090 (24 GB) |
| Gemma 4 31B | 31.3B | 21.9 GB | 80 KB | RTX 3090 (24 GB) |
| Nemotron 3 Nano 30B-A3B | 31.6B, 3.5B active | 20.1 GB | 6 KB | RTX 3090 (24 GB) |
Xing 4.0 29B-A4B VRAM questions
How much VRAM do I need to run Xing 4.0 29B-A4B locally?
About 20.2 GB at Q4_K_M with an 8K context and one request, by the bits-per-weight estimate, and 64.8 GB at BF16. The smallest setup listed here that holds Q4_K_M is the RTX 3090 (24 GB).
How much more VRAM does Xing 4.0 29B-A4B need for 32K or 128K tokens of context?
Its FP16 KV cache is 360 MB at 8K, 1.41 GB at 32K, 5.63 GB at 128K for one request, so 128K adds 5.27 GB over 8K, plus 10% overhead. A q8_0 cache
Does Xing 4.0 29B-A4B need less VRAM because only 4B parameters are active?
No. All 64 experts stay loaded, so the Q4_K_M weights take 17.6 GB. The active parameters decide speed: each generated token reads about 2.25 GB of them.
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.