Spark-X2.5 4B VRAM requirements
Spark-X2.5 4B has 4.1B parameters. With an 8K-token context and one request it needs about 3.47 GB of GPU memory at Q4_K_M, 5.14 GB at FP8 and 9.35 GB at FP16/BF16. The published weights take 7.66 GB (BF16). The smallest setup here that holds it at Q4_K_M with an 8K context is the RTX 4060 8GB; on one 24 GB RTX 3090 / 4090 it runs with up to 525K tokens of context.
3.47 GB at Q4_K_M, 8K context, one request
- FP8
- 5.14 GB
- FP16 / BF16
- 9.35 GB
- Published weights
- 7.66 GB
- Smallest setup, Q4_K_M
- RTX 4060 8GB
Worked example: Spark-X2.5 4B with 32K tokens
Inputs: Spark-X2.5 4B · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (4.1B at Q4_K_M)
- 2.32 GB
- KV cache (36 KB per token)
- 1.23 GB
- Buffers and runtime (0.5 GB + 10%)
- 875 MB
- Total
- 4.40 GB
The smallest setup here that holds it is the RTX 4060 8GB, with about 3.60 GB to spare. Change the inputs in the calculator
What makes Spark-X2.5 4B's memory use different
Its 27 sliding-window layers keep only the last 512 tokens (llama.cpp gives them 1,024 cells: the window plus a 512-token batch, rounded up to 256): at its 1M limit they hold 108 MB of cache instead of the 108 GB they would need with full attention.
Per token of context it adds 36 KB of FP16 cache; Granite 4.2 3B (3.7B), the nearest-sized model here with plain full attention, adds 80 KB, so Spark-X2.5 4B needs 45% as much.
How much VRAM does Spark-X2.5 4B need?
Spark-X2.5 4B needs 3.47 GB at Q4_K_M, 5.40 GB at Q8_0 and 9.35 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (BF16) | 7.66 GB | 9.35 GB | RTX 3060 12GB |
| FP16 / BF16 | 7.66 GB | 9.35 GB | RTX 3060 12GB |
| FP8 / INT8 | 3.83 GB | 5.14 GB | RTX 4060 8GB |
| INT4 (AWQ / GPTQ) | 2.03 GB | 3.16 GB | RTX 4060 8GB |
| GGUF Q8_0 | 4.07 GB | 5.40 GB | RTX 4060 8GB |
| GGUF Q6_K | 3.14 GB | 4.38 GB | RTX 4060 8GB |
| GGUF Q5_K_M | 2.71 GB | 3.91 GB | RTX 4060 8GB |
| GGUF Q4_K_M | 2.32 GB | 3.47 GB | RTX 4060 8GB |
| GGUF IQ4_XS | 2.08 GB | 3.22 GB | RTX 4060 8GB |
| GGUF Q3_K_M | 1.87 GB | 2.98 GB | RTX 4060 8GB |
| GGUF IQ3_XXS | 1.58 GB | 2.66 GB | RTX 4060 8GB |
| GGUF Q2_K | 1.60 GB | 2.69 GB | RTX 4060 8GB |
Fine-tuning Spark-X2.5 4B? Spark-X2.5 4B VRAM for LoRA, QLoRA and full training.
Spark-X2.5 4B GGUF files on Hugging Face
The 2 GGUF files llama.cpp picks for Spark-X2.5 4B with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.
| Quant | File | Size | vs estimate | 8K | 32K | 128K |
|---|---|---|---|---|---|---|
| Q4_K_M | XHToken/ | 2.42 GB2,600,224,352 bytes | +5% | 3.59 GB | 4.52 GB | 8.23 GB |
| Q8_0 | XHToken/ | 4.07 GB4,375,021,152 bytes | ±0% | 5.41 GB | 6.34 GB | 10.0 GB |
GB here is GiB (1024³ bytes), as everywhere on this site; Hugging Face shows the Q4_K_M file as 2.6 GB, in 10⁹ bytes.
Which Spark-X2.5 4B GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac
From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for Q4_K_M (2.42 GB). One request, FP16 KV cache, -np 1.
| Hardware | Largest GGUF, 8K | Its longest context | Q4_K_M, longest context |
|---|---|---|---|
| RTX 5060 Ti 16GB | Q8_0, 4.07 GB | 268K | 315K |
| RTX 4090 (24 GB) | Q8_0, 4.07 GB | 475K | 522K |
| RTX 5090 (32 GB) | Q8_0, 4.07 GB | 682K | 729K |
| L40S (48 GB) | Q8_0, 4.07 GB | 1M (full) | 1M (full) |
| H100 SXM (80 GB) | Q8_0, 4.07 GB | 1M (full) | 1M (full) |
| M4 Pro Mac (64 GB, 48 GB usable) | Q8_0, 4.07 GB | 1M (full) | 1M (full) |
| M4 Max Mac (128 GB, 96 GB usable) | Q8_0, 4.07 GB | 1M (full) | 1M (full) |
Run Spark-X2.5 4B with llama.cpp
llama-server -hf XHToken/Spark-X2.5-4B-GGUF:Q4_K_M -c 534528 -np 1Spark-X2.5-4B-Q4_K_M.gguf, 2.6 GB, from XHToken/
Spark-X2.5 4B at 8K, 32K, 128K and 1M (full) tokens of context
9 of its 36 layers use full attention and 27 keep a sliding window of 512 tokens. Each extra token of context adds 36 KB of FP16 cache per request once the sliding windows are full. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.
| Context | KV cache, FP16 | Total at Q4_K_M | Total at Q8_0 | Smallest setup, Q4_K_M |
|---|---|---|---|---|
| 8K tokens | 396 MB | 3.47 GB | 5.40 GB | RTX 4060 8GB |
| 32K tokens | 1.23 GB | 4.40 GB | 6.33 GB | RTX 4060 8GB |
| 128K tokens | 4.61 GB | 8.11 GB | 10.0 GB | RTX 3060 12GB |
| 1M tokens | 36.1 GB | 42.8 GB | 44.7 GB | M4 Pro Mac (64 GB, 48 GB usable) |
How fast Spark-X2.5 4B writes
Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth. A dash means it does not fit on one card. Try other settings in the speed calculator.
| Hardware | Bandwidth | Q4_K_M | FP8 |
|---|---|---|---|
| RTX 3060 12GB | 360 GB/s | 62–89 | 41–58 |
| RTX 4090 | 1,008 GB/s | 148–230 | 103–154 |
| RTX 5090 | 1,792 GB/s | 225–376 | 164–258 |
| M4 Max Mac (128 GB) | 546 GB/s | 90–132 | 60–87 |
| M3 Ultra Mac Studio (512 GB) | 819 GB/s | 126–191 | 87–127 |
| H100 SXM | 3,350 GB/s | 325–604 | 253–434 |
| H200 | 4,800 GB/s | 385–765 | 311–569 |
Longest context on one GPU
How many tokens of context Spark-X2.5 4B fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.
| GPU | Memory | Q4_K_M | Q8_0 | FP8 |
|---|---|---|---|---|
| RTX 3060 | 12 GB | 215K | 165K | 172K |
| RTX 4060 Ti 16GB | 16 GB | 318K | 268K | 275K |
| RTX 3090 / 4090 | 24 GB | 525K | 475K | 482K |
| RTX 5090 | 32 GB | 732K | 682K | 689K |
| A100 40GB | 40 GB | 939K | 889K | 896K |
| Mac, 64 GB unified memory | 48 GB | 1M (full) | 1M (full) | 1M (full) |
| L40S / RTX 6000 Ada | 48 GB | 1M (full) | 1M (full) | 1M (full) |
| A100 / H100 80GB | 80 GB | 1M (full) | 1M (full) | 1M (full) |
| Mac, 128 GB unified memory | 96 GB | 1M (full) | 1M (full) | 1M (full) |
| H200 | 141 GB | 1M (full) | 1M (full) | 1M (full) |
| B200 | 180 GB | 1M (full) | 1M (full) | 1M (full) |
Model details
- Parameters
- 4.1B (4,112,079,360)
- Layers
- 9 of its 36 layers use full attention and 27 keep a sliding window of 512 tokens
- Attention cache
- 4 KV heads × 256
- Context length
- 1,048,576 tokens
- Published weights
- 7.66 GB (BF16)
- On Hugging Face
- XHToken/
Spark-X2.5-4B
Why this estimate looks this way
At Q4_K_M and an 8K-token context, Spark-X2.5 4B uses 2.32 GB for weights, 396 MB for its FP16 KV cache and 789 MB for estimated runtime overhead, totaling 3.47 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
Its attention layout matters for long context: 9 of its 36 layers use full attention and 27 keep a sliding window of 512 tokens. The FP16 cache grows by about 36 KB per additional token per request once the sliding windows are full.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
Spark-X2.5 4B next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| Spark-X2.5 4B | 4.1B | 3.47 GB | 36 KB | RTX 4060 8GB |
| Nemotron 3 Nano 4B | 4.0B | 3.10 GB | 16 KB | RTX 4060 8GB |
| Granite 4.2 3B | 3.7B | 3.46 GB | 80 KB | RTX 4060 8GB |
| MiniCPM5 2B | 2.5B | 2.42 GB | 42 KB | RTX 4060 8GB |
Spark-X2.5 4B VRAM questions
How much VRAM do I need to run Spark-X2.5 4B locally?
About 3.47 GB at Q4_K_M with an 8K context and one request, by the bits-per-weight estimate, and 9.35 GB at BF16. The Q4_K_M GGUF Spark-X2.5-4B-Q4_K_M.gguf in XHToken/
Can I run Spark-X2.5 4B on a 24 GB GPU?
Yes. On the RTX 4090 (24 GB), the largest listed GGUF that fits with an 8K context and 0.5 GB free is Q8_0 (4.07 GB), with up to 475K tokens of context. Q4_K_M (2.42 GB) runs with up to 522K tokens of context.
Which quantization of Spark-X2.5 4B fits in 16 GB of VRAM?
On the RTX 5060 Ti 16GB, the largest listed GGUF that fits with an 8K context and 0.5 GB free is Q8_0 (4.07 GB), with up to 268K tokens of context. Q4_K_M (2.42 GB) runs with up to 315K tokens of context.
Can I run Spark-X2.5 4B on a Mac?
On the M4 Pro Mac (64 GB, 48 GB usable), the largest listed GGUF that fits with an 8K context and 0.5 GB free is Q8_0 (4.07 GB), with its full 1M-token context. On the M4 Max Mac (128 GB, 96 GB usable), the largest listed GGUF that fits with an 8K context and 0.5 GB free is Q8_0 (4.07 GB), with its full 1M-token context. By default macOS lets the GPU use about 75% of unified memory.
How much more VRAM does Spark-X2.5 4B need for 32K or 128K tokens of context?
Its FP16 KV cache is 396 MB at 8K, 1.23 GB at 32K, 4.61 GB at 128K for one request, so 128K adds 4.22 GB over 8K, plus 10% overhead. A q8_0 cache
Why does Spark-X2.5 4B use more VRAM than its GGUF file size?
The file holds only the weights. At Q4_K_M with an 8K context: 2.42 GB of weights (the file's 2,600,224,352 bytes) + 396 MB of FP16 KV cache + 800 MB of compute buffers and runtime (0.5 GB + 10%) = 3.59 GB.
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.