Kolibri-1 VRAM requirements
Kolibri-1 needs about 49.2 GB of VRAM at Q4_K_M with 8K context, so it needs a 64 GB setup such as 2× RTX 5090; a 24 GB card is too small. Kolibri-1 has 78.1B parameters, of which about 3.5B are used per token; all 384 experts still have to be in memory. With the same 8K context and one request it needs 80.8 GB at FP8 and 161 GB at FP16/BF16. The published weights take 73.4 GB (FP8). It does not fit on a 24 GB card at Q4_K_M; the smallest setup here that holds it with an 8K context is 2× RTX 5090 (64 GB).
49.2 GB at Q4_K_M, 8K context, one request
- FP8
- 80.8 GB
- FP16 / BF16
- 161 GB
- Published weights
- 73.4 GB
- Smallest setup, Q4_K_M
- 2× RTX 5090 (64 GB)
Worked example: Kolibri-1 with 32K tokens
Inputs: Kolibri-1 · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (78.1B at Q4_K_M)
- 44.0 GB
- KV cache (20 KB per token)
- 740 MB
- Buffers and runtime (0.5 GB + 10%)
- 4.97 GB
- Total
- 49.7 GB
The smallest setup here that holds it is 2× RTX 5090 (64 GB): split across the 2 cards, with the 0.5 GB runtime base on each of them, it takes 50.2 GB and leaves about 13.8 GB to spare. Change the inputs in the calculator
What makes Kolibri-1's memory use different
Its 40 sliding-window layers keep only the last 513 tokens (llama.cpp gives them 1,280 cells: the window plus a 512-token batch, rounded up to 256): at its 256K limit they hold 100 MB of cache instead of the 20.0 GB they would need with full attention.
Per token of context it adds 20 KB of FP16 cache; Llama 3.1 70B (70.6B), the nearest-sized model here with plain full attention, adds 320 KB, so Kolibri-1 needs 6% as much.
As a mixture-of-experts model it reads about 1.95 GB of its 44.0 GB Q4_K_M weights per generated token (4%), so it writes like a much smaller model while needing memory for all of them.
How much VRAM does Kolibri-1 need?
Kolibri-1 needs 49.2 GB at Q4_K_M, 85.8 GB at Q8_0 and 161 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (FP8) | 73.4 GB | 81.5 GB | Ryzen AI Max+ 395 (128 GB, 96 GB usable) |
| FP16 / BF16 | 145 GB | 161 GB | B200 (180 GB) |
| FP8 / INT8 | 72.7 GB | 80.8 GB | Ryzen AI Max+ 395 (128 GB, 96 GB usable) |
| INT4 (AWQ / GPTQ) | 38.6 GB | 43.3 GB | M4 Pro Mac (64 GB, 48 GB usable) |
| GGUF Q8_0 | 77.3 GB | 85.8 GB | Ryzen AI Max+ 395 (128 GB, 96 GB usable) |
| GGUF Q6_K | 59.6 GB | 66.4 GB | A100 80GB |
| GGUF Q5_K_M | 51.6 GB | 57.5 GB | 2× RTX 5090 (64 GB) |
| GGUF Q4_K_M | 44.0 GB | 49.2 GB | 2× RTX 5090 (64 GB) |
| GGUF IQ4_XS | 39.6 GB | 44.3 GB | M4 Pro Mac (64 GB, 48 GB usable) |
| GGUF Q3_K_M | 35.6 GB | 39.9 GB | M4 Pro Mac (64 GB, 48 GB usable) |
| GGUF IQ3_XXS | 30.0 GB | 33.8 GB | M4 Pro Mac (64 GB, 48 GB usable) |
| GGUF Q2_K | 30.5 GB | 34.3 GB | M4 Pro Mac (64 GB, 48 GB usable) |
Fine-tuning Kolibri-1? Kolibri-1 VRAM for LoRA, QLoRA and full training.
Run Kolibri-1 with Aleph Alpha’s vLLM plugin
The official FP8 checkpoint requires the aleph-alpha-inference package, which installs the supported vLLM version. Aleph Alpha lists two A100/H100 80 GB cards or one H200/B200/B300 as minimum setups. The command below uses two cards and an FP8 KV cache, matching the publisher’s cache precision.
pip install 'aleph-alpha-inference>=1'vllm serve Aleph-Alpha/Kolibri-1 --tensor-parallel-size 2 --kv-cache-dtype fp8 --reasoning-parser kolibri1 --tool-call-parser kolibri1 --enable-auto-tool-choice --max-model-len 8192Start with an 8K context to leave more runtime headroom. On a single H200 or B200, set tensor-parallel-size to 1. The site’s tables use FP16 KV cache, so their total differs from this FP8-cache command. The official model card describes contexts up to 1,048,576 tokens and the required overrides; those settings need more memory. ModelVRAM has not executed this command on a GPU.
Kolibri-1 at 8K, 32K, 128K and 256K (full) tokens of context
10 of its 50 layers use full attention and 40 keep a sliding window of 513 tokens. Each extra token of context adds 20 KB of FP16 cache per request once the sliding windows are full. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.
| Context | KV cache, FP16 | Total at Q4_K_M | Total at Q8_0 | Smallest setup, Q4_K_M |
|---|---|---|---|---|
| 8K tokens | 260 MB | 49.2 GB | 85.8 GB | 2× RTX 5090 (64 GB) |
| 32K tokens | 740 MB | 49.7 GB | 86.3 GB | 2× RTX 5090 (64 GB) |
| 128K tokens | 2.60 GB | 51.8 GB | 88.4 GB | 2× RTX 5090 (64 GB) |
| 256K tokens | 5.10 GB | 54.5 GB | 91.1 GB | 2× RTX 5090 (64 GB) |
How fast Kolibri-1 writes
Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.
| Hardware | Bandwidth | Q4_K_M | FP8 |
|---|---|---|---|
| RTX 3060 12GB | 360 GB/s | — | — |
| RTX 4090 | 1,008 GB/s | — | — |
| RTX 5090 | 1,792 GB/s | — | — |
| M4 Max Mac (128 GB) | 546 GB/s | 63–109 | 41–71 |
| M3 Ultra Mac Studio (512 GB) | 819 GB/s | 90–159 | 60–104 |
| H100 SXM | 3,350 GB/s | 260–523 | — |
| H200 | 4,800 GB/s | 318–673 | 244–487 |
Longest context on one GPU
How many tokens of context Kolibri-1 fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.
| GPU | Memory | Q4_K_M | Q8_0 | FP8 |
|---|---|---|---|---|
| RTX 3060 | 12 GB | No | No | No |
| RTX 4060 Ti 16GB | 16 GB | No | No | No |
| RTX 3090 / 4090 | 24 GB | No | No | No |
| RTX 5090 | 32 GB | No | No | No |
| A100 40GB | 40 GB | No | No | No |
| Mac, 64 GB unified memory | 48 GB | No | No | No |
| L40S / RTX 6000 Ada | 48 GB | No | No | No |
| A100 / H100 80GB | 80 GB | 256K (full) | No | No |
| Mac, 128 GB unified memory | 96 GB | 256K (full) | 256K (full) | 256K (full) |
| H200 | 141 GB | 256K (full) | 256K (full) | 256K (full) |
| B200 | 180 GB | 256K (full) | 256K (full) | 256K (full) |
Model details
- Parameters
- 78.1B (78,103,074,560)
- Experts
- 384 routed experts, all loaded
- Active per token
- 3.5B
- Layers
- 10 of its 50 layers use full attention and 40 keep a sliding window of 513 tokens
- Attention cache
- 4 KV heads × 128
- Context length
- 262,144 tokens
- Published weights
- 73.4 GB (FP8)
- On Hugging Face
- Aleph-Alpha/
Kolibri-1 - Reported benchmarks
- None published in the model card for GPQA, MMLU-Pro or LiveCodeBench.
Why this estimate looks this way
At Q4_K_M and an 8K-token context, Kolibri-1 uses 44.0 GB for weights, 260 MB for its FP16 KV cache and 4.93 GB for estimated runtime overhead, totaling 49.2 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
This is a mixture-of-experts model: all 384 experts contribute to the 78.1B parameters held in memory, even though only about 3.5B parameters run per token. Using only active parameters would understate VRAM.
Its attention layout matters for long context: 10 of its 50 layers use full attention and 40 keep a sliding window of 513 tokens. The FP16 cache grows by about 20 KB per additional token per request once the sliding windows are full.
Kolibri-1 is served with Aleph Alpha’s vLLM plugin. The tables use this site’s FP16 cache and runtime allowance; the publisher evaluates an FP8 cache and recommends up to 262,144 tokens for serving. Its validated 1,048,576-token setting needs explicit overrides. GGUF sizes here are estimates, not verified compatible downloads.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
Kolibri-1 next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| Kolibri-1 | 78.1B, 3.5B active | 49.2 GB | 20 KB | 2× RTX 5090 (64 GB) |
| Qwen3-Coder-Next | 79.7B, 3B active | 50.1 GB | 24 KB | 2× RTX 5090 (64 GB) |
| AliceAI Foundation 80B-A3B | 81.3B, 3B active | 51.1 GB | 24 KB | 2× RTX 5090 (64 GB) |
| Llama 3.1 70B | 70.6B | 47.0 GB | 320 KB | M4 Pro Mac (64 GB, 48 GB usable) |
Kolibri-1 VRAM questions
How much VRAM do I need to run Kolibri-1 locally?
About 49.2 GB at Q4_K_M with an 8K context and one request, by the bits-per-weight estimate, and 161 GB at BF16. The smallest setup listed here that holds Q4_K_M is the 2× RTX 5090 (64 GB).
How much more VRAM does Kolibri-1 need for 32K or 128K tokens of context?
Its FP16 KV cache is 260 MB at 8K, 740 MB at 32K, 2.60 GB at 128K for one request, so 128K adds 2.34 GB over 8K, plus 10% overhead. A q8_0 cache
Does Kolibri-1 need less VRAM because only 3.5B parameters are active?
No. All 384 experts stay loaded, so the Q4_K_M weights take 44.0 GB. The active parameters decide speed: each generated token reads about 1.95 GB of them.
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.