LFM2.5 8B-A1B VRAM requirements

LFM2.5 8B-A1B needs about 5.85 GB of VRAM at Q4_K_M with 8K context, so it fits an 8 GB card such as the RTX 4060 8GB. LFM2.5 8B-A1B has 8.5B parameters, of which about 1.5B are used per token; all 32 experts still have to be in memory. With the same 8K context and one request it needs 9.28 GB at FP8 and 18.0 GB at FP16/BF16. The published weights take 15.8 GB (BF16). The smallest setup here that holds it at Q4_K_M with an 8K context is the RTX 4060 8GB; on one 24 GB RTX 3090 / 4090 it runs with its full 128,000-token context.

5.85 GB at Q4_K_M, 8K context, one request

FP8
9.28 GB
FP16 / BF16
18.0 GB
Published weights
15.8 GB
Smallest setup, Q4_K_M
RTX 4060 8GB
Open LFM2.5 8B-A1B in the calculator

Worked example: LFM2.5 8B-A1B with 32K tokens

Inputs: LFM2.5 8B-A1B · Q4_K_M weights · 32,768 tokens of context · one request · FP16 KV cache

Weights (8.5B at Q4_K_M)
4.77 GB
KV cache (12 KB per token)
384 MB
Buffers and runtime (0.5 GB + 10%)
1.01 GB
Total
6.16 GB

The smallest setup here that holds it is the RTX 4060 8GB, with about 1.84 GB to spare. Change the inputs in the calculator

What makes LFM2.5 8B-A1B's memory use different

18 of its 24 layers keep a fixed-size short-convolution state, so the cache grows by 12 KB per token where caching every layer would take 48 KB.

Per token of context it adds 12 KB of FP16 cache; Qwen3 8B (8.2B), the nearest-sized model here with plain full attention, adds 144 KB, so LFM2.5 8B-A1B needs 8% as much.

As a mixture-of-experts model it reads about 865 MB of its 4.77 GB Q4_K_M weights per generated token (18%), so it writes like a much smaller model while needing memory for all of them.

How much VRAM does LFM2.5 8B-A1B need?

LFM2.5 8B-A1B needs 5.85 GB at Q4_K_M, 9.82 GB at Q8_0 and 18.0 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.

PrecisionWeightsTotalSmallest setup
As published (BF16) 15.8 GB 18.0 GB RTX 3090 (24 GB)
FP16 / BF16 15.8 GB 18.0 GB RTX 3090 (24 GB)
FP8 / INT8 7.89 GB 9.28 GB RTX 3060 12GB
INT4 (AWQ / GPTQ) 4.19 GB 5.21 GB RTX 4060 8GB
GGUF Q8_0 8.38 GB 9.82 GB RTX 3060 12GB
GGUF Q6_K 6.47 GB 7.72 GB RTX 4060 8GB
GGUF Q5_K_M 5.59 GB 6.75 GB RTX 4060 8GB
GGUF Q4_K_M 4.77 GB 5.85 GB RTX 4060 8GB
GGUF IQ4_XS 4.29 GB 5.32 GB RTX 4060 8GB
GGUF Q3_K_M 3.85 GB 4.84 GB RTX 4060 8GB
GGUF IQ3_XXS 3.25 GB 4.18 GB RTX 4060 8GB
GGUF Q2_K 3.30 GB 4.24 GB RTX 4060 8GB

Fine-tuning LFM2.5 8B-A1B? LFM2.5 8B-A1B VRAM for LoRA, QLoRA and full training.

LFM2.5 8B-A1B GGUF files on Hugging Face

The 7 GGUF files llama.cpp picks for LFM2.5 8B-A1B with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.

QuantFileSizevs estimate 8K32K128,000 (full)
Q3_K_M unsloth/LFM2.5-8B-A1B-GGUF 3.67 GB3,939,626,592 bytes −5% 4.64 GB4.95 GB6.15 GB
IQ4_XS unsloth/LFM2.5-8B-A1B-GGUF 3.97 GB4,264,996,448 bytes −7% 4.97 GB5.28 GB6.48 GB
Q4_K_M unsloth/LFM2.5-8B-A1B-GGUF 4.96 GB5,322,223,200 bytes +4% 6.06 GB6.36 GB7.56 GB
Q5_K_M unsloth/LFM2.5-8B-A1B-GGUF 5.92 GB6,361,886,304 bytes +6% 7.12 GB7.43 GB8.63 GB
Q6_K unsloth/LFM2.5-8B-A1B-GGUF 6.60 GB7,088,287,328 bytes +2% 7.86 GB8.17 GB9.37 GB
Q8_0 unsloth/LFM2.5-8B-A1B-GGUF 8.39 GB9,010,196,064 bytes ±0% 9.83 GB10.1 GB11.3 GB
BF16 unsloth/LFM2.5-8B-A1B-GGUF 15.8 GB16,947,260,768 bytes ±0% 18.0 GB18.3 GB19.5 GB

GB here is GiB (1024³ bytes), as everywhere on this site; Hugging Face shows the Q4_K_M file as 5.3 GB, in 10⁹ bytes.

Which LFM2.5 8B-A1B GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac

From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for Q4_K_M (4.96 GB). One request, FP16 KV cache.

HardwareLargest GGUF, 8KIts longest contextQ4_K_M, longest context
RTX 5060 Ti 16GB Q8_0, 8.39 GB 128,000 (full) 128,000 (full)
RTX 4090 (24 GB) BF16, 15.8 GB 128,000 (full) 128,000 (full)
RTX 5090 (32 GB) BF16, 15.8 GB 128,000 (full) 128,000 (full)
L40S (48 GB) BF16, 15.8 GB 128,000 (full) 128,000 (full)
H100 SXM (80 GB) BF16, 15.8 GB 128,000 (full) 128,000 (full)
M4 Pro Mac (64 GB, 48 GB usable) BF16, 15.8 GB 128,000 (full) 128,000 (full)
M4 Max Mac (128 GB, 96 GB usable) BF16, 15.8 GB 128,000 (full) 128,000 (full)

Run LFM2.5 8B-A1B with llama.cpp

llama-server -hf unsloth/LFM2.5-8B-A1B-GGUF:Q4_K_M -c 128000

LFM2.5-8B-A1B-UD-Q4_K_M.gguf, 5.3 GB, from unsloth/LFM2.5-8B-A1B-GGUF (checked 2026-09-29). On a 24 GB RTX 3090 / 4090, Q4_K_M with -c 128000 takes 7.56 GB and leaves 16.4 GB.

LFM2.5 8B-A1B at 8K, 32K and 128,000 (full) tokens of context

6 of its 24 layers use full attention and 18 are short-convolution layers with no growing cache. Each extra token of context adds 12 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.

ContextKV cache, FP16Total at Q4_K_MTotal at Q8_0Smallest setup, Q4_K_M
8K tokens 96 MB 5.85 GB 9.82 GB RTX 4060 8GB
32K tokens 384 MB 6.16 GB 10.1 GB RTX 4060 8GB
128,000 tokens 1.46 GB 7.36 GB 11.3 GB RTX 4060 8GB

How fast LFM2.5 8B-A1B writes

Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.

HardwareBandwidthQ4_K_MFP8
RTX 3060 12GB 360 GB/s 92–16461–106
RTX 4090 1,008 GB/s 207–400147–272
RTX 5090 1,792 GB/s 296–615223–437
M4 Max Mac (128 GB) 546 GB/s 131–23889–157
M3 Ultra Mac Studio (512 GB) 819 GB/s 178–338125–227
H100 SXM 3,350 GB/s 399–908323–687
H200 4,800 GB/s 455–1,087383–857

Longest context on one GPU

How many tokens of context LFM2.5 8B-A1B fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.

GPUMemoryQ4_K_MQ8_0FP8
RTX 3060 12 GB 128,000 (full)128,000 (full)128,000 (full)
RTX 4060 Ti 16GB 16 GB 128,000 (full)128,000 (full)128,000 (full)
RTX 3090 / 4090 24 GB 128,000 (full)128,000 (full)128,000 (full)
RTX 5090 32 GB 128,000 (full)128,000 (full)128,000 (full)
A100 40GB 40 GB 128,000 (full)128,000 (full)128,000 (full)
Mac, 64 GB unified memory 48 GB 128,000 (full)128,000 (full)128,000 (full)
L40S / RTX 6000 Ada 48 GB 128,000 (full)128,000 (full)128,000 (full)
A100 / H100 80GB 80 GB 128,000 (full)128,000 (full)128,000 (full)
Mac, 128 GB unified memory 96 GB 128,000 (full)128,000 (full)128,000 (full)
H200 141 GB 128,000 (full)128,000 (full)128,000 (full)
B200 180 GB 128,000 (full)128,000 (full)128,000 (full)

Model details

Parameters
8.5B (8,467,856,832)
Experts
32 routed experts, all loaded
Active per token
1.5B
Layers
6 of its 24 layers use full attention and 18 are short-convolution layers with no growing cache
Attention cache
8 KV heads × 64
Context length
128,000 tokens
Published weights
15.8 GB (BF16)
On Hugging Face
LiquidAI/LFM2.5-8B-A1B
Reported benchmarks
None published in the model card for GPQA, MMLU-Pro or LiveCodeBench.

Why this estimate looks this way

At Q4_K_M and an 8K-token context, LFM2.5 8B-A1B uses 4.77 GB for weights, 96 MB for its FP16 KV cache and 1,010 MB for estimated runtime overhead, totaling 5.85 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.

This is a mixture-of-experts model: all 32 experts contribute to the 8.5B parameters held in memory, even though only about 1.5B parameters run per token. Using only active parameters would understate VRAM.

Its attention layout matters for long context: 6 of its 24 layers use full attention and 18 are short-convolution layers with no growing cache. The FP16 cache grows by about 12 KB per additional token per request.

The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.

LFM2.5 8B-A1B next to similar-sized models

The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.

ModelParametersTotalKV per tokenSmallest setup
LFM2.5 8B-A1B 8.5B, 1.5B active 5.85 GB 12 KB RTX 4060 8GB
Qwen3 8B 8.2B 6.81 GB 144 KB RTX 4060 8GB
Granite 4.2 8B 8.8B 7.32 GB 160 KB RTX 4060 8GB
Llama 3.1 8B 8.0B 6.58 GB 128 KB RTX 4060 8GB

LFM2.5 8B-A1B VRAM questions

How much VRAM do I need to run LFM2.5 8B-A1B locally?

About 5.85 GB at Q4_K_M with an 8K context and one request, by the bits-per-weight estimate, and 18.0 GB at BF16. The Q4_K_M GGUF LFM2.5-8B-A1B-UD-Q4_K_M.gguf in unsloth/LFM2.5-8B-A1B-GGUF is 5,322,223,200 bytes (4.96 GB); with an 8K context it needs 6.06 GB. The smallest setup listed here that holds Q4_K_M is the RTX 4060 8GB.

Can I run LFM2.5 8B-A1B on a 24 GB GPU?

Yes. On the RTX 4090 (24 GB), the largest listed GGUF that fits with an 8K context and 0.5 GB free is BF16 (15.8 GB), with its full 128,000-token context.

Which quantization of LFM2.5 8B-A1B fits in 16 GB of VRAM?

On the RTX 5060 Ti 16GB, the largest listed GGUF that fits with an 8K context and 0.5 GB free is Q8_0 (8.39 GB), with its full 128,000-token context.

Can I run LFM2.5 8B-A1B on a Mac?

On the M4 Pro Mac (64 GB, 48 GB usable), the largest listed GGUF that fits with an 8K context and 0.5 GB free is BF16 (15.8 GB), with its full 128,000-token context. On the M4 Max Mac (128 GB, 96 GB usable), the largest listed GGUF that fits with an 8K context and 0.5 GB free is BF16 (15.8 GB), with its full 128,000-token context. By default macOS lets the GPU use about 75% of unified memory.

How much more VRAM does LFM2.5 8B-A1B need for 32K or 128,000 tokens of context?

Its FP16 KV cache is 96 MB at 8K, 384 MB at 32K, 1.46 GB at 128,000 for one request, so 128,000 (its limit) adds 1.37 GB over 8K, plus 10% overhead. A q8_0 cache (-ctk q8_0 -ctv q8_0) takes 797 MB at 128,000. Its 18 short-convolution layers keep a fixed-size state, so only 6 of 24 layers add cache.

Why does LFM2.5 8B-A1B use more VRAM than its GGUF file size?

The file holds only the weights. At Q4_K_M with an 8K context: 4.96 GB of weights (the file's 5,322,223,200 bytes) + 96 MB of FP16 KV cache + 1.01 GB of compute buffers and runtime (0.5 GB + 10%) = 6.06 GB.

Does LFM2.5 8B-A1B need less VRAM because only 1.5B parameters are active?

No. All 32 experts stay loaded, so the Q4_K_M file takes 4.96 GB. The active parameters decide speed: each generated token reads about 865 MB of them.

Badge for your model card

Paste it into a Hugging Face or GitHub README; it shows the Q4_K_M total above and links to this page. Q8_0 and FP16 badges.

LFM2.5 8B-A1B VRAM: 5.85 GB at Q4_K_M, 8K context

Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.