AliceAI Foundation 80B-A3B VRAM requirements

AliceAI Foundation 80B-A3B has 81.3B parameters, of which about 3B are used per token; all 512 experts still have to be in memory. With an 8K-token context and one request it needs about 51.1 GB of GPU memory at Q4_K_M, 84.0 GB at FP8 and 167 GB at FP16/BF16. The published weights take 151 GB (BF16). It does not fit on a 24 GB card at Q4_K_M; the smallest single GPU that holds it with an 8K context is the 80 GB A100 / H100 80GB.

Open AliceAI Foundation 80B-A3B in the calculator

VRAM by quantization

Weights plus the KV cache for 8,192 tokens in FP16 and the runtime overhead (0.5 GB plus 10%). Each row opens the calculator with that setting. What the GGUF names mean.

Precision	Weights	Total	Smallest setup
As published (BF16)	151 GB	167 GB	B200
FP16 / BF16	151 GB	167 GB	B200
FP8 / INT8	75.7 GB	84.0 GB	H200
INT4 (AWQ / GPTQ)	40.2 GB	44.9 GB	L40S / RTX 6000 Ada
GGUF Q8_0	80.4 GB	89.2 GB	H200
GGUF Q6_K	62.1 GB	69.0 GB	A100 / H100 80GB
GGUF Q5_K_M	53.7 GB	59.7 GB	A100 / H100 80GB
GGUF Q4_K_M	45.8 GB	51.1 GB	A100 / H100 80GB
GGUF Q3_K_M	37.0 GB	41.4 GB	L40S / RTX 6000 Ada
GGUF Q2_K	31.7 GB	35.6 GB	A100 40GB

KV cache at long context

12 of its 48 layers use full attention and 36 are KDA recurrent layers with no growing cache. Each extra token of context adds 24 KB of FP16 cache per request. How the KV cache works.

Context	KV cache, FP16	KV cache, FP8	Total at Q4_K_M
4K tokens	96 MB	48 MB	51.0 GB
32K tokens	768 MB	384 MB	51.7 GB
128K tokens	3.00 GB	1.50 GB	54.2 GB
256K tokens	6.00 GB	3.00 GB	57.5 GB

Which GPUs can run AliceAI Foundation 80B-A3B

With 8,192 tokens of context. Several GPUs means one tensor-parallel group of 2, 4 or 8 cards.

GPU	Memory	Q4_K_M	FP8
RTX 3060	12 GB	Needs 8	Needs 8
RTX 4060 Ti 16GB	16 GB	Needs 4	Needs 8
RTX 3090 / 4090	24 GB	Needs 4	Needs 4
RTX 5090	32 GB	Needs 2	Needs 4
A100 40GB	40 GB	Needs 2	Needs 4
Mac, 64 GB unified memory about 75% of it is usable by the GPU by default	48 GB	Needs 2	Needs 2
L40S / RTX 6000 Ada	48 GB	Needs 2	Needs 2
A100 / H100 80GB	80 GB	Fits on one	Needs 2
Mac, 128 GB unified memory about 75% of it is usable by the GPU by default	96 GB	Fits on one	Fits on one
H200	141 GB	Fits on one	Fits on one
B200	180 GB	Fits on one	Fits on one

How fast AliceAI Foundation 80B-A3B writes

Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.

Hardware	Bandwidth	Q4_K_M	FP8
RTX 3060 12GB	360 GB/s	—	—
RTX 4090	1,008 GB/s	—	—
RTX 5090	1,792 GB/s	—	—
M4 Max Mac (128 GB)	546 GB/s	72–127	48–82
M3 Ultra Mac Studio (512 GB)	819 GB/s	103–184	69–120
H100 SXM	3,350 GB/s	285–587	—
H200	4,800 GB/s	345–746	269–545

Longest context on one GPU

How many tokens of context fit on a single card with one request and an FP16 KV cache. "Full" means the model's whole context window fits.

GPU	Memory	Q4_K_M	Q8_0	FP8
RTX 3060	12 GB	No	No	No
RTX 4060 Ti 16GB	16 GB	No	No	No
RTX 3090 / 4090	24 GB	No	No	No
RTX 5090	32 GB	No	No	No
A100 40GB	40 GB	No	No	No
Mac, 64 GB unified memory	48 GB	No	No	No
L40S / RTX 6000 Ada	48 GB	No	No	No
A100 / H100 80GB	80 GB	256K (full)	No	No
Mac, 128 GB unified memory	96 GB	256K (full)	256K (full)	256K (full)
H200	141 GB	256K (full)	256K (full)	256K (full)
B200	180 GB	256K (full)	256K (full)	256K (full)

Model details

Parameters: 81.3B (81,286,433,408)
Experts: 512 routed experts, all loaded
Active per token: 3B
Layers: 12 of its 48 layers use full attention and 36 are KDA recurrent layers with no growing cache
Attention cache: 2 KV heads × 256
Context length: 262,144 tokens
Published weights: 151 GB (BF16)
On Hugging Face: yandex/AliceAI-Foundation-80B-A3B-Base

Why this estimate looks this way

At Q4_K_M and an 8K-token context, AliceAI Foundation 80B-A3B uses 45.8 GB for weights, 192 MB for its FP16 KV cache and 5.10 GB for estimated runtime overhead, totaling 51.1 GB. The overhead is a 0.5 GiB base plus 10% of weights and cache.

This is a mixture-of-experts model: all 512 experts contribute to the 81.3B parameters held in memory, even though only about 3B parameters run per token. Using only active parameters would understate VRAM.

Its attention layout matters for long context: 12 of its 48 layers use full attention and 36 are KDA recurrent layers with no growing cache. The FP16 cache grows by about 24 KB per additional token per request once any sliding windows are full.

The 36 KDA layers keep a fixed recurrent state, which this estimate does not include; only the 12 gated-attention layers add a context-growing KV cache. Real serving memory may be higher.

The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.

Other models

Numbers read from the model files on Hugging Face on 2026-09-27. Compare it with other models in the reproducible model and GPU report.