What LLMs can an RTX 4080 Super 16GB run?

An RTX 4080 Super 16GB runs 21 of the 69 open models tracked here at Q4_K_M with 8K context; the largest is gpt-oss-20b (20.9B parameters, 14.8 GB at MXFP4). It reads memory at 736 GB/s. 13 more fit at a lower precision. Each fit leaves at least 0.5 GB free on the RTX 4080 Super 16GB (why a card does not give a model all its memory).

Not sure your PC has an RTX 4080 Super 16GB? Detect your GPU in the browser.

Can I run it on an RTX 4080 Super 16GB with 32K context?

Gemma 4 26B-A4B: Only Q3_K_M
Gemma 4 31B: No, 2 cards
Qwen3.8 27B: Only Q2_K
gpt-oss-20b: Yes, MXFP4
Llama 3.1 8B: Yes, up to Q8_0
gpt-oss-120b: --n-cpu-moe 29
Qwen3.6 35B-A3B: --n-cpu-moe 13
Qwen3.6 27B: Only Q2_K
Gemma 4 12B: Yes, up to Q8_0
GLM-4.7 Flash: --n-cpu-moe 12
Qwen3.8 Flash Next: --n-cpu-moe 42
Nemotron 3 Nano 30B-A3B: Only Q2_K
Qwen3-Coder-Next: --n-cpu-moe 35
Llama 3.1 70B: No, 4 cards

Models that fit on one RTX 4080 Super 16GB

The most precise weights that still fit with 8K tokens of context and one request, the memory that takes, the longest context at that precision and the writing speed. Bigger models come first. gpt-oss-20b (14.8 GB, 76–133 tokens/s on the RTX 4080 Super 16GB) is listed at MXFP4 only, as published.

Model	Best precision	Memory	Longest context	Tokens/s
LLM-jp-4.1 32B-A3B Thinking 32.1B, 3.8B active	GGUF Q2_K	14.8 GB	17K	89–158
Nemotron 3 Nano 30B-A3B 31.6B, 3.5B active	GGUF Q2_K	14.1 GB	225K	120–216
GLM-4.7 Flash 31.2B, 3B active	GGUF Q2_K	14.3 GB	28K	109–195
Xing 4.0 29B-A4B 31.2B, 4B active	GGUF Q2_K	14.3 GB	33K	93–165
Qwen3-Coder 30B-A3B 30.5B, 3.3B active	GGUF Q2_K	14.4 GB	18K	88–155
Qwen3 30B-A3B 30.5B, 3.3B active	GGUF Q2_K	14.4 GB	18K	88–155
Muse Glimmer 30B 29.8B	GGUF Q2_K	13.5 GB	128K (full)	30–43
Granite 4.2 30B 29.3B	GGUF Q2_K	15.3 GB	8K	27–38
ThinkingCap Qwen3.8 27B 27.8B	GGUF Q3_K_M	15.0 GB	15K	27–38
Qwen3.6 27B 27.8B	GGUF Q3_K_M	15.0 GB	15K	27–38
Qwen3.8 27B 27.8B	GGUF Q3_K_M	15.0 GB	15K	27–38
Hemmingway-1 27B 27.3B	GGUF Q3_K_M	14.7 GB	19K	28–39
Gemma 4 26B-A4B 25.8B, 4B active	GGUF Q3_K_M	13.9 GB	81K	80–140
gpt-oss-20b 20.9B, 3.6B active	As published (MXFP4)	14.8 GB	34K	76–133
Gemma 4 12B 12.0B	GGUF Q8_0	14.2 GB	85K	29–41
ZDTaichu 5.0 9B 9.8B	GGUF Q8_0	11.4 GB	126K	36–50
Ornith 1.5 9B 9.7B	GGUF Q8_0	11.3 GB	130K	36–51
Qwen3.5 9B 9.7B	GGUF Q8_0	11.3 GB	130K	36–51
LensVLM 9B 9.4B	GGUF Q8_0	11.0 GB	138K	37–52
Ornith 1.0 9B 9.4B	GGUF Q8_0	11.0 GB	138K	37–52
MiMo V2.6 Distill Qwen 9B 9.4B	GGUF Q8_0	11.0 GB	138K	37–52
Bespoke Nimble 9B 9.0B	GGUF Q8_0	10.5 GB	8K (full)	39–55
Granite 4.2 8B 8.8B	GGUF Q8_0	11.4 GB	31K	36–50
LFM2.5 8B-A1B 8.5B, 1.5B active	GGUF Q8_0	9.82 GB	128,000 (full)	109–196
Qwen3 8B 8.2B	GGUF Q8_0	10.7 GB	39K	38–54
Llama 3.1 8B 8.0B	GGUF Q8_0	10.3 GB	45K	40–56
Gemma 4 E4B 8.0B	GGUF Q8_0	9.38 GB	128K (full)	44–62
Ling 3.0 Tiny 7.9B-A1.3B 7.9B, 1.3B active	GGUF Q8_0	9.15 GB	128K (full)	125–227
LightOnOCR-3 4B 4.5B	FP16 / BF16	10.1 GB	165K	41–57
Spark-X2.5 4B 4.1B	FP16 / BF16	9.35 GB	166K	44–62
Nemotron 3 Nano 4B 4.0B	FP16 / BF16	8.78 GB	256K (full)	47–66
Granite 4.2 3B 3.7B	FP16 / BF16	8.69 GB	87K	47–67
MiniCPM5 2B 2.5B	FP16 / BF16	6.02 GB	128K (full)	68–97
Limite 1B Violetto 1.0B	FP16 / BF16	2.79 GB	128K (full)	142–220

Too big for one RTX 4080 Super 16GB at 8K context

How many RTX 4080 Super 16GB cards these need at Q4_K_M (gpt-oss-120b at MXFP4) with 8K context and 0.5 GB to spare, in one tensor-parallel group; some need more than 8.

Gemma 4 31B: 2 cards
Ornith 1.0 35B: 2 cards
Ornith 1.5 35B-A3B: 2 cards
Qwen3.6 35B-A3B: 2 cards
K2-Horizon MoVA 36B-A4B: 2 cards
Llama 3.1 70B: 4 cards
Kolibri-1: 4 cards
Qwen3-Coder-Next: 4 cards
AliceAI Foundation 80B-A3B: 4 cards
gpt-oss-120b: 8 cards
Nemotron 3 Super 120B-A12B: 8 cards
Nemotron 3.5 Super VL 120B-A12B: 8 cards
Qwen3.5 122B-A10B: 8 cards
Mistral Medium 3.5 128B: 8 cards
Qwen3.8 Flash Next: 8 cards
Step 3.7 Flash 196B-A11B: more than 8 cards
MiniMax M2.7: more than 8 cards
DeepSeek V4 Flash: more than 8 cards
DeepSeek V4 Flash 0731: more than 8 cards
MiMo V2.6 Flash: more than 8 cards
IQuest-Q1 320B-A15B: more than 8 cards
GLM-5.3 Flash: more than 8 cards
MiniMax M3: more than 8 cards
Ling 3.1 Flash 560B-A25B: more than 8 cards
DeepSeek V3 / R1: more than 8 cards
DeepSeek V3.2: more than 8 cards
GLM-5.3: more than 8 cards
GLM-5.2: more than 8 cards
DeepSeek V4.1 Flash 1017: more than 8 cards
DeepSeek V4.1 Flash: more than 8 cards
Hy4 Preview 770B-A49B: more than 8 cards
MiMo V2.6 Pro: more than 8 cards
DeepSeek V4 Pro: more than 8 cards
Qwen3.8 2.4T-A95B: more than 8 cards
Kimi K3: more than 8 cards

Best models for RTX 4080 Super 16GB with room for long context

The biggest models that still fit at Q4_K_M with 32,768 tokens of context (or their whole window, if shorter), the memory left over, the longest context the card takes at that precision and the writing speed with that context in the cache, from 736 GB/s of bandwidth. gpt-oss-20b is at MXFP4, as published.

Model	Context	Memory	Left over	Longest context	Tokens/s
gpt-oss-20b 20.9B, MXFP4	32K	15.4 GB	571 MB	34K	63–109
Gemma 4 12B 12.0B	32K	8.98 GB	7.02 GB	256K (full)	46–65
ZDTaichu 5.0 9B 9.8B	32K	7.67 GB	8.33 GB	128K (full)	53–76
Ornith 1.5 9B 9.7B	32K	7.58 GB	8.42 GB	256K (full)	54–77
Qwen3.5 9B 9.7B	32K	7.58 GB	8.42 GB	256K (full)	54–77

llama-server commands for one RTX 4080 Super 16GB

GGUF repos checked 2026-09-29; -c is the longest context with 0.5 GB free on the RTX 4080 Super 16GB. No -ngl: llama.cpp’s -fit, on by default since b7440, places the layers.

gpt-oss-20b, MXFP4 with 34K tokens: 62–107 tokens/s on the RTX 4080 Super 16GB

llama-server -hf ggml-org/gpt-oss-20b-GGUF:MXFP4 -c 34816 -np 1

gpt-oss-20b-MXFP4.gguf, 12.1 GB: 15.5 GB used and 518 MB free of the RTX 4080 Super 16GB's 16 GB, 62–107 tokens/s once the 34K cache is full. -np 1: one slot, one sliding window.

Gemma 4 12B, Q4_K_M with 256K tokens: 32–45 tokens/s on the RTX 4080 Super 16GB

llama-server -hf unsloth/gemma-4-12b-it-GGUF:Q4_K_M -c 262144 -np 1

gemma-4-12b-it-Q4_K_M.gguf, 7.1 GB: 12.8 GB used and 3.17 GB free of the RTX 4080 Super 16GB's 16 GB, 32–45 tokens/s once the 256K cache is full. -np 1: one slot, one sliding window.

How fast the RTX 4080 Super 16GB writes as the context fills

Tokens per second at Q4_K_M (gpt-oss-20b at MXFP4) for the biggest models that fit with 32K tokens, with 8K and 32K tokens in the cache and at the longest context the card holds, and at Q8_0 with 8K where that fits. Each token reads the active weights and the whole cache once, so the speed follows the 736 GB/s of bandwidth and falls as the cache grows. The ceiling is that bandwidth divided by the bytes read per token, which no runtime reaches; the reply time is for 1,000 new tokens with 32K (or the longest context) in the cache, prompt processing aside. With the same memory, RTX 4060 Ti 16GB, 288 GB/s, writes 59% slower on average; RTX 5060 Ti 16GB, 448 GB/s, writes 37% slower on average; RX 9070 XT 16GB, 644 GB/s, writes 12% slower on average; RTX 4070 Ti Super 16GB, 672 GB/s, writes 8% slower on average; RTX 5070 Ti 16GB, 896 GB/s, writes 20% faster on average; RTX 5080 16GB, 960 GB/s, writes 28% faster on average.

Model	8K	32K	Longest	At the longest	Q8_0 at 8K	Bandwidth ceiling at 8K	1,000-token reply at 32K
gpt-oss-20b	76–133	63–109	34K	62–107	MXFP4 only	284	9–16 s
Gemma 4 12B	48–68	46–65	256K	32–45	29–41	93	15–22 s
ZDTaichu 5.0 9B	60–85	53–76	128K	37–53	36–50	119	13–19 s
Ornith 1.5 9B	60–86	54–77	256K	27–38	36–51	120	13–19 s
Qwen3.5 9B	60–86	54–77	256K	27–38	36–51	120	13–19 s
LensVLM 9B	62–88	55–78	256K	27–38	37–52	123	13–18 s
Ornith 1.0 9B	62–88	55–78	256K	27–38	37–52	123	13–18 s
MiMo V2.6 Distill Qwen 9B	62–88	55–78	256K	27–38	37–52	123	13–18 s
Bespoke Nimble 9B	64–93	—	8K	64–93	39–55	129	11–16 s
Granite 4.2 8B	56–80	36–50	55K	27–37	36–50	110	20–28 s

RTX 4080 Super 16GB against the other 16 GB cards

The same models fit on every card with 16 GB, so speed is what separates them. By bandwidth the RTX 4080 Super 16GB ranks 3 of 7 at 736 GB/s. Over the 12 biggest models that fit at Q4_K_M with 8K context, the RTX 4060 Ti 16GB (288 GB/s) writes 59% slower, the RTX 5060 Ti 16GB (448 GB/s) writes 37% slower, the RX 9070 XT 16GB (644 GB/s) writes 12% slower, the RTX 4070 Ti Super 16GB (672 GB/s) writes 8% slower, the RTX 5070 Ti 16GB (896 GB/s) writes 20% faster and the RTX 5080 16GB (960 GB/s) writes 28% faster than the RTX 4080 Super 16GB. Each cell below is the other card's tokens per second minus this one's, midpoints of the estimated ranges.

Model	RTX 4080 Super 16GB tokens/s	RTX 4060 Ti 16GB	RTX 5060 Ti 16GB	RX 9070 XT 16GB	RTX 4070 Ti Super 16GB	RTX 5070 Ti 16GB	RTX 5080 16GB
gpt-oss-20b	76–133	−61	−39	−12	−8.4	+20	+28
Gemma 4 12B	48–68	−34	−22	−6.9	−4.8	+12	+16
ZDTaichu 5.0 9B	60–85	−43	−27	−8.6	−5.9	+15	+20
Ornith 1.5 9B	60–86	−44	−28	−8.7	−6.0	+15	+21
Qwen3.5 9B	60–86	−44	−28	−8.7	−6.0	+15	+21
LensVLM 9B	62–88	−44	−28	−8.9	−6.1	+15	+21
Ornith 1.0 9B	62–88	−44	−28	−8.9	−6.1	+15	+21
MiMo V2.6 Distill Qwen 9B	62–88	−44	−28	−8.9	−6.1	+15	+21
Bespoke Nimble 9B	64–93	−46	−29	−9.2	−6.4	+16	+22
Granite 4.2 8B	56–80	−40	−26	−8.0	−5.6	+14	+19
LFM2.5 8B-A1B	165–309	−132	−81	−25	−17	+40	+56
Qwen3 8B	60–86	−43	−27	−8.6	−6.0	+15	+20

Near misses on RTX 4080 Super 16GB

Models that miss at Q4_K_M with 32,768 tokens of context (or their whole window), or leave under 0.5 GB free there, but fit with a shorter context or 3-bit weights, or are short by at most a quarter of the memory and fit with MoE offload. Smallest shortfall first.

Model	Short by at Q4_K_M	Fits instead
Gemma 4 26B-A4B 25.8B	1.50 GB	GGUF Q3_K_M with 32K, 14.4 GB
Muse Glimmer 30B 29.8B	3.51 GB	GGUF IQ3_XXS with 32K, 13.6 GB
Hemmingway-1 27B 27.3B	3.63 GB	GGUF IQ3_XXS with 32K, 14.2 GB
ThinkingCap Qwen3.8 27B 27.8B	3.92 GB	GGUF IQ3_XXS with 32K, 14.4 GB
Qwen3.6 27B 27.8B	3.92 GB	GGUF IQ3_XXS with 32K, 14.4 GB

What more cards or MoE offload unlock

One, two and four RTX 4080 Super 16GB cards in one tensor-parallel group (vLLM or SGLang), with the models each step adds at Q4_K_M and 8K context and their tokens per second on that group.

Setup	Memory	Bandwidth	Models at Q4_K_M	Adds (tokens/s)
RTX 4080 Super 16GB	16 GB	736 GB/s	21	This page
2× RTX 4080 Super 16GB	32 GB	1,472 GB/s	39	Gemma 4 26B-A4B 99–191, Hemmingway-1 27B 41–60, ThinkingCap Qwen3.8 27B 40–59, Qwen3.6 27B 40–59, Qwen3.8 27B 40–59, Granite 4.2 30B 36–52, Muse Glimmer 30B 38–56, Qwen3-Coder 30B-A3B 102–196, Qwen3 30B-A3B 102–196, Xing 4.0 29B-A4B 102–196, GLM-4.7 Flash 116–229, Gemma 4 31B 34–50, Nemotron 3 Nano 30B-A3B 119–236, LLM-jp-4.1 32B-A3B Thinking 100–193, Ornith 1.0 35B 125–250, Ornith 1.5 35B-A3B 125–250, Qwen3.6 35B-A3B 125–250, K2-Horizon MoVA 36B-A4B 79–148
4× RTX 4080 Super 16GB	64 GB	2,944 GB/s	43	Llama 3.1 70B 32–46, Kolibri-1 162–344, Qwen3-Coder-Next 173–375, AliceAI Foundation 80B-A3B 173–375

Every mixture-of-experts GGUF file the --n-cpu-moe planner has measured, on one RTX 4080 Super 16GB with 32K tokens of context (or the model's whole window): the smallest --n-cpu-moe that fits the card (0 means the whole file fits), the memory on the card and in system RAM, and tokens per second from 736 GB/s of card bandwidth with DDR5-5600, 2 channels (89.6 GB/s) or DDR4-3200, 2 channels (51.2 GB/s). 10 of these models do not fit whole and need offload.

Model	GGUF	--n-cpu-moe	On the card	In system RAM	Tokens/s, DDR5	Tokens/s, DDR4
gpt-oss-20b	MXFP4	0	12.5 GB	587 MB	59–103	59–103
Gemma 4 26B-A4B	UD-Q4_K_M	3	15.6 GB	2.06 GB	47–82	42–73
Qwen3 30B-A3B	Q4_K_M	15	15.9 GB	5.34 GB	28–47	22–37
Qwen3 30B-A3B	Q8_0	31	15.4 GB	18.8 GB	14–24	9–16
GLM-4.7 Flash	UD-Q4_K_XL	10	15.8 GB	3.13 GB	37–63	31–52
GLM-4.7 Flash	Q4_K_M	12	15.8 GB	3.94 GB	35–59	28–48
Nemotron 3 Nano 30B-A3B	Q4_K_M	21	15.4 GB	8.70 GB	36–61	25–43
Qwen3.6 35B-A3B	UD-IQ4_XS	5	15.9 GB	2.24 GB	56–98	52–89
Ornith 1.5 35B-A3B	Q4_K_M	13	15.6 GB	6.20 GB	50–85	39–66
Qwen3.6 35B-A3B	UD-Q4_K_M	13	15.8 GB	6.39 GB	44–75	35–60
Qwen3.6 35B-A3B	Q8_0	25	15.6 GB	20.4 GB	24–41	17–28
Ornith 1.5 35B-A3B	Q8_0	26	15.6 GB	21.2 GB	24–40	16–27
Qwen3-Coder-Next	IQ4_XS	32	15.8 GB	25.7 GB	31–52	21–35
Qwen3-Coder-Next	Q4_K_M	35	15.3 GB	31.6 GB	26–45	18–30
Qwen3-Coder-Next	UD-Q4_K_XL	35	15.9 GB	32.0 GB	24–41	17–28
gpt-oss-120b	MXFP4	29	14.8 GB	46.4 GB	14–23	9–14
Qwen3.5 122B-A10B	Q4_K_M	42	15.2 GB	57.8 GB	10–16	6–11
Qwen3.8 Flash Next	UD-Q2_K_XL	36	15.7 GB	32.6 GB	21–36	15–25
Qwen3.8 Flash Next	UD-IQ3_XXS	37	15.8 GB	35.4 GB	19–33	14–23
Qwen3.8 Flash Next	GSQ-RCO IQ3_S	39	15.3 GB	37.5 GB	19–31	13–22
Qwen3.8 Flash Next	UD-Q3_K_XL	40	15.2 GB	43.5 GB	16–28	12–20
Qwen3.8 Flash Next	UD-IQ4_XS	40	15.8 GB	46.4 GB	16–27	11–19
Qwen3.8 Flash Next	UD-Q4_K_XL	42	15.5 GB	63.1 GB	13–22	9–15
Qwen3.8 Flash Next	UD-Q5_K_XL	43	15.8 GB	82.7 GB	11–18	7–12

In system RAM counts the offloaded experts plus the input embedding (token_embd), which llama.cpp keeps in RAM in any case.

How these numbers are worked out

Memory is the weights at each precision plus the KV cache for 8,192 tokens in FP16 and runtime overhead (0.5 GB plus 10%), computed from each model's files on Hugging Face with the LLM VRAM Calculator. Speed is estimated from memory bandwidth and the parameters read per token with the LLM Speed Calculator. Both tools take any other model from Hugging Face.

Nearby GPUs and setups

How many of the tracked models each one holds at Q4_K_M with 8K context, against 21 here.

Setup	How it relates	Memory	Bandwidth	Models at Q4_K_M
RTX 4060 Ti 16GB	Same memory	16 GB	288 GB/s	21 (same)
RTX 5060 Ti 16GB	Same memory	16 GB	448 GB/s	21 (same)
RX 9070 XT 16GB	Same memory	16 GB	644 GB/s	21 (same)
RTX 4070 Ti Super 16GB	Same memory	16 GB	672 GB/s	21 (same)
RTX 5070 Ti 16GB	Same memory	16 GB	896 GB/s	21 (same)
RTX 5080 16GB	Same memory	16 GB	960 GB/s	21 (same)
RTX 5070 12GB	Next size down	12 GB	672 GB/s	20 (−1)
2× RTX 3060 12GB	Next size up	24 GB	720 GB/s	38 (+17)

Model numbers read from Hugging Face on 2026-10-20.