What LLMs can an RTX 3090 run?

An RTX 3090 (24 GB) runs 38 of the 69 open models tracked here at Q4_K_M with 8K context; the largest is Ornith 1.5 35B-A3B (36.0B parameters, 23.0 GB). It reads memory at 936 GB/s. 1 more fits at a lower precision. Each fit leaves at least 0.5 GB free on the RTX 3090 (why a card does not give a model all its memory).

Not sure your PC has an RTX 3090? Detect your GPU in the browser.

Can I run it on an RTX 3090 with 32K context?

Gemma 4 26B-A4B: Yes, up to Q6_K
Gemma 4 31B: Q4_K_M, 26K context
Qwen3.8 27B: Yes, up to Q5_K_M
gpt-oss-20b: Yes, MXFP4
Llama 3.1 8B: Yes, up to BF16
gpt-oss-120b: --n-cpu-moe 24
Qwen3.6 35B-A3B: Yes, Q4_K_M
Qwen3.6 27B: Yes, up to Q5_K_M
Gemma 4 12B: Yes, up to Q8_0
GLM-4.7 Flash: Yes, Q4_K_M
Qwen3.8 Flash Next: --n-cpu-moe 37
Nemotron 3 Nano 30B-A3B: Yes, Q4_K_M
Qwen3-Coder-Next: --n-cpu-moe 26
Llama 3.1 70B: No, 4 cards

Models that fit on one RTX 3090

The most precise weights that still fit with 8K tokens of context and one request, the memory that takes, the longest context at that precision and the writing speed. Bigger models come first. gpt-oss-20b (14.8 GB, 93–166 tokens/s on the RTX 3090) is listed at MXFP4 only, as published.

Model	Best precision	Memory	Longest context	Tokens/s
K2-Horizon MoVA 36B-A4B 37.4B, 4B active	GGUF Q3_K_M	20.9 GB	20K	70–123
Ornith 1.5 35B-A3B 36.0B, 3B active	GGUF Q4_K_M	23.0 GB	33K	117–211
Qwen3.6 35B-A3B 36.0B, 3B active	GGUF Q4_K_M	23.0 GB	33K	117–211
Ornith 1.0 35B 35.1B, 3B active	GGUF Q4_K_M	22.4 GB	57K	117–211
LLM-jp-4.1 32B-A3B Thinking 32.1B, 3.8B active	GGUF Q4_K_M	21.0 GB	44K	86–152
Nemotron 3 Nano 30B-A3B 31.6B, 3.5B active	GGUF Q5_K_M	23.5 GB	10K	95–169
Gemma 4 31B 31.3B	GGUF Q4_K_M	21.9 GB	26K	24–33
GLM-4.7 Flash 31.2B, 3B active	GGUF Q4_K_M	20.3 GB	64K	105–188
Xing 4.0 29B-A4B 31.2B, 4B active	GGUF Q4_K_M	20.2 GB	75K	87–154
Qwen3-Coder 30B-A3B 30.5B, 3.3B active	GGUF Q5_K_M	23.5 GB	7K	79–139
Qwen3 30B-A3B 30.5B, 3.3B active	GGUF Q5_K_M	23.5 GB	8K	79–139
Muse Glimmer 30B 29.8B	GGUF Q5_K_M	22.3 GB	91K	23–32
Granite 4.2 30B 29.3B	GGUF Q4_K_M	20.8 GB	17K	25–35
ThinkingCap Qwen3.8 27B 27.8B	GGUF Q5_K_M	21.2 GB	41K	25–34
Qwen3.6 27B 27.8B	GGUF Q5_K_M	21.2 GB	41K	25–34
Qwen3.8 27B 27.8B	GGUF Q5_K_M	21.2 GB	41K	25–34
Hemmingway-1 27B 27.3B	GGUF Q5_K_M	20.9 GB	45K	25–35
Gemma 4 26B-A4B 25.8B, 4B active	GGUF Q6_K	22.7 GB	46K	67–117
gpt-oss-20b 20.9B, 3.6B active	As published (MXFP4)	14.8 GB	128K (full)	93–166
Gemma 4 12B 12.0B	GGUF Q8_0	14.2 GB	256K (full)	36–51
ZDTaichu 5.0 9B 9.8B	FP16 / BF16	20.8 GB	85K	25–35
Ornith 1.5 9B 9.7B	FP16 / BF16	20.6 GB	93K	25–35
Qwen3.5 9B 9.7B	FP16 / BF16	20.6 GB	93K	25–35
LensVLM 9B 9.4B	FP16 / BF16	20.1 GB	108K	26–36
Ornith 1.0 9B 9.4B	FP16 / BF16	20.1 GB	108K	26–36
MiMo V2.6 Distill Qwen 9B 9.4B	FP16 / BF16	20.1 GB	108K	26–36
Bespoke Nimble 9B 9.0B	FP16 / BF16	19.1 GB	8K (full)	27–38
Granite 4.2 8B 8.8B	FP16 / BF16	19.9 GB	28K	26–36
LFM2.5 8B-A1B 8.5B, 1.5B active	FP16 / BF16	18.0 GB	128,000 (full)	80–140
Qwen3 8B 8.2B	FP16 / BF16	18.5 GB	40K (full)	28–39
Llama 3.1 8B 8.0B	FP16 / BF16	18.1 GB	47K	29–40
Gemma 4 E4B 8.0B	FP16 / BF16	17.1 GB	128K (full)	30–42
Ling 3.0 Tiny 7.9B-A1.3B 7.9B, 1.3B active	FP16 / BF16	16.7 GB	128K (full)	91–162
LightOnOCR-3 4B 4.5B	FP16 / BF16	10.1 GB	256K (full)	51–72
Spark-X2.5 4B 4.1B	FP16 / BF16	9.35 GB	373K	55–78
Nemotron 3 Nano 4B 4.0B	FP16 / BF16	8.78 GB	256K (full)	58–83
Granite 4.2 3B 3.7B	FP16 / BF16	8.69 GB	128K (full)	59–84
MiniCPM5 2B 2.5B	FP16 / BF16	6.02 GB	128K (full)	84–122
Limite 1B Violetto 1.0B	FP16 / BF16	2.79 GB	128K (full)	171–271

Too big for one RTX 3090 at 8K context

How many RTX 3090 cards these need at Q4_K_M (gpt-oss-120b at MXFP4) with 8K context and 0.5 GB to spare, in one tensor-parallel group; some need more than 8.

Llama 3.1 70B: 2 cards
Kolibri-1: 4 cards
Qwen3-Coder-Next: 4 cards
AliceAI Foundation 80B-A3B: 4 cards
gpt-oss-120b: 4 cards
Nemotron 3 Super 120B-A12B: 4 cards
Nemotron 3.5 Super VL 120B-A12B: 4 cards
Qwen3.5 122B-A10B: 4 cards
Mistral Medium 3.5 128B: 4 cards
Qwen3.8 Flash Next: 8 cards
Step 3.7 Flash 196B-A11B: 8 cards
MiniMax M2.7: 8 cards
DeepSeek V4 Flash: 8 cards
DeepSeek V4 Flash 0731: more than 8 cards
MiMo V2.6 Flash: more than 8 cards
IQuest-Q1 320B-A15B: more than 8 cards
GLM-5.3 Flash: more than 8 cards
MiniMax M3: more than 8 cards
Ling 3.1 Flash 560B-A25B: more than 8 cards
DeepSeek V3 / R1: more than 8 cards
DeepSeek V3.2: more than 8 cards
GLM-5.3: more than 8 cards
GLM-5.2: more than 8 cards
DeepSeek V4.1 Flash 1017: more than 8 cards
DeepSeek V4.1 Flash: more than 8 cards
Hy4 Preview 770B-A49B: more than 8 cards
MiMo V2.6 Pro: more than 8 cards
DeepSeek V4 Pro: more than 8 cards
Qwen3.8 2.4T-A95B: more than 8 cards
Kimi K3: more than 8 cards

Best models for RTX 3090 with room for long context

The biggest models that still fit at Q4_K_M with 32,768 tokens of context (or their whole window, if shorter), the memory left over, the longest context the card takes at that precision and the writing speed with that context in the cache, from 936 GB/s of bandwidth.

Model	Context	Memory	Left over	Longest context	Tokens/s
Ornith 1.5 35B-A3B 36.0B	32K	23.5 GB	542 MB	33K	97–172
Qwen3.6 35B-A3B 36.0B	32K	23.5 GB	542 MB	33K	97–172
Ornith 1.0 35B 35.1B	32K	22.9 GB	1.05 GB	57K	97–172
LLM-jp-4.1 32B-A3B Thinking 32.1B	32K	22.6 GB	1.38 GB	44K	57–100
Nemotron 3 Nano 30B-A3B 31.6B	32K	20.3 GB	3.72 GB	256K (full)	102–183

llama-server commands for one RTX 3090

GGUF repos checked 2026-09-29; -c is the longest context with 0.5 GB free on the RTX 3090. No -ngl: llama.cpp’s -fit, on by default since b7440, places the layers.

Ornith 1.5 35B-A3B, Q4_K_M with 27K tokens: 100–179 tokens/s on the RTX 3090

llama-server -hf bartowski/Ornith-1.5-35B-A3B-GGUF:Q4_K_M -c 27648

Ornith-1.5-35B-A3B-Q4_K_M.gguf, 21.9 GB: 23.5 GB used and 534 MB free of the RTX 3090's 24 GB, 100–179 tokens/s once the 27K cache is full.

Qwen3.6 35B-A3B, Q4_K_M with 33K tokens: 96–171 tokens/s on the RTX 3090

llama-server -hf ggml-org/Qwen3.6-35B-A3B-GGUF:Q4_K_M -c 33792

Qwen3.6-35B-A3B-Q4_K_M.gguf, 20.4 GB: 23.5 GB used and 520 MB free of the RTX 3090's 24 GB, 96–171 tokens/s once the 33K cache is full.

Ornith 1.0 35B, Q4_K_M with 50K tokens: 85–151 tokens/s on the RTX 3090

llama-server -hf bartowski/deepreinforce-ai_Ornith-1.0-35B-GGUF:Q4_K_M -c 51200

deepreinforce-ai_Ornith-1.0-35B-Q4_K_M.gguf, 21.4 GB: 23.5 GB used and 523 MB free of the RTX 3090's 24 GB, 85–151 tokens/s once the 50K cache is full.

How fast the RTX 3090 writes as the context fills

Tokens per second at Q4_K_M for the biggest models that fit with 32K tokens, with 8K and 32K tokens in the cache and at the longest context the card holds, and at Q8_0 with 8K where that fits. Each token reads the active weights and the whole cache once, so the speed follows the 936 GB/s of bandwidth and falls as the cache grows. The ceiling is that bandwidth divided by the bytes read per token, which no runtime reaches; the reply time is for 1,000 new tokens with 32K (or the longest context) in the cache, prompt processing aside. With the same memory, RX 7900 XTX, 960 GB/s, writes 2% faster on average; RTX 4090, 1,008 GB/s, writes 7% faster on average; 2× RTX 3060 12GB, 720 GB/s, writes 23% slower by bandwidth.

Model	8K	32K	Longest	At the longest	Q8_0 at 8K	Bandwidth ceiling at 8K	1,000-token reply at 32K
Ornith 1.5 35B-A3B	117–211	97–172	33K	96–171	Does not fit	472	6–10 s
Qwen3.6 35B-A3B	117–211	97–172	33K	96–171	Does not fit	472	6–10 s
Ornith 1.0 35B	117–211	97–172	57K	82–144	Does not fit	472	6–10 s
LLM-jp-4.1 32B-A3B Thinking	86–152	57–100	44K	49–85	Does not fit	328	10–17 s
Nemotron 3 Nano 30B-A3B	108–195	102–183	256K	68–118	Does not fit	432	5–10 s
GLM-4.7 Flash	105–188	70–122	64K	49–84	Does not fit	414	8–14 s
Xing 4.0 29B-A4B	87–154	65–112	75K	44–76	Does not fit	335	9–15 s
Qwen3-Coder 30B-A3B	87–154	50–86	39K	44–76	Does not fit	334	12–20 s
Qwen3 30B-A3B	87–154	50–86	39K	44–76	Does not fit	334	12–20 s
Muse Glimmer 30B	27–38	27–37	128K	25–35	Does not fit	51	27–38 s

RTX 3090 against the other 24 GB cards

The same models fit on every card with 24 GB, so speed is what separates them. By bandwidth the RTX 3090 is the slowest of the 3 cards with 24 GB at 936 GB/s. Over the 12 biggest models that fit at Q4_K_M with 8K context, the RX 7900 XTX (960 GB/s) writes 2% faster and the RTX 4090 (1,008 GB/s) writes 7% faster than the RTX 3090. Each cell below is the other card's tokens per second minus this one's, midpoints of the estimated ranges.

Model	RTX 3090 tokens/s	RX 7900 XTX	RTX 4090
Ornith 1.5 35B-A3B	117–211	+3.6	+11
Qwen3.6 35B-A3B	117–211	+3.6	+11
Ornith 1.0 35B	117–211	+3.6	+11
LLM-jp-4.1 32B-A3B Thinking	86–152	+2.7	+8.2
Nemotron 3 Nano 30B-A3B	108–195	+3.4	+10
Gemma 4 31B	24–33	+0.7	+2.1
GLM-4.7 Flash	105–188	+3.3	+9.9
Xing 4.0 29B-A4B	87–154	+2.8	+8.3
Qwen3-Coder 30B-A3B	87–154	+2.8	+8.3
Qwen3 30B-A3B	87–154	+2.8	+8.3
Muse Glimmer 30B	27–38	+0.8	+2.4
Granite 4.2 30B	25–35	+0.7	+2.2

Near misses on RTX 3090

Models that miss at Q4_K_M with 32,768 tokens of context (or their whole window), or leave under 0.5 GB free there, but fit with a shorter context or 3-bit weights, or are short by at most a quarter of the memory and fit with MoE offload. Smallest shortfall first.

Model	Short by at Q4_K_M	Fits instead
Gemma 4 31B 31.3B	Tight, 80 MB free	Q4_K_M with 26K context
Granite 4.2 30B 29.3B	3.45 GB	Q4_K_M with 17K context
K2-Horizon MoVA 36B-A4B 37.4B	6.31 GB	GGUF IQ3_XXS with 32K, 22.9 GB

What more cards or MoE offload unlock

One, two and four RTX 3090 cards in one tensor-parallel group (vLLM or SGLang), with the models each step adds at Q4_K_M and 8K context and their tokens per second on that group, and the median on-demand rent for that many cards (getdeploying.com, 2026-09-29).

Setup	Memory	Bandwidth	Models at Q4_K_M	Rent per hour	Adds (tokens/s)
RTX 3090	24 GB	936 GB/s	38	$0.27	This page
2× RTX 3090	48 GB	1,872 GB/s	40	$0.54	K2-Horizon MoVA 36B-A4B 94–178, Llama 3.1 70B 21–30
4× RTX 3090	96 GB	3,744 GB/s	48	$1.08	Kolibri-1 178–390, Qwen3-Coder-Next 189–421, AliceAI Foundation 80B-A3B 189–421, gpt-oss-120b 158–334, Nemotron 3 Super 120B-A12B 100–192, Nemotron 3.5 Super VL 120B-A12B 100–192, Qwen3.5 122B-A10B 110–216, Mistral Medium 3.5 128B 24–33

Every mixture-of-experts GGUF file the --n-cpu-moe planner has measured, on one RTX 3090 with 32K tokens of context (or the model's whole window): the smallest --n-cpu-moe that fits the card (0 means the whole file fits), the memory on the card and in system RAM, and tokens per second from 936 GB/s of card bandwidth with DDR5-5600, 2 channels (89.6 GB/s) or DDR4-3200, 2 channels (51.2 GB/s). 4 of these models do not fit whole and need offload.

Model	GGUF	--n-cpu-moe	On the card	In system RAM	Tokens/s, DDR5	Tokens/s, DDR4
gpt-oss-20b	MXFP4	0	12.5 GB	587 MB	74–129	74–129
Gemma 4 26B-A4B	UD-Q4_K_M	0	17.0 GB	748 MB	69–120	69–120
Qwen3 30B-A3B	Q4_K_M	0	21.1 GB	167 MB	50–87	50–87
Qwen3 30B-A3B	Q8_0	17	23.8 GB	10.5 GB	21–36	15–25
GLM-4.7 Flash	UD-Q4_K_XL	0	18.8 GB	170 MB	61–106	61–106
GLM-4.7 Flash	Q4_K_M	0	19.5 GB	170 MB	61–106	61–106
Nemotron 3 Nano 30B-A3B	Q4_K_M	0	23.8 GB	231 MB	88–156	88–156
Qwen3.6 35B-A3B	UD-IQ4_XS	0	17.6 GB	515 MB	78–138	78–138
Ornith 1.5 35B-A3B	Q4_K_M	0	21.6 GB	273 MB	90–160	90–160
Qwen3.6 35B-A3B	UD-Q4_K_M	0	21.7 GB	515 MB	76–133	76–133
Qwen3.6 35B-A3B	Q8_0	15	23.5 GB	12.5 GB	35–59	25–42
Ornith 1.5 35B-A3B	Q8_0	16	23.6 GB	13.3 GB	33–57	24–40
Qwen3-Coder-Next	IQ4_XS	22	23.8 GB	17.7 GB	41–69	29–49
Qwen3-Coder-Next	Q4_K_M	26	23.3 GB	23.6 GB	34–57	23–39
Qwen3-Coder-Next	UD-Q4_K_XL	27	23.2 GB	24.8 GB	30–51	21–36
gpt-oss-120b	MXFP4	24	22.7 GB	38.5 GB	16–27	10–17
Qwen3.5 122B-A10B	Q4_K_M	36	23.3 GB	49.7 GB	11–19	7–13
Qwen3.8 Flash Next	UD-Q2_K_XL	27	23.8 GB	24.6 GB	27–46	19–33
Qwen3.8 Flash Next	UD-IQ3_XXS	29	23.3 GB	27.9 GB	24–41	17–29
Qwen3.8 Flash Next	GSQ-RCO IQ3_S	31	23.2 GB	29.6 GB	23–39	17–28
Qwen3.8 Flash Next	UD-Q3_K_XL	32	23.5 GB	35.2 GB	20–34	14–24
Qwen3.8 Flash Next	UD-IQ4_XS	33	23.5 GB	38.6 GB	19–32	13–23
Qwen3.8 Flash Next	UD-Q4_K_XL	37	22.8 GB	55.8 GB	15–26	10–17
Qwen3.8 Flash Next	UD-Q5_K_XL	39	23.4 GB	75.1 GB	12–21	8–13

In system RAM counts the offloaded experts plus the input embedding (token_embd), which llama.cpp keeps in RAM in any case.

Run locally or rent an RTX 3090?

Renting an RTX 3090 costs a median of $0.27 an hour on demand, across the providers getdeploying.com tracks (checked 2026-09-29), so every $1,000 of purchase price equals about 3,704 hours of rent, 154 days around the clock. At that rate Ornith 1.5 35B-A3B at Q4_K_M with 32K tokens, the biggest model here that fits with a long context, writes 97–172 tokens/s for one request: about $0.44–0.78 per million generated tokens. Electricity and resale value are left out, and no card prices are listed because 2026 street prices vary widely between sellers; compare current ones before you decide.

Serving with vLLM on the RTX 3090

The KV cache pool vLLM builds at its default --gpu-memory-utilization 0.92 and --max-model-len 32768, weights as published, from the vLLM KV cache calculator. The pool is shared by every request; the last column assumes 4,096 tokens each.

Model	KV cache per GPU	Pool (tokens)	At 32K	Requests at 4K
gpt-oss-20b	6.6 GB	254,933	7.78x	35
Qwen3 8B	4.9 GB	35,808	1.09x	8

How these numbers are worked out

Memory is the weights at each precision plus the KV cache for 8,192 tokens in FP16 and runtime overhead (0.5 GB plus 10%), computed from each model's files on Hugging Face with the LLM VRAM Calculator. Speed is estimated from memory bandwidth and the parameters read per token with the LLM Speed Calculator. Both tools take any other model from Hugging Face.

Nearby GPUs and setups

How many of the tracked models each one holds at Q4_K_M with 8K context, against 38 here.

Setup	How it relates	Memory	Bandwidth	Models at Q4_K_M
RX 7900 XTX	Same memory	24 GB	960 GB/s	38 (same)
RTX 4090	Same memory	24 GB	1,008 GB/s	38 (same)
2× RTX 3060 12GB	Same memory	24 GB	720 GB/s	38 (same)
RTX 5080 16GB	Next size down	16 GB	960 GB/s	21 (−17)
RTX 5090	Next size up	32 GB	1,792 GB/s	39 (+1)
2× RTX 3090	More of this card	48 GB	1,872 GB/s	40 (+2)
4× RTX 3090	More of this card	96 GB	3,744 GB/s	48 (+10)

Model numbers read from Hugging Face on 2026-10-20.