Can I run gpt-oss-20b on an RTX 3090?

Yes: gpt-oss-20b needs about 15.4 GB at MXFP4 with 32K tokens of context, which fits the 24 GB RTX 3090 with 8.56 GB to spare. MXFP4 is the format gpt-oss-20b is published in, and on the RTX 3090 it runs up to 128K (full) tokens with 0.5 GB to spare.

Why 15.4 GB with vLLM, but 12.5 GB on the RTX 3090 with llama.cpp

The 15.4 GB counts the published checkpoint as vLLM loads it: 12.8 GB of weights, the 32K KV cache, 0.5 GB and 10% overhead. llama.cpp's measured MXFP4 GGUF is 11.3 GB, and its 587 MB of token embeddings stay in RAM. With every layer on the RTX 3090 it takes 12.5 GB at 32K with 1 GB of buffers, 11.5 GB under its 24 GB.

Yes MXFP4 with 32K tokens of context

MXFP4, 32K: 15.4 GB
RTX 3090: 24 GB, 936 GB/s
To spare: 8.56 GB
Tokens/s: 78–137 tokens/s

At MXFP4 with 32K tokens of context it writes about 78–137 tokens/s for one request on an RTX 3090.

Best precision for gpt-oss-20b on an RTX 3090

The most precise setting that leaves at least 0.5 GB free; one that fits with less is marked tight.

Context	Best fit	Memory	Free	Tokens/s
8K	MXFP4	14.8 GB	9.18 GB	93–166
32K	MXFP4	15.4 GB	8.56 GB	78–137
128K (full)	MXFP4	17.9 GB	6.08 GB	47–80

gpt-oss-20b on the RTX 3090 as the context fills

One request, FP16 KV cache, 0.5 GB plus 10% overhead; the MXFP4 GGUF columns are the measured llama.cpp file with every layer on the GPU and 1 GB of buffers; a minus sign is memory missing, and tight is less than 0.5 GB free.

Context	MXFP4	Free	Tokens/s	MXFP4 GGUF	Free	Tokens/s
4K	14.7 GB	9.28 GB	97–172	11.8 GB	12.2 GB	90–160
8K	14.8 GB	9.18 GB	93–166	11.9 GB	12.1 GB	87–155
16K	15.0 GB	8.97 GB	87–155	12.1 GB	11.9 GB	82–145
32K	15.4 GB	8.56 GB	78–137	12.5 GB	11.5 GB	74–129
64K	16.3 GB	7.73 GB	64–111	13.2 GB	10.8 GB	61–105
128K (full)	17.9 GB	6.08 GB	47–80	14.7 GB	9.29 GB	45–77

--n-cpu-moe for gpt-oss-20b on the RTX 3090

The measured MXFP4 GGUF fits the RTX 3090 whole at 32K (12.5 GB), so --n-cpu-moe is not needed there. Measured MXFP4 file, 1 GB of buffers; tokens/s by system RAM speed.

Context	On the card	In RAM	DDR4-3200	DDR5-5600	DDR5-6400
8K	11.9 GB	587 MB	87–155	87–155	87–155
32K	12.5 GB	587 MB	74–129	74–129	74–129
64K	13.2 GB	587 MB	61–105	61–105	61–105
128K (full)	14.7 GB	587 MB	45–77	45–77	45–77

gpt-oss-20b on more than one RTX 3090

Cards	Best at 32K	Tokens/s	MXFP4 longest	MXFP4 tokens/s	Rent per hour
2× (48 GB)	MXFP4	109–212	128K (full)	109–212	$0.54
4× (96 GB)	MXFP4	158–333	128K (full)	158–333	$1.08

Tensor parallel with the combined bandwidth, as in vLLM; llama.cpp splits layers by default and runs at about one card's speed.

Other options

Renting an RTX 3090 costs about $0.27 an hour (median on getdeploying.com, 2026-09-29): $0.55–0.97 per million tokens at 78–137 tokens/s.
Smallest setup for MXFP4 at 32K: RTX 4060 Ti 16GB.

Run gpt-oss-20b on the RTX 3090 with llama-server

llama-server -hf ggml-org/gpt-oss-20b-GGUF:MXFP4 -c 131072 -np 1

gpt-oss-20b-MXFP4.gguf, 12.1 GB, from ggml-org/gpt-oss-20b-GGUF (checked 2026-09-29). At -c 131072 on the RTX 3090: 17.9 GB of 24 GB, 6.08 GB free. -np 1: one slot, one sliding window.

Questions

Can I run gpt-oss-20b on an RTX 3090?

How fast is gpt-oss-20b on an RTX 3090?

At MXFP4 with 32K tokens of context it writes about 78–137 tokens/s for one request on an RTX 3090.

Does gpt-oss-20b need --n-cpu-moe on an RTX 3090?

The measured MXFP4 GGUF fits the RTX 3090 whole at 32K (12.5 GB), so --n-cpu-moe is not needed there.

What does a second RTX 3090 change for gpt-oss-20b?

Two RTX 3090 cards (48 GB in one tensor-parallel group) hold gpt-oss-20b at MXFP4 with 32K, and MXFP4 up to 128K (full) tokens, at about 109–212 tokens/s.

Try other settings in the VRAM calculator, the speed calculator or the MoE offload planner. See also gpt-oss-20b VRAM requirements, what LLMs an RTX 3090 can run and every pair, or detect your own GPU. Model data checked 2026-09-26.