Can I run gpt-oss-20b on an RTX 4090?

Yes: gpt-oss-20b needs about 15.4 GB at MXFP4 with 32K tokens of context, which fits the 24 GB RTX 4090 with 8.56 GB to spare. MXFP4 is the format gpt-oss-20b is published in, and on the RTX 4090 it runs up to 128K (full) tokens with 0.5 GB to spare.

Why 15.4 GB with vLLM, but 12.5 GB on the RTX 4090 with llama.cpp

The 15.4 GB counts the published checkpoint as vLLM loads it: 12.8 GB of weights, the 32K KV cache, 0.5 GB and 10% overhead. llama.cpp's measured MXFP4 GGUF is 11.3 GB, and its 587 MB of token embeddings stay in RAM. With every layer on the RTX 4090 it takes 12.5 GB at 32K with 1 GB of buffers, 11.5 GB under its 24 GB.

Yes MXFP4 with 32K tokens of context

MXFP4, 32K: 15.4 GB
RTX 4090: 24 GB, 1,008 GB/s
To spare: 8.56 GB
Tokens/s: 83–146 tokens/s

At MXFP4 with 32K tokens of context it writes about 83–146 tokens/s for one request on an RTX 4090.

Best precision for gpt-oss-20b on an RTX 4090

The most precise setting that leaves at least 0.5 GB free; one that fits with less is marked tight.

Context	Best fit	Memory	Free	Tokens/s
8K	MXFP4	14.8 GB	9.18 GB	99–177
32K	MXFP4	15.4 GB	8.56 GB	83–146
128K (full)	MXFP4	17.9 GB	6.08 GB	50–86

gpt-oss-20b on the RTX 4090 as the context fills

One request, FP16 KV cache, 0.5 GB plus 10% overhead; the MXFP4 GGUF columns are the measured llama.cpp file with every layer on the GPU and 1 GB of buffers; a minus sign is memory missing, and tight is less than 0.5 GB free.

Context	MXFP4	Free	Tokens/s	MXFP4 GGUF	Free	Tokens/s
4K	14.7 GB	9.28 GB	103–184	11.8 GB	12.2 GB	96–171
8K	14.8 GB	9.18 GB	99–177	11.9 GB	12.1 GB	93–165
16K	15.0 GB	8.97 GB	93–166	12.1 GB	11.9 GB	88–155
32K	15.4 GB	8.56 GB	83–146	12.5 GB	11.5 GB	78–138
64K	16.3 GB	7.73 GB	68–119	13.2 GB	10.8 GB	65–113
128K (full)	17.9 GB	6.08 GB	50–86	14.7 GB	9.29 GB	48–83

--n-cpu-moe for gpt-oss-20b on the RTX 4090

The measured MXFP4 GGUF fits the RTX 4090 whole at 32K (12.5 GB), so --n-cpu-moe is not needed there. Measured MXFP4 file, 1 GB of buffers; tokens/s by system RAM speed.

Context	On the card	In RAM	DDR4-3200	DDR5-5600	DDR5-6400
8K	11.9 GB	587 MB	93–165	93–165	93–165
32K	12.5 GB	587 MB	78–138	78–138	78–138
64K	13.2 GB	587 MB	65–113	65–113	65–113
128K (full)	14.7 GB	587 MB	48–83	48–83	48–83

gpt-oss-20b on more than one RTX 4090

Cards	Best at 32K	Tokens/s	MXFP4 longest	MXFP4 tokens/s	Rent per hour
2× (48 GB)	MXFP4	114–224	128K (full)	114–224	$1.06
4× (96 GB)	MXFP4	163–347	128K (full)	163–347	$2.12

Tensor parallel with the combined bandwidth, as in vLLM; llama.cpp splits layers by default and runs at about one card's speed.

Other options

Renting an RTX 4090 costs about $0.53 an hour (median on getdeploying.com, 2026-09-29): $1.0–1.8 per million tokens at 83–146 tokens/s.
Smallest setup for MXFP4 at 32K: RTX 4060 Ti 16GB.

Run gpt-oss-20b on the RTX 4090 with llama-server

llama-server -hf ggml-org/gpt-oss-20b-GGUF:MXFP4 -c 131072 -np 1

gpt-oss-20b-MXFP4.gguf, 12.1 GB, from ggml-org/gpt-oss-20b-GGUF (checked 2026-09-29). At -c 131072 on the RTX 4090: 17.9 GB of 24 GB, 6.08 GB free. -np 1: one slot, one sliding window.

Questions

Can I run gpt-oss-20b on an RTX 4090?

How fast is gpt-oss-20b on an RTX 4090?

At MXFP4 with 32K tokens of context it writes about 83–146 tokens/s for one request on an RTX 4090.

Does gpt-oss-20b need --n-cpu-moe on an RTX 4090?

The measured MXFP4 GGUF fits the RTX 4090 whole at 32K (12.5 GB), so --n-cpu-moe is not needed there.

What does a second RTX 4090 change for gpt-oss-20b?

Two RTX 4090 cards (48 GB in one tensor-parallel group) hold gpt-oss-20b at MXFP4 with 32K, and MXFP4 up to 128K (full) tokens, at about 114–224 tokens/s.

Try other settings in the VRAM calculator, the speed calculator or the MoE offload planner. See also gpt-oss-20b VRAM requirements, what LLMs an RTX 4090 can run and every pair, or detect your own GPU. Model data checked 2026-09-26.