gpt-oss-20b VRAM requirements
gpt-oss-20b needs about 14.8 GB of VRAM at MXFP4 with 8K context, so it fits a 16 GB card such as the RTX 4060 Ti 16GB or the RTX 5060 Ti 16GB. gpt-oss-20b has 20.9B parameters, of which about 3.6B are used per token; all 32 experts still have to be in memory. With one request it needs 43.6 GB at FP16/BF16. The published MXFP4 weights take 12.8 GB; llama.cpp's measured MXFP4 GGUF is 11.3 GB. The smallest setup here that holds it at MXFP4 with an 8K context is the RTX 4060 Ti 16GB; on one 24 GB RTX 3090 / 4090 it runs with its full 128K-token context.
14.8 GB at MXFP4 as published, 8K context, one request
- MXFP4 GGUF file
- 11.3 GB
- FP16 / BF16
- 43.6 GB
- Published weights
- 12.8 GB
- Smallest setup, MXFP4
- RTX 4060 Ti 16GB
Worked example: gpt-oss-20b with 32K tokens
Inputs: gpt-oss-20b · MXFP4 weights · 32,768 tokens of context · one request · FP16 KV cache
- Weights (20.9B at MXFP4)
- 12.8 GB
- KV cache (24 KB per token)
- 786 MB
- Buffers and runtime (0.5 GB + 10%)
- 1.86 GB
- Total
- 15.4 GB
The smallest setup here that holds it is the RTX 4060 Ti 16GB, with about 571 MB to spare. Change the inputs in the calculator
Can I run gpt-oss-20b on…
- RTX 4090: Yes, MXFP4
- RTX 3090: Yes, MXFP4
- RTX 5090: Yes, MXFP4
- RTX 3060 12GB: --n-cpu-moe 2
- RTX 4060 Ti 16GB: Yes, MXFP4
- RTX 5060 Ti 16GB: Yes, MXFP4
- RTX 4070 12GB: --n-cpu-moe 2
- RTX 5080 16GB: Yes, MXFP4
- RTX 4080 Super 16GB: Yes, MXFP4
- RX 7900 XTX: Yes, MXFP4
What makes gpt-oss-20b's memory use different
Its 12 sliding-window layers keep only the last 128 tokens (llama.cpp gives them 768 cells: the window plus a 512-token batch, rounded up to 256): at its 128K limit they hold 18 MB of cache instead of the 3.00 GB they would need with full attention.
Per token of context it adds 24 KB of FP16 cache; Granite 4.2 30B (29.3B), the nearest-sized model here with plain full attention, adds 256 KB, so gpt-oss-20b needs 9% as much.
As a mixture-of-experts model it reads about 2.21 GB of its 12.8 GB MXFP4 weights per generated token (17%), so it writes like a much smaller model while needing memory for all of them.
Running gpt-oss-20b with experts in system RAM
llama.cpp's --n-cpu-moe N keeps the experts of the first N layers in RAM. For the measured
MXFP4 GGUF with 32K tokens of context, the smallest N
that fits each card, and the speed with 89.6 GB/s of system RAM.
Plan the offload for another GPU.
The VRAM here can sit below the 15.4 GB of the worked example because it counts the file's measured
bytes, keeps the input embedding (token_embd) in system RAM as llama.cpp does and adds a flat 1 GB of buffers, where the
estimate above adds 0.5 GB plus 10% to the preset's weights.
| GPU | --n-cpu-moe | VRAM used | In system RAM | Tokens/s |
|---|---|---|---|---|
| RTX 5060 Ti 16GB 16 GB | 0 (all on the GPU) | 12.5 GB | 587 MB | 37–64 |
| RTX 4090 24 GB | 0 (all on the GPU) | 12.5 GB | 587 MB | 78–138 |
| RTX 5090 32 GB | 0 (all on the GPU) | 12.5 GB | 587 MB | 128–233 |
In system RAM counts the offloaded experts plus the input embedding (token_embd), which stays in RAM even with --n-cpu-moe 0.
How much VRAM does gpt-oss-20b need?
gpt-oss-20b needs 14.8 GB at MXFP4 as published and 43.6 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.
| Precision | Weights | Total | Smallest setup |
|---|---|---|---|
| As published (MXFP4) | 12.8 GB | 14.8 GB | RTX 4060 Ti 16GB |
| FP16 / BF16 | 39.0 GB | 43.6 GB | M4 Pro Mac (64 GB, 48 GB usable) |
No GGUF rows: gpt-oss-20b's GGUF quants keep the experts in MXFP4, so every GGUF type, Q2_K to Q8_0, is about the size of the measured MXFP4 GGUF (11.3 GB), and a size worked out from bits per weight would describe a file that does not exist. Every figure on this page, the GPU pages and the "Can I run" pages is at MXFP4.
Fine-tuning gpt-oss-20b? gpt-oss-20b VRAM for LoRA, QLoRA and full training.
gpt-oss-20b GGUF files on Hugging Face
The 1 GGUF files llama.cpp picks for gpt-oss-20b with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.
| Quant | File | Size | vs estimate | 8K | 32K | 128K |
|---|---|---|---|---|---|---|
| MXFP4 | ggml-org/ | 11.3 GB12,109,566,624 bytes | −12% | 13.1 GB | 13.8 GB | 16.2 GB |
GB here is GiB (1024³ bytes), as everywhere on this site; Hugging Face shows the MXFP4 file as 12.1 GB, in 10⁹ bytes.
Which gpt-oss-20b GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac
From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for MXFP4 (11.3 GB). One request, FP16 KV cache, -np 1. Where nothing fits, the --n-cpu-moe table above keeps experts in system RAM. The MXFP4 file is 1.54 GB under its bits-per-weight estimate; the estimate tables and the llama-server command on this page count the estimate, so they show less context.
| Hardware | Largest GGUF, 8K | Its longest context | MXFP4, longest context |
|---|---|---|---|
| RTX 5060 Ti 16GB | MXFP4, 11.3 GB | 99K | 99K |
| RTX 4090 (24 GB) | MXFP4, 11.3 GB | 128K (full) | 128K (full) |
| RTX 5090 (32 GB) | MXFP4, 11.3 GB | 128K (full) | 128K (full) |
| L40S (48 GB) | MXFP4, 11.3 GB | 128K (full) | 128K (full) |
| H100 SXM (80 GB) | MXFP4, 11.3 GB | 128K (full) | 128K (full) |
| M4 Pro Mac (64 GB, 48 GB usable) | MXFP4, 11.3 GB | 128K (full) | 128K (full) |
| M4 Max Mac (128 GB, 96 GB usable) | MXFP4, 11.3 GB | 128K (full) | 128K (full) |
Run gpt-oss-20b with llama.cpp or Ollama
llama-server -hf ggml-org/gpt-oss-20b-GGUF:MXFP4 -c 131072 -np 1gpt-oss-20b-MXFP4.gguf, 12.1 GB, from ggml-org/
ollama run gpt-oss:20b
Ollama library tag gpt-oss:20b; its quant is Ollama's choice. Same context: /set parameter num_ctx 131072.
gpt-oss-20b at 8K, 32K and 128K (full) tokens of context
12 of its 24 layers use full attention and 12 keep a sliding window of 128 tokens. Each extra token of context adds 24 KB of FP16 cache per request once the sliding windows are full. The last column is the smallest setup here, one card or a group, that holds it at MXFP4. How the KV cache works.
| Context | KV cache, FP16 | Total at MXFP4 | Smallest setup, MXFP4 |
|---|---|---|---|
| 8K tokens | 210 MB | 14.8 GB | RTX 4060 Ti 16GB |
| 32K tokens | 786 MB | 15.4 GB | RTX 4060 Ti 16GB |
| 128K tokens | 3.02 GB | 17.9 GB | RTX 3090 (24 GB) |
How fast gpt-oss-20b writes
Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth and the parameters read per token. A dash means it does not fit on one card. Try other settings in the speed calculator.
| Hardware | Bandwidth | MXFP4 |
|---|---|---|
| RTX 3060 12GB | 360 GB/s | — |
| RTX 4090 | 1,008 GB/s | 99–177 |
| RTX 5090 | 1,792 GB/s | 158–295 |
| M4 Max Mac (128 GB) | 546 GB/s | 58–100 |
| M3 Ultra Mac Studio (512 GB) | 819 GB/s | 83–147 |
| H100 SXM | 3,350 GB/s | 245–489 |
| H200 | 4,800 GB/s | 303–633 |
Longest context on one GPU
How many tokens of context gpt-oss-20b fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.
| GPU | Memory | MXFP4 |
|---|---|---|
| RTX 3060 | 12 GB | No |
| RTX 4060 Ti 16GB | 16 GB | 34K |
| RTX 3090 / 4090 | 24 GB | 128K (full) |
| RTX 5090 | 32 GB | 128K (full) |
| A100 40GB | 40 GB | 128K (full) |
| Mac, 64 GB unified memory | 48 GB | 128K (full) |
| L40S / RTX 6000 Ada | 48 GB | 128K (full) |
| A100 / H100 80GB | 80 GB | 128K (full) |
| Mac, 128 GB unified memory | 96 GB | 128K (full) |
| H200 | 141 GB | 128K (full) |
| B200 | 180 GB | 128K (full) |
Model details
- Parameters
- 20.9B (20,914,757,184)
- Experts
- 32 routed experts, all loaded
- Active per token
- 3.6B
- Layers
- 12 of its 24 layers use full attention and 12 keep a sliding window of 128 tokens
- Attention cache
- 8 KV heads × 64
- Context length
- 131,072 tokens
- Published weights
- 12.8 GB (MXFP4)
- On Hugging Face
- openai/gpt-oss-20b
- Reported benchmarks
- GPQA Diamond (no tools) 71.5 (reasoning high). Vendor-reported: from the maker’s own model card, not rerun here; read 2026-09-30.
Why this estimate looks this way
At MXFP4 as published and an 8K-token context, gpt-oss-20b uses 12.8 GB for weights, 210 MB for its FP16 KV cache and 1.80 GB for estimated runtime overhead, totaling 14.8 GB. The overhead is a 0.5 GB base plus 10% of weights and cache.
This is a mixture-of-experts model: all 32 experts contribute to the 20.9B parameters held in memory, even though only about 3.6B parameters run per token. Using only active parameters would understate VRAM.
Its attention layout matters for long context: 12 of its 24 layers use full attention and 12 keep a sliding window of 128 tokens. The FP16 cache grows by about 24 KB per additional token per request once the sliding windows are full.
The architecture and published checkpoint size come from the model's config.json and weight files. These are estimates rather than measured peak memory; inference engines can reserve extra buffers or preallocate the full configured cache.
gpt-oss-20b next to similar-sized models
The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context (MXFP4 checkpoints at MXFP4, as published).
| Model | Parameters | Total | KV per token | Smallest setup |
|---|---|---|---|---|
| gpt-oss-20b | 20.9B, 3.6B active | 14.8 GB, MXFP4 | 24 KB | RTX 4060 Ti 16GB |
| Gemma 4 26B-A4B | 25.8B, 4B active | 17.0 GB | 20 KB | RTX 3090 (24 GB) |
| Hemmingway-1 27B | 27.3B | 18.0 GB | 64 KB | RTX 3090 (24 GB) |
| ThinkingCap Qwen3.8 27B | 27.8B | 18.3 GB | 64 KB | RTX 3090 (24 GB) |
gpt-oss-20b VRAM questions
How much VRAM do I need to run gpt-oss-20b locally?
About 14.8 GB at MXFP4 with an 8K context and one request, by the bits-per-weight estimate, and 43.6 GB at BF16. The MXFP4 GGUF gpt-oss-20b-MXFP4.gguf in ggml-org/
Can I run gpt-oss-20b on a 24 GB GPU?
Yes. On the RTX 4090 (24 GB), the largest listed GGUF that fits with an 8K context and 0.5 GB free is MXFP4 (11.3 GB), with its full 128K-token context.
Which quantization of gpt-oss-20b fits in 16 GB of VRAM?
On the RTX 5060 Ti 16GB, the largest listed GGUF that fits with an 8K context and 0.5 GB free is MXFP4 (11.3 GB), with up to 99K tokens of context.
Can I run gpt-oss-20b on a Mac?
On the M4 Pro Mac (64 GB, 48 GB usable), the largest listed GGUF that fits with an 8K context and 0.5 GB free is MXFP4 (11.3 GB), with its full 128K-token context. On the M4 Max Mac (128 GB, 96 GB usable), the largest listed GGUF that fits with an 8K context and 0.5 GB free is MXFP4 (11.3 GB), with its full 128K-token context. By default macOS lets the GPU use about 75% of unified memory.
How much more VRAM does gpt-oss-20b need for 32K or 128K tokens of context?
Its FP16 KV cache is 210 MB at 8K, 786 MB at 32K, 3.02 GB at 128K for one request, so 128K adds 2.81 GB over 8K, plus 10% overhead. A q8_0 cache
Why does gpt-oss-20b use more VRAM than its GGUF file size?
The file holds only the weights. At MXFP4 with an 8K context: 11.3 GB of weights (the file's 12,109,566,624 bytes) + 210 MB of FP16 KV cache + 1.65 GB of compute buffers and runtime (0.5 GB + 10%) = 13.1 GB.
Does gpt-oss-20b need less VRAM because only 3.6B parameters are active?
No. All 32 experts stay loaded, so the MXFP4 file takes 11.3 GB. The active parameters decide speed: each generated token reads about 2.21 GB of them.
Badge for your model card
Paste it into a Hugging Face or GitHub README; it shows the MXFP4 total above and links to this page. FP16 badge.
Numbers read from the model files on Hugging Face on . Compare it with other models in the reproducible model and GPU report.