Jev Alternatives You Can Run Locally: Laya, Zero-Shot Encoders, Small LLMs
Jev is TypeSafe's closed decision model, reachable only through an API. These open models do the same classification, yes/no and scoring work on your own machine, listed from closest to Jev to furthest.
What Jev is, and why it has no local version
Jev is TypeSafe's "System One" decision model: you send a state (text or JSON, up to 32,000 tokens) and typed questions, and it returns a Choice, a yes/no probability (Noul) or a Score with calibrated probabilities instead of generated text. It is served only through APIs (typesafe/jev-1.13 on OpenRouter, TypeSafe's own API and Cloudflare), billed per input token; there are no weights to download. The models below run on your own GPU or CPU.
Encoders: the closest local replacements
Small classification encoders do what Jev does, in one forward pass and under 2 GB. Laya is the only one built to Jev's interface: typed Choice, Noul and Score questions with calibrated probabilities.
| Model | Answers | Calibrated | Weights | Memory | License |
|---|---|---|---|---|---|
| Laya (English, typed-decisions, multilingual) | Choice, Noul, Score | Yes | 249 MB–850 MB 24 files, see the Laya page | about 1–2 GB on a GPU or CPU | Apache-2.0 |
| ModernBERT-large zeroshot v2.0 | Choice, Noul | No | 792 MB | about 1–2 GB on a GPU or CPU | Apache-2.0 |
| DeBERTa-v3-large zeroshot v2.0 | Choice, Noul | No | 870 MB | about 1–2 GB on a GPU or CPU | MIT |
| GLiClass ModernBERT large v3.0 | Choice, Noul | No | 1,596 MB | about 1–2 GB on a GPU or CPU | Apache-2.0 |
The zero-shot encoders turn each label into a hypothesis ("This text is about billing.") and score it with natural language inference, so they answer Choice and yes/no questions but have no ordered Score and were not trained for calibrated probabilities. They run with Hugging Face Transformers' zero-shot-classification pipeline (GLiClass with its own gliclass package).
How fast: milliseconds per decision and decisions per second
Memory is rarely the limit for these encoders; speed is. Pick the model, the GPU, how many tokens each row reads and how many rows one decision needs, and see the time per decision and how many decisions per second the GPU can serve.
Laya: one row per question you ask about the text.
– per decision
One request at a time: – decisions/s
Many requests batched together: – decisions/s
Laya (English), 256 tokens, one row per decision
| GPU | Per decision | One at a time | Batched |
|---|---|---|---|
| RTX 4070 12GB | 9.7–54 ms | 19–103 decisions/s | 73–103 decisions/s |
| RTX 5070 12GB | 7.3–53 ms | 19–138 decisions/s | 77–138 decisions/s |
| RTX 5070 Ti 16GB | 5.4–49 ms | 20–184 decisions/s | 110–184 decisions/s |
| RTX 5080 16GB | 5.1–47 ms | 21–197 decisions/s | 141–197 decisions/s |
| RTX 3090 | 5.2–51 ms | 20–192 decisions/s | 89–192 decisions/s |
| RTX 4090 | 4.8–45 ms | 22–207 decisions/s | 207 decisions/s |
| RTX 5090 | 2.7–44 ms | 23–367 decisions/s | 262–367 decisions/s |
| L40S | 2.2–46 ms | 22–453 decisions/s | 177–453 decisions/s |
| A100 80GB | 2.4–43 ms | 23–418 decisions/s | 390–418 decisions/s |
| H100 SXM | 0.8–41 ms | 24–1,238 decisions/s | 687–1,238 decisions/s |
| H200 | 0.8–41 ms | 24–1,238 decisions/s | 984–1,238 decisions/s |
Not estimated: CPUs, Apple Silicon, AMD and Intel GPUs, and NVIDIA cards without a published BF16 tensor figure (RTX 3060, 4060, 4060 Ti, 5060 Ti, 4070 Ti Super, 4080 Super, RTX PRO 6000, B200, DGX Spark).
How the estimate works
- Measured anchor. The ModernBERT paper (Table 2) measured tokens per second on one RTX 4090: ModernBERT-large 52,300 at 512 tokens and 46,800 at 8,192; ModernBERT-base 148,100 and 123,700; DeBERTa-v3-large 24,600 at 512, and half that when inputs of mixed length are padded. Laya English and typed-decisions are ModernBERT-large; Laya multilingual is mmBERT-base, which has ModernBERT-base's architecture and non-embedding size (mmBERT paper, section 3.1).
- Other GPUs. An encoder's time is partly matrix work, which scales with tensor throughput, and partly memory traffic, which scales with bandwidth. So the speed relative to the RTX 4090 lies between the two ratios, and the range uses both. Tensor figures are NVIDIA's dense BF16 numbers (RTX Blackwell whitepaper appendix, L40S, A100, H100 and H200 pages, sparsity figures halved); bandwidths are the ones the LLM speed calculator uses.
- Per decision. The low end is the GPU fully busy on your request, never less than reading the weights once. The high end adds 40 ms of fixed cost per call: the laya package's published single-question times are 39.5 ms and 32.8 ms on a Tesla T4 and 25 ms with ggmlc on an RTX 4050 Laptop (model card), and for a short question most of that is tokenizing, Python and kernel launches, not arithmetic. A batched server with a compiled model gets close to the low end; the laya package called once per request lands near the high end.
- Batched. Decisions per second when enough requests share each forward pass to keep the GPU busy: tokens per second divided by the tokens of one decision. On the RTX 4090 itself this is the paper's measured rate, so both ends are equal; elsewhere expect real servers within this range, lower if requests are padded to a common length (except ModernBERT, which unpads).
- Not modelled: ggml and llama.cpp runtimes (F32 activations, different kernels), INT8/FP8 engines such as TensorRT, laptop GPUs and power limits. Checked on 2026-09-29.
Small LLMs with JSON-schema output
Any chat model can answer a Jev question if you force its reply into a schema, for example llama-server --json-schema or Ollama's format field, and read the probability of each option from the token log-probabilities. It works on the same GPU you already use for chat, but each answer is generated token by token, so it is slower than an encoder and the probabilities are not calibrated. VRAM is the site's estimate at Q4_K_M (MXFP4 for gpt-oss) with an 8K-token context; each model page has every quantization and context.
| Model | GGUF file | VRAM |
|---|---|---|
| MiniCPM5 2B | 1,616 MB | 2.42 GB |
| Granite 4.2 3B | 2,317 MB | 3.46 GB |
| Nemotron 3 Nano 4B | 2,900 MB | 3.10 GB |
| Gemma 4 E4B | 4,977 MB | 5.64 GB |
| Qwen3 8B | 5,028 MB | 6.81 GB |
| Granite 4.2 8B | 5,539 MB | 7.32 GB |
| Qwen3.5 9B | 5,681 MB | 6.76 GB |
| Gemma 4 12B | 7,122 MB | 8.57 GB |
| gpt-oss-20b | 12,110 MB | 14.8 GB |
Which one to pick
- You call Jev today and want the same questions offline: Laya, which takes the same typed questions.
- Only English labels or yes/no checks, and you already use Transformers: a zero-shot encoder.
- Many labels per text (tags): GLiClass, which scores all labels in one pass.
- The decision needs world knowledge or reasoning more than speed, and you have 3–16 GB of VRAM: a small LLM with JSON-schema output.
Jev facts from OpenRouter's Jev guide and the TypeSafe docs; encoder sizes from the Hugging Face API; all read on .
Frequently asked questions
Can Jev run locally?
No. Jev is served only through APIs (OpenRouter, TypeSafe, Cloudflare) and its weights are not published. The closest local model is the open Laya, which takes the same Choice, Noul and Score questions.
What is the closest open-source model to Jev?
Laya, an Apache-2.0 decision encoder that reproduces Jev's interface and returns calibrated probabilities. Its files are under 1 GB, and it runs on a 4 GB GPU or a CPU.
Can a regular LLM replace Jev?
Yes, with trade-offs. With output forced into a JSON schema, small models such as Gemma 4 and Qwen3 answer choice and yes/no questions, but they are slower, need more VRAM (about 2.4–15 GB at Q4_K_M, MXFP4 for gpt-oss) and their probabilities are not calibrated.
More calculators
- FLUX 3 Image Klein VRAM Calculator Estimate FLUX 3 Image Klein VRAM by build, CPU offload, prompt upsampler and working-memory budget. Compare INT8, FP8, NVFP4 and BF16 files. Open →
- d1-omni VRAM LiquidAI d1-omni weights: 1.09 GiB in FP16 on GPU, 2.19 GiB in FP32 on CPU. Text, image and audio decisions, input limits, license and official Python usage. Open →
- pplx-decider v1.1 VRAM pplx-decider v1.1 needs 48.6 GiB for its BF16 checkpoint before working memory. GPU fit, decision head, noncausal attention and the official Python loader. Open →
- Speculative Decoding VRAM Calculator (MTP & draft models) Extra VRAM of MTP heads, DFlash drafters and draft models in llama.cpp, for Qwen3.8 27B and Gemma 4 31B. Open →
- Qwen3.8 27B GGUF Quants: Bonsai 2 vs GSQ-RCO vs UD Ternary Bonsai 2, GSQ-RCO and Unsloth UD files of Qwen3.8 27B: VRAM, quality, speed and the engine each needs. Open →
- Bonsai 2 27B VRAM Exact file sizes of Ternary Bonsai 2 27B, the longest context on 8–24 GB GPUs, and why sites say 5.95 GB or 8.60 GB. Open →
Updated