LensVLM 9B VRAM requirements

How much VRAM does LensVLM 9B need?

LensVLM 9B needs 6.61 GB at Q4_K_M, 11.0 GB at Q8_0 and 20.1 GB at FP16 with 8,192 tokens of context and one request. Each total below is the weights plus the FP16 KV cache and the runtime overhead (0.5 GB plus 10%), and each row opens the calculator with that setting. What the GGUF names mean.

Fine-tuning LensVLM 9B? LensVLM 9B VRAM for LoRA, QLoRA and full training.

LensVLM 9B GGUF files on Hugging Face

The 8 GGUF files llama.cpp picks for LensVLM 9B with -hf <repo>:<QUANT>, with the bytes the Hugging Face file list reported on 2026-09-29. Each total is that file plus the FP16 KV cache and 0.5 GB + 10%, one request. "vs estimate" compares the file with the bits-per-weight size the table above uses.

GB here is GiB (1024³ bytes), as everywhere on this site; Hugging Face shows the Q4_K_M file as 5.8 GB, in 10⁹ bytes.

LensVLM 9B GGUF files and visual text compression

LensVLM-9B is Apple's fine-tune of Qwen3.5-9B that reads long text as rendered images: in its paper it keeps accuracy close to reading the full text at 4.3× effective compression. For memory that is the point: a 131,072-token document read as text needs 4.00 GB of FP16 KV cache on top of the weights, and the same document compressed 4.3× (about 30,482 tokens) needs 953 MB. The paper measures this on seven text QA benchmarks; other documents may compress less.

With images, the bartowski Q4_K_M file and its BF16 mmproj at 32K context come to 8.44 GB, which no longer fits an 8 GB card with 0.5 GB free; an RTX 3060 12GB leaves 3.56 GB for longer documents.

Apple publishes only the BF16 safetensors (18,819,722,392 bytes, vision tower included) under its research-only license. The GGUF builds below hold the language model; for images, llama.cpp also loads the 921,704,992-byte (0.86 GB) mmproj file from the same repos. The main GGUF files, with the totals this page's method gives:

The two Q4_K_M files differ by 211,943,712 bytes (0.20 GB): bartowski keeps more tensors at higher precision. The mmproj file adds 0.86 GB to every total when you pass images. On a Mac, mlx-community/LensVLM-9B-OptiQ-4bit is 7,100,710,126 bytes of weights plus a 912,058,009-byte vision file. File sizes from the Hugging Face file lists, read on 2026-09-29; totals as in the table above, one request, FP16 KV cache.

Which LensVLM 9B GGUF fits a 16, 24, 32, 48 or 80 GB GPU or a Mac

From the file sizes above: the largest file that fits each machine with an 8K context and 0.5 GB left free, the longest context it then has room for, and the longest context for Q4_K_M (5.44 GB). One request, FP16 KV cache.

Run LensVLM 9B with llama.cpp

llama-server -hf bartowski/LensVLM-9B-GGUF:Q4_K_M -c 262144

LensVLM-9B-Q4_K_M.gguf, 5.8 GB, from bartowski/LensVLM-9B-GGUF (checked 2026-09-29). On a 24 GB RTX 3090 / 4090, Q4_K_M with -c 262144 takes 15.3 GB and leaves 8.72 GB.

LensVLM 9B at 8K, 32K, 128K and 256K (full) tokens of context

8 of its 32 layers use full attention and 24 are linear-attention layers with no growing cache. Each extra token of context adds 32 KB of FP16 cache per request. The last column is the smallest setup here, one card or a group, that holds it at Q4_K_M. How the KV cache works.

How fast LensVLM 9B writes

Tokens per second for one request with 8,192 tokens of context, estimated from memory bandwidth. A dash means it does not fit on one card. Try other settings in the speed calculator.

Longest context on one GPU

How many tokens of context LensVLM 9B fits on a single card with one request, an FP16 KV cache and 0.5 GB left free. "Full" means the model's whole context window fits.

Model details

Parameters

9.4B (9,409,813,744); Apple's vision-language model built on Qwen3.5-9B; the weights are under Apple's research-only license

Layers

8 of its 32 layers use full attention and 24 are linear-attention layers with no growing cache

Attention cache

4 KV heads × 256

Context length

262,144 tokens

Published weights

17.5 GB (BF16)

On Hugging Face

apple/LensVLM-9B

Reported benchmarks

None published in the model card for GPQA, MMLU-Pro or LiveCodeBench.

LensVLM 9B next to similar-sized models

The three models here closest to it in parameter count, at Q4_K_M with 8,192 tokens of context.