VRAM Calculator
This page answers one specific question: given the GPU you have, which size of local language model can you run, and how much context can you afford. Enter your VRAM, the context length you want, and how many concurrent requests you expect. The calculator then breaks down VRAM usage for every Gemma 4 variant in every quantization format, and tells you whether it fits.
All model sizes are real file byte counts pulled from the HuggingFace API, not estimates from multiplying parameter count by bit width. That distinction matters more than it sounds — different quantization formats of the same model often compare in exactly the opposite direction from what intuition suggests, and the reason is explained below.
Model sizes are real file byte counts from the HuggingFace API converted to GiB, not estimates from parameter counts. When running GGUF, the E-series PLE table and token embedding stay in system RAM, and that share is already deducted here — which is why "Weights" is much smaller than the file size. KV cache is computed from each variant's layer count, KV heads, shared layers and sliding window. The whole formula was calibrated against measured VRAM for E4B at 8K/32K/64K/128K context on an 8 GB laptop, landing within 0.15 GB; other variants are extrapolated from the same architecture, so if a row says "Tight", try a smaller context first. Multi-GPU is modeled as pooled VRAM with a separate compute buffer per card and has not been verified on real hardware — actual results also depend on interconnect bandwidth and how the model is split.
How to read the numbers
Weights is what the model parameters themselves occupy in VRAM. It is not the same as the model file size. When running GGUF, the Gemma 4 E-series keeps its PLE (per-layer embedding) table and token embedding in system RAM rather than on the GPU. The E4B q4_0 file is 4.8 GiB, but only about 2.2 GiB actually reaches the card, because that PLE table alone accounts for 2.8B parameters. Estimating VRAM from file size overshoots by more than 2 GB — enough to wrongly tell someone with an 8 GB card that E4B will not fit, when in practice it runs comfortably.
KV cache holds the cached attention keys and values during inference, and grows linearly with context length and concurrency. The E-series is unusually frugal here for two reasons: some layers share the KV of the layer above and take no space of their own, and others use sliding-window attention with a 512-token window, so no matter how long the context is, those layers only retain that short window. E4B going from 8K to 128K context adds only about 1.1 GB of KV. A same-size model without those design choices would grow several times faster.
Overhead covers compute buffers, memory fragmentation, and the inference engine itself. This one cannot be derived exactly from model structure, so it is fitted from measurements: a constant term plus a component that scales linearly with context.
How far off can it be
The formula was calibrated on a laptop with 8 GB of VRAM, running E4B quantized to q4_0 through llama.cpp’s Vulkan backend with the KV cache quantized to q8_0. Predicted versus measured across four context lengths:
| Context | Predicted | Measured | Error |
|---|---|---|---|
| 8K | 2.96 GiB | 2.90 GiB | +0.06 |
| 32K | 4.16 GiB | 4.25 GiB | −0.09 |
| 64K | 4.52 GiB | 4.54 GiB | −0.02 |
| 128K | 5.26 GiB | 5.14 GiB | +0.12 |
To be clear about the limits: this is a calibration against one variant on one software stack. Other variants are extrapolated from the same architectural formulas and were not each measured, so their error may well be larger. That is why the verdict has three levels rather than a binary yes/no — a row only reads “Fits” with a real margin left over, and thin margins are marked “Tight”. When you see “Tight”, drop the context one step and actually run it before pushing higher.
The quantization size trap
The easiest mistake when picking a variant is assuming that 4-bit quantization roughly halves a model to a size proportional to its parameter count. Google’s official Gemma 4 QAT checkpoints use the compressed-tensors format, whose config specifies targets: ["Linear"] — only linear layers get quantized, and embeddings stay in bf16. For models where embeddings are a large fraction of the parameters, the quantized result is far bigger than expected.
The most counterintuitive pair: E4B in QAT W4A16 is 10.72 GiB, while 12B Unified — three times the parameter count — is only 9.56 GiB in the same format. Pick by “fewer parameters must be smaller” and you end up with a model that is both larger and slower. The E-series’ enormous PLE table sits uncompressed in bf16 and drives the size up on its own.
GGUF q4_0 does not have this problem because it quantizes the embeddings too, bringing E4B down to 4.8 GiB. So on a small-VRAM machine, reach for GGUF plus llama.cpp rather than QAT weights plus vLLM. The latter is built for machines with VRAM to spare — its advantage is throughput and concurrency, not footprint.
Two constraints on multi-GPU
Set the GPU count above 1 and available VRAM becomes the sum across cards — but two things do not simply add up.
vLLM tensor parallelism requires the KV head count to divide by the GPU count. It splits attention heads across cards, and if they do not divide evenly it will not start. KV head counts across Gemma 4 are: E2B 1, E4B 2, 12B and 26B A4B 8 each, 31B 16. So E2B can never use tensor parallelism on multiple GPUs, E4B tops out at 2, and any non-power-of-two count like 3, 5, or 6 divides none of them. The calculator says so explicitly in that case rather than letting the affected rows silently disappear.
llama.cpp splits by layer, which has no divisibility constraint but also brings no speedup. It places different layers on different cards and streams through them layer by layer, so only one card computes at any instant. Extra GPUs let it fit a bigger model, not run one faster. If you want throughput from multiple GPUs, you need vLLM’s tensor parallelism and have to live with the divisibility rule above.
Each card also needs its own compute buffer, so overhead scales with GPU count rather than being amortized across them. I have no multi-GPU hardware to verify this against, so that part is modeled rather than measured and is coarser than the single-card figures.
How much headroom to leave
The “already used” field is easy to skip past. On a machine with a monitor attached, the desktop compositor and browser typically consume 0.3 to 1 GB before the model loads at all, and that has to come off the top. Set it to 0 for a headless compute card, or when display output runs through integrated graphics.
Also note that the multimodal projector (mmproj) is a separate file, loaded only when you need image input. For text-only inference, unchecking that box saves roughly 0.9 GB — not a rounding error on an 8 GB card.
Related reading
- Every Gemma 4 Variant: Performance, Efficiency, and How to Choose — architectural differences, benchmark results, and use cases for E2B, E4B, 12B Unified, 26B-A4B, and 31B, one at a time.
- Self-Hosting a Gemma 4 Gateway: Pitfalls You Only Hit by Running It — from miscalculating model size during selection to a tunnel hijacking DNS routing, the problems that actually came up during deployment.
- Local LLM Deployment, Part 1: Choosing a Model and Preparing the Environment — the first entry in a step-by-step tutorial series, covering how to match a variant and inference engine to your hardware.
- Tools — other browser-side utilities on this site.
The calculator solves exactly one problem: whether it fits. How fast the model then runs, how much first-token latency grows once context gets long, and at what concurrency throughput starts to collapse all depend on inference engine configuration and memory bandwidth. The calculator cannot answer those — they have to be measured. The tutorial series records the benchmarking method and parameter tuning that go with it.