LLM VRAM Calculator

To run a model, its weights must sit in GPU memory (VRAM), along with the KV cache: the model's notes on every token of the conversation so far, which grows with the context length and with every chat running at once. On top of that comes working memory for the GPU software, usually 10–20%. Fine-tuning needs more: gradients, the optimiser's saved state and the activations kept for the backward pass.

The formula

weights = parameters × bits per weight ÷ 8
KV cache = 2 × layers × KV heads × head size × bytes per number × context tokens × batch
overhead = (weights + KV cache + training extras) × overhead %
total = weights + KV cache + training extras + overhead
full fine-tune extras = parameters × (2 gradients + 4 master copy + 8 AdamW) bytes + activations
LoRA extras = adapter weights × 16 bytes + activations
GPUs needed = total ÷ memory of one GPU, rounded up
speed ≈ memory bandwidth ÷ (active weights + KV cache)

Worked example

Llama 3.3 70B at Q4_K_M (about 4.8 bits per weight) with an 8,192-token context, FP16 KV cache and 10% overhead:

• Weights: 70.6 billion × 4.8 ÷ 8 = 42.36 billion bytes = 39.45 GiB (÷ 1,073,741,824).
• KV cache: 2 × 80 layers × 8 KV heads × 128 × 2 bytes = 327,680 bytes per token; × 8,192 tokens = 2,684,354,560 bytes = 2.50 GiB.
• Overhead: 10% × (39.45 + 2.50) = 10% × 41.95 = 4.195 GiB.
• Total: 41.95 + 4.195 = 46.145, about 46.1 GiB (49.5 GB).
• One RTX 4090 has 24 GiB, so you need 2 (48 GiB). A 64 GB Mac lets the GPU use about ¾ of it, 48 GiB, so it just fits.
• Speed: each new token reads the weights and cache once, 42.36 + 2.68 = 45.04 GB. The 4090 moves 1,008 GB/s, so 1,008 ÷ 45.04 ≈ 22 tokens a second at most (the two cards take turns, layer by layer).

Things that change the answer

• Mixture-of-experts (MoE) models like DeepSeek V3, Mixtral, Qwen3 30B-A3B and gpt-oss need memory for all their weights, but only read the active ones for each token, so they run much faster than their size suggests.
• Sliding-window attention (Gemma 3, gpt-oss, Llama 4) keeps only recent tokens in most layers, and DeepSeek's compressed cache stores far less per token, so long contexts cost them much less.
• Offloading: llama.cpp, Ollama and LM Studio can keep some layers in system RAM when the model doesn't fit. It works, but system RAM is 10–30 times slower than VRAM, so speed drops sharply.
• Several GPUs split the model by layers (they take turns, so speed stays near one card's) or by tensor parallel in vLLM (faster, but usually needs 2, 4 or 8 cards and a little extra memory on each).
• QLoRA stores the frozen model in 4-bit (about 4.25 bits per weight here) and trains small 16-bit adapters, which is how a 70B model can be fine-tuned on a single 48–80 GB card.
• Full fine-tuning with mixed precision and AdamW needs about 16 bytes per weight before activations, so even an 8B model needs over 120 GB. DeepSpeed ZeRO or FSDP split this across GPUs.

Model shapes used (approximate)

GPUs and Macs used (approximate)

Model shapes come from each model's published configuration and GPU figures from makers' spec sheets, checked 10 October 2026. Bits per weight for GGUF files are typical whole-file averages. Mac figures are one chip option for each memory size; other chips with the same memory have different bandwidth. All results are estimates: real use depends on the software (llama.cpp, Ollama, vLLM, PyTorch), its settings and the driver, so leave some headroom.