1. Weight memory
Parameters × effective bytes per parameter. Quantized formats include a small allowance for scales and metadata.
Open Model AtlasKnow the memory bill
Inference hardware planner · Aug 28, 2026
Know the memory bill
before the GPU bill.
Estimate weight memory, KV cache, runtime reserve, minimum GPU count, and system RAM for decoder-only model inference. Every input and formula stays visible.
6
weight formats
2
KV formats
90%
usable VRAM budget
0
signup required
Workload inputs
Describe the model and box
Available GPU hardware
Advanced architecture inputs
Use the checkpoint config for accurate KV-cache math. Changing these fields switches the preset to custom.
Reproducible math
The formulas behind the number
This is a planning estimate, not a benchmark. The calculator keeps each memory component separate so you can replace representative architecture values with the model’s actual config.
2. KV cache
2 × layers × KV heads × head dimension × context × concurrent sequences × KV bytes.
3. Runtime reserve
Weight memory × selected overhead percentage, plus a fixed 1.5 GB runtime buffer.
4. Usable hardware
VRAM per GPU × GPU count × 90%. Minimum GPU count rounds the estimated footprint up to whole GPUs.
What the estimate includes
- • Decoder weight memory at the selected effective precision
- • KV cache for context length and concurrent sequences
- • Configurable runtime and allocator reserve
- • A conservative usable-VRAM budget and whole-GPU rounding
What still needs a real benchmark
- • Training, fine-tuning, optimizer state, and gradients
- • Vision/audio towers, speculative draft models, adapters, or rerankers
- • Tensor-parallel communication, uneven sharding, CPU offload, and framework kernels
- • Throughput, latency, power, and provider-specific cost
Next decision
Now choose the model that earns that hardware.
Return to the atlas to compare task fit, licenses, deployment targets, and production cautions.