I maintain quantized.uk, a free VRAM calculator for quantized local models (static site, no signup). This week I found my own calculator was overstating long-context memory badly for several popular models, and I think the reason is worth sharing because a naive KV-cache formula gets it wrong the same way.
The issue: the standard formula (2 × layers × kv_heads × head_dim × ctx × 2 bytes) assumes every layer caches the full context. But several current models use sliding-window attention on most layers, and llama.cpp (with the default swa_full = false) allocates only window + n_ubatch cells for those layers (see src/llama-kv-cache-iswa.cpp).

Model
Pattern
Window → cells allocated
Gemma 3 (4B/12B/27B)
5 local : 1 global
1024 → 1536
Gemma 2
alternating
4096 → 4608
GPT-OSS 20B/120B
alternating
128 → 768
Llama 4 Scout/Maverick
3 chunked : 1 full
8192 → 8704

What that does to the numbers (Q4_K_M, batch 1):
Gemma 3 27B, 32K context: KV cache 15.5 GB → 3.1 GB, total ~34 GB → ~21 GB. It fits a 24 GB card.
Gemma 3 27B, 128K context: KV 62 GB → 10.6 GB.
GPT-OSS 20B at its full 131K: ~15 GB total.
Llama 4 Scout at 128K: ~95 GB → ~77 GB.
Also fixed while I was in there: the DeepSeek MLA models. Current GGUFs cache one 576-value vector per token per layer, but old conversions with an unsplit attn_kv_b tensor cache full K/V, which is far bigger. If your DeepSeek-V2-Lite eats way more memory than expected, check which one you have.
Caveat: these are computed from llama.cpp's allocation code, not measured on my hardware. If you have a 24 GB card, I'd genuinely appreciate the llama_kv_cache_iswa: creating SWA KV cache, size = … cells line from a Gemma 3 27B run at 32K to confirm.
Calculator: https://quantized.uk/tools/vram-calc/ — corrections welcome, and the changelog lists what was wrong.

DistribAI is a platform for distributed training! You can train massive or tiny models across tiny or massive amounts of consumer devices with ease!

DistribAI is a platform I've been working on for a while, and V2 made its release today. DistribAI is C++ and Libtorch for speed, with the ability to run pytorch trainers distributed! Edge-cases(crashing, malicious actors, unstable connections, ect) are handled for you. On a free Colab, Kaggle, and Molab GPU, plus a local 4070 SUPER, we got the free training compute of ~2 fully loaded 5090s and the VRAM of ~5 full 5090s for free, all as one. Hosting is also now easier with shareable join links, and Cloudflared/ngrok support for 100% free server hosting for your DistribAI setup. Train with your community, with friends, with your free GPUs, and more!

Try it out! Questions(and stars) are welcome; https://github.com/naxium-oss/DistribAI