How much RAM does a local LLM need on a Mac? Measurements on 16 GB and a 16–128 GB sizing table

Posted on Oct 28 • Originally published at klukyanov.ru

"Will this model run on my Mac?" usually gets answered with the model's file size. That's only a third of the answer. While a model runs, memory goes to three things: the weights, the context (KV cache) and macOS with everything else you have open. I measured three models on my MacBook M5 with 16 GB, calculated the context cost for classic 8B–70B models, and put together what actually fits into 16 to 128 GB.

So the working rule is: weights + context ≤ ¾ of your Mac's RAM. You can raise the GPU limit with sysctl iogpu.wired_limit_mb, but then apps get less and macOS starts swapping.

Setup: Apple M5, 16 GB, Ollama 0.31, MLX builds. Each model ran in three modes: a short request (a 150-word business email) with a 4K window; a 4.2K-token document (~10 pages) with an 8K window; an 18K-token document (~45 pages) with a 32K window, asking for five key takeaways. Memory is what ollama ps reports. Answers were capped at 200 tokens.

In classic architectures (Llama 3, Qwen 3 and most 2025 models) every layer caches the whole window. The size follows from the config: 2 × layers × KV heads × head dim × 2 bytes. From the config.json files on Hugging Face, at 16-bit (the default in Ollama and llama.cpp):

At a full 128K window, a classic model's cache weighs as much as the 4-bit model itself. A 70B model with that window needs 80 GB.

Applying "weights + context ≤ ¾ RAM" with the classic-model cache numbers above (newer models with a lean cache fit with room to spare). This is a calculation, not a measurement on every machine:

How much memory does your Mac have, and which local model do you actually run every day?

Shorter weekly write-ups (in Russian) — on Telegram.