Mistral Large 4 lokal betreiben: 52B aktive Parameter, aber rund 400 GB RAM nötig

Posted on Oct 22 • Originally published at blog.authon.dev

Mistral Large 4 is a 1.05 trillion parameter MoE with 52B active, and Mistral calls it open-weight. As of today, October 22, you can't download a single byte of it. The Hugging Face repo is a countdown page.

So "can I run Mistral Large 4 locally?" has two answers. Today: no. On October 31, if the date holds: yes, if you have somewhere around 400GB of fast memory and some patience. Below I sort the confirmed facts from the guesses, run the hardware math, and give the llama.cpp offload command to use once GGUFs land. That should tell you whether to start clearing a shelf or just pay for tokens.

The announcement describes an "open-weight hybrid instruct-and-reasoning MoE with multimodal input", trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs. The model docs page adds the details the blog post skips:

That last row bothers me more than the missing weights. The docs list the license as "Open" with no name. On Mistral's models overview, Mistral Large 3 and Mistral Small 4 sit right next to it, both tagged Apache 2.0. If Large 4 were going to be Apache 2.0 too, the easy move was to just say so. Until the license file exists, "open-weight" is a promise, and you shouldn't plan a product around it.

Context is also murkier than the 1M figure suggests. That number is for Mistral's hosted API. The HF placeholder lists no context length at all, so what the downloadable checkpoint natively supports is unknown until the config.json lands.

All self-reported, all from the announcement:

The cyber numbers are the interesting part. Mistral claims 82% on a reproduce-and-patch test where, per the announcement, Claude Opus 5.5 and GPT-6 Astra score near zero because they refuse the task. Whether that's a feature depends on which side of the firewall you sit.

The Hacker News thread was less impressed. One commenter said it "performed pretty poorly (on par with Kimi K3)" on their private adversarial programming benchmark. Another pointed out it costs "about 10x of Deepseek" on OpenRouter and asked why they'd use it. The defense was mostly about EU data residency and GDPR, which is a real reason for some teams and no reason at all for a hobbyist with a GPU.

The usual mistake with MoE models is reading "52B active" as "52B-sized hardware". Every expert has to be resident somewhere, because the router picks different ones per token. You need memory for all 1.05T parameters. The 52B only decides how many bytes you read per token, which is what sets your speed.

No GGUFs exist yet, so the closest real-world anchor is Kimi K2, another ~1T MoE. Unsloth's Kimi K2 guide publishes actual file sizes. Scaling those by 1.05 gives a rough picture. These are my estimates, not published Mistral numbers:

Add KV cache on top. How much depends on the attention design, which Mistral hasn't published, and at long context it won't be small.

Speed is the other catch. Kimi K2 has 32B active parameters. Mistral Large 4 has 52B. On the same box at the same quant, expect it to run roughly 1.6x slower per token, because decode on these setups is limited by memory bandwidth and you're reading about 60% more weights for each token.

Unsloth says full Q8 Kimi K2 needs at least 8x H200. An 8x H200 node has 1,128GB of HBM, which fits Q4 Mistral Large 4 comfortably with room for KV cache. 8x H100 (640GB) technically holds ~617GB of Q4 weights and leaves you almost nothing for context. Realistically that's Q3 territory.

The lineup moved around this year. In early March, Cult of Mac reported that Apple had quietly dropped the 512GB M3 Ultra option, capping it at 256GB. The M5 Ultra Mac Studio brought 512GB back in September. A 512GB box fits the Q2 or Q3 estimate above, but not Q4. A 256GB Mac can't fully hold even the ~257GB 1-bit estimate, so expect to spill to SSD.

This is how most people will run it. Keep attention and shared layers on the GPU and push the expert tensors to system RAM. llama.cpp has dedicated flags for this: --cpu-moe keeps all MoE weights on CPU, --n-cpu-moe N keeps the experts of the first N layers there, and -ot / --override-tensor gives you regex control.

Once GGUFs appear, the command will look like Unsloth's Kimi K2 recipe. The repo name below is a guess, so swap in whatever actually gets published:

./llama.cpp/llama-cli \ -hf <publisher>/Mistral-Large-4-GGUF:UD-Q2_K_XL \ --n-gpu-layers 99 \ --ctx-size 16384 \ -ot ".ffn_.*_exps.=CPU" Enter fullscreen mode Exit fullscreen mode If you have spare VRAM, use --n-cpu-moe and lower N until you run out. Every expert layer you pull back onto the GPU helps.

For expectations, Unsloth reports 1 to 2 tokens/s for the 1-bit Kimi K2 on one 24GB GPU plus 256GB of RAM, 5+ tokens/s once combined RAM+VRAM covers the whole quant (about 247GB for that one), and under 1 token/s once it spills to disk. Mistral Large 4 reads more bytes per token, so knock those numbers down. A consumer desktop that tops out at 128 or 192GB of RAM is not in the game. Look at used EPYC or Xeon boards with lots of memory channels.

If you want to know whether it's worth 400GB of RAM, spend a few dollars first. Mistral's docs use the slug mistral-large-4-0, but I couldn't find Mistral confirming the exact API model ID, so check the list before you hardcode anything:

curl -s https://api.mistral.ai/v1/models \ -H "Authorization: Bearer $MISTRAL_API_KEY " | jq -r '.data[].id' | grep large curl -s https://api.mistral.ai/v1/chat/completions \ -H "Authorization: Bearer $MISTRAL_API_KEY " \ -H "Content-Type: application/json" \ -d '{ "model": "mistral-large-4-0", "messages": [{"role": "user", "content": "Find the race condition in this Go snippet:..."}] }' Enter fullscreen mode Exit fullscreen mode On OpenRouter it's mistralai/mistral-large-4-0. Run your own evals on your own code, not the pelican test everyone already ran.

Some napkin math: at $4.18 per million output tokens, the $12,299 M5 Ultra config Guru3D lists (256GB memory, 4TB SSD) buys you about 2.9 billion output tokens. That Mac can't even hold the Q2 quant. Unless you need offline inference, EU-only data handling on your own metal, or you just like owning the thing (valid), the API wins on cost for a long time.

Stop looking at the 52B. The 1.05T total decides your hardware, and at any quant worth using that means 400GB or more; the active count only tells you how slow it will be once loaded. For a local daily driver, a 30B to 120B MoE that fits in VRAM and runs at 30+ tokens/s will be more useful than a 1T model at 2 tokens/s. I'll download the Q2 on the 31st anyway, because of course I will. But until a license file shows up next to the weights, I'm treating "open-weight" as marketing.

So: will you run a 1T MoE at home when the weights drop, or is 52B active still out of reach for your rig? Post your setup (GPU, RAM, memory channels) and what quant you think you can fit. And if you think I'm wrong about the API being cheaper, show me your power bill.