Mercury 3: Diffusion-Sprachmodell von Inception zeigt im Benchmark erstaunliche Schwaechen

Signup Write New Story The AI Model That Promised to Crush GPT-6 Has a Surprising Weakness by Bruce Li by Bruce Li | @ zbruceli Engineer, building stuff

We test Inception's Mercury 3 against the fastest autoregressive models from Anthropic, Google Gemini, OpenAI, and Cerebras to find out

On October 19, 2026, Inception Labs launched Mercury 3 with a bold pitch: a diffusion language model that beats GPT-6 Luna and Claude Haiku 5.5 on benchmarks while running 4 to 8 times faster, at more than 1,000 tokens per second ( Inception blog, "Diffusion at Scale: Introducing Mercury 3", Oct 19, 2026).

Faster and smarter than the big labs' speed models. That is the kind of sentence that makes a benchmark nerd reach for a credit card. So I did. About $34 of API spend and more than 48,000 model calls later, here is what I found.

Short version: Mercury 3 is genuinely fast and genuinely good. It is not the fastest thing on the menu, it is not the cheapest per right answer, and it has one very odd habit.

Normal (autoregressive) LLMs write like a typist: one token at a time, left to right. A diffusion LLM writes like a sculptor: it starts with a rough block of placeholder text and refines many tokens in parallel over a few passes, until the answer "comes into focus." In theory that means far fewer sequential steps, and so much more speed. If you want the longer explainer, I wrote one back when Mercury first launched: What is Diffusion LLM and why it matters (Feb 2025).

Here is the 2026 version doing its thing, on the same kind of prompt I used in 2025 ("write a Python function that splits an image into left and right halves"):

Across 30 such captures, answers resolved in 5–11 visible steps, in well under a second at the median. One detail the marketing animations skip: refinement is not a smooth fade-in. In every one of the 30 calls, at least one step moved further from the final text before coming back. The sculptor occasionally un-chisels.

I picked two jobs where speed actually matters to a person waiting on a screen.

Test 1: "Write me this function" (code generation). Each model gets a programming task, for example "write a Python function that checks whether any two numbers in a list are closer to each other than a given threshold." We run its code against hidden tests. Pass or fail, no partial credit. That's 637 problems from three well-known sets (HumanEval+, MBPP+, and LiveCodeBench), 5 tries each, for every model. This is the coding-assistant use case: you ask, you wait, you hope.

Test 2: "Answer this question from our database" (a SQL agent). The model gets a real database and a question like "What is the total amount of money spent on food?" (from BIRD, a standard text-to-SQL benchmark). It can't see the data upfront. It has to look around: list tables, peek at columns, run a few test queries, then submit the query that answers the question. Up to 8 turns. We check whether its query returns the same rows as the official answer. 150 questions, 3 tries each.

Why an agent? Because agents are where speed compounds. If one call takes a second and the agent needs six of them, you're waiting six seconds. A model that is fast per call but needs more calls can lose the race. More on that in a moment.

The contenders, each the "fast" tier of its maker:

Same prompts, same settings, randomised order, everything measured end-to-end over the public APIs from one machine. Full code, configs and raw results: github.com/zbruceli/mercury3.

On code generation, Mercury 3 at its best setting ( low ) passes 86.0% of problems with a median response time of 0.83 seconds. That ties Claude Haiku with thinking off (86.1% at 0.97 s) and edges out GPT-6 Luna with reasoning off (83.2% at 1.28 s).

Then the control walks in. gpt-oss-120b on Cerebras passes 84.9% at 0.34 seconds. Within about a point of Mercury, at well under half the wait.

Raw generation speed tells the same story. Measured as visible text streamed after the first token, Mercury 3 writes about 850–900 tokens per second. That's roughly 2× Gemini Flash-Lite, 2.8× Claude Haiku and 6× GPT-6 Luna, which lands neatly inside Inception's "4–8× faster" for Luna, and short of it for Haiku. But gpt-oss on Cerebras writes about 2,400–2,500 tokens per second, ~2.7× Mercury, at every output length we tried. Diffusion's advantage is a constant factor, and it doesn't grow on longer outputs.

One nice touch you can feel: short Mercury answers arrive in a single block. Total time (0.30 s) is basically the time to first token (0.29 s). The answer doesn't stream in; it just appears.

In the SQL agent, Mercury 3 ( low ) finishes a whole task in 2.06 seconds at the median. Claude Haiku takes 3.86 s, Gemini 4.43 s, GPT-6 Luna 5.95 s. That's 1.9–2.9× faster than the frontier labs' fast models per task. Diffusion, or at least this product, delivers in the agent loop.

Claude Haiku is 3.2× slower than Mercury per call, but only 1.9× slower per task, because it solves the problem in about 3.2 turns where Mercury takes 5.1. Haiku thinks less often and better. Meanwhile gpt-oss on Cerebras is 1.7× faster than Mercury per call, but takes 6.3 turns, and ends up only slightly faster per task (1.82 s). At its medium setting it's actually slower than Mercury per task.

Lesson for agent builders: task latency ≈ call latency × number of turns, and the number of turns is a property of the model, not something your speed chart will tell you.

On accuracy, everyone lands within a few points (47.6% to 59.1% correct). Claude Haiku is the most accurate, about 5 points ahead of Mercury 3 when compared on the same questions, but that lead is borderline: its confidence interval reaches down to zero. With 150 questions, the per-model error bars are ±7–8 points, so I wouldn't crown anyone here.

Mercury 3 is priced at $0.10 / $0.35 per million input / output tokens, a 50% launch promo off the regular $0.20 / $0.70. Low per-token prices, but what matters is cost per correct answer:

Mercury isn't expensive, but it's also not the bargain the per-token sticker suggests, and the promo price won't last.

Here's the odd habit. Mercury 3 has reasoning settings, and you'd expect more reasoning to mean better answers. It's the opposite. On code, accuracy drops from 86.0% at low to 82.3% at medium and 81.4% at high.

The reason: at higher settings, Mercury 3 sometimes reasons silently until it hits the 32,768-token output cap and returns nothing at all. That happened on 7.5% of calls at medium and 8.6% at high. Those runaway calls burned 61% of all the output tokens Mercury 3 used in the code test, and pushed its slowest-5% response times past 50 seconds. Giving it 4× more room barely helped: in a small 12-call probe, only one extra call finished. No other model in the code test ran away even once.

My favourite example: an easy-looking problem asking for the "directrix of a parabola," whose single worked example doesn't match the textbook formula. The autoregressive models shrug, write the standard formula, and fail. Mercury 3 at medium and high tries to reconcile the math with the example on all 10 attempts, and thinks until it runs out of budget. Same score as a wrong guess, but at ~32,000 tokens and close to a minute per attempt. Autoregressive models fail by committing; this one fails by not stopping.

Practical advice: use Mercury 3 at low or instant. At low it is also 9 points better than its predecessor Mercury 2.5 at the same setting (86.0% vs 76.9%), and faster (0.83 s vs 1.36 s). Real progress in 20 months.

Benchmarking five vendors side by side turns up some gossip.

Verdict: Mercury 3 is faster than the big labs' fast models, about as accurate as them with reasoning off, and especially good in agent loops. It's a speed demon, just not the only one in town.

All code, configs, run data summaries, and the scripts that generate every chart here are on GitHub: github.com/zbruceli/mercury3. If you find a bug in my numbers, I'd genuinely like to hear about it.

Meet Moxie Marlinspike: The Anarchist Sailor Who Built Signal and Encrypted Billions of Chats