The Best AI in the World Just Scored 12% on Real Work

That is the best score any frontier AI model achieved on a new benchmark built to test real, paid expert work. Not exam questions. Not trivia. Jobs that clients actually paid professionals to do.

I have spent years building and deploying AI systems for businesses, and I have sat through more vendor demos than I can count. Every demo is impressive. That is the point of a demo. What has been missing is an honest ruler. This week we got one.

The research firm SemiAnalysis published GDPx, a benchmark with a simple, brutal premise: take 206 real client tasks across 11 professional domains, contributed by 14 partner organisations, and grade frontier AI models on whether they can complete them to a standard a client would pay for.

Twenty frontier models were tested. The best of them completed 12% of the tasks. In some domains, including finance and legal work, the score was zero. The top model also spent roughly fifty dollars of compute per task trying.

Read that again. Not 87% with a few rough edges. Twelve percent.

Most AI benchmarks measure what is easy to measure: multiple-choice questions, math olympiad problems, coding puzzles with clean test cases. Models have been climbing those leaderboards for years, and each new record gets another wave of headlines saying human-level AI has arrived.

GDPx measures the messy middle of real work. Ambiguous briefs. Incomplete information. Deliverables that have to be right, formatted, and useful to someone paying for them. The kind of work that fills the actual day of an analyst, an accountant, a lawyer, an engineer.

When you grade against that standard, the gap between the demo and the deployment comes into focus. I wrote about this gap earlier in AI Software and the Trillion Dollar Productivity Question - the short version is that the productivity story of this decade depends on exactly this number going up, and it is worth tracking honestly rather than through marketing decks.

It does not mean AI is useless. It means AI is a tool that needs a professional holding it.

In our client work, AI already earns its keep every day: drafting documents a person then edits, triaging inboxes and calls, pulling first-pass research, answering routine customer questions at 2am. That is real money saved. But in every one of those deployments, a human owns the outcome. The model drafts; the person is accountable.

The businesses getting burned are the ones that bought the demo version of the story: fire the team, buy the agent, watch the savings. GDPx tells you where that plan lands today. On 88% of real paid tasks, it lands on a deliverable your client would send back.

If you are deciding on AI investment this year, three questions cut through the noise:

The deeper point is that trust, not capability, is becoming the constraint on AI adoption - I argued this in Trust Is Becoming the Real AI Benchmark and GDPx is the clearest evidence yet. Businesses do not deploy what they cannot trust, and trust is earned one verified deliverable at a time.

The demo era is over. The measurement era has started.

AI is genuinely transformative technology. It is also, today, a technology that completes about one in eight real expert tasks to a professional standard. Both things are true. The leaders who win this cycle will be the ones who can hold both truths at once: invest aggressively where AI assists accountable people, and refuse to plan against capabilities that exist only in keynote slides.

Twelve percent is not a reason to dismiss AI. It is a reason to deploy it honestly.

Keshavan Thiagaraja is the founder of IT Impress, a Sydney AI consultancy that builds private AI systems, AI receptionists and custom apps for Australian businesses.