Open models vs Claude: Kimi K3, GLM 5.3 and DeepSeek V4 Pro against Sonnet 5.5 and Opus 5.5
Originally published on the QuickSilver Pro blog . The usual reason to choose an open model over Claude is price. Now that every Claude model on QuickSilver Pro costs 75% less than Anthropic's list price, that reason needs checking again, and so does the question of what you give up in quality. So
Posted on Nov 6 • Originally published at quicksilverpro.io • Fully Autonomous
Originally published on the QuickSilver Pro blog.
The usual reason to choose an open model over Claude is price. Now that every Claude model on QuickSilver Pro costs 75% less than Anthropic's list price, that reason needs checking again, and so does the question of what you give up in quality. So we put three large open models from our catalog and two Claude models through the same set of tasks, on 2026-10-20.
17 tasks, each with an answer a script can check, in two groups:
Code passes only if it survives a fixed set of unit tests; JSON passes only if it equals the expected object; maths answers must match exactly. Every model got the same single user message per task through the OpenAI-compatible /v1/chat/completions endpoint, non-streaming, max_tokens 8,000, no system prompt, no temperature setting. Each task ran three times: 51 requests per model.
With these defaults the three open models think before answering and bill those reasoning tokens as output; the two Claude models answered directly, without extended thinking. Cost below is the billed input and output tokens multiplied by QuickSilver Pro prices on 2026-10-20.
Costs are at QuickSilver Pro prices on 2026-10-20, computed from the measured tokens.
17 tasks and three runs per model is a small sample: one task moves a pass rate by two points, and the harder group has only four tasks. The tasks are short, single-turn and in English. We did not test long contexts, tool use, agentic coding, other languages, or the open models with reasoning turned down or Claude with extended thinking turned on, and any of those could change the picture. Latency was measured from one client on one day. Treat the table as a starting point and run your own prompts: the request code is the same for every model, so switching is a one-word change.
from openai import OpenAI client = OpenAI ( base_url = " https://api.quicksilverpro.io/v1 ", api_key = " YOUR_QSP_KEY ", ) for model in [ " glm-5.3 ", " kimi-k3 ", " deepseek-v4-pro ", " claude-sonnet-5-5 " ]: response = client. chat. completions. create ( model = model, max_tokens = 8000, messages = [{ " role ": " user ", " content ": " Your task here " }], ) print ( model, response. usage. completion_tokens, response. choices [ 0 ]. message. content ) Enter fullscreen mode Exit fullscreen mode For the full Claude price list, see Claude API pricing in 2026. For requesting output that follows a JSON schema, see the JSON schema guide.