Running website A/B tests from Claude Code with an MCP server

Coding agents can now ship a landing page in minutes. The slow part is finding out whether the new page actually converts better than the old one. That needs a split test on real traffic, which usually means a testing tool, a dashboard, and someone babysitting it.

Here is how to let the agent run that loop itself over MCP: add the site, install the tag, start a test, and read the winner, all from the terminal.

The Model Context Protocol lets a client (Claude Code, Claude, ChatGPT, Cursor, Codex) call tools on a remote server. For split testing, the useful tools are roughly:

I will use Luria's MCP server because I build it, but the workflow is the same idea with any testing tool that exposes an API (PostHog has an MCP server for feature-flag experiments, for example).

claude mcp add --transport http luria https://www.luriart.com/api/mcp Enter fullscreen mode Exit fullscreen mode The first call opens a browser: sign in with an emailed link and press Allow. No API key to paste. (Clients without OAuth can send Authorization: Bearer lk_... instead.) Cursor, Codex, VS Code and others take the same URL as a remote Streamable HTTP server:

{ "mcpServers": { "luria": { "url": "https://www.luriart.com/api/mcp" } } } Enter fullscreen mode Exit fullscreen mode 2. Add the site and the tag Prompt:

Add example.com to Luria and install the tag in this repo's layout.

The agent calls add_site, gets back a one-line script tag and per-framework install notes, edits your layout file, then calls verify_install until the tag reports in. Until a site is connected, every other tool answers setup_needed with the exact next steps, so the agent cannot wander off and create tests on nothing.

Start a test on the homepage headline. Keep the current one as the control.

The agent calls create_test (or auto_test to let Luria write the version from the page and the site's data). Two things worth knowing:

Clicks are not wins. If checkout happens on your backend (Stripe, a booking tool), send the conversion server-side:

curl -X POST https://www.luriart.com/api/v1/conversions \ -H "Authorization: Bearer $LURIA_KEY " -H "Content-Type: application/json" \ -d '{"site":"SITE_ID","visitor":"<luria_sid from the browser>","name":"purchase","value":49.99,"order_id":"1042"}' Enter fullscreen mode Exit fullscreen mode Or just ask the agent: "Wire purchase conversions from our Stripe webhook to Luria." It has the record_conversion tool and your codebase.

How is the headline test doing? If there is a winner, keep it and start the next test you suggest.

The agent calls get_test (visits, conversions and the chance each version beats the original), and next_tests / start_next_test to keep going. A change is kept only when it is 95% likely to beat the original after at least 200 visits per version. If you want to sanity-check how long that should take, here is a sample size calculator.

Disclosure: I'm the founder of Luria. New accounts get 1,000 free credits if you want to try the loop above.