Your AI agent writes the test once. Optics runs it with no model.

Posted on Oct 20 • Originally published at optics-framework.org

This post first appeared on optics-framework.org, the home of Optics Framework. I work on Optics Framework.

Many AI tools for UI tests put a model in control of each test run. Thus, each run has a model cost. Also, the model can go through the app in a different sequence each time.

Optics Framework uses the model at a different point. Your AI agent goes through the app one time through the Optics MCP server. Then it writes a CSV test suite. After that, each run replays the suite and uses no model.

I did this procedure on saucedemo.com, a sample web site for tests. The agent was Claude Code with claude-opus-5-5, and the server was Optics Framework 1.11.0. This post shows the recorded session, the files that the agent wrote and the data from 10 replays.

Watch the recording on optics-framework.org

The recorded agent session at 1.5× speed. The left panel shows each Optics tool call, and the right panel shows the browser that the agent controls.

I gave Claude Code one prompt and one MCP server. Skills and user settings were off. Thus, the agent had only the information from Optics.

Use the optics tools to open https://www.saucedemo.com in Playwright with headless false. Sign in as standard_user / secret_sauce, add the Sauce Labs Backpack to the cart, open the cart, check out as Ava Birch with postcode 560001, finish the order, and check that "Thank you for your order!" appears. Then write it as an Optics suite in./checkout, run optics dry_run on it, and tell me how to run it. Enter fullscreen mode Exit fullscreen mode What the MCP server gives the agent optics mcp is a Model Context Protocol server. In version 1.11.0 it has 41 tools. 3 tools control the session, and 38 tools are Optics keywords such as press_element, enter_text and assert_presence. The agent can also read these MCP resources:

On a web page, an item in the element list can also have a loc value. Optics makes this CSS selector in the browser. Then Optics makes sure that the selector finds only that element. This is the item for the Login button on saucedemo.com:

{ "i": 3, "label": "Login", "cls": "input", "bounds": [ 494, 302, 786, 351 ], "act": [ "tap" ], "rid": "login-button", "loc": "[data-test= \" login-button \" ]" } Enter fullscreen mode Exit fullscreen mode The agent used a loc value for each of its 11 clicks and text entries.

The server has no tool to record or save a test. The agent writes the files, and it uses the rules from optics://project-format. The server also does not show the tools that tap a screen coordinate. On a different screen size, a coordinate from a screenshot can point to a different element.

The time for the session was 71 seconds, with 32 model turns. The agent made 21 Optics tool calls and read the project format one time. It read the element list 6 times, one time on each new page. Claude Code showed a model cost of $0.44.

First, the agent did the full checkout in the browser. It found "Thank you for your order!" on the last page. Then it wrote 4 files and ran optics dry_run. The dry run passed.

The test case uses 5 modules in sequence. The modules file has one row for each action, and all the values are variables:

module_name,module_step,param_1,param_2 Open Site,Launch App,${site_url} Login,Enter Text,${username_field},${username} Login,Enter Text,${password_field},${password} Login,Press Element,${login_button} Add Backpack To Cart,Press Element,${add_backpack_button} Add Backpack To Cart,Press Element,${cart_link} Add Backpack To Cart,Press Element,${checkout_button} Enter Checkout Details,Enter Text,${first_name_field},${first_name} Enter Checkout Details,Enter Text,${last_name_field},${last_name} Enter Checkout Details,Enter Text,${postal_code_field},${postal_code} Enter Checkout Details,Press Element,${continue_button} Finish Order,Press Element,${finish_button} Finish Order,Assert Presence,${order_complete_header} Enter fullscreen mode Exit fullscreen mode The elements file holds the test data and the locators. For most elements, the agent put the verified loc value first and the label second:

Element_Name,Element_ID,Element_ID_2 site_url,https://www.saucedemo.com, username,standard_user, first_name,Ava, username_field,"[data-test=""username""]",Username login_button,"[data-test=""login-button""]",Login checkout_button,"[data-test=""checkout""]",Checkout finish_button,"[data-test=""finish""]",Finish order_complete_header,text=Thank you for your order!,"[data-test=""complete-header""]" Enter fullscreen mode Exit fullscreen mode For a different user or a different product, write new values in this file. The modules stay the same.

These 4 files are the full result of the session. You can read them, review them in a pull request and keep them in Git with the code of your app.

Then I ran the suite 10 times with optics execute. The environment had no API keys. All 10 runs passed. The shortest run was 2.8 seconds, the longest run was 4.2 seconds, and the median was 3.3 seconds. Each time includes the start of Python and the start of the browser.

3 recorded replays of the suite at 1× speed, with the terminal output of each run.

To count the model calls, I recorded each DNS lookup and each network connection from the Optics Python process. In all 10 runs, the record was empty. Only the browser connected to the network, to load saucedemo.com. To make sure that the record works, I sent one request to example.com from the same process. The record showed that request.

The source code agrees with this result. Optics makes an LLM client only when you set ai_self_heal to true. The value is false if you do not set it.

Data from the agent session and from the 10 replays.

A suite from an agent has the same problem as a suite from a person: the app can change. To simulate a refactor, I replaced the login button locator in elements.csv with [data-test="login-btn-v2"]. That selector is not on the page.

The run passed. Optics did not find the first locator. Then it tried the second locator, the label "Login". The log shows each locator that Optics tried:

Locating element: [data-test="login-btn-v2"] of type: CSS... Locating element: Login of type: Text... Pressing element 'Login' Enter fullscreen mode Exit fullscreen mode The time for this step was 10.2 seconds. With a correct first locator, the time for the same step is approximately 0.1 seconds. Optics waits for the first locator until its time limit, and then it tries the label. The run passes with the second locator, but the run is slower. When the log shows a fallback, update the first locator.

Python 3.12 (minimum) and an MCP client are necessary. These steps use Claude Code.

For Cursor, VS Code and Claude Desktop, refer to Let your AI agent build tests. For CI, set headless: true in config.yaml and refer to Run tests in CI.

For an introduction to Optics Framework, read Introducing Optics Framework. To find error screens that no step examines, read Your tests passed. Your app didn't.