Solving Flaky Playwright Tests: How We Automated E2E Generation with an AI Agent and MCP

Every developer knows the dread of seeing a green CI pipeline turn red because of an end-to-end test failure after a harmless frontend update.

You change a Tailwind class, reorder a flexbox container, or update an icon, and suddenly three Playwright specs break. The underlying logic works perfectly, but the tests were coupled to brittle DOM structures or arbitrary class names.

Maintaining E2E test suites is one of those engineering tasks we know is critical, but teams often push aside because scaffolding and debugging selectors feels like an endless chore.

In this post, I want to break down an architectural approach to solving this: using an autonomous AI agent that explores web apps, infers user journeys, and generates standard Playwright code with MCP support.

Brittle Locators: Tests relying on CSS classes (.btn-primary-v2) or deep XPath selectors break constantly. Modern frontends with utility CSS or CSS modules make this even worse.

Scaffolding Friction: Writing boilerplate for auth flows, form validations, and navigation paths takes valuable engineering hours.

The Vendor Lock-in Trap: Many modern AI testing tools hide your tests inside proprietary black-box runners or cloud dashboards. If you cancel your subscription, you lose your entire test suite.

Traditional test recorders simply watch where you click and record raw coordinates or selectors. Instead, an agentic model interacts with the web application like a human QA engineer:

Rather than scraping raw CSS selectors, the agent analyzes the page's Accessibility Tree and semantic structure. It identifies interactive elements by their functional roles (role="button", accessible labels, and ARIA attributes), matching how actual users perceive the UI.

The agent crawls the application dynamically, discovering actionable paths: signup flows, navigation states, filter updates, and checkout steps.

The most important design decision: no proprietary formats. The output must be idiomatic Playwright TypeScript code using standard practices like page.getByRole() and page.getByLabel():

import { test, expect } from ' @playwright/test '; test ( ' completes user onboarding flow ', async ({ page }) => { await page. goto ( ' /signup ' ); // Using resilient accessible locators rather than brittle class names await page. getByLabel ( ' Email address ' ). fill ( ' dev@example.com ' ); await page. getByLabel ( ' Password ' ). fill ( ' SecurePassword123! ' ); await page. getByRole ( ' button ', { name: ' Create account ' }). click (); // Validate state progression await expect ( page. getByRole ( ' heading ', { name: ' Welcome aboard ' })). toBeVisible (); }); Enter fullscreen mode Exit fullscreen mode Because it outputs standard files, you can commit them directly to your Git repository, run them locally, or plug them into your existing GitHub Actions pipeline.

Developers don't want to live inside another isolated web dashboard. Testing belongs inside the developer loop.

To bridge this gap, we built support for two developer interfaces:

A CLI tool ( @pluswithai /cli): Run explorations and trigger isolated cloud test executions straight from your terminal.

Model Context Protocol (MCP) Server: By providing an MCP server, environments like Cursor, Claude Desktop, or agent-enabled IDEs can directly interface with the test engine. You can ask your agentic IDE to "explore my staging app and generate a Playwright suite for the checkout flow", and the tool will execute the exploration and return the code to your editor.

We built this architecture into Pluswithai ( https://pluswithai.com ), an agent that explores your site, writes standard Playwright suites, and runs them on a schedule in isolated cloud browsers.

Building autonomous testing agents is an ongoing engineering challenge. Right now, our focus is on:

We're currently running an open trial (no credit card required, no waitlist) so developers can test it on their own web apps and see how it handles their flows:

👉 Visit Pluswithai - https://pluswithai.com

I'd love to hear from other engineers maintaining Playwright or Cypress suites:

Leave a comment below—I'll be actively answering technical questions about the architecture and how it works under the hood!