Building a test management tool where AI agents are first-class citizens, not an afterthought

TestPulse 1.0.0 just shipped. It's an open-source (AGPLv3) test management platform — projects, versioned test cases, suites, runs, reporting, a Jira connector — but the part worth writing about is how it treats AI agents.

Most test management tools have a REST API. Some have a decent one. But "has an API" and "is designed for an AI agent to operate it safely" are different bars. An agent acting on your test suite needs:

None of that is exotic, but almost nothing ships with it by default. So I built it in from the start.

TestPulse exposes an MCP server at /mcp, inside the same binary as the REST API — not a separate service, not a plugin. It's authenticated exactly the way REST is: a Bearer token, scoped to one project and one role, or unscoped and acting at its creator's current permissions, resolved fresh on every call.

Fourteen tools, deliberately coarse-grained — not a CRUD mirror of every REST endpoint, and not a handful of do-everything workflow mega-tools:

upsert_test_case, upsert_step_group, upsert_test_suite, update_suite_membership, bulk_add_cases, create_test_run, record_result, update_run_item_status, search, get_details, get_run_progress, list_run_items, list_unit_test_results, list_projects Enter fullscreen mode Exit fullscreen mode Deletes are REST-only, deliberately — no delete tool exists.

Three of those tools ( upsert_test_case 's create path, bulk_add_cases, create_test_run ) accept an optional idempotencyKey. If a call times out and you retry with the same key, the server returns the original result instead of creating a duplicate — keys are scoped per caller and tool, expire after 24 hours, and only a successful call gets cached, so retrying a failed (validation/permission) call is always safe.

The scenario I kept coming back to while designing this: "Read requirements.md and create a test suite in TestPulse."

An agent connected via MCP does this in two calls:

That atomicity matters more than it sounds like it should — the alternative (loop over a create-case call per case) leaves you debugging a partially-populated suite when call #14 of 20 fails on a schema issue.

git clone https://github.com/Barayo/TestPulse cd TestPulse docker compose -f deploy/compose/docker-compose.yml up --build -d Enter fullscreen mode Exit fullscreen mode That gets you a running instance with a seeded Sample Project — every MCP tool above has real data to operate on immediately, no setup required.

Repo: https://github.com/Barayo/TestPulse Docs: https://github.com/Barayo/TestPulse/blob/main/docs/mcp-server.md

Feedback, issues, and PRs welcome — it's a 1.0, not a finished product.