Building an AI bug-fixing agent that runs inside your GitHub Actions runner

Disclosure: I'm building a product around this (bugzero). This article was drafted with AI assistance, then reviewed and edited by me.

Production errors are a good fit for an AI agent. There is a stack trace, there is a repository, and "done" has a clear definition: a pull request that a human reviews.

What bothered me was the usual shape of these tools. You give a SaaS read access to your repositories, and often your AI key as well, and hope for the best. So I started from one rule:

The service that coordinates the agent should never be able to read the code.

Everything below follows from that rule. I'll walk through the design, the code that enforces it, and the things that surprised me.

Two parts. A control plane knows about errors, configuration and run history. A runner, your own GitHub Actions runner, has the code and the AI key.

Sentry / PostHog / webhook │ new error ▼ control plane ── workflow_dispatch ──▶ your GitHub Actions runner (errors, config, run history) · checks out your repo ▲ · runs the agent (your AI key) │ · opens a pull request └── status · PR link · summary · token counts ◀──┘ Enter fullscreen mode Exit fullscreen mode An error arrives from the error tracker. The control plane picks the mapped repository and starts a workflow in it. In your runner, the agent explores the checked-out code and writes a fix.

The fix goes to a new branch, and a pull request is opened. The runner reports status, PR link, a summary and token counts back. The control plane can only press a button The control plane is a GitHub App with two permissions: Metadata: read and Actions: read & write.

No Contents. It cannot clone your repository or read a file. All it can do is start a workflow with workflow_dispatch.

The workflow lives in your repo, so you can read exactly what runs:

on: workflow_dispatch: inputs: run_id: description: bugzero run id required: true type: string base_branch: description: Branch to fix required: true type: string token: description: Per-run callback token required: true type: string control_plane: description: URL to report back to required: false type: string permissions: contents: write pull-requests: write jobs: fix: runs-on: ubuntu-latest timeout-minutes: 25 steps: - uses: actions/checkout@v4 with: ref: ${{ inputs.base_branch }} fetch-depth: 0 - uses: runde-software/bugzero-action@v1 with: run-id: ${{ inputs.run_id }} callback-token: ${{ inputs.token }} control-plane: ${{ inputs.control_plane }} env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} Enter fullscreen mode Exit fullscreen mode The AI key is a secret in your repository. The control plane never sees it.

The runner needs to fetch the error context and report the result, and GitHub gives it no identity toward my service. So the control plane mints a per-run token and passes it as a workflow input:

The part I had to think about is that workflow inputs are shown on the run page. Anyone who can read your repository's Actions runs can see the token until it expires. That is why it is short-lived and narrowly scoped, rather than a long-lived credential I hope nobody looks at.

There is no shell and no way to make network requests of its own. The reason is that the model's input is partly attacker-influenced: error messages, stack traces and code comments can contain text written by someone who is not you. An agent that can only read files and produce a diff has a small blast radius, and the diff still goes through code review.

Reads go through a path guard that rejects../ escapes out of the checkout:

private resolve ( rel: string ): string { const root = path. resolve ( this. opts. cwd ); const abs = path. resolve ( root, rel ); if ( abs!== root &&! abs. startsWith ( root + path. sep )) { throw new Error ( `Path " ${ rel } " is outside the repository.` ); } return abs; } Enter fullscreen mode Exit fullscreen mode Search is git grep -l -I -F, capped at 20 file names. Fixes are written in one place only: create_pull_request creates a new branch, commits as a bot user, pushes, and opens the PR. It never pushes to your base branch and never merges.

The cost of this design is real: the agent cannot run your tests. It can read a test file and mention it, but it cannot execute anything. I think that is the right default for now, and I'm not sure it is the right default forever. More on that at the end.

The loop stops at 30 tool calls or 20 minutes of wall-clock time, whichever comes first. The workflow's job timeout is 25 minutes, which leaves headroom to open the PR and report back.

The detail that mattered: every model call gets only the time that is left, not a fixed timeout. From the loop (trimmed):

const remaining = deadline - Date. now (); if ( remaining <= 0 ) { stopReason = TIME_LIMIT_REASON; break; } // Every call gets only the time left, so one slow response can't outlive the // job and leave the run unreported. turn = await adapter. next ( AbortSignal. timeout ( remaining )); Enter fullscreen mode Exit fullscreen mode Without that, one slow response at minute 19 can push the job past its timeout, and the run is never reported as finished.

1. GitHub refuses PRs from Actions by default. Repositories ship with "Allow GitHub Actions to create and approve pull requests" turned off, and permissions: pull-requests: write in the workflow cannot override it. The push succeeds, then opening the PR returns a 403. I now treat that as its own case: the branch stays pushed, the run fails with a message that says exactly which setting to flip, and the proposed PR text is kept so the work isn't lost. If an organization locks the setting, you can pass a fine-grained token instead.

2. Keep volatile data out of the runner. Model prices change often. Early on, the action computed the cost of a run itself, so every price change meant a new release. Now the action only reports the token usage of each model request, and the control plane prices the run with its own catalog. Adding a model or changing a price needs no action release.

3. Newer OpenAI models do tool calls through the Responses API. The newest models I support don't call tools through Chat Completions, so the OpenAI adapter uses the Responses API. If you support several providers, keep each one behind a small adapter that returns the same normalized "turn" (text, tool calls, usage).

5. git add -A is a trap. The PR commit stages everything in the working tree. If someone adds a build step before the action, its output ends up in the pull request. The workflow keeps that job minimal, and the docs say why.

The claim "your code never leaves your infrastructure" would be wrong, so here is the precise version:

The control plane never receives your source code. The agent's summary is the model's own text, so it can mention file names or short fragments.

The action's source is MIT-licensed: runde-software/bugzero-action. I'd like to hear where you think the design is wrong.