Using Mastra Agents With a Custom OpenAI-Compatible API: the url Model Config, the Silent 5-Step Limit and Two Retry Layers That Multiply

I'm Aman Kumar. Mastra is a TypeScript framework for agents and workflows, and most of its examples use model strings like "openai/gpt-4o" that resolve through a built-in provider registry. That's convenient until you want the same agent to talk to something that isn't in the registry: a self-hosted vLLM or LiteLLM server, Ollama's /v1 endpoint, or a hosted OpenAI-compatible gateway.

Mastra handles this with a small config object instead of a separate provider package. It works, but a few defaults surprised me, so I tested them with @mastra/core 1.78.0 on Node 20.19.2 against a local mock server that logs every request body and can return tool calls, 401, 429 with Retry-After, and 503 on demand. The examples point at APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions accordingly. Its base URL is https://apiclaw.biz/v1; I use YOUR_MODEL_ID below, so take real IDs from /v1/models or your provider's model list.

npm i @mastra/core zod Enter fullscreen mode Exit fullscreen mode import { Agent } from " @mastra/core/agent "; const agent = new Agent ({ id: " support ", name: " support ", instructions: " You answer in two sentences. ", model: { id: " apiclaw/YOUR_MODEL_ID ", url: " https://apiclaw.biz/v1 ", apiKey: process. env. APICLAW_API_KEY, }, }); const res = await agent. generate ( " What does HTTP 429 mean? " ); console. log ( res. text, res. usage ); Enter fullscreen mode Exit fullscreen mode The model field accepts a registry string, an AI SDK model instance, or this object. There is also a { providerId, modelId, url, apiKey } form. In my tests it produced exactly the same request.

POST /v 1 /chat/completions { "model": "YOUR_MODEL_ID", "messages": [{ "role": "system", "content": "You answer in two sentences." }, { "role": "user", "content": "What does HTTP 429 mean?" }]} Enter fullscreen mode Exit fullscreen mode The user agent was ai-sdk/openai-compatible, so under the hood this is the Vercel AI SDK's OpenAI-compatible provider. There was no GET /v1/models call and no extra parameters.

The part before the first slash is a label, not a lookup. apiclaw/ gets stripped and only the rest is sent as model. Only the first slash counts, so id: "apiclaw/deepseek/deepseek-chat" sends "model": "deepseek/deepseek-chat". That's what you want for gateways whose model IDs contain slashes, but it means you can't drop the prefix. Without url, Mastra tries the prefix against its registry, and an unknown one fails with No API URL found for apiclaw/YOUR_MODEL_ID.

url must include /v1. Mastra appends /chat/completions to whatever you pass, nothing more. With url: "http://127.0.0.1:8766" the request went to /chat/completions, which most servers answer with a 404. A trailing slash (.../v1/ ) was harmless: the request still went to /v1/chat/completions.

apiKey doesn't fall back to an environment variable. When I left it out and set APICLAW_API_KEY in the environment, the request went out with no Authorization header at all and the server returned 401. For a custom provider, read the variable yourself, as in the example above.

api: "responses" switches endpoints. With that flag the agent posted to /v1/responses with an input array instead of messages. Many OpenAI-compatible servers only implement Chat Completions, so leave it off unless your server documents the Responses API. My mock returned 404 and the agent threw AI_APICallError: Unknown path /v1/responses.

Custom headers work as you'd expect. headers: { "X-Team": "docs" } showed up on every request.

Tools are where I lost the most time. A normal tool call looked fine: the first request carried tools and "tool_choice": "auto", the model returned a tool call, Mastra ran my execute function and sent a second request with four messages (system, user, assistant tool call, tool result). That's two steps.

Then I had the mock keep requesting the tool forever, which is what a weaker model does when it can't decide it's finished:

import { createTool } from " @mastra/core/tools "; import { z } from " zod "; const weather = createTool ({ id: " get_weather ", description: " Current weather for a city ", inputSchema: z. object ({ city: z. string () }), execute: async ({ city }) => ({ city, tempC: 31 }), }); const agent = new Agent ({ /*...same model... */ tools: { weather } }); const res = await agent. generate ( " Weather in Pune? " ); console. log ( JSON. stringify ( res. text ), res. steps. length, res. finishReason ); // "" 5 tool-calls Enter fullscreen mode Exit fullscreen mode The agent made five requests, ran the tool five times, and then returned normally. text was an empty string and finishReason was "tool-calls". It didn't throw an error or log a warning.

If your code only reads res.text, the user just gets a blank answer.

// raise or lower the budget per call const res = await agent. generate ( prompt, { maxSteps: 3 }); // and treat a run that ended on a tool call as unfinished if ( res. finishReason === " tool-calls " ||! res. text ) { // retry with a stronger model, or return a clear error } Enter fullscreen mode Exit fullscreen mode With maxSteps: 3 I got exactly three requests and three tool executions. Keep in mind that each step resends the whole conversation, so the fifth request carried ten messages. On per-token pricing that's where the cost goes. On a request-capped plan, one stuck question uses five requests.

This was the most surprising result. I pointed an agent at a model that always returned 503:

The first layer is the model-call retry from the AI SDK, set by modelSettings.maxRetries (or maxRetries on an entry in a model array). It's 0 by default in Mastra, it backs off exponentially (1, 2, 4, 8 and 16 seconds in my log), and it honoured Retry-After: 2 on a 429.

The second layer is Mastra's default error processors. They retry a failed generation twice, about three seconds apart, whatever maxRetries says. With both layers on, each of the three outer attempts ran its own six inner attempts, so one bad upstream turned into 18 requests and a 99-second wait. A 401 failed at once, with no retry.

What I settled on is one layer, set explicitly:

const agent = new Agent ({ //... errorProcessorDefaults: false, // turn off the outer retry }); await agent. generate ( prompt, { modelSettings: { maxRetries: 2 } }); // 3 attempts, honours Retry-After Enter fullscreen mode Exit fullscreen mode You can also keep the defaults and leave maxRetries at 0. Just don't turn up both, and if you have your own retry around generate(), that makes a third layer.

model also accepts an array, and Mastra moves to the next entry when one fails:

model: [ { model: { id: " apiclaw/YOUR_PRIMARY_MODEL ", url, apiKey }, maxRetries: 1 }, { model: { id: " apiclaw/YOUR_BACKUP_MODEL ", url, apiKey }, maxRetries: 0 }, ], Enter fullscreen mode Exit fullscreen mode That worked, but with the default error processors still on, the primary got six requests (two inner attempts times three outer) before the backup got its first one, and the whole call took 9 seconds. If you rely on fallbacks, turn off the outer retry so the switch happens after one or two failures.

agent.stream() streamed text fine, but the request had no stream_options, so a server that follows OpenAI's behaviour sends no usage chunk, and await stream.usage came back with empty token counts. If you log tokens, build the model with the AI SDK provider directly and pass that instance to Mastra:

import { createOpenAICompatible } from " @ai-sdk/openai-compatible "; const apiclaw = createOpenAICompatible ({ name: " apiclaw ", baseURL: " https://apiclaw.biz/v1 ", apiKey: process. env. APICLAW_API_KEY, includeUsage: true, }); const agent = new Agent ({ id: " a ", name: " a ", instructions: "... ", model: apiclaw ( " YOUR_MODEL_ID " ) }); Enter fullscreen mode Exit fullscreen mode With that, the request included "stream_options": {"include_usage": true} and usage reported input, output and total tokens. generate() reported usage either way.

Disclosure: I build APIClaw, the OpenAI-compatible gateway in these examples (flat monthly plans; crypto on the site, other payment methods through support). Everything above applies to any compatible endpoint.