Your library isn't in the training data: making AI agents use it well

Posted on Oct 13 • Originally published at kevinmartinez.dev

A model that doesn't know your library doesn't fail by doing nothing. It fails by confidently writing code for a library it does know.

Ask an AI agent to write a React component with Jest tests and it'll do a decent job. There are millions of examples of both in its training data.

Now ask it to use something it has never seen:

You'll get code that looks right, compiles in the agent's head, and is wrong in small, confident ways. The agent isn't guessing at random. It fills every gap with the closest thing it knows. An option name from a similar library. An import from the popular alternative. A return shape it assumed.

I've spent the last months on this problem with TWD, a testing library I maintain that almost no model knows yet, and a Claude Code plugin that writes tests with it. This is the toolkit I ended up with. None of it is specific to my library.

Whatever you hand the agent (a skill, an AGENTS.md, an llms.txt, a docs page), it is the only place the model can learn your API. So spend that space on what teaches:

What you shouldn't fill it with is warnings. A list of "never do X" written three times doesn't stop the mistake; a program checking for it does. I wrote a whole post about why: Prose does not beat determinism.

Context isn't free, and the agent pays for every byte on every request. My first reference doc was 32 KB, and the agent loaded all of it to write a three-test file.

Now it's split: a core of about 8 KB with what most tasks need, and a second file with the rest that the agent only reads when the task calls for it. Claude Code skills support this directly, since files under references/ are loaded on demand. For any other setup it's the same idea: a short core, with links to the details.

A typed API is documentation the model can't ignore, as long as somebody runs the compiler.

When the agent invents an option name or passes arguments in the wrong order, a type check turns that into an error in seconds, instead of a bug found at runtime. So:

Some mistakes are valid code: a missing await, an import from the popular alternative library, a debugging leftover like.only, reading a property that's always undefined for your API.

That's what a small, focused linter is for. For my library it's four rules. Three properties made it work well with agents:

Before trusting it, run it against every real project you can find. A rule that fires on working code teaches the agent something false.

An agent reading a console log will grep for checkmarks, count lines, and sometimes get it wrong. Your CLI has two readers now: a human and a program.

Instructions regress like any code. A well-meant edit can quietly drop a rule the agent depended on, and nothing tells you.

An eval is a test for an agent: set up a small project, give the agent a task, then grade what it produced. In Claude Code that's claude plugin eval. A case is a folder with a scaffold script, a prompt and a list of graders:

prompt: | Write tests for the todo page: the list, the empty state, a failed load, and adding a todo. graders: - name: right-imports type: regex pattern: ' from\s+["' ' ]twd-js/runner["' ' ]' target: { source: file, path: src/twd-tests/todos.twd.test.ts } - name: linter-ran type: tool_used tool: Bash input_match: ' twd-lint' Enter fullscreen mode Exit fullscreen mode What I learned running them:

The quietest failure is drift: the library moves on, and the instructions don't.

When I compared my agent's instructions with the library's docs and source, I found an option documented that the library ignores, and a default stated backwards. The agent followed the instructions faithfully. The instructions were wrong.

If your library, SDK or design system isn't something models know yet:

Maybe one day the models will know your library the way they know React. Until then, your library has to make up for what the model doesn't know.