How to Choose Your First AI Project

Posted on Nov 1 • Originally published at pykero.com

Your first AI project should be a narrow, repetitive, text-heavy task that a human already does slowly or inconsistently, where the inputs already exist and the output can be checked by someone on your team. It should ship in weeks, not quarters, and it should have one metric you can read six weeks after launch. Everything else about "AI strategy" follows from what that first result teaches you.

Most founders we talk to get this backwards. They start with the most visible idea (a chatbot on the homepage, an "AI copilot" for the whole product) and end up with a demo that impresses in a meeting and never reaches production. The filter below is what we use to steer them toward something that actually ships.

Run every candidate through these in order. A project that fails any one of them is a second or third project, not a first.

The best first projects are already happening manually. Someone on your team reads inbound leads and sorts them. Someone summarizes support tickets for the weekly review. Someone copies fields from PDFs into a CRM. Someone writes near-identical follow-up emails forty times a week.

If nobody is doing the task today, you are not automating a workflow, you are inventing a product. That is a legitimate thing to do, but it is a product bet with product risk, and it should not be your first AI project.

The complaint matters as much as the task. A workflow people hate gives you a motivated internal user who will test the output, report failures, and defend the project when it is imperfect in week two.

Text in emails, documents, support tickets, call transcripts, and web pages is usable. Data scattered across six spreadsheets with inconsistent column names is not, and "we just need to clean the data first" is the most common cause of death for a first AI initiative.

Pick a task where the model can read what already exists. If the honest answer is that the data needs a migration project before any model can touch it, choose a different first project and put the migration on the roadmap separately.

Your first project needs a human who can look at the output and say "yes" or "no" in under a minute. A drafted email, a classification label, an extracted set of fields, a one-paragraph summary: all checkable. A multi-step agent that negotiates with vendors or edits your database autonomously is not checkable by glancing at it, and you have no evidence yet about how your model behaves on your data.

This is also why we tell founders to stay internal for the first project. When the output is wrong, the cost is a few minutes of a colleague's time, not a customer's trust. Once you have a few weeks of real error rates, the decision to go customer-facing becomes a decision based on data. We cover what that transition takes in moving an AI agent from pilot to production.

Hours saved per week. Reply rate. Percentage of tickets routed correctly. Days from inbound lead to first response. One number, measured before and after.

If the only honest success criterion is "the team feels more productive", the project cannot be evaluated and cannot justify a second one. For the finance conversation behind this, see our guide on calculating the ROI of an AI agent.

Our own first internal AI project was not glamorous. We send outreach to prospective clients, and the slow part was research: someone had to open each company's site, understand what they do, and write an email that referenced it. That checked every box. A human did it and disliked it. The input, a public website, already existed. A colleague could read the draft and judge it in thirty seconds. The metric was reply rate.

We built a small engine that crawls each prospect's site with a self-hosted crawler, hands the content to a locally hosted model, and produces one tailored email per company. The first architecture we tried was a chain: one step to extract facts about the company, another to pick an angle, another to draft. It worked, but when we collapsed it into a single prompt that extracted the relevant facts and drafted the email in one pass, the drafts got better and the cost per email dropped. That finding shaped how we build client projects now, and we wrote it up in single call vs agent chains.

The point is not the specific architecture. The point is that a narrow, checkable, measurable first project taught us something general in a few weeks, which is exactly what a first project is for.

A first project that passes the filter is typically a few weeks of agency work: a short discovery to pin down the task and the metric, a build, and a couple of weeks of supervised use with real inputs. If the quote you receive runs beyond six to eight weeks, or requires standing up new infrastructure before anything is usable, the scope has drifted. Push back on scope before you push back on price.

Model costs for an internal task at this scale are usually a rounding error next to the build. Our outreach engine runs on a locally hosted model and a self-hosted crawler, so the per-email cost is effectively hardware we already own. The architecture change mattered more than the model choice: collapsing three model calls into one cut the cost per email, and it did so while improving the drafts. The exception is high-volume document processing, where model choice does matter; even there, a small or self-hosted model is often enough once the task is narrow, and you can test that during the build rather than deciding up front.

Also budget for the boring part: logging every input and output from day one. We could only compare the chain against the single prompt because every crawled page and every draft had been stored. Without that log you cannot measure error rates, you cannot compare two approaches, and you cannot answer the "did it work" question in week six.

If it worked, the second project is usually obvious: the next step in the same workflow, or the same task for a neighboring team. If it did not, you have learned where your data or your domain breaks a model, for the price of a few weeks rather than a quarter.

If you have a candidate in mind and want a second opinion on whether it passes the filter, let's talk.

Originally published on the Pykero blog.