Timeout is not failure: the email bug every AI writes

I build back-office systems for small Italian companies: CRMs, booking flows, the boring machinery that sends the right email to the right person. I write most of that code with an AI assistant. I'm not hiding it, it's how I work.

And there is one bug my assistant writes almost every time, in every project, unless I stop it. It looks like good engineering. It passes the tests. And in production it sends your customer the same email twice.

Ask an AI to "send the confirmation email, and make it robust", and you'll get something like this:

async function sendConfirmation ( booking: Booking ) { for ( let attempt = 1; attempt <= 3; attempt ++ ) { try { await emailProvider. send ( confirmationFor ( booking )); return; } catch ( err ) { if ( attempt === 3 ) throw err; await sleep ( 1000 * attempt ); } } } Enter fullscreen mode Exit fullscreen mode Retries, backoff, a final throw. A reviewer would approve it. So would I, on a tired day.

The problem is hidden in that catch. It treats every error the same way: the email didn't go out, try again. That's true for some errors. For the most common one in production, it's not.

Say the request to your email provider times out after 10 seconds. What do you know?

Not much. Maybe the request never left your server. Maybe it reached the provider, the provider queued the email, and the response got lost on the way back. Maybe the provider is just slow today and your email is already in the customer's inbox.

A timeout doesn't mean "failed". It means "I don't know". This isn't news: it's one of the first things Martin Kleppmann explains about unreliable networks in Designing Data-Intensive Applications. When you get no answer, you can't tell a lost request from a lost response. But it's easy to forget when the code that handles it is three lines long and written by someone else, or by something else.

And when you retry on "I don't know", you sometimes send it twice. For a newsletter, that's embarrassing. For a booking confirmation, a password reset, an invoice or a medical report, it's a support ticket, or worse, a customer who stops trusting your system.

The tests don't catch it because the fake email provider in your tests either succeeds or fails cleanly. Real networks have a third state, and nobody wrote a test for it.

If your provider supports an idempotency key, use it. You send a unique key with each email, the provider remembers it, and a retry with the same key is ignored instead of sent again. That's how payment APIs make retries safe (Brandur Leach's write-up of Stripe-style idempotency keys is the classic explanation), and it's the answer Malcolm Featonby gives in the Amazon Builders' Library: make the write idempotent, then retry freely.

Many transactional email APIs don't offer that. The one I use doesn't document one. So the rest of this post is what you do when the other side can't deduplicate for you.

Sending an email then has three possible results:

In code, I make the third one a type of its own, so nobody (human or AI) can treat it like the others by accident:

/** The provider rejected it, or the request never reached it: the email did not go out. Safe to retry. */ export class DeliveryRefused extends Error { override readonly name = ' DeliveryRefused '; } /** We don't know if the email went out: no answer in time, or the connection broke after the request was sent. */ export class DeliveryUncertain extends Error { override readonly name = ' DeliveryUncertain '; } Enter fullscreen mode Exit fullscreen mode The interesting part is deciding which is which. My rule: an error is "refused" only if it proves the email didn't go out.

On the network side, that means the request never arrived: the DNS name didn't resolve, nobody was listening on the port, the network had no route, the connection was never opened. On the HTTP side, it means the provider looked at the request and said no (a 4xx). Everything else is uncertain, the timeout first of all, and a 5xx too: a gateway that answers 502 or 504 may have passed your request on before giving up.

// network errors that prove the request never reached the provider const NEVER_ARRIVED = new Set ([ ' ENOTFOUND ', ' EAI_AGAIN ', ' ECONNREFUSED ', ' EHOSTUNREACH ', ' ENETUNREACH ', ' UND_ERR_CONNECT_TIMEOUT ', ]); // the email adapter: the only place that knows how the provider talks async function sendEmail ( msg: Message, config: EmailConfig ): Promise < void > { let res: Response; try { res = await fetch ( config. providerUrl, { method: ' POST ', body: JSON. stringify ( msg ), signal: AbortSignal. timeout ( config. timeoutMs ), }); } catch ( err ) { const code = networkErrorCode ( err ); // reads err.code or err.cause.code if ( code && NEVER_ARRIVED. has ( code )) throw new DeliveryRefused ( `never arrived ( ${ code } )` ); throw new DeliveryUncertain ( ' no answer, the email may have gone out ' ); } if ( res. status >= 500 ) throw new DeliveryUncertain ( `provider answered ${ res. status }, it may have gone out` ); if (! res. ok ) throw new DeliveryRefused ( `provider rejected it ( ${ res. status } )` ); } Enter fullscreen mode Exit fullscreen mode Notice the default. When in doubt, the code says "uncertain". An AI tends to write the opposite: it lists the scary errors and treats everything else as "retry".

The safe default has to be written down, because it's not the one that comes naturally.

(Reads are different. If a request that only reads times out, nothing changed on the other side, so you can retry it, with backoff, a little random jitter and a limit, as the Google SRE book and Marc Brooker's piece on backoff and jitter both recommend, so your retries don't pile onto a provider that is already struggling. The rule here is only about calls that write: send an email, charge a card, post a message.)

Three outcomes are not enough on their own. If your server restarts in the middle of a send, the retry logic is gone, and a background job might pick up the "unsent" email and send it again.

So before sending, the code claims the email in the database. A claim is a row that says "this email is being sent, hands off". It's written and committed on its own, and the send happens after, with no database transaction left open while we wait for the provider. The send happens only if the claim succeeds, and what happens next depends on the outcome:

type SendOutcome = ' sent ' | ' already-claimed ' | ' uncertain '; async function sendEmailOnce ( id: string, send: () => Promise < void > ): Promise < SendOutcome > { if (! ( await claimEmail ( id ))) return ' already-claimed '; // someone else has it, or it's already done try { await send (); } catch ( err ) { if ( err instanceof DeliveryUncertain ) { alertHuman ( `email ${ id }: outcome unknown, stays claimed, will NOT be resent` ); return ' uncertain '; // the claim stays: nobody resends it } await releaseClaim ( id ); // only a sure "no" frees the claim throw err; } await markEmailSent ( id ). catch (() => alertHuman ( `email ${ id } went out but the mark failed: stays claimed, will not be resent` ), ); return ' sent '; } Enter fullscreen mode Exit fullscreen mode Read the branches again, because each one is a decision:

In the vocabulary of distributed systems, this is choosing at-most-once delivery: an email may occasionally need a human, but it never goes out twice. The opposite choice, at-least-once (the transactional outbox pattern that Chris Richardson describes in Microservices Patterns, for example), never loses a message but needs the receiver to deduplicate. Without an idempotency key, the receiver here is your customer's inbox, and inboxes don't deduplicate.

Two details that make the human part cheaper:

Here's the part that matters most when an AI writes your code: a rule in a comment is a rule the AI will forget. Next month I'll ask for a new email flow, the assistant will write the friendly retry loop again, and it will look fine.

So the rule becomes a test that fails. The fake email provider in my tests can do the nasty thing real networks do: accept the email, then time out.

it ( ' never resends an email whose outcome is uncertain ', async () => { const provider = fakeProviderThatAcceptsThenTimesOut (); await sendConfirmation ( booking, { provider }); await runRecoveryJob ({ provider }); // the job that picks up "unsent" emails await sendConfirmation ( booking, { provider }); // and a user clicking twice expect ( provider. accepted ). toHaveLength ( 1 ); // one email reached the customer, not three }); Enter fullscreen mode Exit fullscreen mode This test is the real guardrail. When a future change brings back the naive retry, the test goes red, and the AI has to explain itself. That's a pattern I use everywhere now: when I find a trap that matters, it doesn't go into a note. It goes into a check that blocks.

A read can be retried. A write can be retried only if it's idempotent, or if you're sure it never arrived.

It applies far beyond email: payments, SMS, WhatsApp messages, webhooks you call, anything that changes the world outside your database. Your AI assistant knows how to write a retry loop. What it doesn't know is which of your errors mean "I don't know". You have to tell it, and then you have to make the test suite remember.

The ideas in this post aren't mine. I learned them from these, and they explain them better:

What I added is small: the habit of turning each of these rules into a test, so the AI that writes my code can't forget them.

I'm Matteo Mazzocchi, an Italian developer. I build CRMs and back-office systems for small businesses, with an AI pair and a lot of tests that say no. More about my work: mazzocchimatteo.com.