Your LLM call works 99% of the time. Your 100-call job fails 63% of the time.

You write a script that summarizes a 100-page PDF, one LLM call per page. It dies on page 71 with a timeout. You run it again, and it starts from page 1. No single part is broken: the API had one bad second, which happens. But the rerun pays for pages 1 to 70 a second time, and its odds of dying are the same as the first run's.

Model APIs bill per token, a chunk of text about three-quarters of an English word. Input tokens are what you send, output tokens are what the model writes back. A call can fail with nobody at fault: a timeout, an overloaded server, an answer your code can't parse. Call the share of calls that succeed p. At p = 99%, a single call is safe enough.

A job finishes only if every call in it succeeds, so the odds multiply: a job of N calls finishes with probability p to the power N. Two calls at 99%: 0.99 × 0.99 ≈ 98%. A hundred calls: 0.99¹⁰⁰ = 36.6%, so the job fails in 63% of runs. At 95% per call, a hundred-call job finishes 0.6% of the time. Both rates are my assumptions, not measurements: 99% is a call that fails only on network trouble, 95% is closer to one whose answer must also parse as JSON. Short jobs barely notice: five calls at 99% still finish 95% of the time.

What a failure costs depends on what your code does next. Start again from call 1, and every call before the failure is bought again, as many times as it takes. Retry only the call that failed, and nothing finished is paid for twice. Here are both, for jobs of 20 and 100 calls:

PRICE_IN, PRICE_OUT = 2.00, 12.00 # GPT-5.6 list price, $ per 1M tokens CALL = ( 1_000 * PRICE_IN + 250 * PRICE_OUT ) / 1e6 # one page: $0.005 def job ( n, p ): finish = p ** n # every call has to succeed start_over = ( 1 - finish ) / (( 1 - p ) * finish ) # calls paid, restarting from call 1 retry = n / p # calls paid, retrying the failed call return finish, start_over, retry for n, p in [( 20, 0.99 ), ( 20, 0.95 ), ( 100, 0.99 ), ( 100, 0.95 )]: finish, start_over, retry = job ( n, p ) print ( f " { n: 3 } calls at { p:. 0 % }: finishes { finish: 5.1 % }, " f " start over $ { start_over * CALL: 5.2 f }, retry $ { retry * CALL:. 2 f } " ) Enter fullscreen mode Exit fullscreen mode A page is 1,000 tokens in and 250 out.

On GPT-5.6 at its list price, the one the vendor publishes, that's $2 per million input tokens and $12 per million output: $0.005 a page, $0.50 for the PDF. The prices come from the catalog of the LLM gateway I work on, read on 29 October 2026. The start_over line is the textbook formula for the expected wait until N successes in a row.

Each × compares with a run without errors: $0.10 for 20 pages, $0.50 for 100. Retrying costs the same small extra at any length, because you pay for N ÷ p calls. Starting over grows with the job. At 95%, the 100-page PDF takes 169 runs on average, and each failed run dies about 19 pages in: 3,358 paid calls, $16.79 for a $0.50 job. By hand you'd quit after a few runs, with nothing saved.

The fix has two parts. Retry the call that failed, a couple of times with a growing pause, instead of failing the whole job. And write each answer to disk as soon as you have it. Saved progress like this is a checkpoint: if the job dies anyway, the next run reads the file, skips finished pages and starts from the one that failed.

import json, pathlib, time def run_job ( pages, call_model, out = pathlib. Path ( " results.jsonl " ), attempts = 3 ): done = set () if out. exists (): # an earlier run already paid for these done = { json. loads ( line )[ " page " ] for line in out. read_text (). splitlines ()} with out. open ( " a " ) as f: for i, page in enumerate ( pages ): if i in done: continue for attempt in range ( attempts ): try: answer = call_model ( page ) break except Exception: # narrow this to timeouts and server errors if attempt == attempts - 1: raise # give up; the next run starts from this page time. sleep ( 2 ** attempt ) f. write ( json. dumps ({ " page ": i, " answer ": answer }) + " \n " ) f. flush () # on disk before the next paid call Enter fullscreen mode Exit fullscreen mode call_model is your own function: it sends one page to the model and returns the answer as text. Each line of results.jsonl is one finished page, so a crash costs at most the call in progress, and the next run carries on from there.

The job now costs N ÷ p calls at any length: 105 for the PDF at 95%.

Why not catch the error and skip the page? The job would always finish and never pay twice, but at 95% about 5 pages in 100 would be quietly missing, and the summary would have holes you can't see.

A page too long for the model fails on every attempt: let the job stop, fix the cause, and the checkpoint keeps the rest. The math treats failures as independent, but in an outage they come together; that's why the snippet waits longer each time and then gives up. Retry only idempotent steps, ones that are safe to run twice: a call that also emails a customer isn't. And no gateway can do this for you, including the one I work on: it sees single calls, not your job, so it can't know which 70 pages you already have.

A job of N calls finishes with probability p to the power N. Make one call, not the whole job, the thing you retry, and save each answer before you pay for the next.