Your nightly LLM job pays double — every big vendor gives 50% off for waiting a day

Most LLM code I see is written as if a human is staring at a spinner. Every request is synchronous, every job fans out into 50 parallel workers, and the bill comes in at full price.

But a lot of that work runs at 3 a.m. Nobody is waiting for it. And all three big vendors will cut the price in half for exactly that kind of work — you just have to tell them you're willing to wait.

Two words first. A token is roughly ¾ of a word; models bill per million of them, with input (what you send) cheaper than output (what the model writes). A Batch API is a separate endpoint where you upload many requests at once, walk away, and download the answers later.

Same lever, same size, everywhere: you give up the guarantee of an answer in seconds, and you get half the price. That's the whole trade.

A realistic background job: every night, summarize the day's 20,000 support tickets for an internal dashboard. Each ticket is about 1,500 tokens in and 200 tokens out. I'll use GPT-5.6, whose official price is $2 per million input tokens and $12 per million output.

TICKETS, IN_TOK, OUT_TOK, NIGHTS = 20_000, 1_500, 200, 30 PRICE_IN, PRICE_OUT = 2.00, 12.00 # GPT-5.6, $ per 1M tokens, official def monthly ( discount ): m_in = TICKETS * IN_TOK / 1e6 # 30M input tokens per night m_out = TICKETS * OUT_TOK / 1e6 # 4M output tokens per night night = m_in * PRICE_IN + m_out * PRICE_OUT return night * ( 1 - discount ) * NIGHTS print ( f " sync: $ { monthly ( 0.0 ):,. 0 f } /month " ) # $3,240 print ( f " batch: $ { monthly ( 0.5 ):,. 0 f } /month " ) # $1,620 Enter fullscreen mode Exit fullscreen mode $3,240 versus $1,620 a month, for the same summaries landing on the same dashboard before anyone arrives at work. The only code change is where you send the requests and when you collect them.

Notice what parallelism does to this number: nothing. Fifty workers finish the job in twenty minutes instead of three hours, but they pay full price for every token. Concurrency buys speed; batch buys price. At 3 a.m. you don't need speed.

A quick test: if the answer arrived tomorrow morning, would anyone notice? If not, it belongs in a batch. Typical candidates:

One objection I'd raise myself: "24 hours is too unpredictable for a nightly job." In practice the vendors say most batches return far sooner, and you can always fall back to the synchronous API for whatever hasn't returned by your deadline. You pay full price only for the stragglers.

Batch is wrong whenever the next step depends on the previous answer.

The rule: batch the wide part of the job (thousands of independent items), keep the narrow part (dependent steps, humans waiting) synchronous.

Batch doesn't make a bad prompt cheaper — a 1,500-token ticket you could trim to 600 is still waste at half price. Results can come back in any order, so you match them by your own ID. Anthropic notes that batch and prompt caching discounts stack, but cache hits inside a batch are "best-effort", so don't budget on them.

And an honest gap on our side. We run a gateway that resells these same models 10–25% below the vendors' official prices — GPT-5.6 at $1.70/$10.20 against $2/$12, which puts the job above at $2,754 a month. We have no batch endpoint and no embeddings. A gateway discount and a vendor batch discount are different levers: ours applies to synchronous, interactive traffic; batch's 50% applies only to work that can wait, and right now you get it only by going to the vendor directly. For the 3 a.m. job, vendor batch beats us.

Open your billing page and find the job that runs while everyone sleeps. Then ask it whether it really needs an answer in seconds.