Price Free AI Capacity in Deadline Slack, Not in Tokens

It is 02:14. Your nightly batch window closes at 02:30. The free lane is still running, and the token meter reads zero.

Last week that felt like a win. Tonight the window is gone. Which operational action follows from that evidence?

Free capacity converts the first into the second. That conversion has a rate. Measure the rate, not the sticker price.

slack_used = free_p95_duration * ( 1 + retry_ratio ) - paid_p50_duration Enter fullscreen mode Exit fullscreen mode Three terms matter. The free lane p95 per attempt. The extra attempts per completed job. The paid baseline you gave up.

A retry is not a retry. It is wall-clock time plus a new queue position plus a fresh attempt.

Each event resets the attempt clock. Each event consumes slack, not tokens. That is how a free lane stays cheap and still blows the window.

If your provider returns 429, stop and back off. Do not retry around the limit.

Here is a small script. It reads measured job-class statistics from one fixed window and prints a routing decision.

It does not call a model. It prices time.

#!/usr/bin/env python3 # breakeven.py - route batch job classes by deadline slack, not list price. import json import sys from dataclasses import dataclass @dataclass class ClassStats: name: str attempts: int # free-lane attempt starts in the window completed: int # jobs that finished inside the class deadline free_wall_p50: float # seconds per attempt, free lane free_wall_p95: float # seconds per attempt, free lane paid_wall_p50: float # seconds per job, reserved lane paid_cost: float # currency units per completed job, reserved lane slack_budget: float # seconds of negative slack this class may spend miss_penalty: float # currency units charged when the deadline slips def retry_ratio ( s ): if s. completed == 0: return None return max ( s. attempts - s. completed, 0 ) / s. completed def free_duration_p95 ( s ): r = retry_ratio ( s ) return None if r is None else s. free_wall_p95 * ( 1.0 + r ) def slack_used ( s ): d = free_duration_p95 ( s ) return None if d is None else d - s. paid_wall_p50 def miss_proxy ( s ): # Crude upper-bound proxy, not a fitted probability model. used = slack_used ( s ) if used is None: return 1.0 if used <= s. slack_budget: return 0.0 return min ( 1.0, ( used - s. slack_budget ) / max ( used, 1e-9 )) def route ( s ): used = slack_used ( s ) if used is None: return ' hold ', ' no completed jobs in window ' if used > s. slack_budget: return ' reserved ', ' p95 slack use exceeds budget ' risk = miss_proxy ( s ) * s. miss_penalty if risk >= s. paid_cost: return ' reserved ', ' risk-adjusted free cost >= paid cost ' return ' free ', ' slack within budget and risk below paid cost ' def main ( path ): with open ( path ) as fh: rows = [ ClassStats ( ** json. loads ( l )) for l in fh if l. strip ()] print ( ' class slack_used budget miss_p free_risk decision ' ) for s in sorted ( rows, key = lambda r: r. name ): used = slack_used ( s ) decision, why = route ( s ) risk = miss_proxy ( s ) * s. miss_penalty used_s = ' n/a ' if used is None else ' %ds ' % used print ( ' %-17s %9s %6ds %7.2f %10.2f %s ' % ( s. name, used_s, s. slack_budget, miss_proxy ( s ), risk, decision )) print ( ' %-17s reason: %s ' % ( '', why )) if __name__ == ' __main__ ': main ( sys. argv [ 1 ] if len ( sys. argv ) > 1 else ' jobs.jsonl ' ) Enter fullscreen mode Exit fullscreen mode Declared workload for the sample fixture State the conditions before you quote any result.

Create the fixture. These are sample values, not published measurements.

{ "name": "nightly-embed", "attempts": 1180, "completed": 1000, "free_wall_p50": 38.0, "free_wall_p95": 96.0, "paid_wall_p50": 41.0, "paid_cost": 0.42, "slack_budget": 120.0, "miss_penalty": 180.0 } { "name": "ocr-backfill", "attempts": 960, "completed": 900, "free_wall_p50": 22.0, "free_wall_p95": 150.0, "paid_wall_p50": 24.0, "paid_cost": 0.11, "slack_budget": 60.0, "miss_penalty": 40.0 } Enter fullscreen mode Exit fullscreen mode Run the router:

python3 breakeven.py jobs.jsonl Enter fullscreen mode Exit fullscreen mode Expected output on the fixture above. Label it expected until your own window reproduces it.

class slack_used budget miss_p free_risk decision nightly-embed 72s 120s 0.00 0.00 free reason: slack within budget and risk below paid cost ocr-backfill 136s 60s 0.56 22.36 reserved reason: p95 slack use exceeds budget Enter fullscreen mode Exit fullscreen mode The second class is the interesting one. Its per-attempt p95 is slower, and its retry ratio adds about 7 percent. Slack use lands at 136 seconds against a 60 second budget.

Free is the expensive lane here, and tokens never entered the calculation.

A routing rule you never broke is a guess. Break the lane first.

# worker.py is a throwaway harness, not a model client. python3 worker.py --class ocr-backfill --concurrency 4 & WPID = $! for i in 1 2 3; do sleep 60 pkill -f 'worker.py --class ocr-backfill' -n sleep 5 python3 worker.py --class ocr-backfill --concurrency 4 & WPID = $! done kill $WPID python3 breakeven.py jobs.jsonl Enter fullscreen mode Exit fullscreen mode Watch one thing: does a killed attempt show up as attempt=fail or does it vanish?

If it vanishes, your attempts counter is wrong. Your retry ratio is then wrong. Your routing decision is then wrong.

A minimal harness so the smoke test runs:

# worker.py - throwaway harness for the ledger drill only. import argparse, random, time p = argparse. ArgumentParser () p. add_argument ( ' --class ', dest = ' cls ', required = True ) p. add_argument ( ' --concurrency ', type = int, default = 1 ) p. add_argument ( ' --attempt-seconds ', type = float, default = 30.0 ) p. add_argument ( ' --fail-rate ', type = float, default = 0.15 ) a = p. parse_args () while True: time. sleep ( a. attempt_seconds ) ok = random. random () > a. fail_rate print ( ' %s attempt=%s ' % ( a. cls, ' ok ' if ok else ' fail ' ), flush = True ) Enter fullscreen mode Exit fullscreen mode This harness proves your ledger counts. It proves nothing about any model. Swap in your real worker for that.

Without these fields the ledger is unreadable a week later.

Keep tokens_in on free runs. You need it later to compute the exchange rate between free capacity and reserved capacity.

Why 50 percent as a default? Because retry ratio is usually noisier than wall time. If your retry ratio p95 moves more than your duration p95, tighten the threshold.

If the miss penalty is contractual, do not run the free lane at any threshold. Slack budgets are for engineering tradeoffs, not for contracts.

Run these in order. Write them down before the drill, not after.

Step 3 matters most. A rollback typed by hand at 02:20 is a second incident.

The drill needs a free lane before you can price one. MonkeyCode is an open-source project with free model access; the operator states 10M free tokens and a free server option.

I treat both availability claims as operator-supplied. Limits and terms change, so confirm them on the project page before you plan around them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

In this workflow the free tier is exactly the lane whose retry ratio you are measuring. Point the harness at one small class, run one window, read the ledger. Start with one class and one window on the free tier, then decide.

Every number here is a threshold with a rationale, and yours will differ. The useful question is not whether free capacity is cheap.

It is: how many seconds of slack does a retry buy, and who pays when the budget runs out?

Answer that with a measured window, and routing stops being a mood.