What To Do When a Free LLM API Says 429
Every free LLM API says yes right up until the moment it says no. The no arrives as an HTTP 429, usually forty requests into a demo or halfway through a batch job, and most tutorials end the story at "add a retry loop." Here is what actually works when you are building on free tiers, provider by pr
Every free LLM API says yes right up until the moment it says no. The no arrives as an HTTP 429, usually forty requests into a demo or halfway through a batch job, and most tutorials end the story at "add a retry loop." Here is what actually works when you are building on free tiers, provider by provider and layer by layer.
A 429 is not one problem, it is four. Free tiers meter you on several axes at once, and the fix depends on which one fired:
Read the response body and headers before you touch your code. Groq sends x-ratelimit-remaining-requests (daily) and x-ratelimit-remaining-tokens (per minute) plus retry-after, so the 429 tells you which window is empty and when to come back. The Gemini API returns 429 RESOURCE_EXHAUSTED and names the violated window in details[].quotaId, with retryDelay set for per-minute limits; its daily quotas reset at midnight Pacific. OpenRouter free-model caps jump from a small daily allowance to a larger one once the account has purchased credits, and its error text and x-ratelimit-reset-requests header say which window closed. Some providers, sadly, send nothing: if a 429 comes back bare, treat every window as closed and back off to the hour.
One provider's free tier is one quota pool. The structural fix is to treat free tiers as a pool of pools:
Hand-rolling this means a provider abstraction, a per-provider window tracker, a queue, and health checks for every one of them, which is most of a side project by itself. If you would rather not own that layer, there are routers that do it: I build FreeLLMAPI, an open-source, self-hosted router that keeps your provider keys, handles 429s and retries with backoff, falls back to other providers and models, and exposes one OpenAI-compatible endpoint at the end. The same job can be done by hand with the playbook above; the router just means you write it once and self-host it.
Whatever you choose, log per-request: provider, model, latency, status, and the rate-limit headers. A week of logs tells you which window you actually bump, and that single fact decides whether you need smoothing, shorter prompts, a second provider, or honestly, the $5 paid tier. Most "we need more quota" problems turn out to be "one long prompt repeated in a loop" problems. The 429 is information. Read it before you reroll the key.