llm-free-cascade verkettet 20 kostenlose LLM-APIs als Fallback-Kette

In September I shared a few small Node packages. One of them was llm-free-cascade. The idea is simple: send a chat request to Groq. If Groq says no, try Gemini. Then Cerebras. And so on, through every free tier you have a key for.

The idea was simple. Getting it right was not. Here is what I learned.

My first version always started with key #1. Once key #1 ran out for the day, every request still tried it first, failed, and only then moved on. More keys made it slower, not faster.

Fix: take turns between keys. When a key is rate-limited, let it rest and skip it until it's ready again.

A 429 rarely means "this key is done." It usually means "this model is done for this key." So try another model on the same key before giving up on the provider.

new LLMCascade ({ keys: { groq: process. env. GROQ_API_KEY }, models: { groq: [ ' openai/gpt-oss-120b ', ' openai/gpt-oss-20b ' ] }, // 2nd is tried if the 1st is limited }); Enter fullscreen mode Exit fullscreen mode 3. Know how long to wait A fixed cooldown is wrong. It's too long for a per-minute limit and too short for a daily one.

Providers tell you when to come back, each in their own way:

The cascade reads all of these. A per-minute limit waits about a minute. A daily limit waits until that provider resets.

That last one matters. Without it, a provider that is down makes every request wait for the full timeout.

You could send the same request to two providers at once and keep the faster answer. On free tiers that spends twice the quota for one answer, so I left it out.

This year alone, Cerebras' free tier became a 30-day trial, SambaNova started asking for a card, and GitHub Models shut down. Model names change even more often.

So the provider list lives in one data file, providers.json. Each entry says when it was last checked and where the numbers came from. It also warns you about things like "non-commercial use only" or "needs a phone number". A small script checks every model name against each provider's live list.

One note: many providers don't allow several free accounts to get around their limits. Check each provider's terms.

Someone opened an issue asking to use it like the OpenAI API, from their own tools. So now you can run it as a local server:

npx llm-free-cascade serve # http://127.0.0.1:8787/v1 Enter fullscreen mode Exit fullscreen mode Then point any OpenAI client at it, for example from Python:

from openai import OpenAI client = OpenAI ( base_url = " http://127.0.0.1:8787/v1 ", api_key = " unused " ) reply = client. chat. completions. create ( model = " cascade ", # or "groq" to use only Groq messages = [{ " role ": " user ", " content ": " Explain photosynthesis in one sentence. " }], ) print ( reply. choices [ 0 ]. message. content ) Enter fullscreen mode Exit fullscreen mode It works with streaming too. The server only listens on your own machine by default, since it holds all your keys.

If every free tier is busy at the same time, Ollama can answer from your own computer. No limits, just slower.

new LLMCascade ({ custom: { ollama: { baseUrl: ' http://localhost:11434/v1 ', model: ' llama3.2 ', apiKey: ' ollama ', // any text works, but it can't be empty tier: 5, // try it last }, }, }); Enter fullscreen mode Exit fullscreen mode 9. You can see what it's doing npx llm-free-cascade keys # which providers you're missing, and where to sign up npx llm-free-cascade status # your providers, keys and limits npx llm-free-cascade probe # quick live test of every key Enter fullscreen mode Exit fullscreen mode In code, cascade.stats() shows calls, failures, speed and cooldowns for each provider and key.

llm-free-cascade is free (MIT), has zero dependencies, and runs on Node 18+.

Found a model name that no longer works? Fixing it is a one-file PR. Want tool calling in the server? Open an issue and tell me what you're using it with.