Team-Level LLM API Key Management: The Guide I Wish Someone Gave Me

When your app was a side project, one.env file with an OpenAI key was fine. The moment a second engineer joins, that setup turns into a small disaster: keys leak into git, a junior's experiment blows the budget, and no one knows who spent what.

Here's the system we ended up with after burning two budgets and rotating three keys.

The instinct is to put the production key in 1Password and call it a day. The problem:

Give each developer their own key. Scope it down.

That's a matrix of keys, not one key. A CI key should never have production write access. An intern's dev key should never work in staging.

OpenAI's account-level limit is too coarse — one runaway loop burns everyone's quota. You want per-key spend caps:

# Pseudocode: what the gateway should enforce per_key_budget = { " dev-llong ": { " monthly ": 20, " models ": [ " gpt-4o-mini " ]}, " prod-service-a ": { " monthly ": 500, " models ": [ " gpt-4o ", " claude-3-5-sonnet " ]}, " ci-bot ": { " monthly ": 5, " models ": [ " gpt-4o-mini " ]}, } Enter fullscreen mode Exit fullscreen mode 4. Rotate, but make it cheap Key rotation should be a 5-minute operation, not a sprint. If every rotation means updating 30 service configs, people stop doing it. A gateway layer with one ANTHROPIC_API_KEY -equivalent that routes to upstream providers lets you rotate the upstream key without touching the app.

Without this, "who burned $400 last night?" becomes a he-said-she-said.

We ran through Envoy + custom Lua scripts, then a vendor-specific dashboard, before landing on a gateway that does per-key budgets, per-model routing, and a unified audit log out of the box. Agent Token is what we use today — one OpenAI-compatible key, per-key spend caps, team usage dashboard across GPT / Claude / Gemini.

The pattern matters more than the tool, though: one key per person per environment, hard budgets, and an audit log you'd actually check.

How does your team manage LLM keys? Genuinely curious what works for bigger orgs.