OpenAI API Cost Too High? Six Fixes, Ordered by Effort

Posted on Oct 20 • Originally published at promptunit.ai

I am the founder of PromptUnit. This post first appeared on the PromptUnit blog, where it is kept up to date.

If your OpenAI API cost feels too high, it usually is, and the fix is rarely "switch providers." OpenAI's current GPT-6 lineup alone spans a 100x price range: GPT-6 Astra costs $10.00 per million input tokens and $50.00 per million output tokens, while GPT-6 Luna costs $0.10 and $0.50. A team that sends every call to one strong model can often cut its bill sharply on the tasks where a cheaper model holds up. Moving a call from GPT-5.4 ($2.50/$15.00) to GPT-5.4 mini ($0.75/$4.50), for example, cuts that call's cost to 30% of what it was. The savings come from six approaches, none of which require rebuilding the product. They stack on top of each other, and the first two require almost no engineering work. Start there.

The short version, in the order we would do it. Prices here and below are OpenAI's standard rates per million tokens (input/output) for prompts up to 272K input tokens, checked on OpenAI's pricing page on October 16, 2026.

The rest of this post goes through the six main fixes in order of effort, then the newer settings (Flex, reasoning effort, Batch limits), current prices, a cost formula and the measuring code.

The biggest single lever is also the easiest to pull. If your application is using GPT-5.4 ($2.50/$15.00) for tasks that do not require frontier reasoning, GPT-5.4 mini ($0.75/$4.50) costs 3.3 times less on both input and output. That is not a rounding error. A workload costing $3,000 per month on GPT-5.4 costs roughly $900 per month on GPT-5.4 mini for the same call volume, if the cheaper model passes your test.

The same logic applies one generation up. OpenAI's models page points to GPT-6 Astra ($10.00/$50.00) for complex reasoning and coding, GPT-6.1 Sol ($2.00/$10.00) to balance intelligence and cost, and GPT-6 Luna ($0.10/$0.50) for cost-sensitive, high-volume workloads. Astra to 6.1 Sol is 5 times cheaper per token. 6.1 Sol to Luna is another 20 times.

The practical test is simple: run 200 representative samples from your production traffic through both models. Score the outputs using whatever quality metric matters for your use case: accuracy on a labeled test set, human review ratings, downstream task performance. Classification, extraction, summarization, and Q&A with well-defined source context are where a cheaper model most often holds up, but how close it gets depends on your prompts, and OpenAI does not publish a gap you can rely on for your task. Let your own eval decide. For tasks requiring complex multi-step reasoning, extended chains of logic, or synthesis across long documents, expect a larger gap.

If you are running GPT-5.4 mini today and looking to go further, GPT-6 Luna at $0.10 per million input tokens and $0.50 per million output tokens is 7.5 times cheaper on input and 9 times cheaper on output. It also costs less than GPT-4o-mini ($0.15/$0.60). A workload costing $900 per month on GPT-5.4 mini would cost roughly $100 to $120 per month on GPT-6 Luna for the same tokens, if the task allows it.

One catch: GPT-5.4 mini defaults to no reasoning, while GPT-6 Luna defaults to medium reasoning effort, and reasoning tokens are billed as output. Set the effort you need before you compare (Luna accepts none ). Run the eval and find out.

The downside of a failed experiment is a few hours of engineering time. The "GPT-4o vs GPT-4o-mini" section below gives a framework for making this decision.

Output tokens cost more than input tokens on every GPT and o-series model on OpenAI's price list. On GPT-5.4, output costs $15.00 per million versus $2.50 per million for input, a 6:1 ratio. On the GPT-6 models it is 5:1 (GPT-6.1 Sol: $10.00 versus $2.00). On GPT-4o-mini, it is $0.60 versus $0.15, a 4:1 ratio. Reducing output token count has an outsized impact on total cost compared to reducing input by the same amount.

Adding output constraints to your existing prompts takes 10 minutes and costs nothing. "Respond in under 100 words" cuts verbose responses. "Return only the JSON object, no explanation or preamble" eliminates all the conversational wrapper around structured outputs. "Use at most three sentences" bounds conversational replies. If your application currently generates 300 tokens of output on average and these constraints bring it to 180 tokens, you have cut output costs by 40%. On a $2,000 monthly OpenAI bill where 70% is output token cost, that is $560 per month saved from prompt edits.

On reasoning models, the output also includes thinking you never see. OpenAI bills reasoning tokens as output tokens, and depending on the problem a model can generate anywhere from a few hundred to tens of thousands of them. Two settings help. reasoning.effort is covered in the Flex and reasoning section below. max_output_tokens caps reasoning and visible output together, so leave room: if the model hits the cap while still reasoning, the response comes back incomplete and you have still paid for the input and the reasoning. The Responses API also has a text.verbosity setting that changes how much detail the model writes.

Test the constrained prompts against your quality metric before deploying. For many structured tasks, shorter outputs are not worse outputs, they are more direct outputs. The model can often answer the question in 100 words. The extra 200 tokens are hedging, context-setting, and preamble that your application discards anyway.

OpenAI's prompt caching is on by default for supported models, but a cache hit is not guaranteed and the discount depends on the model. Cached input costs 50% less on GPT-4o and GPT-4o-mini, 90% less on the GPT-5 and GPT-6 models (GPT-5.4: $0.25 instead of $2.50 per million), and 95% less on GPT-6.1 Sol. From GPT-5.6 on there is also a write charge: putting a prefix into the cache costs 1.25 times the normal input rate. So caching pays when a prefix is reused. OpenAI's own example: one write and nine full reads cost 2.15 times the ordinary input cost of that prefix, against 10 times without caching.

The requirement is structural. Stable content (the system prompt, tool definitions, shared reference material) goes first, and anything that changes, such as timestamps or user-specific data, goes at the end. On GPT-5.6 and later the shared prefix must be at least 1,024 tokens; on earlier models the minimum varies with request settings. A cached prefix on GPT-5.6 and later stays available for at least 30 minutes after it was last written or reused.

One catch on the newer models. With implicit caching, OpenAI places the cache breakpoint at the end of the latest eligible message, which in a simple request is the user message. If every request is the same system prompt followed by a different user message, the shared system prompt on its own is not reused. OpenAI's fix is an explicit breakpoint ( prompt_cache_breakpoint ) after the static part, which works on GPT-5.6 and later. Changing tools, reasoning.effort or text.verbosity between requests can also break the match. Check usage.input_tokens_details.cached_tokens instead of assuming caching works.

The savings add up for high-volume workloads with consistent system prompts. Consider a workload on GPT-6 Luna with a 2,000-token system prompt and 50,000 calls per day. Uncached input cost for the system prompt alone: 2,000 times 50,000 divided by 1,000,000 times $0.10 equals $10 per day. Now assume 9 in 10 calls read the prompt from the cache at $0.01 per million and 1 in 10 writes it again at $0.125 per million. That is $0.90 plus $1.25, or $2.15 per day. The saving is $7.85 per day, about $235 per month, from a stable prompt and one breakpoint. The 9-in-10 hit rate is an assumption: OpenAI's guide reports example deployments at about 70% and above 90% and says real rates depend on your application.

Anthropic's Claude models also discount cached input, under their own pricing rules. If you are evaluating whether to stay on OpenAI or move some workloads with large, stable system prompts to Anthropic, compare the cached rates on both sides, not just the list prices. The prompt caching cost savings guide covers both providers in detail.

After caching, the next target is the non-cached portions of your prompt: the user-specific context, the conversation history, and the query itself. Prompt compression applies systematic token reduction to this content before it reaches the API.

The three highest-impact compression techniques are system prompt consolidation (removing redundant instructions), context trimming (summarizing or truncating old conversation turns), and few-shot reduction (testing whether fewer examples maintain quality). How much they save depends on how much padding your prompts carry today.

A concrete example: a summarization endpoint with a 1,200-token system prompt, 2,800-token document context, and 500-token user query totals 4,500 tokens per call. After compression: 400-token system prompt, 1,400-token trimmed context, 300-token query totals 2,100 tokens. A 53% reduction. At $2.50 per million input tokens on GPT-5.4 and 100,000 calls per month, that is $600 per month in input cost savings from prompt optimization. For the detailed technique breakdown, see LLM token reduction techniques.

Two caveats from OpenAI's caching guide. Summarizing or truncating conversation history changes the prompt prefix and can reset cache reuse, so compare total input cost before and after, not just token counts. And on GPT-5.6 and later a prefix under 1,024 tokens is not cached at all: cutting a 1,200-token system prompt to 400 tokens can turn a discounted cached prefix into a full-price one. Fewer tokens can still come out cheaper. Measure it.

OpenAI is not always the right provider for every call. For tasks where an alternative model performs adequately, routing those calls to a cheaper provider can produce substantial savings. The question is which tasks qualify.

The usual candidates for classification, extraction, and simple Q&A are the low-cost tiers at other providers, such as Google's Gemini Flash-Lite models and Anthropic's Claude Haiku models. Their prices change often, so check today's numbers on the Gemini 3.5 Flash-Lite and Claude Haiku 5.5 price pages rather than a table in a blog post, this one included. Before you add a second provider, compare against OpenAI's own cheap tier: GPT-6 Luna costs $0.10 per million input tokens and $0.50 per million output tokens. Cheaper is not automatically good enough, either. In our own early checks below, a cheaper model was judged worse on a research step.

Cross-provider routing requires maintaining API integrations with multiple providers and building quality validation for the routed calls. It is more engineering work than the earlier options, but for high-volume workloads, the payoff is proportionally larger. See the cross-provider LLM routing guide for implementation patterns that handle fallback, quality scoring, and provider selection logic.

OpenAI's Batch API takes 50% off standard rates on the models that support it, which is most of them. Requests are submitted in bulk as a file, and each batch completes within 24 hours, often sooner. For workloads that do not require real-time responses, this is one of the simplest cost reductions available.

The use cases that qualify for batching are broader than most teams assume. Nightly data enrichment pipelines, content moderation queues, document classification for uploaded files, embedding generation, and analytics processing are all candidates. If a workload runs on a schedule or processes a queue rather than responding to live user requests, it likely qualifies.

The code change is small: write the requests to a.jsonl file, upload it, create the batch, and collect the results when it finishes. For a workload currently costing $2,000 per month that can be moved to batch, the savings are $1,000 per month. The implementation often takes a day. If a file and a 24-hour window do not fit your job, Flex processing (next section) gives the same 50% rate on ordinary requests. For more on the routing decision between batch and real-time, see the batch API routing decision guide.

Three settings that change the bill without changing the model. All the facts here are from OpenAI's Flex processing, reasoning and Batch API guides.

Set service_tier to "flex" on a normal Responses or Chat Completions request, and the tokens are billed at Batch API rates, half the standard price, with prompt caching discounts on top. The trade is slower responses and occasional resource unavailability. Flex is in beta with limited model availability. On the pricing page today that includes the GPT-6, GPT-6.1, GPT-5.6, GPT-5.5 and GPT-5.4 models, but not GPT-4o or GPT-4o-mini.

from openai import OpenAI client = OpenAI () response = client. with_options ( timeout = 900.0 ). responses. create ( model = " gpt-6-luna ", input = " Summarize this support thread in two sentences:... ", service_tier = " flex ", ) print ( response. output_text ) Enter fullscreen mode Exit fullscreen mode When capacity runs short, Flex returns a 429 Resource Unavailable error, and OpenAI does not charge for that request. Retry with exponential backoff if the job can wait, or retry with service_tier set to "auto" to pay the standard rate and get the answer now. Raise the SDK timeout as well: the default is 10 minutes, and OpenAI's own example uses 15 (the 900 seconds above).

Flex fits jobs that already run as ordinary API calls and can wait: model evaluations, data enrichment, background summaries. Batch fits large offline jobs you can package into a file.

The GPT-6 and GPT-5.6 models are reasoning models. They think in reasoning tokens before they answer, and OpenAI bills those as output tokens even though the API does not return them. reasoning.effort sets how much they think: lower effort favors speed and fewer tokens, higher effort a more thorough answer.

The defaults matter if you never set it. GPT-5.5, GPT-5.6, GPT-6 Sol, GPT-6.1 Sol and GPT-6 Luna all default to medium. Accepted values differ by model: GPT-6 Luna accepts none, GPT-6 Astra does not, GPT-6.1 Sol starts at low, and GPT-5.4 mini and nano default to none.

Fast mode deserves a check of its own. OpenAI lets you set Fast as a project's default service tier, and requests that do not name a tier then run in Fast mode at the higher price. If your bill looks double what the token counts suggest, look at the project setting.

This is an illustration with assumed percentages, not a measurement. Your numbers depend on your traffic, and each step only applies where quality holds. Applying all six options in sequence to an example high-volume workload:

Starting point: $5,000 per month on GPT-5.4 for mixed classification, extraction, and generation workloads.

Model downgrade (classification and extraction to GPT-5.4 mini or GPT-6 Luna where quality holds): saves 50 to 60% of those workloads' costs. New baseline: roughly $3,200 per month.

Output length control: reduces output costs by 35% across all remaining workloads. New baseline: roughly $2,500 per month.

Prompt caching: assume the repeated system prompt is read from the cache on most calls. Cached input on GPT-5.4 costs $0.25 instead of $2.50 per million, and here that trims roughly another 15% of the remaining bill. New baseline: roughly $2,100 per month.

Prompt compression: reduces variable token count by 40%. New baseline: roughly $1,500 per month.

Batching or Flex for workloads that can wait: cuts those workloads' costs by 50%. New baseline: roughly $1,100 per month.

Cross-provider routing for tasks that pass quality checks on cheaper models: reduces remaining workload costs by another 20 to 30%. Final baseline: roughly $800 per month.

The path from $5,000 to $800 per month is not a single dramatic change. It is six incremental optimizations applied in order of effort. Each one compounds on the others. The first two take a day. The last two take a week or more. But the direction is clear, and the arithmetic adds up.

Per million tokens, Standard tier, prompts up to 272K input tokens, from OpenAI's pricing page and models page on October 16, 2026:

Three things to notice. First, output tokens cost far more than input on every model (5:1 on the GPT-6 models, 6:1 on GPT-5.4), which is why output length control (Option 2) punches above its weight. Second, the gap between GPT-6 Astra and GPT-6 Luna is 100x, which is the whole reason model choice (Option 1) is the biggest lever. Third, newer is not always pricier: GPT-6.1 Sol costs less per token than GPT-5.4, GPT-5.5 and GPT-5.6 Sol, and less than GPT-5.6 Terra on output.

If you buy OpenAI models through Azure instead, they are billed through Azure, not OpenAI. The Azure OpenAI pricing comparison has the side-by-side table and a worked monthly example. Note that the data residency surcharge is not Azure-only: OpenAI's own regional processing endpoints add 10% on newer models, as listed above.

monthly cost = calls per month × (avg input tokens × input price + avg output tokens × output price) ÷ 1,000,000

Worked example, 200,000 calls per month at 1,000 input and 500 output tokens each:

The distance between those rows is the opportunity. One warning: on reasoning models, the 500 output tokens must include the reasoning tokens, and at the default medium effort the real number can be much higher than the visible answer. The only way to know which row your traffic can safely sit in, and what it really costs there, is to measure it per task, not per app.

Before you change models or settings, record the real cost of each call. The Responses API returns everything you need in response.usage: input_tokens, input_tokens_details.cached_tokens and, on GPT-5.6 and later, input_tokens_details.cache_write_tokens, plus output_tokens and output_tokens_details.reasoning_tokens. This Python function turns that into dollars:

from openai import OpenAI client = OpenAI () # USD per 1M tokens, Standard tier, prompts up to 272K input tokens. # From developers.openai.com/api/docs/pricing on Oct 16, 2026. Recheck before you rely on them. PRICES = { " gpt-6.1-sol ": { " input ": 2.00, " cached ": 0.10, " write ": 2.50, " output ": 10.00 }, " gpt-6-luna ": { " input ": 0.10, " cached ": 0.01, " write ": 0.125, " output ": 0.50 }, " gpt-5.4-mini ": { " input ": 0.75, " cached ": 0.075, " write ": 0.75, " output ": 4.50 }, # no write charge before GPT-5.6 } def request_cost ( model: str, usage ) -> dict: p = PRICES [ model ] details = usage. input_tokens_details cached = details. cached_tokens or 0 written = getattr ( details, " cache_write_tokens ", 0 ) or 0 uncached = usage. input_tokens - cached - written reasoning = usage. output_tokens_details. reasoning_tokens or 0 cost = ( uncached * p [ " input " ] + cached * p [ " cached " ] + written * p [ " write " ] + usage. output_tokens * p [ " output " ] # output_tokens already includes reasoning tokens ) / 1_000_000 return { " model ": model, " cost_usd ": round ( cost, 6 ), " cache_hit_rate ": round ( cached / usage. input_tokens, 2 ) if usage. input_tokens else 0.0, " reasoning_tokens ": reasoning, " output_tokens ": usage. output_tokens, } model = " gpt-6-luna " response = client. responses. create ( model = model, reasoning = { " effort ": " low " }, input = [ { " role ": " developer ", " content ": " Label the support ticket as billing, bug or other.

Reply with the label only. " }, { " role ": " user ", " content ": " I was charged twice this month. " }, ], ) print ( request_cost ( model, response. usage )) Enter fullscreen mode Exit fullscreen mode Log that result with a feature name for every call, then add it up per feature per day.

Three numbers tend to explain a high bill: which features cost the most, how much of the input comes from the cache, and how much of the output is reasoning. Fix the biggest one first. The LLM cost tracking guide covers storing and grouping these numbers.

Not sure which of these fixes matters most for you? PromptUnit shows the cost of every request and checks which ones a cheaper model can answer just as well. You change two settings, the API base URL and a PromptUnit key (your own OpenAI key stays), and it is free until it has saved you $5 (individual accounts) or $100 (business accounts with a work email). Start free

GPT-4o-mini is about 16.7 times cheaper than GPT-4o on both input and output ($0.15 versus $2.50 per million input tokens, $0.60 versus $10.00 output). For a large share of production work, the cheaper model can hold up, but only your own test can tell you where:

The decision rule is simple: send the bounded, verifiable tasks to mini and keep the open-ended, high-stakes ones on the stronger model. Do not switch blind, run a sample of your real traffic through both and compare on your own quality metric before you move anything.

Both are older models now. If you are choosing today, the same split applies one generation up: GPT-6 Luna ($0.10/$0.50) costs less than GPT-4o-mini on both input and output, and GPT-6.1 Sol ($2.00/$10.00) costs less than GPT-4o on input and the same on output. The newer models are reasoning models, though, and reasoning tokens are billed as output, so compare cost per request on your own traffic, not just the price per token.

We do not have to argue this from theory. With PromptUnit's optional side-by-side checks, after the answer has gone back to the app, cheaper models answer a sample of its real requests and the model the app asked for judges whether each cheaper answer is as good. This is early data from one production account:

In an app the PromptUnit team runs itself (checks up to October 12, 2026, again judged by Claude Sonnet 5, the model the app asked for), GPT-5.4 mini was judged as good on the app's video-prompt step in 23 of 23 checks, at about 73% lower cost. On its web-research step, Claude Haiku 4.5 was judged worse than Sonnet 5 in 2 of 3 checks, so PromptUnit kept that step on Sonnet 5. Claude Haiku 5.5 was judged as good in only 4 of 8 checks on the same step. Cheaper models win on some steps and lose on others, which is why the test has to be per step, not per app.

Across the whole bill, the measured drop on the live account is about 11%, because most of its spend is long writing steps that stay on the stronger model. For a workload with many simple requests, we estimate a 40 to 70 percent reduction across the bill. That is an estimate, not a measurement.

The samples are small, so treat them as a signal, not a benchmark. The method and limits are in What We Measured Routing GPT-4o Traffic to Cheaper Models.

That is the pattern behind every fix above: most of the bill is spent on calls that are simpler than the model running them. The hidden cost of defaulting to your flagship model turns that into dollars per month of waiting.

PromptUnit automates the per-request part of options 1 and 5: it checks each call, and when a cheaper model is just as good for that request, including one at another provider whose key you connected, it uses that model; otherwise the call goes to the model you asked for. You change two settings, your API base URL and your PromptUnit key, instead of building the routing yourself.

If your OpenAI bill has grown faster than your usage or revenue, PromptUnit gives you the per-call cost data to identify where to apply these optimizations first.

Measure what each request costs first, then work down the list: move simple tasks to a cheaper model after testing a sample of real traffic (GPT-6 Luna costs $0.10 per million input tokens and $0.50 per million output tokens), set reasoning effort per task, cap output length, put stable content first so prompt caching hits, and send work that can wait to Flex processing or the Batch API at half the standard rate. None of this requires leaving OpenAI.

Why is my OpenAI API cost so high in the first place?

Usually it's one default nobody revisited: every call goes to the flagship model regardless of task, output length is uncapped, reasoning effort stays at the default even for simple tasks, and caching either isn't set up or isn't structured to actually hit. None of those are hard problems, they just need someone to look.

Do I need to switch away from OpenAI to fix a high API bill?

No. Five of the six fixes here work entirely within OpenAI. Cross-provider routing is the one exception, and it comes late in the list because it's the most engineering work for the least additional savings once the other fixes are in place.

How fast can I see savings after fixing a too-high OpenAI API cost?

Model downgrade and output length control are same-day changes that need little engineering. Test quality on a sample of real requests first. Once they are live, you'll see the drop on your next invoice.

It is on by default for supported models, but a cache hit is not guaranteed. Requests must start with the same content, and on GPT-5.6 and later the shared prefix must be at least 1,024 tokens. On those models a cache read costs 10% of the input price (5% on GPT-6.1 Sol) and a cache write costs 1.25 times the input price. If each request is a fixed system prompt plus a new user message, add an explicit cache breakpoint after the fixed part. Check usage.input_tokens_details.cached_tokens to see what was actually cached.

Flex is a service tier for requests that can wait. You set service_tier to flex on a normal Responses or Chat Completions request and pay Batch API rates, which are half the standard price. Responses are slower, and when capacity is short you get a 429 Resource Unavailable error, which OpenAI does not charge for. Flex is in beta and only some models support it, including the GPT-6, GPT-5.6, GPT-5.5 and GPT-5.4 models.