Bedrock Prompt Shortening: Optimierer verlaengert Prompts statt sie zu kuerzen

Posted on Oct 26 • Originally published at gerardo.dev

In the AgentCore Optimization Loop article, I let AWS rewrite the system prompt of my DevOps agent. The recommendation was good, but it had an effect I didn't measure at the time: the prompt grew. The optimizer kept everything I had written and added two new blocks.

The Advanced Prompt Optimization documentation acknowledges this: by default it optimizes for quality only, with no length constraint, and an optimized prompt can end up longer than the original. Longer means more input tokens on every call and, potentially, a longer time to first token.

On October 20, 2026, AWS announced an option for that, shortenPrompt, which asks the optimizer for a prompt shorter than the original with an equal or better score. If it can't find one, it gives you yours back.

I wanted to answer a concrete question: how much does shortening a prompt that already works actually save me, and at what volume is it worth it? To find out, I used the real classifier prompt of my news agent, the 205 classifications it made in production as the reference, and three target models. The result wasn't what I expected, and neither were the reasons.

Advanced Prompt Optimization is Bedrock's full optimizer, different from the simple OptimizePrompt. You give it a JSONL file with your templates (variables in double braces, {% raw %}{{variable}}{% endraw %} ), up to 100 evaluation samples per template, and an evaluation method: the default judge, free-text criteria ( steeringCriteria ), an LLM judge with your rubric, or your own Lambda. The service runs an inference, evaluation, and rewrite loop on each target model and returns one optimized prompt per model. A job accepts up to 10 templates and 5 models, and the documentation talks about 15 to 20 minutes for a prompt with few samples and several hours for many. My jobs took 14 and 29 minutes.

shortenPrompt is enabled per template, inside the JSONL, not in the API call:

{ "version": "bedrock-2026-05-14", "templateId": "news-classifier-shorten", "promptTemplate": "...", "optimizationConfig": { "shortenPrompt": true }, "customEvaluationMetricLabel": "clasificacion_vs_produccion", "evaluationMetricLambdaArn": "arn:aws:lambda:us-east-1:<cuenta>:function:apo-news-classification-evaluator", "evaluationSamples": [ "..." ] } Enter fullscreen mode Exit fullscreen mode The return logic is simple: the shortened prompt is returned only if it is shorter and scores at least as high as the original. If either condition fails, you get your original prompt back with compressionRatio = 1.0. The ratio is optimizedTokenCount / originalTokenCount, measured on the template and not on the prompt with the variables filled in.

On cost, the documentation says there is no separate service charge: you pay for the tokens it consumes. The pricing page gives the formula: about 16 × N target model inferences per template, where N is the number of samples, plus the optimizer (currently Claude Sonnet 4.6) with 101 × (3700 + 0.35 × P) input tokens, where P is the token count of your template. The optimizer's output tokens have no formula.

My agent processes the AWS RSS feed three times a day. Since late September the model no longer computes relevance: it only classifies each news item into a topic ( topic_tier ) and a change type ( change_type ), and a rubric with editable weights computes the 0 to 10 score. If the score reaches 6, a post is generated; if it reaches 7, the item goes into the newsletter.

I chose this prompt for three reasons. It has an objective answer, so it can be evaluated without a subjective judge. It runs every day, so it has real volume to compute savings against. And it was already compact, about 1,900 characters: if shortening finds something to cut here, it will find it in almost any prompt. A fragment:

topic_tier: el tema al que pertenece el servicio o la capacidad (elige el más específico): • genai_agentes: Bedrock, IA generativa, agentes, MCP, Amazon Quick, AgentCore, SageMaker AI • serverless_core: Lambda, Step Functions, API Gateway, EventBridge, DynamoDB... change_type: qué tipo de cambio anuncia la noticia (juzga el cambio, no el servicio): • nueva_capacidad: una función o servicio que antes no existía • mejora: una mejora incremental de algo que ya existía (límites, rendimiento, integraciones)... Enter fullscreen mode Exit fullscreen mode The reference: what production decided I didn't label anything by hand. The reference for each news item is the classification production made, stored in DynamoDB: 205 news items classified between September 30 and October 22, 2026.

I split the set deterministically, by ID hash: 60 items for the optimizer and 145 as a holdout it never sees.

With this reference I'm not measuring whether the shortened prompt is more accurate, but whether it behaves the same as the current one. For a shortening, that seems like the right question to me: I'm looking for a cheaper prompt that doesn't break anything, which makes it a regression test. The downside is that if production was wrong, the optimizer learns to repeat the mistake.

Advanced Prompt Optimization offers four ways to evaluate, and I ruled out three. The default judge scores accuracy, completeness, and Expression Quality, and that last criterion rewards short answers, so it would have optimized the brevity of the output instead of the classification. steeringCriteria are too vague for a task with an exact answer. An LLM judge adds cost, because the judge reads prompt, response, and reference on every evaluation, and it adds its own noise to something you can settle by comparing two values.

I went with a Lambda. The contract the service requires is simple: you receive {"preds":, "golds":} and return {"score": float, "scores":}, higher is better. My rule: if the output isn't valid JSON with all the allowed keys and values, the sample scores 0; if it is, it gets 0.5 for the right topic and 0.5 for the right change type.

def compute_score ( preds, golds ): """ Mandatory JSON contract; 0.5 for the right topic_tier and 0.5 for the right change_type. """ scores = [ score_one ( p, g ) for p, g in zip ( preds, golds )] return { " score ": sum ( scores ) / len ( scores ) if scores else 0.0, " scores ": scores } Enter fullscreen mode Exit fullscreen mode The service reads the docstring as the metric description and reviews the full code before using it. That review cost me my first job; I cover it in the gotchas.

In production the prompt goes in as the system prompt, the news item as the user message, and the output uses Structured Outputs. Advanced Prompt Optimization takes a single text template and doesn't accept structured outputs, so the JSON contract has to live inside the prompt. I protected it with <advpo:exclude>: the optimizer can rewrite the classification instructions, but not the news item or the output format.

{% raw %}<advpo:exclude>Noticia a clasificar: Título: {{title}} Descripción: {{description}} Responde ÚNICAMENTE con un objeto JSON, sin texto antes ni después, con exactamente estas claves: change_type, topic_tier, topic, reason, keyPoints (lista de textos) y emojis (lista de textos). En topic_tier y change_type usa solo las claves listadas arriba.</advpo:exclude>{% endraw %} Enter fullscreen mode Exit fullscreen mode In the same job I put two identical copies of the template, one optimized for quality only (the default behavior) and one with shortenPrompt: true, to compare them under the same conditions.

This part of the design changed along the way. Since October 8, my agent consumes Claude Haiku 5.5 through the Anthropic API, not through Bedrock. Advanced Prompt Optimization only optimizes Bedrock models, and my working account can't invoke Haiku 5.5 on Bedrock. The account that can hadn't submitted the Anthropic use case form, so Haiku 4.5 didn't run there.

I ended up running one job per account, with the same data and the same evaluator: Claude Haiku 5.5 (the production model) in one, and Claude Haiku 4.5 (the production model until October 8) plus Amazon Nova 2 Lite in the other. Nova is the model I dropped in March because it didn't support structured outputs; since Advanced Prompt Optimization works with plain text, that limitation doesn't matter here. That same missing form ended up affecting the Haiku 5.5 job too, as I explain in Finding 3.

I followed the same method as in previous articles: before reading the job results, I wrote down each hypothesis with the criterion that would refute it.

📋 Recorded deviations: (1) the original design used only Nova 2 Lite, because Haiku 5.5 wasn't available on Bedrock in my working account; (2) I moved to three models split across two accounts; (3) the job runs Haiku 5.5 with its default configuration, which uses thinking, unlike production (see gotchas); (4) the first two jobs failed because of the Lambda code validator; (5) I stopped a job running the old evaluator, and its window overlaps by a few minutes with the good job in that account, which slightly inflates the costs measured there; (6) my holdout baseline ran in parallel with the Haiku 4.5 job.

All the code is at github.com/codecr/bedrock-prompt-shortening: scripts numbered 01 to 07, the evaluator, and the cleanup.

Before looking at any optimized prompt, I ran the original on the 145 holdout news items, twice on each model. The second run matters because the model isn't deterministic: if two variants differ by less than the original prompt differs from itself, that difference means nothing. Haiku 5.5 ran as it does in production, without thinking and with low effort.

Nova is the most stable and the one that agrees least with production: it repeats its answer 96.6% of the time, but being consistent doesn't make it more accurate. Haiku 5.5 doesn't reach 1.0 even against references it mostly generated itself, because in production it runs with a separate system prompt and structured outputs, and changing how you call the model already moves the result. And the noise floor is high: the same prompt changes the classification of between 5 and 17 of 145 news items from one run to the next. An improvement smaller than that can't be told apart from chance.

A business data point: with the active rubric, Haiku 5.5's classification changes the decision to generate a post on 10 of the 145 news items compared with what production decided. That margin of error already exists in my pipeline today, and I hadn't measured it.

🔢 ProTip #1: The same text doesn't measure the same on every model. My prompt with the same news item is 1,621 tokens on Haiku 5.5 versus 1,132 on Haiku 4.5: 43% more. Before calculating how much trimming a prompt saves you, measure its tokens on the model where it will run. And don't use the job's originalTokenCount for that: the service counted my template as 600 tokens on all three models, with its own count that isn't the tokenizer you're billed for.

This is the complete result of the two jobs. "Identical" means identical character for character to the original, once the <advpo:exclude> tags are removed, which the service always strips.

H1 is refuted: no model returned a ratio below 1.0. But of the six rows, only the two Nova rows are a clean result.

On Nova 2 Lite the optimizer worked, the model answered almost every inference, and the service found neither a better prompt nor a shorter one that scored as high. For a prompt that was already compact, evaluated with a strict criterion, that's a reasonable answer: there was nothing to cut without losing quality. When it returns the original, the service copies the original's measurement into the "optimized" metrics, which is why both columns match down to the last decimal.

Haiku 4.5 and Haiku 5.5 aren't clean cases, for two different reasons the job result doesn't mention.

Back to the third row. For Haiku 4.5, optimizing for quality only, the job summary reports that the score rose from 0.575 to 0.833, a 45% gain, and that average time to first token dropped from 1.12 s to 0.60 s. The "optimized" prompt is identical to the original.

I went through the results file sample by sample. On the original side, 12 of the 60 inferences came back empty: a blank response, inputTokens: 0, outputTokens: 0, zero latency. The evaluator received an empty string, which isn't valid JSON, and scored it 0. On the "optimized" side none failed. The entire improvement comes from those 12 failed calls.

Bedrock's CloudWatch metrics show what happened: during the job window, Haiku 4.5 logged 607 server errors ( InvocationServerErrors ). Some may be mine: my holdout baseline ran 230 calls against the same model in parallel, and my client retried 5xx errors. But the job's 25 empty inferences belong to the job. The quota wasn't the problem: Haiku 4.5 has 10,000 requests per minute in that account.

There's another detail in that same row. Even excluding the 12 failures, the original scores 0.719 versus 0.833 for the same text in the same job. It's the same noise floor as the baseline, now inside the job: with 60 samples and a model that agrees with itself 88% of the time, two measurements of the same prompt differ by more than 10 points.

🚨 ProTip #2: Don't trust the summary's averageScore. Before accepting an optimized prompt, do two things the service doesn't do for you: compare the prompt text against the original (in my six cases it was identical), and count how many samples in dataset[] have outputTokens: 0. A failed inference enters the average as one more 0.0, and in my case it produced a 45% improvement on a prompt that didn't change. The repo's 05_fetch_results.py script runs both checks.

The Haiku 5.5 job finished as Completed, with both entries at SUCCESS, no errors in the results file, and a +5% improvement on the quality template (0.808 → 0.850) for, once again, a prompt identical to the original. That +5% is the size of how much the same prompt varies against itself: in my baseline, Haiku 5.5 changed the classification of 12 of 145 news items between two runs. On its own it says nothing. The problem was in the optimizer's metrics.

Haiku 5.5 ran 1,925 inferences during the job window. The pricing formula predicts 16 × 60 × 2 = 1,920: the target model did exactly the expected work. But the optimizer, Claude Sonnet 4.6, logged 689 invocations and 684 client errors ( InvocationClientErrors ), with only 7,756 input tokens in total. In the other account, the same optimizer consumed 2.49 million tokens.

I found the cause by invoking Sonnet 4.6 by hand in that account: "Model use case details have not been submitted for this account". It's the same Anthropic form that kept me from using Haiku 4.5 there. The optimizer runs with my credentials, not a service role, and my account couldn't invoke it. The job didn't catch it at creation, and didn't report it in failureMessage or in the result entries. It evaluated the original prompt, couldn't generate candidates, and handed it back to me with SUCCESS.

This kind of failure is hard to spot because the result looks normal. If I hadn't checked CloudWatch to calculate costs, I would have written that Haiku 5.5 found nothing to cut.

🔍 ProTip #3: Advanced Prompt Optimization doesn't use a service role: it invokes the target models, the optimizer, and your Lambda with the credentials of whoever creates the job (Forward Access Session). Before launching a job, verify that your account can invoke Claude Sonnet 4.6 with a test call. At job creation the service validates that you can invoke the target models ( "No inference API is accessible for model" ), but it doesn't validate the optimizer. And after each job, check InvocationClientErrors for us.anthropic.claude-sonnet-4-6 in Bedrock's CloudWatch metrics.

No prompt changed, so there are no measured savings. Even so, the original question can be answered with the jobs' real costs, assuming shortening had achieved the announcement's example: compressionRatio = 0.7, 30% fewer tokens in the template.

According to CloudWatch, the jobs consumed:

Total: $23.61, plus about $1.40 for the holdout validation. The Haiku 4.5 and Nova figures include a few minutes of the job I stopped, so a clean job comes out somewhat cheaper. The optimizer is the most expensive part: about $3.14 per template-model pair.

With that data I calculated the cost of optimizing one template for one model and what it would save per month at my real volume (205 news items in 23 days, about 267 calls a month), with the 30% cut applied to the template's 600 tokens:

H3 is confirmed by a wide margin. The table also shows that shortening pays off more on the more expensive model: Haiku 4.5 costs 10 times more per input token than Haiku 5.5 on Bedrock. On the production model, which is the cheapest, shortening matters less. My table slightly underestimates the Haiku 5.5 savings, because it calculates on the 600 tokens the service counts and not on the ones its tokenizer bills; even correcting for that, the order of magnitude doesn't change.

This doesn't mean shortening is useless; it means the math depends more on volume than on the cut percentage. At about 30,000 calls a month on Haiku 4.5, or about 176,000 on Haiku 5.5, the job pays for itself in a month. At 267, it doesn't.

As for latency, the other benefit the announcement mentions, I have nothing to measure: without a shorter prompt there's no TTFT difference. The improvement the job summary shows (1.12 s → 0.60 s on Haiku 4.5) is two measurements of the same text, taken while the model was returning server errors; I don't attribute it to the prompt.

Beyond the findings above, these are the gotchas that cost me a job or a good while.

🧪 ProTip #4: In modelConfigurations, the additionalModelRequestFields field only accepts topK. I tried passing thinking: disabled and output_config.effort: low for Haiku 5.5, the way I run it in production, and the job was rejected at creation: "contains unsupported keys: [output_config, thinking]. Supported keys: [topK]". The optimizer tunes the prompt for Haiku 5.5 with its default configuration, which uses thinking: on a sample of 12 news items that meant 57% more output tokens and 0.75 s more latency than the production configuration, with the same score. In practice, you're optimizing for a different configuration than the one you'll run.

🛑 ProTip #5: The service reviews your Lambda evaluator's code before using it, and rejects it if it finds anything it considers dangerous. My first job failed with "Metric code validation failed: Uses potentially dangerous builtin: compile()" because of a re.compile(): the analysis doesn't distinguish the re method from the compile builtin. Use re.match(pattern, text) directly. The job doesn't fail at creation either: it fails about three minutes later, with a generic failureMessage that points you to the results file in S3, where the real per-entry reason is.

⏱️ ProTip #6: A job's terminal states are Completed, Failed, Stopped, and PartiallyCompleted. The official sample's wait loop only considers the first three, so with a partial result it waits forever.

I also found some differences between the user guide and the real API:

Two of the four hypotheses went untested. The portability matrix I had ready made no sense with six identical prompts, so I didn't run it.

Back to the opening question: how much does shortening a prompt that already works save me?

In my case, nothing, for two reasons. My prompt was already compact, and the experiment's only clean result returned the original. And even if it had been cut by 30%, at 267 calls a month the savings would be cents per year against a job costing several dollars. Shortening prompts is a volume optimization, and it's worth doing the math before launching the job: tokens saved per call, times calls per month, times price per token, against roughly $3 to $6 per template and model.

What helped me most from the exercise was seeing how the service reports. In one day I got a +45% improvement that came from failed inferences counted as zero, and a SUCCESS from a job whose optimizer never managed to run. Neither case shows up in the job result. I found them by comparing the text of the returned prompt and checking CloudWatch's error metrics.

Something similar happened to me with the Optimization Loop A/B test: the number an optimizer reports has to be verified independently before you make a decision with it.

🚀 Final ProTip: Before optimizing a prompt for cost, calculate the break-even point with your real volume and the price of the model where it runs. If it doesn't pay for itself within a year, there's no point running it for cost; run it only if you have a quality problem to solve. And if you run it, check three things before accepting the result: whether the text changed, how many inferences came back empty, and whether the optimizer logged errors in CloudWatch.

Have you measured how many calls a month your production prompts make? Is your volume at the scale where this math actually matters? If anyone is seeing real savings with shortening, I'd really like to know at what volume — the comments are open.