15 Checks fuer den Umzug von AI-Agents zu Amazon Bedrock und AgentCore

Posted on Oct 22 • Originally published at Medium • AI-assisted

A readiness assessment for AI teams moving agents from a prototype or another provider to Amazon Bedrock and AgentCore. Score yourself out of 30, with the evidence, AWS service, and command for each check.

By Evgeny Anikiev, AWS-certified DevOps engineer with 10+ years running build automation, CI/CD, and Kubernetes in production. AWS service details verified October 22, 2026.

TL;DR: Score 15 checks from 0 to 2. They cover cost per task, evaluations, model access, quotas, AgentCore Runtime, tools, identity, data, observability, and rollback. A total of 24 or more means you can move production traffic. From 15 to 23, fix the zeros and run a shadow or canary pilot. Below 15, keep the agent where it is and build the foundations first.

Three changes from the past year that older checklists miss:

An AI migration readiness assessment is a review, done before production traffic moves, of whether an AI team can run its agents on a new platform safely and within budget. For AWS, that means checking cost per task, evaluation baselines, model governance, Bedrock quotas, AgentCore hosting, tool access, identity, data handling, monitoring, and rollback on Amazon Bedrock and Amazon Bedrock AgentCore.

It follows the six perspectives of the AWS Cloud Adoption Framework (Business, People, Governance, Platform, Security, Operations), applied to agents instead of servers. Typical starting points are an agent calling the OpenAI or Anthropic API directly, a LangGraph or CrewAI service in a container, or a notebook prototype that leadership now wants in front of customers.

One rule overrides the total: a 0 on check 2 (evaluation baseline), check 11 (identity and tool policy), or check 15 (rollback) blocks production at any score. Without a baseline you can't tell whether the move made answers worse. Without tool policy an agent can call anything its credentials allow. Without rollback a bad model or prompt change stays live until someone fixes it by hand.

check, perspective, score, evidence, owner 01 cost per task, business,,, 02 evaluation baseline, business,,, 03 owners and on - call, people,,, 04 framework and AgentCore skills, people,,, 05 model access controls, governance,,, 06 cost attribution, governance,,, 07 quotas and inference routing, platform,,, 08 runtime hosting, platform,,, 09 tools behind Gateway, platform,,, 10 memory and RAG data, platform,,, 11 identity and tool policy, security,,, 12 data protection and network, security,,, 13 observability and logging, operations,,, 14 online evaluations, operations,,, 15 rollout and rollback, operations,,, Enter fullscreen mode Exit fullscreen mode Business 1. Do you know the cost per task?

Agents bill per token, and one user request often makes several model calls. Measure input and output tokens per task on your current setup, then price them on Bedrock. Score 2 if finance has seen a cost per task and a monthly estimate at expected volume.

def cost_per_task ( calls, in_tokens, out_tokens, in_price, out_price, cached = 0, cache_discount = 0.9 ): """ Prices are USD per million tokens. cached = input tokens per call served from the prompt cache. """ uncached = in_tokens - cached per_call = ( uncached * in_price + cached * in_price * ( 1 - cache_discount ) + out_tokens * out_price ) / 1_000_000 return calls * per_call # Example prices, not a quote: $3 input and $15 output per million tokens. print ( round ( cost_per_task ( 6, 8000, 1000, 3, 15 ), 3 )) # 0.234 print ( round ( cost_per_task ( 6, 8000, 1000, 3, 15, cached = 6000 ), 3 )) # 0.137 Enter fullscreen mode Exit fullscreen mode The numbers that move this figure:

2. Do you have an evaluation baseline from the current setup?

Build a golden set of real requests with expected answers or grading rules, and score your current agent on it before you change anything. Run the same set on Bedrock after the move. Score 2 if the set has at least a few hundred cases, covers your main task types, and has a pass threshold the product owner agreed to. A 0 here blocks production.

Someone owns the prompts, someone owns the tools and their APIs, and someone gets paged when the agent fails at 2 a.m. Score 2 if all three are named and the on-call runbook covers model errors, throttling, and tool failures.

4. Can the team build and run on AgentCore?

AgentCore Runtime is framework-agnostic. AWS documents agents built with Strands Agents, LangGraph, and CrewAI running on it, so most teams keep their code. Score 2 if at least two engineers have deployed a test agent to AgentCore Runtime and called it with real credentials.

5. Who controls which models can be called?

Since October 2025, serverless Bedrock models are enabled by default. Anthropic models still need a one-time use-case form, which covers member accounts when submitted from the management account. Restrict models with a service control policy. Score 2 if an allow-list exists and is enforced at the organization level.

{ "Version": "2012-10-17", "Statement": [ { "Sid": "DenyUnapprovedBedrockModels", "Effect": "Deny", "Action": [ "bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream" ], "NotResource": [ "arn:aws:bedrock:*::foundation-model/anthropic.*", "arn:aws:bedrock:*::foundation-model/amazon.nova-*", "arn:aws:bedrock:*:*:inference-profile/us.anthropic.*", "arn:aws:bedrock:*:*:inference-profile/us.amazon.nova-*" ] } ] } Enter fullscreen mode Exit fullscreen mode Replace the model families with your approved list. The Converse API uses the same bedrock:InvokeModel permission, so this policy covers it too.

6. Can you split the Bedrock bill by team and agent?

Create an application inference profile per agent or team, tag it, and activate the tags as cost allocation tags. Calls made through the profile then show up per tag in Cost Explorer. Score 2 if every production agent calls models through a tagged profile.

7. Are quotas and inference routing sized?

Bedrock quotas are tokens per minute and requests per minute, set per model and per Region. Cross-Region inference spreads load across Regions in one geography (geo) or worldwide (global), and not every model supports global. Score 2 if you have compared peak tokens per minute against your quotas in Service Quotas, requested increases in advance, and chosen geo or global routing to match your data residency rules. AWS gives priority on increases to accounts that already use their current quota, so build up real traffic before asking.

AgentCore Runtime runs each session in its own microVM with separate compute, memory, and filesystem, and wipes it when the session ends. A session ends after 15 minutes of inactivity by default and lives up to 8 hours. For longer jobs, AgentCore Runtime instances run up to 14 days. Score 2 if your longest task fits these limits and you have load-tested cold starts at expected concurrency.

9. Are tools exposed through AgentCore Gateway?

Gateway turns existing APIs and Lambda functions into MCP tools that agents can discover and call. It is also where AgentCore Policy checks each call. Score 2 if every tool the agent can call goes through Gateway, and none are hard-coded HTTP calls inside the agent.

Decide what goes into AgentCore Memory (conversation history and long-term facts per user) and what goes into a Bedrock Knowledge Base (documents for retrieval). For each, write down the data source, refresh schedule, retention period, and who may read it. Score 2 if the retrieval quality on your golden set matches or beats the current setup.

11. Are identity and tool policy in place?

Use AgentCore Identity for inbound authentication (who may call the agent) and for outbound credentials to tools, so no API keys live in agent code or environment variables. Add AgentCore Policy rules on the Gateway, for example limiting a refund tool to amounts below a threshold. Score 0 if the agent holds long-lived keys or can call tools without a policy check. A 0 here blocks production.

12. Are data protection and network paths decided?

AWS states that Bedrock does not use your prompts and completions to train AWS models or share them with model providers. Your security team will still ask about three more things:

Score 2 if security has signed off on all three in writing.

AgentCore Observability records traces of each step: model calls, tool calls, and latency. Bedrock model invocation logging is disabled by default. Turn it on for the audit trail, and treat the logs as sensitive, because they contain full prompts and responses.

aws bedrock put-model-invocation-logging-configuration \ --logging-config '{ "cloudWatchConfig": { "logGroupName": "/bedrock/model-invocations", "roleArn": "arn:aws:iam::111122223333:role/BedrockInvocationLogging" }, "textDataDeliveryEnabled": true }' Enter fullscreen mode Exit fullscreen mode Score 2 if traces, invocation logs, and alarms on errors, throttling, and latency exist in the production account.

14. Are evaluations running on live traffic?

AgentCore Evaluations samples live traces and scores them with built-in evaluators for response quality, safety, task completion, and tool use, or with your own. Score 2 if online evaluation runs on production traffic with an alarm when scores drop below the threshold from check 2.

15. Can you roll back a model, prompt, or agent change?

Pin model versions instead of relying on whatever is newest. Deploy agent changes as new AgentCore Runtime versions behind an endpoint, so you can move traffic back. Track Bedrock model end-of-life dates. Score 2 only if you rehearsed a rollback in a non-production account. A 0 here blocks production.

This example is hypothetical, built to show the scoring. A team runs a LangGraph customer support agent in a container, calling a model provider's API directly with a key in an environment variable. It handles about 10,000 tasks a month. Its first scoring pass:

Total: 7 out of 30, with blockers on checks 2, 11, and 15. A sensible next four to six weeks:

With the example prices from check 1, caching 6,000 of 8,000 input tokens per call would take the cost per task from about $0.23 to about $0.14, or from about $2,340 to about $1,370 a month at 10,000 tasks. Confirm with your model's real cache pricing.

Do I need AgentCore to use Amazon Bedrock?

No. You can call Bedrock models from any application with the Converse or InvokeModel APIs. AgentCore adds hosting, memory, tool gateway, identity, policy, observability, and evaluations for agents, and works with models from Bedrock or other providers.

Can I run LangGraph or CrewAI agents on AgentCore?

Yes. AgentCore Runtime is framework-agnostic, and AWS documents agents built with Strands Agents, LangGraph, and CrewAI running on it.

Does Amazon Bedrock train on my prompts?

AWS states that Bedrock does not use your prompts and completions to train AWS models and does not share them with third-party model providers.

Do I still need to request model access in Bedrock?

Not for most models. Since October 2025, serverless models are enabled by default in commercial Regions. Anthropic models need a one-time use-case form. Some AWS Marketplace models need a subscription, which Bedrock tries to create on first use.

How long can an AgentCore Runtime session run?

Up to 8 hours per session on microVMs, with a default idle timeout of 15 minutes. AgentCore Runtime instances support work up to 14 days.

Request increases in Service Quotas per model and Region, and use cross-Region inference profiles to spread load. AWS gives priority to accounts that already use their existing quota.

What is the difference between AgentCore Policy and Bedrock Guardrails?

Policy controls which tool calls an agent may make, with Cedar rules checked at AgentCore Gateway. Guardrails filter model inputs and outputs, for example blocking topics or redacting PII.

Moving an agent to Bedrock and AgentCore? I run readiness assessments and build the AWS foundations for AI teams at DevOps Cloud Consult: landing zone, model access controls, observability, and cost tracking. If you scored under 24, that is the work I can help with.