Building Production-Grade
AI Agents and Agentic
Workflows on AWS

The PoC-to-Production Wall

Almost every team building agentic AI in 2026 is stuck on the same step. The proof of concept works — a single notebook, a few foundation-model calls, a handful of tools, a demo that gets applause in a Friday review. The production version does not work. It does not scale, it does not remember, it cannot be observed, it cannot be governed, and the team that built the PoC is now staring at six weeks of infrastructure work before a single real user ever talks to it.

The gap is not a model gap. Foundation models on Amazon Bedrock — Claude, Nova, Llama, Mistral — are more than capable. The gap is everything that surrounds the model: the runtime that executes the agent loop, the memory that persists context across sessions, the gateway that turns enterprise APIs into agent-callable tools, the observability that lets you debug a four-hour multi-agent run, the identity layer that keeps an agent from doing something the user is not entitled to do.

This article is a working blueprint for how to cross that wall. We will build a real-time customer-support agent end-to-end: a multi-agent crew authored in CrewAI, powered by Anthropic's Claude on Amazon Bedrock, hosted on AgentCore Runtime, given persistent memory through AgentCore Memory, equipped with enterprise tools through AgentCore Gateway, and instrumented for production through AgentCore Observability. The reference architecture is the one shown in the diagram below.

The AgentCore Service Family

AgentCore is best understood as a set of independent services that compose. You can adopt them one at a time. Many teams begin with Runtime alone — host the agent, do nothing else differently — and add Memory and Gateway over the following sprints as the system's needs emerge.

Reference Architecture

The architecture below is the one we deploy for production agent workloads. Each box is an AWS service with a well-defined contract; arrows are real network paths, not abstractions. A user-facing client application — a web SPA hosted on AWS Amplify, authenticated through Amazon Cognito — invokes the agent through a streamable HTTP endpoint. The agent runs on AgentCore Runtime and reaches the rest of the AgentCore family for memory, tools, code execution, and telemetry.

Two design principles drive this layout. First, the agent path and the application path are separate. The browser talks to the agent over a streaming endpoint to AgentCore Runtime; it talks to traditional CRUD APIs through API Gateway and Lambda to DynamoDB. Mixing them — letting the agent be the only path into your backend — is the most common production mistake we see. Don't.

Second, the agent never reaches a backend service directly. It reaches them through AgentCore Gateway, which is the only place tool catalogues, OAuth tokens, and rate limits are configured. This is the trust boundary. When a compliance reviewer asks "what can this agent actually do?", the answer is the Gateway tool list — not "whatever it convinces itself it can do."

Case Study: A Real-Time Customer-Support Crew

Consider a fintech that handles tens of thousands of weekly customer queries about transactions, card limits, and KYC status. The previous system was a rule-based chatbot bolted onto a ticketing tool. We replace it with a multi-agent crew that researches the customer, retrieves transaction context, drafts a response, and escalates to a human when confidence is low. The crew is built with CrewAI, the models are Claude Haiku and Sonnet on Amazon Bedrock, and the entire system runs on AgentCore.

The Crew Composition

Three specialist agents, one supervisor, all defined in CrewAI. Each agent has a narrow role, a small set of tools, and its own system prompt. The supervisor orchestrates the sequence and decides when the response is good enough to send.

Step 1 — Build the Agent Locally with CrewAI and Bedrock

We use the standard CrewAI structure. The model is configured to call Anthropic Claude through Amazon Bedrock, which keeps inference inside the customer's AWS account, removes the need for a separate API key, and inherits IAM-based access control. The crew is wrapped in a BedrockAgentCoreApp with an @app.entrypoint decorator — that single decorator is what makes the same code runnable on localhost:8080 for testing and on AgentCore Runtime in production.

from bedrock_agentcore.runtime import BedrockAgentCoreApp

app = BedrockAgentCoreApp()

# Lazy imports keep AgentCore cold-start under the 30s timeout.
# CrewAI's transitive dependency tree is heavy; never import at module top.

@app.entrypoint
def handler(payload, context):
    from crewai import Agent, Task, Crew, Process
    from crewai.llm import LLM

    bedrock_llm = LLM(
        model="bedrock/anthropic.claude-sonnet-4-20250514-v1:0",
        temperature=0.2,
    )

    profile_agent = Agent(
        role="Customer Profile Specialist",
        goal="Retrieve and summarise the customer's profile and KYC status.",
        backstory="You are precise. You only return verified profile data.",
        tools=[gateway_tool("get_customer_profile")],
        llm=bedrock_llm,
        verbose=True,
    )

    transaction_agent = Agent(
        role="Transaction Analyst",
        goal="Identify the transactions relevant to the customer's question.",
        backstory="You flag anomalies. You never modify records.",
        tools=[gateway_tool("search_transactions"),
               gateway_tool("flag_for_fraud_review")],
        llm=bedrock_llm,
    )

    response_agent = Agent(
        role="Response Author",
        goal="Draft a clear, accurate response or escalate.",
        backstory="You write in the bank's tone. You escalate when uncertain.",
        tools=[gateway_tool("search_policy_kb"),
               gateway_tool("escalate_to_human")],
        llm=bedrock_llm,
    )

    crew = Crew(
        agents=[profile_agent, transaction_agent, response_agent],
        tasks=build_tasks(payload["query"]),
        process=Process.sequential,
        memory=True,  # wire to AgentCore Memory in step 3
    )

    result = crew.kickoff(inputs={"query": payload["query"],
                                  "customer_id": payload["customer_id"]})
    return {"response": str(result), "session_id": context.session_id}

if __name__ == "__main__":
    app.run()  # local: localhost:8080 — same code in prod on AgentCore Runtime

Step 2 — Expose Backend Services as Tools through AgentCore Gateway

The agents above call gateway_tool("get_customer_profile") rather than invoking a Lambda or hitting an internal REST endpoint directly. AgentCore Gateway is the layer that makes this work. We register an OpenAPI spec or a Lambda ARN with the Gateway; it produces an MCP-compatible tool that any agent framework can call, with authentication, rate limits, and per-tool access policies handled centrally.

import boto3

agentcore = boto3.client("bedrock-agentcore-control")

gateway = agentcore.create_gateway(
    name="support-tools",
    protocol="MCP",
    authorizer={
        "type": "COGNITO",
        "userPoolArn": COGNITO_POOL_ARN,
    },
)

agentcore.create_gateway_target(
    gatewayId=gateway["gatewayId"],
    name="get_customer_profile",
    targetConfiguration={
        "lambda": {
            "functionArn": GET_PROFILE_LAMBDA_ARN,
            "toolSchema": {
                "description": "Returns customer profile and KYC status.",
                "inputSchema": {"customer_id": {"type": "string"}},
            },
        }
    },
)

From the agent's perspective, the tool now exists with a clean schema — no Lambda, no IAM policy, no SDK detail. From the platform's perspective, every call through the tool is logged, traced, and rate-limited at the Gateway. This is the single-most-leveraged piece of the architecture: it is the difference between an agent that could reach any service and an agent that can reach exactly the services on its tool list.

Step 3 — Add Persistent Context with AgentCore Memory

Without memory, every customer conversation begins from zero. With AgentCore Memory, the crew remembers that this customer's last three contacts were about a disputed transaction, that they prefer formal language, and that they have a Premier-tier account. We create a memory resource with two strategies — Semantic for facts, UserPreferences for tone and channel preferences — and the crew writes events to it on every turn.

import boto3
from bedrock_agentcore_starter_toolkit.memory import MemoryClient

mc = MemoryClient()

memory = mc.create_memory_and_wait(
    name="support-crew-memory",
    description="Long-term memory for the customer-support crew.",
    event_expiry_days=90,
    strategies=[
        {"type": "SEMANTIC",
         "name": "customer_facts",
         "namespace": "/customers/{customerId}"},
        {"type": "USER_PREFERENCES",
         "name": "customer_prefs",
         "namespace": "/customers/{customerId}/prefs"},
    ],
)

# In the crew handler — write the conversation event after every turn:
mc.create_event(
    memoryId=memory["id"],
    actorId=f"customer-{customer_id}",
    sessionId=context.session_id,
    messages=[
        {"role": "USER", "content": query},
        {"role": "ASSISTANT", "content": response},
    ],
)

# Before answering — retrieve the most relevant long-term memories:
recall = mc.retrieve_memory_records(
    memoryId=memory["id"],
    namespace=f"/customers/{customer_id}",
    searchCriteria={"searchQuery": query, "topK": 5},
)

Long-term extraction runs asynchronously in the background after the event is written. This is a deliberate design choice — it keeps the live request path fast — but it means your application has to tolerate a lag between writing an event and the resulting memory record being retrievable. We use short-term memory for in-session continuity and lean on long-term memory only across sessions.

Step 4 — Deploy to AgentCore Runtime

Deployment is one command. The AgentCore CLI containerises the agent, registers it as an AgentCore Runtime, and returns an endpoint ARN. There is no Dockerfile to write, no autoscaling group to size, no load balancer to provision. Each user session lands on its own microVM, gets its session ID and the eight-hour execution budget, and is sanitized at session end.

# Configure once per project
agentcore configure --entrypoint support_crew.py --name support-crew

# Test locally — same code path as production
agentcore invoke --local '{"query":"why was my card declined?","customer_id":"c-1842"}'

# Deploy to AWS
agentcore deploy

# Invoke the production endpoint
agentcore invoke '{"query":"why was my card declined?","customer_id":"c-1842"}'

# Tail logs in real time
aws logs tail /aws/bedrock-agentcore/runtimes/support-crew-DEFAULT --follow

Step 5 — Turn on Observability

Observability for AgentCore is OpenTelemetry-native through the AWS Distro (ADOT). Once Transaction Search is enabled in CloudWatch and the standard ADOT environment variables are set, the agent emits a three-tier hierarchy of sessions → traces → spans into CloudWatch and X-Ray. The CloudWatch GenAI Observability dashboard then surfaces per-session traces, token usage, latency percentiles, tool-call breakdowns, and error patterns.

AGENT_OBSERVABILITY_ENABLED=true
OTEL_PYTHON_DISTRO=aws_distro
OTEL_PYTHON_CONFIGURATOR=aws_configurator
OTEL_RESOURCE_ATTRIBUTES=service.name=support-crew,\
  aws.log.group.names=/aws/bedrock-agentcore/runtimes/support-crew-DEFAULT
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_TRACES_EXPORTER=otlp

For frameworks AgentCore instruments natively — CrewAI, LangGraph, Strands, LangChain — that is the entirety of the wiring. For others, you add an OpenInference instrumentation library to emit GenAI semantic-convention spans. Either way, every span lands in the same CloudWatch log group, every trace is reachable from X-Ray, and every session is queryable on the GenAI Observability page. When a customer reports that "the agent gave me the wrong answer at 14:32", you can pull the exact session in CloudWatch, see every tool call the agent made, the prompts it sent, the tokens it consumed, and the model's full reasoning trace.

Why CrewAI for Multi-Agent Orchestration

CrewAI's mental model is unusually clear: agents have roles, agents have goals, agents have backstories, and they execute tasks under a process (sequential or hierarchical). That vocabulary maps cleanly onto how human teams actually work, which makes the resulting code reviewable by people who are not LLM specialists.

Two CrewAI features matter most in production. Hierarchical process lets a manager agent dynamically delegate work to specialists rather than running them in a fixed sequence — useful when the right plan depends on the customer's question. And Flows add an event-driven layer above Crews, allowing conditional logic, loops, and state across multiple crew invocations — useful when the workflow is not a single conversation but a multi-stage pipeline.

Communication Between Agents

Agents in a CrewAI crew communicate through the shared crew context: the output of one task becomes input to the next, and a manager agent can ask any specialist for clarification mid-run. With AgentCore Memory wired in, that communication also persists across sessions — agent A from yesterday's session can leave a note that agent B picks up tomorrow, scoped to the same customer namespace.

PoC Stack vs. Production AgentCore Stack

The contrast between a notebook prototype and the production architecture above is sharp on every axis a platform team cares about:

What We Learned Putting This Into Production

The honest version of any production-AI write-up is the list of things that did not work the first time. Ours fall into a few clear categories.

Non-Negotiable Production Guardrails

These are the controls without which we would not run an AgentCore stack in front of real customers:

• All agent tool calls go through Gateway. No direct Lambda invocation, no in-agent SDK calls, no embedded API keys. Gateway is the audit boundary and the rate-limit boundary. There is no second path.
• Identity is JIT-scoped. AgentCore Identity issues the agent only the OAuth scopes the current task requires. The Profile agent cannot move money. The Transaction agent cannot edit the profile.
• Every response carries its session ID. Returned to the client, logged on the server, written to the ticketing system. When something goes wrong, finding the trace is one query, not a forensic exercise.
• Confidence-gated escalation. The Response agent self-scores; below a tuned threshold, the response routes to a human queue with the full reasoning trace attached. We never let the agent send under low confidence.
• Memory has TTLs. Short-term memory defaults to 30 days, long-term to 90 in our deployment. Customer data does not accumulate forever in a memory store just because the service supports it.
• Per-tool rate limits and circuit breakers at the Gateway, not in the agent. An agent in a retry loop should fail fast and loud, not melt a downstream service.

What This Pattern Generalises To

Customer support is a particularly visible use case, but the same architecture fits a much wider class of problems: SRE assistants that triage incidents across Kubernetes, monitoring, and runbooks; coding agents that maintain context across an engineer's full week of work; field-operations agents that combine maps, schedules, and parts inventories; financial-research agents that compose proprietary data with public filings. Anything where the work is multi-step, multi-tool, and benefits from memory across sessions fits the same shape.

The shared blueprint is consistent. Author the agents in the framework that fits the team's mental model — CrewAI for role-based crews, LangGraph for explicit state machines, Strands for AWS-native development. Power them with the right Bedrock foundation model for the cost/quality trade-off the workload demands. Host them on AgentCore Runtime so the infrastructure stops being a distraction. Reach the rest of the enterprise through Gateway, remember through Memory, prove correctness through Observability, and bound permissions through Identity. The pieces are independent; the discipline is treating them as a system from day one.