AWS AgentCore in Action: From Prototype to Production Insights

I was at the Unstructured Data Meetup at Amazon’s LHR16 office at AWS Summit London after-party hosted by Zilliz. Ying Hou’s session on Amazon Bedrock AgentCore is the one I want to write about.

I’ve been building agentic systems for a while now at FlexyWork AI, mostly on LangGraph and Strands, with the usual supporting cast of LangSmith for observability, FastAPI for serving, and a lot of custom plumbing for memory and tool routing. So I came in skeptical. Every cloud provider has shipped some flavour of “agent platform” in the last 18 months, and most of them are either too opinionated (you have to use their model, their framework, their orchestration) or too thin to actually save you time.

AgentCore is genuinely different. Not because it does anything magical, but because of what it refuses to do, it refuses to lock you in. And the problems it tackles are the exact ones I’ve been hand-rolling solutions for.

Here’s what stuck with me.

Ying opened with a framing that anyone who’s shipped an agent will recognise immediately: open-source frameworks (CrewAI, LangGraph, LlamaIndex, OpenAI Agents SDK, Strands) make POCs trivial. You can have a working multi-agent system over coffee. The hard part the part nobody talks about at hackathons is everything between that demo and an agent that 10,000 real users hit in production.

Performance. Scalability. Security. Governance. Observability. Identity. Memory that survives a restart. Tool access that doesn’t expose half your AWS account. Multi-tenancy without users leaking into each other’s sessions.

I’ve felt every one of these. The first time you realise your “stateless” agent actually has to remember things across a week-long conversation, or that your tool calls are inheriting your service account’s IAM permissions, you understand why “from POC to prod” is a real engineering problem and not a marketing slogan.

AgentCore isn’t a framework. It’s nine modular services that you can adopt independently:

Runtime — serverless hosting for agents written in any framework. Strands, LangGraph, CrewAI, Google ADK, LlamaIndex, OpenAI Agents SDK. It supports MCP and A2A as protocols, and OTEL for telemetry. The thing that caught my attention: each session gets its own dedicated microVM. Compute, memory, files fully isolated, deleted after the session ends. If you’ve ever worried about one user’s PII bleeding into another user’s context window through shared state, this is the answer.

Memory -short-term and long-term, with automatic extraction of summaries, preferences, facts, and episodes. You can use the built-in extraction strategies or roll your own pipelines. It plugs straight into LangGraph and Strands.

Identity — both inbound (who is the user, can they use this agent?) and outbound (can this agent’s identity access the downstream tool?). Standard OAuth flows, API keys, IAM Sigv4. This is the part I’ve been hand-rolling for months.

Gateway — turns your existing REST APIs, Lambda functions, MCP servers, and Smithy models into MCP tools the agent can call. Built-in semantic search across your tool catalogue, which is the bit I find genuinely clever.

Code Interpreter and Browser — sandboxed Python execution and headless browser automation. Useful for the “analyse this CSV” or “go book this on the supplier portal” class of tasks.

Observability, Policy, and Evaluations — exactly what they sound like. The Policy service is in preview and uses Cedar (AWS’s automated reasoning policy language) for things like “this agent can read customer records but only for the customer ID in the active session.”

Two services stood out for me, both because they solve problems I’ve actively struggled with.

Gateway with semantic search over tools. If you’ve ever wired up an agent with 200+ tools, you know the pain the model’s context window gets eaten alive by tool descriptions before it ever sees the user’s question, and accuracy collapses. Ying showed a diagram where the agent calls search(“prepare for my meeting”) and gets back the 4 most relevant tools out of 360+ across multiple backends. Tools are automatically indexed and the search runs serverless. Cheaper tokens, faster responses, and this is the part I underestimated the agent can discover new tools on the fly, which is huge for autonomous workflows where you don’t know upfront what the agent will need.

Session isolation in Runtime. The microVM-per-session model. Sounds like overkill until you remember Firecracker boots in ~125ms and AWS already runs Lambda on it. For agents handling regulated data finance, healthcare, anything with PII this is the kind of guarantee you currently have to design an entire architecture around. Getting it as a primitive changes the maths on what you can ship.

I’ll also call out Identity as quietly important. Outbound auth the agent calling Google Drive, Slack, Notion as the user, not as a service account is one of those things that sounds simple until you try to do it cleanly across half a dozen OAuth providers with refresh tokens and scoped consent. Having that as a managed primitive removes a whole category of bugs.

A few honest reservations.

The “any framework” pitch is real but it’s still framework-as-payload you bring your LangGraph or Strands agent and AgentCore runs it. That’s the right design, but it means your agent code still has all the same complexity. AgentCore solves the operational surface, not the authoring surface. Strands is AWS’s own answer to the authoring side, and Ying mentioned it a few times, but it’s not mandatory.

Pricing wasn’t covered in the talk, and that matters. Serverless per-session microVMs sound great until you see the per-invocation cost compared to running a Fargate task. I’d want to model this against my current LangGraph-on-ECS setup before committing.

And the lock-in concern doesn’t fully go away. Yes, you can bring any framework. But once your Memory, Identity, Gateway, Observability, and Policy are all on AgentCore, you’ve stitched yourself into the AWS-shaped hole pretty thoroughly. That’s a fair trade if you’re already on AWS. If you’re multi-cloud or self-hosted, the calculus is different.

The four points Ying closed on are worth keeping:

The first one is the one I keep coming back to. POCs are cheap. Production is where the real engineering and the real value lives. Most of the agentic AI hype cycle right now is stuck at point one because building the rest of the stack is genuinely hard work.

AgentCore is the first managed offering I’ve seen that treats the production-readiness problem as a peer to the authoring problem, instead of an afterthought. Whether it’s the right choice for any given team depends on cloud strategy, cost model, and how much of the stack you’ve already built but if you’re early in your agent journey and you’re already on AWS, it’s a serious accelerator.

Sincere thanks to Ying Hou, Sr. Specialist Solutions Architect, GenAI at AWS, for an excellent session and for making the slides available afterwards. It was a well-structured walkthrough of a complex platform, and the materials will serve as a valuable reference for anyone working on production agentic systems on AWS.

Event Link — Unstructured Data Meetup London (AWS After Party) · Luma