How to Connect a Bedrock Managed Knowledge Base to LangGraph over MCP

A step-by-step guide to grounding a LangGraph agent in Amazon Bedrock's agentic retriever through AgentCore Gateway — without building an MCP server.

Jubin Soni

AI Engineering

Published Aug 18, 2026

Last modified Aug 18, 2026

Why this integration is worth doing?

If you build agents on LangGraph and your enterprise content lives in AWS, you have probably written the same code twice: a chunker, an embedding job, a vector store, a retriever, a reranker, and a permissions filter bolted on at the end. Every new agent gets its own copy. Every copy drifts.

Amazon Bedrock Managed Knowledge Base, launched in June 2026, moves that work behind a managed service. AWS operates the vector store, the embedding and re-ranking models, and the scaling concerns, and the retrieval layer is an agentic retriever that plans and executes multi-step queries rather than matching a single query against the nearest chunks.

The part that matters for anyone outside the Strands ecosystem: you can attach that knowledge base to an AgentCore Gateway as a pre-built target type, and the gateway exposes it over MCP. Any MCP-compatible client discovers the tools with tools/list and calls them with tools/call — LangGraph included, with no custom integration code.

That is the whole integration. The interesting parts are the four places it does not behave like a normal retriever, which this tutorial covers in detail:

The gateway exposes two tools with very different contracts, and picking the wrong one shapes your whole graph.
Authentication is SigV4, not a bearer token, which breaks the standard MCP client setup.
Per-user document filtering silently no-ops unless you pass userContext yourself.
The knowledge base is itself a planner, so you have two planners per turn and need to decide who does what.

By the end you will have a working LangGraph agent grounded in a managed knowledge base, with citations preserved in graph state and per-user access control that actually filters.

What you should already know: LangGraph basics (StateGraph, ToolNode, the ReAct loop), and enough AWS to create a role and attach a policy.

Architecture

Figure 1 — The knowledge base owns retrieval. LangGraph owns orchestration. AgentCore Gateway is the MCP contract between them.

Three things are worth noticing before you write any code.

The gateway hides the knowledge base. Knowledge base IDs sit behind the gateway, so agents discover and call tools by name and your agent code is decoupled from the underlying infrastructure. One gateway URL becomes the access point across the organization, and IAM policy at the gateway replaces per-knowledge-base permission management.

The gateway is an active intermediary, not a proxy. It handles authentication, parameter injection, and schema subsetting. As the administrator, you bind the knowledge base and decide which parameters the calling agent may set and which are hidden behind preconfigured defaults. This is genuinely useful — you can pin numberOfResults or a guardrail and stop an agent from overriding it — but it means the tool schema your agent sees is your schema, not the raw API.

Identity flows in two directions, with different mechanisms. Inbound auth (who may call your gateway) is OAuth/JWT, typically Cognito. Outbound auth (what the gateway may call on your behalf) is an IAM execution role. And a third identity — the end user whose documents get filtered — is neither of those. Conflating the three is the most common source of "why is everyone seeing everything" bugs.

Prerequisites

A managed knowledge base and its knowledge base ID. Note that gateway targets only work with managed knowledge bases — a self-managed vector store knowledge base cannot be attached this way.
An AgentCore Gateway.
Python 3.10+.

Install the client libraries:

1
pip install mcp-proxy-for-aws langchain-mcp-adapters langgraph langchain-aws boto3

The IAM detail that will bite you first

The gateway execution role is a role you create and pass to CreateGateway, which AgentCore assumes to call Bedrock on your behalf. It is not a service-linked role. For the managed knowledge base connector it needs three permissions, and one of them cannot be scoped the way you would expect:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ValidateAndRetrieve",
      "Effect": "Allow",
      "Action": ["bedrock:GetKnowledgeBase", "bedrock:Retrieve"],
      "Resource": "arn:aws:bedrock:us-west-2:123456789012:knowledge-base/KB12345678"
    },
    {
      "Sid": "AgenticRetrievalNotResourceScoped",
      "Effect": "Allow",
      "Action": "bedrock:AgenticRetrieveStream",
      "Resource": "*"
    }
  ]
}

bedrock:GetKnowledgeBase validates the bound knowledge base when the target is created, and bedrock:Retrieve backs the Retrieve tool — both scope cleanly to the knowledge base resource. But bedrock:AgenticRetrieveStream is not scoped to a knowledge base resource, so it has to be granted on *. If your organization runs a "no wildcard resources" policy check, this will fail review, and the fix is a documented exception rather than a tighter ARN. Find that out now rather than in a security review two weeks from now.

Include bedrock:GetKnowledgeBase regardless of which tools you add. If you only add one retrieval tool, include only that tool's action.

Step 1 — Understand the two tools before you choose one

Adding a knowledge base target exposes two tools, and they are not two flavours of the same thing. They have different response shapes, different governance hooks, and different observability.

	Retrieve	AgenticRetrieveStream
What it does	A single search returning the most relevant passages with source references	Multi-step retrieval that plans, retrieves iteratively, and streams results
Answer synthesis	No — passages only	Optional synthesized, citation-backed answer
Response mode	Single response	Streamed events
Latency	Low, predictable	Higher, varies with iteration count
Guardrails hook	Accepts a guardrail configuration on the request	Accepts policyConfiguration with a Bedrock Guardrail
Guardrail actions	Standard	BLOCK only — MASK is not supported on this path
X-Ray tracing	Supported (logType: TRACES, destination XRAY)	Not available
IAM action	bedrock:Retrieve, scopeable to the KB ARN	bedrock:AgenticRetrieveStream, must be granted on *
Best for	Tight loops, evaluation harnesses, latency budgets, anywhere you need traces	Hard multi-hop questions where one search genuinely isn't enough

The two asymmetries worth internalizing: if you standardize on AgenticRetrieveStream you lose X-Ray traces for your retrieval path, and if you were counting on MASK to redact PII rather than block the response, that option doesn't exist there.

My recommendation for a first build: expose both, and let the graph route. Cheap lookups take Retrieve; questions your router flags as multi-hop take AgenticRetrieveStream. That turns a latency/quality trade-off into a graph edge, and it keeps the traced path available for the majority of traffic.

Step 2 — Add the knowledge base as a gateway target

You bind the knowledge base to the gateway and choose which parameters the agent is allowed to set. Everything you don't expose gets a preconfigured default that the agent cannot override.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
aws bedrock-agentcore-control create-gateway-target \
  --gateway-identifier "$GATEWAY_ID" \
  --name "product-docs-kb" \
  --credential-provider-configurations '[{
      "credentialProviderType": "GATEWAY_IAM_ROLE"
  }]' \
  --target-configuration '{
      "mcp": {
        "connector": {
          "connectorType": "BEDROCK_MANAGED_KNOWLEDGE_BASE",
          "knowledgeBaseId": "KB12345678"
        }
      }
  }'

Only GATEWAY_IAM_ROLE is supported for outbound auth on this connector — there is no OAuth path to the knowledge base.

Once the target exists, confirm what your agent will actually see:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# Sanity-check the tool surface before writing any graph code.
import asyncio
from mcp_proxy_for_aws.client import aws_iam_streamablehttp_client
from mcp import ClientSession

GATEWAY_URL = "https://your-gateway-id.gateway.bedrock-agentcore.us-west-2.amazonaws.com/mcp"

async def list_tools():
    async with aws_iam_streamablehttp_client(
        endpoint=GATEWAY_URL,
        aws_region="us-west-2",
        aws_service="bedrock-agentcore",
    ) as (read, write, _):
        async with ClientSession(read, write) as session:
            await session.initialize()
            tools = await session.list_tools()
            for t in tools.tools:
                print(t.name)
                print("  ", sorted(t.inputSchema.get("properties", {}).keys()))

asyncio.run(list_tools())

Read that parameter list carefully. It is the schema after subsetting, so it tells you exactly which knobs you left the agent — and whether userContext is among them, which matters a great deal in Step 5.

Step 3 — Authenticate with SigV4, not a bearer token

This is where the standard MCP client setup breaks. AgentCore Gateway expects SigV4-signed requests, so streamablehttp_client from the mcp package will not work. You swap in aws_iam_streamablehttp_client from mcp-proxy-for-aws:

1
2
3
4
5
6
7
8
9
10
11
12
13
from langchain_mcp_adapters.client import MultiServerMCPClient
from mcp_proxy_for_aws.client import aws_iam_streamablehttp_client

GATEWAY_URL = "https://your-gateway-id.gateway.bedrock-agentcore.us-west-2.amazonaws.com/mcp"
REGION = "us-west-2"

def kb_transport():
    """SigV4-signed streamable HTTP transport for AgentCore Gateway."""
    return aws_iam_streamablehttp_client(
        endpoint=GATEWAY_URL,
        aws_region=REGION,
        aws_service="bedrock-agentcore",   # the signing service name
    )

Three things to get right, all of which produce the same unhelpful 403 when wrong:

aws_service must be bedrock-agentcore, not bedrock.
aws_region must match the gateway's region, not your default profile region.
The credentials in the ambient chain must be the caller's, distinct from the gateway execution role. The role in Step 1's policy is what the gateway uses downstream; it is not what signs your request.

There is a real advantage over bearer-token gateways here, worth noting since it saves you code: SigV4 signs per request from the ambient credential chain, so there is no token to cache and no expiry to handle. On AgentCore Runtime the task role supplies credentials automatically and they rotate underneath you. The equivalent Azure integration needs a custom auth class purely to refresh tokens; here you get that for free.

Step 4 — Load the tools into LangGraph

With signing sorted, wiring the tools in is short.

1
2
3
4
5
6
7
8
9
10
client = MultiServerMCPClient({
    "bedrock_kb": {
        "transport": "http",
        "url": GATEWAY_URL,
        "httpx_client_factory": kb_transport,
    }
})

tools = await client.get_tools()
print([t.name for t in tools])   # ['Retrieve', 'AgenticRetrieveStream']

For a quick check, hand them to a prebuilt agent:

1
2
3
4
5
6
7
8
9
from langgraph.prebuilt import create_react_agent

agent = create_react_agent("bedrock:us.anthropic.claude-sonnet-4-5", tools)

response = await agent.ainvoke({
    "messages": [{"role": "user",
                  "content": "What changed about ingestion rate limits last release?"}]
})
print(response["messages"][-1].content)

If that returns a grounded answer, the integration works. One operational note before the real graph: MultiServerMCPClient is stateless by default — each tool invocation opens a fresh MCP session and tears it down. For knowledge base retrieval that is correct and cheaper than holding a session open.

Step 5 — Fix per-user filtering before you ship

This is the single most important section in this tutorial, because the failure mode is silent.

Managed Knowledge Bases support document-level access control through ACLs. At retrieval time you pass the user's identity in userContext, and only documents that user may read come back. For an S3 data source, you enable this at ingestion by placing a global ACL configuration file in S3 and setting aclEnabled: true with the ACL file URI when you create the data source.

Here is the trap. When the knowledge base is exposed through AgentCore Gateway, the IAM identity of the caller is not passed along as the document-level user. Your application must send userContext explicitly. If it doesn't, the retrieval layer does not error — it simply skips the per-user filtering you were counting on and returns everything the knowledge base can see.

Read that again if you are building anything multi-tenant. A gateway call with no userContext looks identical to a correctly-filtered call, right up until an auditor asks why a contractor's agent surfaced an HR document.

So: confirm userContext survived schema subsetting in Step 2, then thread the caller's identity through per request.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import contextvars
from langchain_core.tools import BaseTool

current_user = contextvars.ContextVar("current_user", default=None)

def bind_user_context(tools: list[BaseTool]) -> list[BaseTool]:
    """Inject the caller's identity into every retrieval call.

    The gateway will not do this for you: without userContext the
    knowledge base returns unfiltered results rather than failing.
    """
    for tool in tools:
        original = tool.coroutine

        async def wrapped(*args, __orig=original, **kwargs):
            user = current_user.get()
            if user is None:
                raise PermissionError(
                    "refusing to retrieve without a resolved user identity")
            kwargs.setdefault("userContext", {"userId": user})
            return await __orig(*args, **kwargs)

        tool.coroutine = wrapped
    return tools

tools = bind_user_context(await client.get_tools())

Two deliberate choices there. The wrapper raises rather than defaulting to anonymous, because a loud failure in development is far better than silent over-retrieval in production. And it sets the value at the edge via a context variable, so you populate current_user once in the handler that already validated the caller's JWT and every downstream retrieval inherits it — including calls made deep inside a graph.

Test this with two users who genuinely have different document access and confirm the result sets differ. Do not assume it works because nothing errored; nothing erroring is precisely the failure mode.

Step 6 — Build the graph and keep the citations

The prebuilt agent hides the thing you most need to control: what happens to retrieved passages on their way into state.

Figure 2 — Two model calls plan this turn: step 2 decides whether to retrieve, step 5 decides how. Both hit a Bedrock foundation model.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
import json
from typing import Annotated, TypedDict

from langchain_aws import ChatBedrockConverse
from langchain_core.messages import ToolMessage
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode

class AgentState(TypedDict):
    messages: Annotated[list, add_messages]
    citations: list

SYSTEM = (
    "You answer questions about our product documentation. "
    "Use Retrieve for direct lookups. Use AgenticRetrieveStream only when a "
    "question spans multiple documents or needs several steps to answer. "
    "Cite every claim with its source reference. "
    "If the retrieved passages do not contain the answer, say you do not know."
)

llm = ChatBedrockConverse(model="us.anthropic.claude-sonnet-4-5",
                          region_name=REGION)
llm_with_tools = llm.bind_tools(tools)

async def agent_node(state: AgentState):
    messages = [{"role": "system", "content": SYSTEM}] + state["messages"]
    return {"messages": [await llm_with_tools.ainvoke(messages)]}

def parse_citations(state: AgentState):
    """Lift source references out of the tool result into graph state.

    Retrieve and AgenticRetrieveStream return different shapes, so handle
    both rather than assuming whichever one you tested first.
    """
    last = state["messages"][-1]
    if not isinstance(last, ToolMessage):
        return {}

    raw = last.content
    if isinstance(raw, list):
        raw = next((b.get("text", "") for b in raw
                    if isinstance(b, dict) and b.get("type") == "text"), "")

    try:
        payload = json.loads(raw)
    except (json.JSONDecodeError, TypeError):
        return {}

    found = []
    passages = payload.get("retrievalResults") or payload.get("citations") or []
    for p in passages:
        loc = p.get("location", {})
        found.append({
            "uri": loc.get("s3Location", {}).get("uri") or loc.get("uri"),
            "score": p.get("score"),
            "snippet": (p.get("content", {}) or {}).get("text", "")[:200],
        })

    return {"citations": state.get("citations", []) + found}

def should_continue(state: AgentState):
    last = state["messages"][-1]
    return "tools" if getattr(last, "tool_calls", None) else END

builder = StateGraph(AgentState)
builder.add_node("agent", agent_node)
builder.add_node("tools", ToolNode(tools))
builder.add_node("cite", parse_citations)

builder.add_edge(START, "agent")
builder.add_conditional_edges("agent", should_continue,
                              {"tools": "tools", END: END})
builder.add_edge("tools", "cite")
builder.add_edge("cite", "agent")

graph = builder.compile()

The cite node is what keeps citations addressable. Without it the passages pass through message history as an opaque blob, the model cites document names in prose, and your UI has nothing to link to.

Note the defensive try/except and the two-key lookup for passages. If someone switches the gateway target's default output mode, or the graph routes to the other tool, a parser that assumes one shape will crash on a configuration change made by a colleague in the console.

Step 7 — Decide who plans

Look again at Figure 2. Your LangGraph agent node calls a model to decide whether to retrieve and how to phrase the query. Then the knowledge base's agentic retriever calls a model again to plan its multi-step query. Two planners, two model calls, two chances to lose the user's intent.

The failure mode is specific: the agent node paraphrases the question before handing it over, dropping a qualifier ("in the 2026 release", "for part XYZ2B"), and the retriever then builds an excellent multi-step plan for the wrong question. Three rules follow:

Pass the question through, don't summarize it. Instruct the agent node to forward the user's wording, including qualifiers, rather than composing a "better" search query. The agentic retriever is better at query formulation than your agent node is; that is what you're paying for.
Cap the iterations. The agentic retriever exposes a maximum-iterations setting. Left unbounded on an ambiguous question it will keep planning, and your p99 latency becomes a function of how vague your users are. Set it, measure, then raise it if recall demands.
Route, don't default. Send lookups to Retrieve and multi-hop questions to AgenticRetrieveStream. Defaulting everything to the agentic path costs latency and tokens on questions a single search would have answered — and, per Step 1, gives up X-Ray traces for the whole workload.

Troubleshooting

Symptom	Likely cause	What to do
403 on every gateway call	aws_service set to bedrock instead of bedrock-agentcore	Fix the signing service name in the transport factory
403 only on AgenticRetrieveStream	The action was scoped to a knowledge base ARN	Grant bedrock:AgenticRetrieveStream on * — it is not resource-scopeable
Target creation fails validation	Missing bedrock:GetKnowledgeBase, or a self-managed KB	Add the permission; confirm the KB is a managed knowledge base
Every user sees every document	userContext not sent, or ACLs never ingested	Thread userContext per request; confirm aclEnabled: true and the ACL file URI on the data source
userContext rejected as an unknown parameter	Schema subsetting hid it at the target	Update the target configuration to expose it
Guardrail masks nothing	MASK on the agentic path	Use BLOCK, or route sensitive queries through Retrieve
No traces for retrieval	X-Ray isn't available for AgenticRetrieveStream	Use Retrieve where tracing matters; lean on AgentCore Observability for the rest
Tool list comes back empty	One failing server in a multi-server client	Give the gateway its own MultiServerMCPClient
Latency spikes on vague questions	Unbounded agentic iterations	Set a maximum-iterations cap and route by question type

Where to go next

The same gateway you just built can front far more than one knowledge base. Web Search is available as another pre-built connector target, so public web grounding becomes another MCP tool on the same endpoint with the same IAM story — and because every tool routes through the gateway, guardrails and policy apply uniformly, evaluated outside your agent's code where the agent cannot reason around them.

Two things worth building next: an evaluation harness that calls Retrieve directly (so you get X-Ray traces and can measure whether your routing rule is actually sending the right questions down the agentic path), and a second gateway target in a non-production account so schema changes get tested before they reach the agent your users talk to.

References

Connect to your knowledge base through AgentCore Gateway — the two tools, IAM-only outbound auth, and the SigV4 client change.
Define the gateway target configuration — gateway execution role permissions, including the unscopeable agentic action.
Use an AgentCore gateway — supported MCP versions and operations.
Amazon Bedrock AgentCore Gateway overview
Amazon Bedrock AgentCore overview
Web Search Tool connector target
Build enterprise search for agents with Amazon Bedrock Managed Knowledge Base — what gateway integration adds, and regional availability.
New in Amazon Bedrock AgentCore: broader knowledge and continuous learning
Build multi-agent SRE assistants with Amazon Bedrock AgentCore — a worked LangGraph + AgentCore + MCP example.
Model Context Protocol — LangChain docs
langchain-mcp-adapters on GitHub
Model Context Protocol specification
MCP security best practices

# langgraph # langchain # amazon-bedrock # mcp # amazon-bedrock-agentcore

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

How to Connect a Bedrock Managed Knowledge Base to LangGraph over MCP

Jubin Soni

Why this integration is worth doing?

Architecture

Prerequisites

The IAM detail that will bite you first

Step 1 — Understand the two tools before you choose one

Step 2 — Add the knowledge base as a gateway target

Step 3 — Authenticate with SigV4, not a bearer token

Step 4 — Load the tools into LangGraph

Step 5 — Fix per-user filtering before you ship

Step 6 — Build the graph and keep the citations

Step 7 — Decide who plans

Troubleshooting

Where to go next

References