Build generative AI agents with LangChain and Anthropic Claude 3 models on Amazon Bedrock
Learn how to leverage the new native tool-calling capabilities in the LangChain AWS package, using Anthropic Claude 3 models with Amazon Bedrock.
Many of our customers often ask me if LangChain supports Amazon Bedrock tool use – it absolutely does! But why is that so important? Tool use is crucial for agent behavior using large language models (LLMs) because it allows the models to perform specific tasks and operations beyond their inherent text generation and conversation capabilities, enhancing their functionality and accuracy. Agents use external tools and APIs to perform specific tasks. For example, an LLM agent can use a calculator tool to solve math problems or access a database to retrieve information. By integrating custom tools, LLMs can execute precise commands, interact with external systems, and deliver more practical and context-aware responses.
LangChain provides base classes, pre-built toolkits, agent runtime classes, and sample code that make it easy to build agent applications using LLMs. The combination of LangChain + Amazon Bedrock + Anthropic Claude 3 models allows you to build agents with the ease of accessing leading LLMs on AWS.
What’s New?
The LangChain AWS package now supports native tool-calling using Anthropic Claude 3 models. This enhancement allows developers to seamlessly integrate powerful LLMs with custom tools and toolkits, providing a robust and flexible environment for various applications using LangChain and LangGraph.
How It Works
Let me walk you through how you can set up and use this feature in your projects:
Set Up Your Environment:
Make sure you have the necessary dependencies installed, including `boto3`, `langchain_aws`, and other related packages.
$ pip install boto3 langchain langchain_aws mypy-boto3-bedrock-runtime pydanticOf course, you also need to enable access to Claude 3 models on Amazon Bedrock in your AWS Account .
Define Your Models and Parameters
Use Pydantic models to define inference parameters and models for validation and default settings. This approach helps maintain consistency and reliability in your AI model configurations.
1
2
3
4
5
6
7
8
9
10
11
12
13
from pydantic import BaseModel, Field
from enum import Enum
class ModelId(str, Enum):
CLAUDE_3_HAIKU = "anthropic.claude-3-haiku-20240307-v1:0"
CLAUDE_3_SONNET = "anthropic.claude-3-sonnet-20240229-v1:0"
CLAUDE_3_OPUS = "anthropic.claude-3-opus-20240229-v1:0"
class ModelKwargs(BaseModel):
temperature: float = Field(default=0.5, ge=0, le=1)
max_tokens: int = Field(default=2048, ge=1, le=4096)
top_p: float = Field(default=0.999, ge=0, le=1)
top_k: int = Field(default=0, ge=0, le=500)
Get Your Bedrock Client:
Set up a Bedrock runtime client using boto3 to leverage the Claude 3 models. This involves configuring your AWS session and client.
1
2
3
4
5
6
7
from boto3 import _get_default_session
from boto3.session import Session
from mypy_boto3_bedrock_runtime.client import BedrockRuntimeClient
def get_bedrock_client(region: str, profile: str | None = None) -> BedrockRuntimeClient:
session = _get_default_session() if profile is None else Session(profile_name=profile)
return session.client("bedrock-runtime", region_name=region)
Integrate with LangChain:
Use the
ChatBedrock class to set up your Bedrock runtime client. This class allows you to pass custom tools and commands seamlessly.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
from langchain_aws.chat_models import ChatBedrock
from langchain_core.callbacks import StreamingStdOutCallbackHandler
def get_chat_bedrock(
client: BedrockRuntimeClient,
model_id: ModelId,
model_kwargs: ModelKwargs,
streaming: bool = False,
verbose: bool = False) -> ChatBedrock:
return ChatBedrock(
client=client,
model_id=model_id.value,
model_kwargs=model_kwargs.__dict__,
streaming=streaming,
verbose=verbose,
callbacks=[StreamingStdOutCallbackHandler()] if streaming else []
)
Define and Use Custom Tools:
Create custom tools like this simple math operation tool and use them with your chat model.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
from langchain.pydantic_v1 import BaseModel as LcBaseModel # use LangChain's vended pydantic
from langchain_core.tools import tool
class MathInputs(LcBaseModel):
first_number: float
second_number: float
class MathOperation(str, Enum):
ADD = "add"
SUBTRACT = "subtract"
MULTIPLY = "multiply"
DIVIDE = "divide"
def do_math(numbers: MathInputs, operation: MathOperation) -> float:
match operation:
case MathOperation.ADD:
return numbers.first_number + numbers.second_number
case MathOperation.SUBTRACT:
return numbers.first_number - numbers.second_number
case MathOperation.MULTIPLY:
return numbers.first_number * numbers.second_number
case MathOperation.DIVIDE:
return numbers.first_number / numbers.second_number
case _:
raise ValueError(f"Invalid operation: {operation}")
Invoke and Execute:
Finally, you can invoke the chat model with your custom tools and get the desired results.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# get a boto3 bedrock-runtime client
client = get_bedrock_client("us-east-1")
# get a chat bedrock model
chat_model = get_chat_bedrock(
client=client,
model_id=ModelId.CLAUDE_3_HAIKU,
model_kwargs=ModelKwargs(),
)
# use the math tool with the chat model
chat_with_tools = chat_model.bind_tools([do_math])
# invoke the chat model with the math tool
result = chat_with_tools.invoke("What is 2 + 2?")
for tool_call in result.tool_calls:
tool_function = eval(tool_call["name"])
result = tool_function.invoke(input=tool_call["args"])
print(result)
Wrapping up
In this post, you learned how to set up and use the LangChain AWS package with native tool-calling support for Claude 3 models on Amazon Bedrock. By following the steps, you can start building agents with the rest of the LangChain package and its capabilities.
Feel free to leave a comment with your experiences or any questions you have as you try out these features!
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article