GenAI, from your Local machine to AWS with LangChain.js

Empowering JavaScript/TypeScript Developers with GenAI

John Wick

Published Aug 9, 2024

Last modified Aug 13, 2024

As Generative AI (GenAI) continues to revolutionize application development, it's crucial to recognize that this technology isn't exclusive to a specific programming language. TypeScript and JavaScript developers can also harness the power of GenAI to create innovative solutions. This article introduces a sample application that serves as an excellent starting point for TypeScript/JavaScript developers looking to experiment with advanced GenAI features.

Overview of the DeepF1 Sample Application

Our sample application, DeepF1, is a Serverless GenAI chat application that demonstrates Retrieval-Augmented Generation (RAG) and Agentic AI capabilities. Built for a fictitious Formula 1 company, DeepF1 allows race engineers to gain insights about driver performance metrics and Formula 1 Championship data.

Key components of the application include:

A web app built with Vite + React and hosted on AWS Amplify
A backend leveraging Amazon Bedrock features and AWS Lambda
A vector database using Amazon OpenSearch Serverless
Document storage using Amazon S3

Notable features:

Serverless architecture for scalability and cost-effectiveness
RAG + Agentic approach for accurate and relevant responses
Local development support using Ollama

Prerequisites

You need to install the following tools to use that sample:

Getting Started with Local Development

One of the strengths of this sample is the ability to experiment locally without incurring Cloud costs. Here's how to get started:

Open your terminal and clone the DeepF1 sample repository: git clone https://github.com/ajohn-wick/deepf1-ai-engineer
Go to the langchain-poc folder and npm install dependencies
Download necessary models on your machine using Ollama:

1
2
ollama pull llama3
ollama pull nomic-embed-text

Go to the src subfolder and run local scripts to test basic functionality using LangChain.js:

LangChain.js is a JavaScript framework providing high-level APIs to interact with AI models / APIs but also many built-in tools to ease the development of complex AI applications.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// 01_local_invoke_model.js
import { HumanMessage, SystemMessage } from "@langchain/core/messages";
import { ChatOllama } from "@langchain/community/chat_models/ollama";

// 1. Initialize the model
const model = new ChatOllama({ model: "llama3", topP: 0.9 });

// 2. Invoke the model and print its response
const prompt = `
You're a helpful assistant dedicated to race engineers and strategists of Formula 1.
You have to provide relevant data points to race engineers in order to better understand 2024 race car using historical Formula 1 data as source of truth.
If you are unsure about something, you can respond that you don't have enough information about that particular topic.
Your main goal is to provide useful information to users.
`;
const response = await model.invoke([
    new SystemMessage(prompt),
    new HumanMessage("Which F1 Driver set the fastest lap time during the Formula 1 Monaco 2023 Race?"),
]);

console.log(response.content);

Run the code thanks to the command node 01_local_invoke_model.js. The response from the model will be displayed in your console!

If you want to use RAG technique to provide a domain-area knowledge (Formula 1 in our case) to the model, update your program with the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
// 02_local_rag_ollama.js
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { ChatOllama } from "@langchain/community/chat_models/ollama";
import { OllamaEmbeddings } from "@langchain/community/embeddings/ollama";
import { DirectoryLoader } from "langchain/document_loaders/fs/directory";
import { CSVLoader } from "@langchain/community/document_loaders/fs/csv";
import { FaissStore } from "@langchain/community/vectorstores/faiss";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { createRetrievalChain } from "langchain/chains/retrieval";

// 1. Initialize the models
const model = new ChatOllama({ model: "llama3", temperature: 0, topP: 0.9 });
const embeddings = new OllamaEmbeddings({ model: "nomic-embed-text" });

// 2. Load F1 Data and split it into smaller chunks
const loader = new DirectoryLoader("./../data",
    {
        ".csv": (path) => new CSVLoader(path),
    }
);
const csvData = await loader.load();
const splitter = new RecursiveCharacterTextSplitter({
    separators: "\n",
    chunkSize: 512,
    chunkOverlap: 200
});
const deepF1Docs = await splitter.splitDocuments(csvData);

// 3. Put the documents into a vector store and convert them to vectors
const vectorStore = await FaissStore.fromDocuments(deepF1Docs, embeddings, {});

// 4. Create the RAG chain that retrieves and combines the prompt with the documents
const prompt = `
You are DeepF1, an AI assistant trained and designed to help the Formula 1 race engineers and strategists of the DeepF1 racing team, a fictional Formula 1 racing team. Your role is to provide relevant data points and strategic insights about:

The DeepF1 Formula 1 car's performance characteristics and settings
Competitors' tyre choices and wear patterns
Pitstop strategies for both DeepF1 and rival teams
Weather conditions and their potential impact on race strategy
Track-specific considerations and historical performance data
Driver feedback and performance metrics
Regulatory considerations and their strategic implications

Your goal is to analyze this information and offer strategic recommendations to ensure victory for the DeepF1 racing team. When responding to queries, prioritize accuracy, timeliness, and actionable insights that can give DeepF1 a competitive edge during races and throughout the season.
`;
const combineDocsChain = await createStuffDocumentsChain({
    llm: model,
    prompt: ChatPromptTemplate.fromMessages([
        ["system", prompt],
        ["human", "Answer the question: {input}\nOnly consisder the following documents as source of truth:\n\n{context}"],
    ]),
});
const chain = await createRetrievalChain({
    retriever: vectorStore.asRetriever(),
    combineDocsChain,
});

// 5. Generate the result
const response = await chain.invoke({ input: "Which F1 Driver set the fastest lap time during the Formula 1 Monaco 2023 Race?" });
console.log(response['answer']);

This code will load some CSV documents containing driver performance metrics and Formula 1 Championship data, split it into smaller chunks, convert them to vectors, and then use them in a multi-step workflow (chain) to perform a vector search and generate a response using the best results.
This code shows how LangChain.js can help you build more advanced AI scenarios in a few lines of code.

That's great but how do I combine LangChain.js and AWS?

Thankfully, there is a @langchain/aws package providing us Bedrock Interfaces in order to use the same JavaScript code we produced. Here are the minor changes we have to implement:

Ollama => Foundation Model available within Amazon Bedrock
Local RAG => Amazon Bedrock Knowledge Base

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
// 03_local_rag_bedrock.js
import { ChatPromptTemplate } from "@langchain/core/prompts";
// import { ChatOllama } from "@langchain/community/chat_models/ollama";
import { ChatBedrockConverse } from "@langchain/aws";
// import { OllamaEmbeddings } from "@langchain/community/embeddings/ollama";
import { AmazonKnowledgeBaseRetriever } from "@langchain/aws";
// import { DirectoryLoader } from "langchain/document_loaders/fs/directory";
// import { CSVLoader } from "@langchain/community/document_loaders/fs/csv";
// import { FaissStore } from "@langchain/community/vectorstores/faiss";
// import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { createRetrievalChain } from "langchain/chains/retrieval";

const knowledgeBaseId = "[YOUR AMAZON BEDROCK KNOWLEDGE BASE ID]";
const modelId = "meta.llama3-8b-instruct-v1:0";

// 1. Initialize the models
const model = new ChatBedrockConverse({
  model: modelId,
  topP: 0.9,
  region: process.env.AWS_REGION,
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    sessionToken: process.env.AWS_SESSION_TOKEN,
  },
});

// 2. Initialize the Bedrock Knowledge Base retriever
const retriever = new AmazonKnowledgeBaseRetriever({
  knowledgeBaseId: knowledgeBaseId,
  region: process.env.AWS_REGION,
});

// 3. Step 3. is not required anymore as we are relying on an Amazon Bedrock Knowledge Base
// 4. Create the RAG chain that retrieves and combines the prompt with the documents
// 5. Generate the result

In order to run that code with success, you will have to deploy the Backend as it is relying on an Amazon Bedrock Knowledge Base instead of local data.

By starting locally, developers can quickly iterate and experiment with GenAI concepts before moving to a Cloud deployment.

Deploying the sample to AWS

When ready to scale, the sample provides a straightforward path to AWS.
Make sure to use one of the AWS region where all AWS Services used by this sample are available.

Deploy the Backend using AWS CDK

To deploy our AWS Backend resources, we are relying on the AWS Cloud Development Kit (CDK) :

If not already done, open your terminal and clone the DeepF1 sample repository: git clone https://github.com/ajohn-wick/deepf1-ai-engineer
Go to the infra folder where our CDK project is stored and npm install dependencies
Execute the following bash script:

./cdk-deploy-to.sh [AWS_ACCOUNT_ID] [AWS_REGION] [AWS_PROFILE_NAME](optional)

This will provision all AWS backend resources (Amazon Bedrock, Amazon S3, AWS Lambda) and build the search index (Amazon OpenSearch Serverless) based on the files found in the ./data subfolder.

If you are not familiar with the Infrastructure as Code concept , do not hesitate to dig into the /infra/lib/genai-stack.ts file. This is where we coded our Cloud application resources using a familiar programming language (TypeScript in our case).
As an example, this file has the definition of our AWS Lambda function along with our LangChain.js code (developed during the Getting Started with Local Development section) in order to interact with the Amazon Bedrock Knowledge Base.

Here is the code of our Lambda function (cf. infra/src/actions-group/agent-deepf1.ts) :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
import {
    MetricUnits,
    Metrics,
    logMetrics,
} from '@aws-lambda-powertools/metrics';
import { Tracer, captureLambdaHandler } from '@aws-lambda-powertools/tracer';
import { errorHandler, logger } from '@shared/index';
import { injectLambdaContext } from '@aws-lambda-powertools/logger';
import { ValidationError } from '@errors/validation-error';
import middy from '@middy/core';

import { ChatPromptTemplate } from "@langchain/core/prompts";
import { ChatBedrockConverse } from "@langchain/aws";
import { AmazonKnowledgeBaseRetriever } from "@langchain/aws";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { createRetrievalChain } from "langchain/chains/retrieval";

import { config } from '@config';

const tracer = new Tracer();
const metrics = new Metrics();
const knowledgeBaseId = config.get('knowledgeBaseId');
const modelId = config.get('modelId');
const promptTemplate = config.get('promptTemplate');

export const agentQueryKB = async (event: any, context: any) => {
    try {
        if (!event) throw new ValidationError('no event payload');
        logger.info('Received event:', JSON.stringify(event, null, 2));
        const { actionGroup, apiPath, httpMethod, inputText } = event;

        // 1. Initialize the models
        const model = new ChatBedrockConverse({
            model: modelId,
            region: process.env.AWS_REGION,
        });

        // 2. Initialize the Bedrock Knowledge Base retriever
        const retriever = new AmazonKnowledgeBaseRetriever({
            knowledgeBaseId: knowledgeBaseId,
            region: process.env.AWS_REGION,
        });
        // 3. Create the RAG chain that retrieves and combines the prompt with the documents
        const combineDocsChain = await createStuffDocumentsChain({
            llm: model,
            prompt: ChatPromptTemplate.fromMessages([
                ["system", promptTemplate],
                ["human", "Answer the question: {input}\nOnly consider the following documents as source of truth:\n\n{context}"],
            ]),
        });
        const chain = await createRetrievalChain({
            retriever: retriever,
            combineDocsChain,
        });

        // 4. Generate the result
        const response = await chain.invoke({ input: inputText });
        logger.info('Received answer:', JSON.stringify(response, null, 2));
        const responseBody = {
            'application/json': {
                body: response.answer
            }
        };
        const actionResponse = {
            actionGroup,
            apiPath,
            httpMethod,
            httpStatusCode: 200,
            responseBody
        };
        metrics.addMetric('SuccessfulAgentQuery', MetricUnits.Count, 1);
        return {
            messageVersion: event.messageVersion,
            response: actionResponse,
        }
    } catch (error) {
        let errorMessage = 'Unknown error';
        if (error instanceof Error) errorMessage = error.message;
        logger.error(errorMessage);

        metrics.addMetric('AgentQueryError', MetricUnits.Count, 1);

        return errorHandler(error);
    }
};

export const handler = middy(agentQueryKB)
    .use(injectLambdaContext(logger))
    .use(captureLambdaHandler(tracer))
    .use(logMetrics(metrics));

Deploy the Frontend using AWS Amplify

Once the Backend deployed with success, go to the the webapp folder within your terminal and execute the following actions:

npm install dependencies
Open the src/App.tsx file and replace the bedrock section with the following one:

1
2
3
4
5
6
7
8
bedrock: {
    region: aws_exports.aws_project_region,
    // modelId: "meta.llama3-8b-instruct-v1:0",
    agent: {
        agentId: "[AMAZON BEDROCK AGENT ID]",
        agentAliasId: "[AMAZON BEDROCK AGENT ALIAS ID]"
    }
},

Make sure to replace [AMAZON BEDROCK AGENT ID] and [AMAZON BEDROCK AGENT ALIAS ID] with Outputs values displayed in your terminal when you deployed the Backend.

If this is the 1st time you are using AWS Amplify CLI, initialize it on your local machine (via the command line amplify configure ) in order to deploy AWS Amplify applications.

Initialize and set up authentication for the Amplify app:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
amplify init
# Here is an example of initializing the AWS Amplify application (make sure the Distribution Directory Path is set to dist):

# - ? Enter a name for the environment dev
# - ? Choose your default editor: Visual Studio Code
# - ✔ Choose the type of app that you're building · javascript
# - Please tell us about your project
# - ? What javascript framework are you using react
# - ? Source Directory Path:  src
# - ? Distribution Directory Path: dist
# - ? Build Command:  npm run-script build
# - ? Start Command: npm run-script start
# - Using default provider  awscloudformation
# - ? Select the authentication method you want to use: AWS profilebash1

amplify add auth
# Here is an example of adding authentication to the AWS Amplify application:

# Using service: Cognito, provided by: awscloudformation
# The current configured provider is Amazon Cognito.
# Do you want to use the default authentication and security configuration? Default configuration
# Warning: you will not be able to edit these selections.
# How do you want users to be able to sign in? Username
# Do you want to configure advanced settings? No, I am done.

Lastly, you can experiment locally your Frontend by running the npm run dev command or you can add hosting and deploy it to AWS:

1
2
3
4
amplify add hosting
# Open the `src` subfolder and rename the `aws-exports.js` file with the following name: `aws-exports.ts`

amplify publish

Once built and artifacts published, you can use the public URL displayed in your terminal to open theDeepF1 Race AI Engineer web app and start chatting with the Foundation Model you specified (llama3 by default) which will leverage Agentic approach when required.

Conclusion

This DeepF1 sample application demonstrates that TypeScript/JavaScript developers are well-positioned to create sophisticated GenAI applications. By providing a path from local development to AWS deployment, it offers a practical starting point for experimentation and innovation in the GenAI space.

We encourage you to explore this sample, adapt it to your needs, and share your experiences with the community. The world of GenAI is open to TypeScript/JavaScript developers, and the possibilities are endless.

P.S: If you have any issue when running or deploying the sample, please have a look at the detailed instructions available within the DeepF1 sample README.md

# typescript # langchainjs # amazon-bedrock # aws-lambda # aws-cdk

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

GenAI, from your Local machine to AWS with LangChain.js

John Wick

Overview of the DeepF1 Sample Application

Prerequisites

Getting Started with Local Development

Deploying the sample to AWS

Deploy the Backend using AWS CDK

Deploy the Frontend using AWS Amplify

Conclusion