Set Up Small Language Model Inference on AWS Lambda Through the AWS Console

Series: Financial Service (19 articles)

1
AWS Lambda MicroVMs and the End of Repeated Application Cold Initialization
…
9
Set Up Small Language Model Inference on AWS Lambda Through the AWS Console This article
10
The Institutional Memory Flywheel: Building a Safer, Smarter Trading Knowledge System with Amazon S3 Vectors and S3 Metadata [ part 1 ]

Martin

Dev @ Vertex Macro

Published Sep 29, 2026

Last modified Sep 29, 2026

Set Up Small Language Model Inference on AWS Lambda Through the AWS Console

This guide builds a CPU-based, serverless inference endpoint for a quantized GGUF small language model using AWS Lambda, llama.cpp, Amazon S3, Lambda Web Adapter, a Lambda Function URL, and response streaming. It also explains the optional SnapStart path for container images and the production controls required for financial-services use.

Native support should be interpreted carefully. Lambda does not provide a native SLM model registry or a managed inference runtime comparable to a dedicated AI hosting service. The practical pattern combines Lambda container images, CPU inference, S3 model storage, Lambda Web Adapter, response streaming, and optional SnapStart.

Target Architecture

Browser or internal client sends an HTTPS streaming request to the Lambda Function URL with invoke mode set to RESPONSE STREAM
Request reaches the AWS Lambda container function configured with 10240 MB of memory
Lambda Web Adapter routes traffic locally to FastAPI running on localhost 8080
FastAPI communicates with the llama.cpp server running inside the container
Server utilizes a quantized GGUF model loaded from Amazon S3 into temp storage or memory

Lambda Web Adapter runs independently from the web application and translates Lambda invocation events into standard HTTP requests. It supports familiar HTTP frameworks such as FastAPI, Flask, Express, Spring Boot, and other HTTP/1.1 applications. Response streaming is enabled with AWS_LWA_INVOKE_MODE=response_stream.

Important Suitability Boundary

This design is appropriate when the model is genuinely small and quantized, CPU inference provides acceptable latency, requests are intermittent or bursty, output length is bounded, scaling to zero is valuable, the workload does not require continuous GPU processing, and one request can finish within the standard Lambda invocation limit.

A Lambda function can use up to 10,240 MB of memory and can execute for up to fifteen minutes per invocation. Memory allocation also controls proportional CPU allocation.

This design is not suitable for large models, very large context windows, sustained high concurrency, continuous batching, GPU-specific kernels, or deterministic low-tail latency. Those workloads should be evaluated on Amazon SageMaker AI, GPU-backed Amazon ECS, Amazon EKS, AWS Batch, or another dedicated inference platform.

Select the AWS Region

Sign in to the AWS Management Console and select a commercial AWS Region that supports Lambda container images, Lambda Function URLs, Lambda response streaming, the desired Lambda architecture, and SnapStart for container images if required.

Response streaming is not available in every AWS Region, so availability must be checked before deployment. SnapStart for container-image functions is available in most commercial Regions, but regional exceptions should be checked against the current AWS documentation before implementation.

Region selection should account for data residency, market-data proximity, client latency, disaster recovery, service availability, and organizational policy.

Choose and Prepare the Model

Use a model that has an appropriate commercial license, has been approved by model-risk management, is small enough for Lambda memory, is available in GGUF format, has been quantized for CPU inference, and has passed task-specific validation.

Possible examples include:

1
2
3
Llama 3.2 1B Instruct

A small DeepSeek R1 distilled model

Another appropriately licensed 1B to 3B parameter model

Do not upload an unverified model directly into production. Calculate and preserve its SHA-256 hash:

1
sha256sum model.gguf

Record the model family, exact filename, model license, quantization type, SHA-256 hash, tokenizer version, prompt-template version, evaluation-report version, and approval record.

Create a KMS Key

Open the AWS Management Console and navigate to:

1
2
3
KMS

Customer managed keys

Create key

Select:

1
2
Key type: Symmetric

Key usage: Encrypt and decrypt

Enter an alias:

1
alias/slm-model-artifacts

Select the key administrators and key users according to the institution's separation-of-duties policy, then choose Finish.

The key protects the S3 model artifact. Encryption does not replace integrity checking. The function should still compare the downloaded artifact with the approved SHA-256 hash.

Create the S3 Model Bucket

Open:

1
2
3
4
AWS Management Console

S3

Buckets

Create bucket

Enter a globally unique bucket name:

1
enterprise-slm-model-artifacts-ACCOUNT_ID-REGION

Configure:

1
2
3
4
5
Object Ownership: ACLs disabled

Block Public Access: Enabled

Bucket Versioning: Enabled

Default encryption: SSE-KMS

KMS key: alias/slm-model-artifacts

Choose Create bucket and create a logical prefix:

1
models/production/

Upload the approved GGUF file under a versioned key, for example:

1
models/production/llama-3.2-1b/2026-09-28/model.gguf

Record the object ARN, Version ID, object size, ETag, and approved SHA-256 hash. Do not overwrite a production model and silently reuse the same identifier.

Create the ECR Repository

Open:

1
2
3
4
AWS Management Console

Elastic Container Registry

Private repositories

Create repository

Enter:

1
2
3
4
Repository name: lambda-slm-inference

Tag immutability: Enabled

Image scanning: Scan on push

Encryption: AES-256 or approved KMS key

Choose Create repository and copy the repository URI:

1
ACCOUNT_ID.dkr.ecr.REGION.amazonaws.com/lambda-slm-inference

Prepare the Container Application

The AWS Console creates and configures cloud resources, but it does not compile llama.cpp or build the container image. Use an approved CodeBuild project, CI/CD pipeline, CloudShell environment with sufficient storage, or development workstation.

Create app.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
import os

import httpx



from fastapi import FastAPI, HTTPException

from fastapi.responses import StreamingResponse

from pydantic import BaseModel



app = FastAPI()



LLAMA_URL = os.getenv(

    "LLAMA_URL",

    "http://127.0.0.1:8081/v1/chat/completions"

)





class InferenceRequest(BaseModel):

    prompt: str

    max_tokens: int = 256

    temperature: float = 0.2





@app.get("/health")

async def health():

    return {"status": "ready"}





@app.post("/inference")

async def inference(request: InferenceRequest):

    if not request.prompt.strip():

        raise HTTPException(status_code=400, detail="prompt is required")



    if request.max_tokens > 512:

        raise HTTPException(

            status_code=400,

            detail="max_tokens must not exceed 512"

        )



    payload = {

        "model": "approved-slm",

        "stream": True,

        "messages": [

            {

                "role": "system",

                "content": (

                    "Use only the evidence supplied by the caller. "

                    "State missing or stale evidence. "

                    "Do not provide autonomous trading instructions."

                )

            },

            {

                "role": "user",

                "content": request.prompt

            }

        ],

        "max_tokens": request.max_tokens,

        "temperature": request.temperature

    }



    async def token_stream():

        async with httpx.AsyncClient(timeout=None) as client:

            async with client.stream(

                "POST",

                LLAMA_URL,

                json=payload

            ) as response:

                response.raise_for_status()

                async for chunk in response.aiter_bytes():

                    yield chunk



    return StreamingResponse(

        token_stream(),

        media_type="text/event-stream"

    )

Create startup.sh:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#!/bin/bash

set -euo pipefail



MODEL_PATH="/models/model.gguf"



if [ ! -f "${MODEL_PATH}" ]; then

  echo "Model file is missing"

  exit 1

fi



echo "${EXPECTED_MODEL_SHA256}  ${MODEL_PATH}" \

  | sha256sum -c -



/opt/llama/llama-server \

  --model "${MODEL_PATH}" \

  --host 127.0.0.1 \

  --port 8081 \

  --ctx-size "${MODEL_CONTEXT_SIZE:-4096}" \

  --threads "${MODEL_THREADS:-6}" \

  --parallel 1 &



exec uvicorn app:app \

  --host 0.0.0.0 \

  --port 8080

For the production S3-to-memory architecture, replace the static model-path check with a controlled initialization stage that retrieves the exact S3 object version, streams the object into /tmp or an approved memfd_create loader, verifies the SHA-256 hash, starts llama-server, and reports readiness only after a deterministic inference test succeeds.

The memfd_create path avoids a traditional persistent disk file, but it does not eliminate the model's memory cost. Lambda still needs sufficient memory for model weights, KV cache, llama.cpp, Web Adapter, the application, buffers, and safety headroom.

Create a multi-stage Dockerfile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
FROM public.ecr.aws/docker/library/ubuntu:24.04 AS builder



RUN apt-get update && \

    apt-get install -y \

        build-essential \

        cmake \

        git \

        libcurl4-openssl-dev && \

    rm -rf /var/lib/apt/lists/*



RUN git clone \

    --depth 1 \

    https://github.com/ggerganov/llama.cpp.git \

    /src/llama.cpp



RUN cmake \

    -S /src/llama.cpp \

    -B /src/llama.cpp/build \

    -DGGML_NATIVE=OFF \

    -DLLAMA_CURL=ON \

    -DBUILD_SHARED_LIBS=OFF && \

    cmake --build \

    /src/llama.cpp/build \

    --config Release \

    -j 4



FROM public.ecr.aws/lambda/python:3.13



COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:1.1.0 \

    /lambda-adapter \

    /opt/extensions/lambda-adapter



COPY --from=builder \

    /src/llama.cpp/build/bin/llama-server \

    /opt/llama/llama-server



COPY app.py ${LAMBDA_TASK_ROOT}/app.py

COPY startup.sh /startup.sh



RUN chmod +x /startup.sh && \

    pip install \

        fastapi \

        uvicorn \

        httpx \

        pydantic



ENV AWS_LWA_PORT=8080

ENV AWS_LWA_READINESS_CHECK_PATH=/health

ENV AWS_LWA_INVOKE_MODE=response_stream



ENTRYPOINT ["/startup.sh"]

Pin exact base-image digests, Lambda Web Adapter versions, llama.cpp commits, dependencies, and compiler settings in production. Do not build production from an unpinned branch.

Select x86_64 or Arm64

Lambda supports x86_64 and arm64. Standard Lambda arm64 uses AWS Graviton2. It does not automatically provide later Graviton generations or their newer vector capabilities.

Build for x86:

1
2
3
4
docker buildx build \

  --platform linux/amd64 \

  -t lambda-slm-inference:1.0.0 \

  --load .

Build for Arm:

1
2
3
4
docker buildx build \

  --platform linux/arm64 \

  -t lambda-slm-inference:1.0.0-arm64 \

  --load .

The image, native binaries, extensions, and all dependencies must match the Lambda architecture. Benchmark both architectures rather than assuming one is always superior.

Push the Image to ECR

Authenticate Docker:

1
2
3
4
5
6
aws ecr get-login-password \

  --region REGION \

  | docker login \

      --username AWS \

      --password-stdin \

      ACCOUNT_ID.dkr.ecr.REGION.amazonaws.com

Tag and push:

1
2
3
4
5
6
docker tag \

  lambda-slm-inference:1.0.0 \

  ACCOUNT_ID.dkr.ecr.REGION.amazonaws.com/lambda-slm-inference:1.0.0



docker push \

  ACCOUNT_ID.dkr.ecr.REGION.amazonaws.com/lambda-slm-inference:1.0.0

In Amazon ECR, verify the image scan status, digest, architecture, immutable tag, and timestamp. Record the digest in the model release manifest.

Create the IAM Execution Role

Open:

1
2
3
4
AWS Management Console

IAM

Roles

Create role

Choose Lambda as the trusted AWS service and attach AWSLambdaBasicExecutionRole. Name the role:

1
lambda-slm-inference-role

Add a narrowly scoped inline policy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
{

  "Version": "2012-10-17",

  "Statement": [

    {

      "Sid": "ReadApprovedModel",

      "Effect": "Allow",

      "Action": [

        "s3:GetObject",

        "s3:GetObjectVersion"

      ],

      "Resource": [

        "arn:aws:s3:::enterprise-slm-model-artifacts-ACCOUNT_ID-REGION/models/production/*"

      ]

    },

    {

      "Sid": "DecryptApprovedModel",

      "Effect": "Allow",

      "Action": [

        "kms:Decrypt"

      ],

      "Resource": [

        "arn:aws:kms:REGION:ACCOUNT_ID:key/KEY_ID"

      ]

    }

  ]

}

Do not grant the inference function permission to change or delete the model.

Create the Lambda Container Function

Open:

1
2
3
4
AWS Management Console

Lambda

Functions

Create function

Select Container image and enter:

1
2
3
Function name: slm-streaming-inference

Container image URI: ECR image URI

Architecture: Match the image architecture

Under permissions, select the existing role lambda-slm-inference-role, then choose Create function.

Configure Memory, Timeout, and Storage

Open:

1
2
3
Configuration

General configuration

Edit

For the initial benchmark, set:

1
2
Memory: 10,240 MB

Timeout: 5 minutes

Testing at 10,240 MB gives the function the largest available proportional CPU allocation. Production memory must be selected using measured latency and cost per accepted inference.

If the model loads through /tmp, configure:

1
2
3
4
Configuration

Ephemeral storage

Edit

Ephemeral storage: 10,240 MB

If the model uses memfd_create, /tmp may be smaller, but overall memory still requires substantial headroom.

Add Environment Variables

Open:

1
2
3
Configuration

Environment variables

Edit

Add:

1
2
3
4
5
6
7
8
9
10
11
AWS_LWA_PORT=8080

AWS_LWA_READINESS_CHECK_PATH=/health

AWS_LWA_INVOKE_MODE=response_stream

MODEL_BUCKET=enterprise-slm-model-artifacts-ACCOUNT_ID-REGION

MODEL_KEY=models/production/llama-3.2-1b/2026-09-28/model.gguf

MODEL_VERSION_ID=APPROVED_VERSION_ID

EXPECTED_MODEL_SHA256=APPROVED_SHA256

MODEL_CONTEXT_SIZE=4096

MODEL_MAX_TOKENS=512

MODEL_THREADS=6

LLAMA_URL=http://127.0.0.1:8081/v1/chat/completions

Do not place broker passwords, API keys, private certificates, or database credentials in environment variables. Use AWS Secrets Manager.

Create the Function URL

Open:

1
2
3
Configuration

Function URL

Create function URL

Select:

1
Auth type: AWS_IAM

Do not choose anonymous access for a financial-services inference endpoint.

Configure CORS only for the exact authorized browser origin and required method. Save the Function URL, edit it, open Additional settings, and set:

1
Invoke mode: RESPONSE_STREAM

Ensure the settings match:

1
2
Function URL: RESPONSE_STREAM

AWS_LWA_INVOKE_MODE: response_stream

Lambda Web Adapter supports streaming with Function URLs and API Gateway. Streaming should be tested through the actual HTTP path because the Lambda console displays test responses as buffered.

Test the Health Endpoint

Because the Function URL uses IAM authentication, call it from a SigV4-capable client, an approved SDK application, or an authenticated backend-for-frontend.

Expected health response:

1
2
3
{

  "status": "ready"

}

A successful process start is not sufficient. Production readiness should execute a tiny deterministic model request and verify that the expected model version is loaded, the expected output schema is returned, the model hash is verified, and memory headroom remains acceptable.

Test Streamed Inference

Send:

1
2
3
4
5
{

  "prompt": "Summarize the supplied order-book evidence. Separate measured facts from statistical interpretations.",

  "max_tokens": 256,

  "temperature": 0.2

}

A browser client can process the stream with response.body.getReader():

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
const response = await fetch(FUNCTION_URL + "inference", {

  method: "POST",

  headers: {

    "content-type": "application/json"

  },

  body: JSON.stringify({

    prompt: "Summarize the approved pre-market evidence.",

    max_tokens: 256,

    temperature: 0.2

  })

});



if (!response.ok || !response.body) {

  throw new Error(`Request failed: ${response.status}`);

}



const reader = response.body.getReader();

const decoder = new TextDecoder("utf-8");

let output = "";



while (true) {

  const { value, done } = await reader.read();



  if (done) {

    break;

  }



  output += decoder.decode(value, {

    stream: true

  });



  document.querySelector("#output").textContent = output;

}



output += decoder.decode();

An ordinary browser call cannot directly use an AWS_IAM protected Function URL unless the application signs the request or uses an authenticated backend. Do not weaken authentication merely to simplify testing.

Optionally Configure API Gateway

Use API Gateway when the institution requires centralized authorization, request validation, throttling, access logging, API governance, or integration with an existing API estate.

Create a supported API and route:

1
POST /inference

Select the Lambda function as the proxy integration and configure the response transfer mode as:

1
STREAM

Apply enterprise authorization, request validation, throttling, access logging, narrow CORS, an appropriate integration timeout, and AWS WAF where supported and appropriate.

Test the complete corporate network path. Proxies and inspection products may buffer chunks even when Lambda and API Gateway stream correctly.

Enable SnapStart for the Container Function

Where supported, open:

1
2
3
4
5
Lambda

slm-streaming-inference

Configuration

SnapStart

Edit

Select:

1
Apply on: Published versions

Save, publish a numbered version, wait for optimization to finish, and create a prod alias pointing to that version. Invoke the alias rather than $LATEST.

The initialization path should download the exact S3 model version, verify its hash, load the GGUF model, start llama.cpp, run deterministic health inference, start the web application, and pass the Web Adapter readiness check before the snapshot is finalized.

State captured before a snapshot can be cloned. Temporary credentials, network connections, randomness, timestamps, and process-unique values require restore-safe handling. A new model release should produce a new Lambda version and new acceptance tests rather than silently changing an existing S3 object.

Configure Monitoring

Open:

1
2
3
4
5
CloudWatch

Metrics

Lambda

By function name

slm-streaming-inference

Monitor invocations, errors, duration, throttles, concurrency, initialization time, restoration time, memory use, model load time, hash verification time, time to first token, tokens per second, prompt tokens, generated tokens, client disconnects, model refusal rate, and evidence age.

Use structured logs such as:

1
2
3
4
5
6
7
8
9
10
{

  "requestId": "briefing-2026-09-28-001",

  "modelVersion": "llama-3.2-1b-q4-2026-09-28",

  "modelSha256": "approved-hash",

  "promptTemplateVersion": "premarket-v4",

  "timeToFirstTokenMs": 820,

  "tokensPerSecond": 18.4,

  "outputTokens": 212,

  "status": "completed"

}

Do not place client positions, customer identifiers, unrestricted prompts, proprietary alpha inputs, or secrets in ordinary logs.

Create alarms for function errors, out-of-memory termination, model-hash mismatch, increased initialization or restoration time, duration near timeout, low token throughput, throttling, concurrency exhaustion, stale evidence, unexpected model version, and policy-validation failures.

Add Concurrency Protection

Open:

1
2
3
Configuration

Concurrency

Edit

Set a conservative initial reserved concurrency, for example:

1
Reserved concurrency: 5

The final value depends on S3 model-download capacity, Lambda quotas, initialization behavior, expected demand, audit capacity, cost tolerance, and downstream protection requirements.

Add Financial-Services Controls

The inference endpoint should not place trades. The first production version should be restricted to approved evidence retrieval, pre-market summarization, model-disagreement explanation, stale-data detection, risk-context narration, source citation, and human review.

It should not receive broker credentials, exchange credentials, order-entry permissions, direct-market-access permissions, unbounded database access, or unrestricted tools.

The model must be instructed to use only supplied evidence, distinguish measurement from inference, identify stale and missing evidence, preserve quantitative uncertainty, avoid autonomous trading instructions, avoid claiming that hidden liquidity is directly visible, avoid presenting delayed off-exchange data as real time, and return a controlled output schema.

For every briefing, retain the business date, authorized user identity, evidence-manifest hash, source timestamps, model object version, model SHA-256, container digest, Lambda version, prompt-template version, decoding parameters, output hash, and human disposition.

Benchmark x86 and Arm

Create otherwise identical x86 and Arm functions and images. Measure cold initialization, SnapStart restoration, time to first token, tokens per second, total response time, maximum memory used, cost per completed response, output equivalence, and failure rate.

Standard Lambda Arm uses Graviton2. Do not claim that it automatically provides SVE from later Graviton generations. Performance depends on llama.cpp kernels, compiler settings, quantization, thread configuration, context length, and model behavior.

Validate Before Production

Run an evaluation set covering normal pre-market sessions, thin books, crossed or stale books, sequence gaps, market-data outages, high off-exchange volume, block trades, disagreement between Hawkes and queue models, high-volatility regime changes, missing risk data, prompt injection, malformed evidence, extreme values, oversized prompts, client disconnection, model initialization failure, S3 failure, KMS failure, and SnapStart restoration.

Measure factual consistency, numerical accuracy, evidence provenance, uncertainty preservation, stale-data detection, unsupported-claim rate, policy adherence, time to first token, tokens per second, and cost per accepted briefing.

Deploy first in shadow mode. Compare generated reports with qualified human briefings. Move to assisted mode only after model-risk, security, compliance, trading leadership, and operational owners approve the evidence.

Move Production Deployment to Infrastructure as Code

The console walkthrough proves the architecture. Production should use AWS CloudFormation, AWS SAM, AWS CDK, Terraform, or another approved enterprise pipeline.

Bind the following into one immutable release:

1
2
3
4
5
6
7
8
9
10
11
12
13
Container digest

Model object version

Model hash

Architecture

Lambda memory

Lambda timeout

SnapStart setting

Function URL setting

IAM role

KMS key

Prompt-template version

Evaluation report

Approval record

This prevents the model, prompt, container, and infrastructure from drifting independently.

Final Production Checklist

Before approval, verify that the exact GGUF artifact is licensed and approved, the image architecture matches Lambda, SHA-256 is checked before inference, memory and limits are benchmarked, output and context limits are enforced, Web Adapter and Function URL streaming modes match, enterprise authentication is active, S3 access follows least privilege, the model bucket is private and versioned, SnapStart restoration has been tested, each model release creates a new Lambda version, Arm and x86 are measured rather than assumed, reserved concurrency limits cost and blast radius, restricted trading information is protected, the model has no order-entry capability, deterministic controls and human review remain authoritative, alarms and incident procedures are active, and testing has traversed the actual corporate network.

References

AWS Samples, “Serverless LLM Inference: Deploy DeepSeek R1 and LLaMA Models on AWS Lambda with Ultra-Fast Cold Starts,” https://github.com/aws-samples/sample-serverless-llama-server

AWS Lambda Developer Guide, “Response streaming for Lambda functions,” https://docs.aws.amazon.com/lambda/latest/dg/configuration-response-streaming.html

AWS Lambda Developer Guide, “Invoking a response streaming enabled function using Lambda function URLs,” https://docs.aws.amazon.com/lambda/latest/dg/config-rs-invoke-furls.html

AWS Lambda Web Adapter User Guide, “Response Streaming,” https://aws.github.io/aws-lambda-web-adapter/configuration/response-streaming.html

AWS Compute Blog, “Using response streaming with AWS Lambda Web Adapter to optimize performance,” https://aws.amazon.com/blogs/compute/using-response-streaming-with-aws-lambda-web-adapter-to-optimize-performance/

AWS, “AWS Lambda now supports SnapStart for container image functions,” September 2, 2026, https://aws.amazon.com/about-aws/whats-new/2026/07/aws-lambda-snapstart-container/

AWS Lambda Developer Guide, “Lambda quotas,” https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html

AWS Lambda Developer Guide, “Selecting and configuring an instruction set architecture for your Lambda function,” https://docs.aws.amazon.com/lambda/latest/dg/foundation-arch.html

AWS Graviton Technical Guide, “AWS Lambda,” https://aws-graviton.github.io/aws-lambda/index.html

AWS, “AWS Lambda Pricing,” https://aws.amazon.com/lambda/pricing/

Series: Financial Service (19 articles)

1
AWS Lambda MicroVMs and the End of Repeated Application Cold Initialization
…
9
Set Up Small Language Model Inference on AWS Lambda Through the AWS Console This article
10
The Institutional Memory Flywheel: Building a Safer, Smarter Trading Knowledge System with Amazon S3 Vectors and S3 Metadata [ part 1 ]

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Set Up Small Language Model Inference on AWS Lambda Through the AWS Console

Series: Financial Service (19 articles)

Martin

Set Up Small Language Model Inference on AWS Lambda Through the AWS Console

Target Architecture

Important Suitability Boundary

Select the AWS Region

Choose and Prepare the Model

Create a KMS Key

Create the S3 Model Bucket

Create the ECR Repository

Prepare the Container Application

Select x86_64 or Arm64

Push the Image to ECR

Create the IAM Execution Role

Create the Lambda Container Function

Configure Memory, Timeout, and Storage

Add Environment Variables

Create the Function URL

Test the Health Endpoint

Test Streamed Inference

Optionally Configure API Gateway

Enable SnapStart for the Container Function

Configure Monitoring

Add Concurrency Protection

Add Financial-Services Controls

Benchmark x86 and Arm

Validate Before Production

Move Production Deployment to Infrastructure as Code

Final Production Checklist

References

Series: Financial Service (19 articles)