Amazon Bedrock Batch Inference with Nova Models: Quick Guide

How to efficiently process large volumes of data using Nova models with Amazon Bedrock's batch inference capability

Jean Malha

Published Oct 17, 2025

Last modified Oct 17, 2025

Overview

This comprehensive guide will teach you how to implement batch inference with Amazon Nova models on Amazon Bedrock, enabling you to process large datasets efficiently while achieving significant cost savings. You'll learn to set up the complete infrastructure, prepare your data, create and monitor batch jobs, and handle the results.

What you'll learn:

Understanding batch inference concepts and benefits
Setting up AWS resources (S3, IAM roles, and policies)
Preparing and formatting input data in JSONL format
Creating and configuring batch inference jobs
Monitoring job progress and handling results
Best practices for cost optimization and error handling

Estimated reading time: 15-20 minutes
Implementation time: 30-45 minutes

Prerequisites & Skill Level

Skill Level: Intermediate
Required Experience:

Basic familiarity with AWS services (S3, IAM, Bedrock)
Understanding of Python programming
Experience with AWS CLI or SDK (boto3)
Knowledge of JSON data formats

Technical Requirements:

AWS account with appropriate permissions
Python 3.7+ environment
boto3 library installed
Basic understanding of machine learning inference concepts

Who Should Read This

This guide is designed for:

Primary Audience:

ML Engineers & Data Scientists looking to scale their inference workloads cost-effectively
Backend Developers integrating AI capabilities into applications with batch processing needs
Solutions Architects designing scalable AI/ML systems on AWS

Secondary Audience:

DevOps Engineers implementing AI/ML infrastructure
Technical Product Managers understanding batch inference capabilities and costs
AWS practitioners expanding their knowledge of Bedrock services

Use Cases:

Processing large datasets for content generation
Batch analysis of customer feedback or documents
Automated report generation
Data transformation and enrichment workflows
Research and experimentation with large prompt sets

Introduction

Amazon Bedrock's batch inference capability allows you to process large volumes of data efficiently and cost-effectively using Nova models. Batch inference is ideal when you need to process multiple prompts asynchronously and can achieve up to 50% cost savings compared to real-time inference, with guaranteed results within 24 hours.

What is Batch Inference?

Batch inference enables you to submit multiple prompts in a single job and receive responses asynchronously. Instead of making individual API calls for each prompt, you can upload a file containing all your prompts to Amazon S3 and let Bedrock process them in batches

Learn more: https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html

Supported Nova Models

Amazon Nova models that support batch inference include:

Nova Lite: Optimized for speed and cost-effectiveness
Nova Micro: Ultra-fast inference for simple tasks
Nova Pro: Advanced reasoning capabilities for complex tasks

Each model has specific quotas and limits for batch inference jobs .

Full list of supported morels and regions: https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-supported.html

Prerequisites

Before creating a batch inference job, ensure you have:

AWS Account: With appropriate permissions for Amazon Bedrock and S3
S3 Bucket: For storing input and output files
Input Data: Formatted as JSONL (JSON Lines) files

Input Data Format

JSONL File Structure

Your input file must be in JSONL format, where each line contains a separate JSON object representing one inference request :

1
2
{"recordId": "record_0", "modelInput": {"messages": [{"role": "user", "content": [{"text": "tell me a joke"}]}], "system": [{"text": "You should respond to all messages in french"}], "inferenceConfig": {"max_new_tokens": 300, "top_p": 0.9, "top_k": 20, "temperature": 0.7}}}
{"recordId": "record_1", "modelInput": {"messages": [{"role": "user", "content": [{"text": "tell me a joke"}]}], "system": [{"text": "You should respond to all messages in french"}], "inferenceConfig": {"max_new_tokens": 300, "top_p": 0.9, "top_k": 20, "temperature": 0.7}}}

Key Components

recordId: Unique identifier for each request that enables tracking and referencing of individual API callsmodelInput: Container object that holds all the input parameters and configuration for the model request, including:

messages: Array of conversation messages following the standard chat format with role-based structure (user, assistant, system)
system: Array of system-level instructions that define the model's behavior and response guidelines
inferenceConfig: Configuration object containing model parameters that control response generation:
- max_new_tokens: Maximum number of tokens to generate in the response
- top_p: Nucleus sampling parameter for controlling response diversity
- top_k: Top-k sampling parameter limiting token selection
- temperature: Controls randomness in response generation

https://docs.aws.amazon.com/nova/latest/userguide/complete-request-schema.html

Constraints

Maximum file size: 1 GB per JSONL file
Minimum records: 100 records per batch job
Maximum records: 50,000 records per file

Sample implementation

Setup and Configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import boto3
import json
import random
import string

region = 'us-east-1'
model = 'us.amazon.nova-lite-v1:0'

# Generate unique identifier for resources
def generate_random_string(length=10):
    return ''.join(random.choice(string.ascii_lowercase) for _ in range(length))

random_string = generate_random_string()
print(f"Using random identifier: {random_string}")
session = boto3.Session()
# Initialize AWS clients
s3_client = session.client('s3', region_name=region)
sts_client = session.client('sts', region_name=region)
iam_client = session.client('iam')
bedrock_client = session.client('bedrock', region_name=region)

# Get account ID
account_id = sts_client.get_caller_identity()['Account']
print(f"Account ID: {account_id}")

Creation of resources

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
bucket_name = f'bedrockbatchdemo{region}{random_string}'

response = s3_client.create_bucket(Bucket=bucket_name)
print(f"Created S3 bucket: {bucket_name}")
role_name = f'BedrockServiceRole{random_string}'
role_description = 'Service role for Bedrock batch processing'

assume_role_policy = {
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Principal": {"Service": "bedrock.amazonaws.com"},
        "Action": "sts:AssumeRole",
        "Condition": {
            "StringEquals": {"aws:SourceAccount": account_id},
            "ArnEquals": {"aws:SourceArn": f"arn:aws:bedrock:*:{account_id}:model-invocation-job/*"}
        }
    }]
}

role_response = iam_client.create_role(
    RoleName=role_name,
    AssumeRolePolicyDocument=json.dumps(assume_role_policy),
    Description=role_description
)

role_arn = role_response['Role']['Arn']
print(f"Created IAM role: {role_arn}")
policy_name = f'BedrockServiceRolePolicy{random_string}'
policy_description = 'Policy for Bedrock batch processing S3 access'

policy_document = {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"],
            "Resource": [
                f"arn:aws:s3:::{bucket_name}",
                f"arn:aws:s3:::{bucket_name}/*"
            ],
            "Condition": {"StringEquals": {"s3:ResourceAccount": account_id}}
        },
         {
            "Sid": "BatchInference",
            "Effect": "Allow",
            "Action": [  
                "bedrock:ListFoundationModels",
                "bedrock:GetFoundationModel",
                "bedrock:ListInferenceProfiles",
                "bedrock:GetInferenceProfile",
                "bedrock:ListCustomModels",
                "bedrock:GetCustomModel",
                "bedrock:TagResource", 
                "bedrock:UntagResource", 
                "bedrock:ListTagsForResource",
                "bedrock:CreateModelInvocationJob",
                "bedrock:GetModelInvocationJob",
                "bedrock:ListModelInvocationJobs",
                "bedrock:StopModelInvocationJob",
                "bedrock:Invoke*"
            ],
            "Resource": "*"
        }
    ]
}

policy_response = iam_client.create_policy(
    PolicyName=policy_name,
    PolicyDocument=json.dumps(policy_document),
    Description=policy_description
)

policy_arn = policy_response['Policy']['Arn']
print(f"Created IAM policy: {policy_arn}")

# Attach policy to role
iam_client.attach_role_policy(
    RoleName=role_name,
    PolicyArn=policy_arn
)
print("Policy attached to role successfully")

Preparation of the data

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# Sample data preparation
prompts = [f"Write a short story about {topic}" for topic in ["space", "ocean", "forest", "city", "mountain", "desert", "island", "cave", "river", "garden"] * 10]

# Default system prompt (can be customized)
default_system_prompt = "You are a helpful AI assistant. Please provide accurate and informative responses."

# Default inference configuration (can be customized)
default_inference_config = {
    "max_new_tokens": 300,
    "top_p": 0.9,
    "top_k": 20,
    "temperature": 0.7
}
filename='batch_input'+random_string+'.jsonl'
# Create JSONL file
with open(filename, 'w') as f:
    for i, prompt in enumerate(prompts):
        record = {
            "recordId": f"request-{i+1}",
            "modelInput": {
                "messages": [
                    {
                        "role": "user",
                        "content": [{"text": prompt}]
                    }
                ],
                "system": [{"text": default_system_prompt}],
                "inferenceConfig": default_inference_config
            }
        }
        f.write(json.dumps(record) + "\n")

print(f"Created batch input with {len(prompts)} records")

input_file = filename
s3_client.upload_file(input_file, bucket_name, input_file)
print(f"Uploaded {input_file} to S3 bucket {bucket_name}")

Run the job

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# Create batch processing job
job_name = f'demo-job{random_string}-2'
model_id = f'arn:aws:bedrock:{region}:{account_id}:inference-profile/{model}'

response = bedrock_client.create_model_invocation_job(
    jobName=job_name,
    roleArn=role_arn,   
    modelId=model_id,
    inputDataConfig={
        's3InputDataConfig': {
            's3InputFormat': 'JSONL',
            's3Uri': f's3://{bucket_name}/{input_file}'
        }
    },
    outputDataConfig={
        's3OutputDataConfig': {
            's3Uri': f's3://{bucket_name}/output/'
        }
    },
    timeoutDurationInHours=24
)

job_arn=response['jobArn']

print(f"Created batch job: {job_name}")
print(f"Job ARN: {response['jobArn']}")

Monitor the job

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import time

# Monitor job status
while True:
    jobstatus = bedrock_client.get_model_invocation_job(jobIdentifier=job_arn)
    status = jobstatus['status']
    print(f"Job status: {status}")
    
    if status in ['Scheduled', 'Failed', 'Completed', 'PartiallyCompleted', 'Stopped', 'Expired']:
        break
    
    time.sleep(30)

print(f"Final status: {status}")
if 'message' in jobstatus:
    print(f"Message: {jobstatus['message']}")

# amazon-bedrock # batch-processing # amazon-nova

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Amazon Bedrock Batch Inference with Nova Models: Quick Guide

Jean Malha

Overview

Prerequisites & Skill Level

Who Should Read This

Introduction

What is Batch Inference?

Supported Nova Models

Prerequisites

Input Data Format

JSONL File Structure

Key Components

Constraints

Sample implementation

Setup and Configuration

Creation of resources