Trying to Bedrock Advanced Prompt Optimization
On May 14, 2026, Amazon Bedrock's new feature "Advanced Prompt Optimization" reached GA. It's a tool that automatically optimizes prompts while comparing up to five models in parallel — by specifying a reference response, it tunes prompts so the output moves closer to your target. I tried it with Claude Haiku 4.5 and Sonnet 4.5, and verbose Markdown outputs were reshaped into concise prose. It also looks useful as a way to decide which model is the right fit for a given task.
What is Advanced Prompt Optimization?
On 2026/05/14, Bedrock's new feature "Advanced Prompt Optimization" was announced and made generally available (GA).
Update information: https://aws.amazon.com/about-aws/whats-new/2026/05/amazon-bedrock-advanced-prompt-optimization-migration-tool/ Official blog: https://aws.amazon.com/blogs/aws/amazon-bedrock-introduces-new-advanced-prompt-optimization-and-migration-tool/
This appears to be a tool that allows you to optimize prompts while comparing up to 5 models on Bedrock.
Trying it out
Let's actually try it out. The official blog mentioned that the Tokyo region had also reached GA, but since it wasn't displayed in the management console, I'll try it in the N. Virginia region.
A menu has been added to the Build section of the Bedrock console! Let me click on
Create prompt optimization right away.This time, I'll run it with
Claude Haiku 4.5 and Claude Sonnet 4.5. For the input file, you need to specify a jsonl file either via an S3 path or by uploading directly.The format is as follows:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
{
"version": "bedrock-2026-05-14", // required; Fixed value
"templateId": "string", // required
"promptTemplate": "string", // required
"steeringCriteria": ["string"], // optional
"customEvaluationMetricLabel": "string", // required if customLLMJConfig or evaluationMetricLambdaArn is used
"customLLMJConfig": { // optional
"customLLMJPrompt": "string", // required if customLLMJConfig present
"customLLMJModelId": "string" // required if customLLMJConfig present
},
"evaluationMetricLambdaArn": "string", // optional
"evaluationSamples": [ // required
{
"inputVariables": [ // required
{
"variableName1": "string",
"variableName2": "string"
}
],
"referenceResponse": "string" // optional
"inputVariablesMultimodal": [ // optional
{
"Arbitrary_Name": { // required for your multimodal variable.
"type": "string", // choose from "PDF" or "IMAGE". Acceptable filetypes for IMAGE = png, jpg,
"s3Uri": "string" // input the S3 path of the file
}
]
}
]
}
This time, I prepared a prompt like the following:
:::note The jsonl file needs to be on a single line, so please remove line breaks before uploading. :::
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
{
"version": "bedrock-2026-05-14",
"templateId": "haiku-sonnet-test",
"promptTemplate": "Write a description about {{aws_service}}",
"evaluationSamples": [
{
"inputVariables": [
{
"aws_service": "EC2"
}
],
"referenceResponse": "Amazon EC2 (Elastic Compute Cloud) is a virtual server service in the cloud provided by AWS. You can launch servers with the required specifications within minutes whenever needed, and you only pay for what you use. You can freely choose the OS and instance type, and scaling up and down can be done flexibly."
},
{
"inputVariables": [
{
"aws_service": "VPC"
}
],
"referenceResponse": "Amazon VPC (Virtual Private Cloud) is a service that allows you to build a private virtual network on AWS. You can freely configure IP address ranges, subnets, and routing, and it also supports connections to the internet and on-premises environments. It is a network foundation for securely isolating and managing AWS resources such as EC2."
}
]
}
It was created successfully!
:::note It takes surprisingly long... it took me a little over 10 minutes. :::
There are tabs for
Graph, Summary tables, and Detailed results. Let me look at the Graph first.
Cost optimization is not displayed — apparently you need to manually configure the cost settings!
In
Summary tables, you can check scores and other metrics for the original and optimized prompts.
The optimized prompt for Haiku:
1
Write a description about {{aws_service}}. The response should be a single concise paragraph of 3-5 sentences. Do not use markdown formatting such as headings, bullet points, tables, or code blocks. Explain the service overview, main value/benefits, and typical use cases in natural Japanese prose.
The optimized prompt for Sonnet:
1
2
3
4
5
6
7
8
Please describe Amazon {{aws_service}} (AWS service) in concise prose of 2-4 sentences.
Requirements:
- Always begin with the name "Amazon {{aws_service}}"
- Explain only the functions, features, and uses as an AWS service
- Do not mention other cloud providers (Google Cloud, Azure, etc.)
- Do not use Markdown headings (#), bullet points (-・), bold text (**), or emojis
- Write the response in plain paragraph form (approximately 100-200 characters)
:::note It's interesting that Haiku had a higher score in the original... :::
In
Detailed results, you can check the actual output for each prompt.
Here are the responses from Haiku for EC2, before and after optimization:
Before optimization:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
About Amazon EC2 (Elastic Compute Cloud)
# Overview
Amazon EC2 is a cloud computing service provided by AWS. You can rent virtual servers (instances) over the internet to run applications.
# Main Features
## 1. **Scalability**
- Increase or decrease the number of servers as needed
- Flexibly respond to traffic fluctuations
## 2. **Pay-as-you-go pricing**
- Pay only for what you use
- No initial investment required
## 3. **Variety of instance types**
- General purpose (t3, m5, etc.)
- Compute optimized (c5, etc.)
- Memory optimized (r5, etc.)
- GPU-equipped types, etc.
# Main Functions
| Function | Description |
|------|------|
| **AMI** | Instance launch template |
| **Security Group** | Firewall function |
| **Elastic IP** | Static public IP address |
| **EBS** | Block storage |
| **Auto Scaling** | Automatic scaling |
# Pricing Structure
- **On-demand**: Pay-as-you-go by the hour
- **Reserved Instances**: Discounts for long-term contracts
- **Spot Instances**: Use surplus capacity at low cost
# Use Cases
- Web application hosting
- Database servers
- Batch processing
- Development and testing environments
EC2 is one of the core services of AWS, used by many companies.
After optimization:
1
2
Amazon EC2 (Elastic Compute Cloud) is a cloud-based virtual server service provided by AWS, allowing you to use compute resources on-demand over the internet. Its main value lies in the flexibility of requiring no initial investment, being able to add and remove servers as needed, and paying only for what you use, and it also enables low-latency service delivery by choosing from data centers around the world. It is used by companies of various scales and purposes, including running web applications and batch processing systems, building development environments, and auto-scaling for traffic fluctuations, with a wide range of organizations from startups to large enterprises leveraging it to streamline infrastructure construction and operations.
Here are the responses from Sonnet for EC2, before and after optimization:
Before optimization:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
EC2 is a virtual server service on the cloud provided by AWS. It is a scalable service that allows you to use computing resources as needed, when needed.
- New servers can be launched in minutes
- Can scale up and down as needed
- Compatible with various OSs and applications
- **General purpose** (t2, t3, m5, etc.): Balanced performance
- **Compute optimized** (c5, etc.): CPU performance focused
- **Memory optimized** (r5, etc.): For when large memory is needed
- **Storage optimized** (i3, etc.): High-speed storage access
- **GPU-equipped** (p3, g4, etc.): Machine learning and graphics processing
- **On-demand**: Pay for what you use
- **Reserved Instances**: Discounts for long-term contracts
- **Spot Instances**: Use surplus resources at very low cost
- **Savings Plans**: Flexible discount plans
- Web servers and application servers
- Development and testing environments
- Data processing and analysis
- Machine learning model training
- Batch processing
EC2 is a foundational service for cloud computing and is used in many systems.
After optimization:
1
Amazon EC2 is a computing service from AWS that allows you to use virtual servers in the cloud. You can flexibly change the number of servers and specifications as needed, and they can be started and stopped within minutes, allowing you to build systems while keeping initial investment low. It is used for a wide range of purposes, including web application hosting, data processing, and development environments.
You can see that both have had their prompts adjusted so that the responses are generated closer to what was specified in the referenceResponse!
Summary
I actually tried out Bedrock's new feature "Advanced Prompt Optimization". Prompt engineering takes quite a lot of time when you try to do it seriously. By using this feature, it seems easy to create prompts working backwards from the final output! It also seems useful as a criterion for judging whether Haiku is sufficient for a certain level of processing, or whether Sonnet is necessary!!
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article