Boost Your LLM Performance on Amazon Bedrock with Self-Reflection
As developers working with Large Language Models (LLMs), we often face a fundamental tradeoff: do we use a larger, more expensive model to get better results, or accept lower accuracy to save costs? What if there was a third option: using a smaller, cost-effective model with self-reflection that can actually outperform a larger model making a single inference call? By giving models time to reflect and refine their responses, you can achieve better results at lower costs.
Applied Scientist, AWS Generative AI Innovation Centre
How It Works
GenAI Reflection for Amazon Bedrock is an open-source Python library that enhances LLM performance by enabling additional compute at inference time with just a few lines of code. While some models offer built-in thinking modes, this library provides a model-agnostic, customizable approach that works across any Amazon Bedrock model. More importantly, it lets you integrate external verification systems (like code execution or database queries) into the reflection loop, often yielding better results than isolated built-in thinking.
When we tackle difficult problems, we rarely get the perfect answer on our first try. We think, reconsider, check our work, and refine our approach. This library enables the same iterative process for LLMs: after generating an initial response, the model reviews its output, identifies potential issues, and produces a refined answer. This process can be repeated multiple times, with each iteration potentially improving the result.
We've tested this approach across multiple domains and observed substantial improvements, particularly in mathematical reasoning tasks where smaller models showed accuracy gains up to 220% with just 1-2 reflection rounds. The approach also works well for sentiment analysis and certain localization tasks, though effectiveness varies by domain (translation and text-to-SQL show mixed results).
The key insight: smaller LLMs benefit more dramatically from reflection than larger ones, and in many cases, a smaller model with 2-3 reflection rounds can match or outperform a larger, more expensive model making a single call; giving you better accuracy at lower overall cost. For detailed benchmarks and experimental results across 10 models and 4 task domains, see our research paper published at NeurIPS 2025 Efficient Reasoning Workshop.
Getting Started
Installing the library is straightforward:
Bash
pip install git+https://github.com/aws-samples/sample-genai-reflection-for-bedrock.git
Basic usage with a single model:
Python
from bhive import Hive, HiveConfigbhive_client = Hive()bhive_config = HiveConfig(bedrock_model_ids=["anthropic.claude-haiku-4-5-20251001-v1:0"],num_reflections=2,)messages = [{"role": "user", "content": [{"text": "What is 2 + 2?"}]}]response = bhive_client.converse(messages, bhive_config)
This enables the model to generate an initial response, then reflect and revise it twice. The
Hive() client acts as a wrapper for the Bedrock client from boto3 library.Key Features
External Verification
One of the powerful features is integrating external verification systems. A verifier is a custom function that checks the model's output and provides feedback—particularly useful when you can programmatically validate responses.
Example for text-to-SQL with database execution feedback:
Python
def text2sql_verifier(context: str) -> str:"""Extracts SQL and validates it against a database."""extracted_sql_query = extract_sql(context, "<SQL>")try:result = execute_sql(db_path=db_path, sql=extracted_sql_query)result_df = pd.DataFrame(result)if not result_df.empty:return f"Query succeeded. Results:\n{result_df.to_string(index=False)}"return "Query succeeded but returned no results."except Exception as e:return f"Error executing SQL: {str(e)}"bhive_config = HiveConfig(bedrock_model_ids=["anthropic.claude-haiku-4-5-20251001-v1:0"],num_reflections=2,verifier=text2sql_verifier)
The model uses this concrete feedback to refine its query in subsequent reflection rounds, creating a powerful feedback loop.
Multi-Model Collaboration
Leverage multiple models to collaboratively solve problems through debate and consensus:
Python
models = ["anthropic.claude-haiku-4-5-20251001-v1:0","amazon.nova-2-lite-v1:0","mistral.mistral-large-2402-v1:0"]bhive_config = HiveConfig(bedrock_model_ids=models,num_reflections=2,aggregator_model_id="anthropic.claude-sonnet-4-5-20250929-v1:0")
The models generate initial thoughts, reflect on each other's responses, and an aggregator synthesizes the final answer.
Automated Optimization
Not sure which configuration works best? Use the built-in optimization feature:
Python
from bhive import Hive, TrialConfigdataset = [("What is the capital of France?", "Paris"),("What is 2 + 2?", "4"),]trial_config = TrialConfig(bedrock_model_combinations=[["anthropic.claude-3-sonnet-20240229-v1:0"],["anthropic.claude-3-haiku-20240307-v1:0"],],reflection_range=[0, 1, 3],)results = Hive().optimise(dataset, trial_config)
The optimizer finds the best setup balancing accuracy, cost, and latency within your constraints.
Additional Features
- Prompt Caching: Reduces costs by avoiding redundant API calls
- Structured Outputs: Validates and formats responses into consistent structured data
- Budget Constraints: Set maximum cost and latency limits per request
Best Practices
- Start with 2-3 reflection rounds: Diminishing returns beyond 3-4 rounds for most tasks
- Test on your specific domain: Reflection effectiveness varies significantly by task (excels in math and classification, mixed results in translation/SQL)
- Implement domain-specific verifiers: Custom verification (code execution, database queries) provides the most value
- Use the optimizer: Let it find the best configuration for your use case
- Monitor costs: More reflections mean more API calls
Conclusion
Self-reflection offers a powerful alternative to always reaching for larger, more expensive models. By giving your existing models more "thinking time," you can achieve better results at lower costs, especially for tasks like mathematical reasoning where smaller models with reflection can dramatically outperform larger models without it. The GenAI Reflection library makes this approach accessible with minimal code changes.
Ready to get started? Install the library, test it on your specific use case, and experiment with different reflection configurations to find your optimal setup.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article