Building Natural Language Query Interfaces with AWS Bedrock
A practical guide to building NL-to-SQL systems for data warehouses. Covers architecture, costs, JSON querying, and when to build custom vs use managed solutions like Quick Suite.
Democratizing data access with AI-powered query translation
The Challenge: Bridging the Gap Between Business Users and Data
Organizations today sit on vast amounts of valuable data locked away in data warehouses like Amazon Redshift. While this data holds critical insights about cloud resources, costs, and operations, accessing it typically requires SQL expertise—a skill that most business users, operations teams, and executives don't possess.
Consider these common scenarios:
- A finance manager needs to understand AWS spending patterns by service and region
- An operations engineer wants to quickly find all running EC2 instances in production
- A security analyst needs to identify S3 buckets without encryption enabled
- A team lead wants to see Lambda function configurations for their team's resources
Each of these requests traditionally requires either SQL knowledge or a ticket to the data team, creating bottlenecks and delays in decision-making.
The question becomes: How can we enable anyone to query complex data warehouses using plain English?
Use Case: Natural Language Querying for AWS Resource and Cost Data
Our specific use case involves querying two types of data stored in Amazon Redshift:
- AWS Resource Configurations - EC2 instances, Lambda functions, S3 buckets with their full JSON configurations
- AWS Cost Data - Daily cost records by service, region, and resource with associated tags
Why Redshift SUPER Type?
AWS resource configurations are inherently heterogeneous—an EC2 instance has
instanceType and state, while a Lambda function has runtime and memorySize. Rather than creating separate tables or flattening every possible field into columns, we use Redshift's SUPER data type.SUPER provides:
- Schema flexibility - Store varying JSON structures without predefined schemas or migrations
- PartiQL querying - Use SQL-like dot notation to traverse JSON (e.g.,
resource_config_json.tags.Environment) - Native performance - Columnar compression and optimized storage for semi-structured data
- Up to 16MB per value - Handles large, deeply nested configurations
This lets us store a single
resource_config_json column that holds different structures per resource type, queryable with standard SQL extended by PartiQL syntax.Example Queries
Users need to ask questions like:
- "Count EC2 instances by region"
- "Show Production Lambda costs"
- "Find S3 buckets with encryption enabled"
- "Show costs for Lambda functions with more than 512MB memory" (requires joining cost and resource data)
Key Requirements
| Requirement | Description |
|---|---|
| Natural Language Input | Accept plain English queries without SQL knowledge |
| JSON Field Querying | Query nested JSON data in Redshift SUPER columns |
| Cross-Table Queries | Join resource configurations with cost data |
| Tag-Based Filtering | Filter by environment, team, cost center tags |
| Aggregations | Support COUNT, SUM, AVG with GROUP BY |
| Low Latency | Target sub-3-second response for simple queries |
| Clear Error Handling | Guide users when queries are ambiguous |
Solution Options Assessment
We evaluated several approaches for implementing natural language data warehouse querying:
Option 1: Amazon Quick Suite
Amazon Quick Suite is AWS's generative AI-powered business intelligence platform.
Fit Assessment:
| Criterion | Score | Notes |
|---|---|---|
| Natural Language Querying | ✅ High | Native Q&A capability with AI |
| Redshift Integration | ✅ High | Direct connector available |
| JSON/SUPER Column Support | ⚠️ Medium | Requires flattening (see below) |
| Custom Query Logic | ⚠️ Medium | Limited customization |
| Cost Model | ⚠️ Medium | Per-user pricing |
SUPER/JSON Limitation: Quick Suite supports only primitive data types (Date, Decimal, Integer, String). To query SUPER columns, you must flatten JSON into views or custom SQL datasets—workable for known patterns but limits ad-hoc exploration.
Best For: Turnkey BI with minimal development, where standard query patterns suffice.
Option 2: RAG — Not Recommended
Some teams consider RAG for text-to-SQL, embedding schemas in a vector database. This doesn't fit:
| Issue | Explanation |
|---|---|
| Wrong problem type | RAG retrieves text chunks; text-to-SQL needs structured translation |
| Schema is small | Schemas fit in a prompt—no retrieval needed |
| Added complexity | Vector DB adds latency without clear benefit |
Verdict: Skip RAG for text-to-SQL—it doesn't solve the right problem.
Option 3: GraphRAG — Not Recommended
GraphRAG is sometimes considered for nested JSON. Also doesn't fit:
| GraphRAG Strength | Our Use Case | Fit? |
|---|---|---|
| Multi-hop reasoning across documents | We're querying structured data | ❌ |
| Entity relationship discovery | Relationships are already defined | ❌ |
Verdict: For text-to-SQL, the schema is your graph. No discovery needed.
Option 4: Direct LLM Translation (Recommended)
Use an LLM to translate natural language to SQL with schema context in the prompt.
| Criterion | Score | Notes |
|---|---|---|
| Natural Language Querying | ✅ High | LLM handles complex queries |
| Latency | ✅ High | Single API call, <3 seconds |
| JSON/SUPER Support | ✅ High | Full PartiQL support |
| Customization | ✅ High | Complete control |
| Error Handling | ✅ High | Clear feedback for ambiguous queries |
Best For: Full control over translation, complex JSON querying, production-grade AI capabilities.
Recommendation: Direct LLM Translation
For querying AWS resource configurations and cost data with complex JSON fields, we recommend Direct LLM Translation using AWS Bedrock with Claude.
Why Not Quick Suite?
- Dynamic JSON Querying - SUPER columns need ad-hoc exploration; Quick Suite requires pre-flattening
- Custom JOIN Logic - Service-to-resource-type mapping is easier in code
- Tag Alignment - Different casing between tables handled seamlessly
- Better Error Handling - Ambiguous queries get clarification prompts instead of incorrect results
- Cost Control - Per-query vs per-user pricing
Cost Considerations
Bedrock (Claude Opus 4.5) Pricing:
- Input: ~$15 per 1M tokens
- Output: ~$75 per 1M tokens
Typical Query:
- Input: ~500 tokens (schema + query)
- Output: ~150 tokens (SQL + reasoning)
- Cost per query: ~$0.012
Monthly estimates:
- 100 queries/day: ~$36/month
- 1,000 queries/day: ~$360/month
Compare to Quick Suite pricing (varies by edition) - break-even depends on query patterns and user count.
When Quick Suite Makes Sense
- Rapid deployment without development resources
- Standard BI dashboards are the primary need
- Data doesn't require complex JSON querying
- Per-user licensing fits your organization
Architecture
Our solution uses a layered architecture:
1. Streamlit Web UI
- Query input with natural language
- Interactive results grid with generated SQL display
- Visualization charts
- CSV/JSON export
2. NL Query Engine
- Intent detection (SELECT, COUNT, AGGREGATE)
- Table selection (resource_table, cost_data)
- Result formatting
3. AWS Bedrock (Claude Opus 4.5)
- Natural language understanding
- JSON field extraction
- Ambiguity detection
- ~2-3 second latency
4. Query Translator
- Converts ParsedQuery to SQL
- Handles PartiQL for JSON fields
- Generates parameterized queries
- Maps service names for JOINs (e.g., 'AWS Lambda' → 'Lambda')
5. Amazon Redshift
resource_tablewithresource_config_json(SUPER)cost_datawithcost_json(SUPER)- Supports complex JOINs between tables
Error Handling and User Guidance
A key design principle is providing clear, actionable feedback when queries can't be processed.
Ambiguous Query Detection:
When Claude detects an ambiguous query, the system:
- Returns a specific clarification message from the AI
- Provides contextual suggestions (specify resource type, add conditions, etc.)
- Guides users toward more precise queries
Example:
- User: "Show me resources"
- System: "🤔 Your query needs clarification. Please specify which resource type (EC2, Lambda, S3) and any filters."
Other Error Scenarios:
| Error Type | User Experience |
|---|---|
| SQL Validation Failure | Shows security/safety issue with suggestions |
| Bedrock Unavailable | Clear setup instructions for AWS configuration |
| Database Connection | Troubleshooting steps for connectivity |
| Parse Errors | Tips for simpler phrasing with examples |
This approach ensures users learn how to write better queries rather than receiving incorrect results.
Scaling to Large Schemas
The approach above works for small schemas (2-10 tables). Enterprise warehouses with hundreds of tables need a different strategy.
The Challenge
| Schema Size | Tables | Fits in Prompt? |
|---|---|---|
| Small | 2-10 | ✅ Yes |
| Medium | 20-50 | ⚠️ Tight |
| Large | 100-500 | ❌ No |
Solution: Two-Stage Table Selection
Rather than sending all table schemas to the LLM, use a lightweight selection layer:
Stage 1: Identify relevant tables using keyword matching or embeddings
Stage 2: Generate SQL using only selected table schemas
Stage 2: Generate SQL using only selected table schemas
This approach:
- Keeps prompts under token limits
- Reduces latency and cost
- Scales to 1000+ tables
Trade-offs:
| Approach | Latency | Accuracy | Best For |
|---|---|---|---|
| Single-stage | Fast | High | < 10 tables |
| Two-stage keyword | Fast | Medium | 10-100 tables |
| Two-stage with LLM | Slower | High | 100+ tables |
Why Not Fine-Tuning?
Fine-tuning is sometimes suggested for text-to-SQL. For schema scaling, it's the wrong tool:
| Concern | Fine-Tuning | Table Selection |
|---|---|---|
| Schema changes | Retrain model | Update catalog |
| Time to implement | Days/weeks | Hours |
| Cost | High (training + hosting) | Near zero |
| Maintenance | Ongoing retraining | Simple config |
Fine-tuning helps when the LLM struggles with query patterns (complex joins, domain-specific syntax). Schema scaling is a context window problem—table selection solves it directly.
Key Implementation Insights
1. JSON Field Inconsistencies Are the Real Challenge
The hardest part isn't the AI—it's handling data inconsistencies. Tags exist in both tables with different casing:
1
2
# cost_data: cost_json.tags.environment (lowercase)
# resource_table: resource_config_json.tags.Environment (CamelCase)Solution: When joining tables, filter both with their respective conventions. This requires understanding your data sources deeply.
2. Redshift SUPER Needs Special Configuration
Enable case-sensitive JSON access:
1
SET enable_case_sensitive_super_attribute TO true;Without this,
tags.Environment and tags.environment are treated as the same field, causing silent failures.3. Service Name Mapping Requires Business Logic
Cost data uses
Amazon EC2, resource data uses EC2. JOIN conditions need CASE statements to map service names to resource types—this is domain knowledge the LLM can't infer.4. Tag Structure Variations
In our implementation, EC2 and S3 tags are stored as arrays (
[{"key": "Environment", "value": "Production"}]), while Lambda uses objects ({"Environment": "Production"}). This matches how we modeled the data from AWS Config.Our current implementation supports Lambda tags directly. For EC2/S3 array tags, you'd need to either:
- Flatten tags into separate columns via views
- Implement array iteration in the query translator
- Focus on Lambda resources for tag-based queries
Note: Actual AWS Config format may vary—verify your specific data structure.
What We Learned
1. The AI Isn't the Hard Part
Modern LLMs like Claude handle text-to-SQL remarkably well. The real challenges are:
- Data inconsistencies (tag casing, service name variations)
- Domain-specific business logic (how services map to resources)
- Edge cases in your schema (nullable fields, nested arrays)
2. Error Handling Is a Product Decision
Silent fallbacks that return incorrect results are worse than clear error messages. When queries are ambiguous, guide users toward better questions rather than guessing intent.
3. Start Small, Validate Early
Don't build for 100 tables on day one. Start with 2-3 tables, test against real user queries, and iterate. You'll discover data quality issues and edge cases that no amount of planning reveals.
4. Bedrock as a Requirement Simplifies Everything
For prototypes, requiring AI availability keeps scope focused. You're demonstrating AI capabilities—own that dependency rather than building elaborate fallback systems.
Conclusion
Natural language interfaces for data warehouses are no longer experimental—they're practical with the right architecture choices.
Key Takeaways:
- Skip RAG and GraphRAG for text-to-SQL—they solve different problems
- Evaluate managed solutions first (Quick Suite) before building custom
- Build custom when you need dynamic JSON querying, complex business logic, or full control
- Plan for scale early if you have 20+ tables—two-stage selection is straightforward
- Prioritize error handling over silent fallbacks—guide users, don't guess
The technology is ready. The question is whether your data is ready—schema consistency, clear relationships, and well-defined business logic matter more than the AI model you choose.
What's Next?
If you're building something similar:
- Start with your top 10 query patterns
- Test against real data early (you'll find surprises)
- Show the generated SQL for transparency
- Iterate on prompts based on what works and what doesn't
The future of data access isn't about teaching everyone SQL—it's about making data as accessible as asking a question.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article