From Lakehouse to GenAI: Query Apache Iceberg Data Using Amazon Bedrock and Athena

Learn how to build a governed natural-language query layer on top of an Apache Iceberg lakehouse using Amazon Bedrock, AWS Glue, Amazon Athena, and RAG. This architecture combines semantic metadata, NL2SQL, SQL validation, and data governance to let users interact with enterprise data using natural language while keeping the underlying data platform secure and auditable.

Gunal K

Published Sep 23, 2026

Last modified Sep 23, 2026

From Lakehouse to GenAI: Query Apache Iceberg Data with Amazon Bedrock and Athena

Modern data platforms make it easier to store, process, govern, and analyze large volumes of enterprise data.

But there is still a gap between having governed analytical data available in a lakehouse and making that data accessible to people who do not write SQL.

A business user may not want to ask:

"Which table contains the revenue metric, and which date column should I use?"

They may simply ask:

"Show me monthly revenue by region for Q2 and identify regions where revenue declined by more than 10% month over month."

That is a very different interaction model.

Instead of requiring users to understand databases, schemas, joins, and SQL, we can provide a natural-language interface over the existing governed data platform.

In this article, we will build a reference architecture that combines:

Apache Iceberg
Amazon S3
AWS Glue Data Catalog
Amazon Athena
Amazon Bedrock
Semantic metadata
Retrieval-Augmented Generation (RAG)
Natural Language to SQL (NL2SQL)
SQL validation
IAM
AWS Lake Formation
Amazon CloudWatch

The goal is not to replace the data platform with an LLM.

The goal is to make the existing data platform easier to interact with while preserving security, governance, validation, observability, and auditability.

Architecture Overview

The architecture separates the solution into two major layers.

Data foundation

The data foundation consists of:

Amazon S3
Apache Iceberg
AWS Glue Data Catalog
Amazon Athena
AWS Lake Formation

GenAI interaction layer

The GenAI layer consists of:

Application/UI
Amazon Bedrock
Semantic metadata
RAG
NL2SQL
SQL validation
Result explanation
Observability and audit logging

The high-level flow is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
                         ┌──────────────────────┐
                         │        User          │
                         │ Natural Language     │
                         │      Question        │
                         └──────────┬───────────┘
                                    │
                                    ▼
                         ┌──────────────────────┐
                         │   Application / UI   │
                         └──────────┬───────────┘
                                    │
                                    ▼
                         ┌──────────────────────┐
                         │   Amazon Bedrock     │
                         │   Foundation Model   │
                         └──────────┬───────────┘
                                    │
                     ┌──────────────┴──────────────┐
                     │                             │
                     ▼                             ▼
            ┌──────────────────┐        ┌──────────────────┐
            │ Semantic Metadata│        │    NL2SQL        │
            │ Retrieval / RAG  │───────►│    Generation    │
            └────────┬─────────┘        └────────┬─────────┘
                     │                           │
                     │                           ▼
                     │                 ┌──────────────────┐
                     │                 │  SQL Validation  │
                     │                 └────────┬─────────┘
                     │                          │
                     │                          ▼
                     │                 ┌──────────────────┐
                     │                 │  Amazon Athena   │
                     │                 └────────┬─────────┘
                     │                          │
                     │                          ▼
                     │                 ┌──────────────────┐
                     │                 │ Apache Iceberg   │
                     │                 │      Tables      │
                     │                 └────────┬─────────┘
                     │                          │
                     │                          ▼
                     │                 ┌──────────────────┐
                     │                 │   Amazon S3      │
                     │                 └──────────────────┘
                     │
                     ▼
             AWS Glue Data Catalog

        IAM + Lake Formation + CloudWatch
               provide governance

The key architectural principle is:

The LLM generates and interprets. The governed data platform remains responsible for authorization, query execution, and data access.

The LLM should never become the security boundary.

From an Iceberg Lakehouse to a GenAI Interface

My previous article focused on building auto-refreshing analytics using AWS Glue, Apache Iceberg, and materialized views.

That architecture addressed an important question:

How do we maintain a fresh analytical data foundation?

The next question is:

How do we make that governed data easier for people to interact with?

The progression looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
AWS Glue
    │
    ▼
Apache Iceberg
    │
    ▼
Amazon Athena
    │
    ▼
Governed Analytics
    │
    ▼
Semantic Metadata
    │
    ▼
RAG
    │
    ▼
Amazon Bedrock
    │
    ▼
Natural Language → SQL
    │
    ▼
Validated Query
    │
    ▼
Governed Results

This creates a separation between the data foundation and the AI interface.

What Are We Building?

Consider a simplified sales dataset.

Our Iceberg table is:

1
analytics.sales_iceberg

with the following columns:

1
2
3
4
5
6
transaction_id
transaction_date
customer_id
product_id
region
revenue

A user asks:

"Show monthly revenue by region for Q2 2026 and identify regions with more than a 10% month-over-month decline."

The system needs to determine:

Which table contains the required data?
Which column represents revenue?
Which column represents region?
Which column represents the transaction date?
What does "Q2 2026" mean?
How should monthly revenue be calculated?
How should month-over-month change be calculated?
Which regions crossed the 10% decline threshold?

Rather than providing the model with an entire enterprise schema, we first retrieve the relevant semantic metadata.

The model then uses that context to generate SQL.

Why RAG + NL2SQL?

A basic natural-language analytics architecture looks like:

1
2
3
4
5
6
7
User Question
      │
      ▼
     LLM
      │
      ▼
     SQL

This may work for a small schema.

It becomes much harder when the organization has hundreds or thousands of tables.

For example, the word "revenue" could refer to:

gross revenue
net revenue
recognized revenue
subscription revenue
product revenue

The model needs business context, not just column names.

Consider this metadata:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
  "table": "analytics.sales_iceberg",
  "description": "Customer sales transactions",
  "business_domain": "Sales",
  "columns": [
    {
      "name": "revenue",
      "description": "Net recognized revenue",
      "business_terms": [
        "revenue",
        "net sales",
        "sales"
      ]
    }
  ]
}

This additional context gives the model a much better representation of how the organization defines its data.

The architecture becomes:

1
2
3
4
5
6
7
8
9
10
11
12
13
Natural Language Question
          │
          ▼
   Metadata Retrieval
          │
          ▼
 Relevant Tables + Columns
          │
          ▼
     Amazon Bedrock
          │
          ▼
      SQL Generation

RAG does not guarantee correct SQL.

It provides the model with more relevant and controlled context.

That distinction is important.

Managed Bedrock Knowledge Bases vs. Custom RAG

There is an important AWS capability to understand here.

Amazon Bedrock Knowledge Bases now support structured data scenarios, including AWS Glue Data Catalog with AWS Lake Formation. AWS documents a managed path where natural-language queries can be converted into SQL against supported structured data stores. AWS also provides a GenerateQuery operation for generating SQL independently.

However, this article demonstrates a custom semantic-metadata/RAG + NL2SQL + Athena execution pattern.

Why?

Because it gives the application explicit control over:

semantic metadata retrieval
table and column allow-lists
prompt construction
SQL generation
SQL parsing
validation
Athena execution
query limits
audit logging
application-specific business rules

These are useful controls when building a governed enterprise data application.

The managed Bedrock structured-data capability and the custom architecture are therefore complementary patterns rather than mutually exclusive approaches.

Designing the Semantic Layer

The semantic layer is one of the most important parts of the architecture.

A database schema might tell us:

1
revenue DECIMAL

A semantic layer can tell us:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
revenue

Business definition:
Net recognized revenue from completed transactions.

Business terms:
revenue
net sales
sales

Domain:
Sales

Allowed aggregation:
SUM

This additional context helps translate business language into technical structures.

A more complete metadata document could look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
  "table": "analytics.sales_iceberg",
  "description": "Customer sales transactions",
  "business_domain": "Sales",
  "columns": [
    {
      "name": "transaction_date",
      "type": "date",
      "description": "Date on which the transaction was recorded"
    },
    {
      "name": "region",
      "type": "string",
      "description": "Geographical sales region"
    },
    {
      "name": "revenue",
      "type": "decimal",
      "description": "Net recognized revenue",
      "business_terms": [
        "revenue",
        "net sales",
        "sales"
      ]
    }
  ]
}

The semantic layer can also contain:

table relationships
join keys
business definitions
synonyms
sensitive-data classifications
example questions
data ownership
business domains
aggregation rules
query usage notes

This is where the solution starts moving beyond simple schema retrieval toward a real enterprise semantic layer.

Building the RAG Layer

The general workflow is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Semantic Metadata
       │
       ▼
Document Preparation
       │
       ▼
Chunking / Indexing
       │
       ▼
Retrieval Layer
       │
       ▼
Relevant Metadata
       │
       ▼
SQL Generation

Amazon Bedrock Knowledge Bases can be used as part of the retrieval architecture.

For this custom pattern, the retrieval corpus can contain:

table descriptions
column descriptions
business terminology
relationships
business rules
data definitions
sample questions

For example:

User question

Which regions had a revenue decline greater than 10%?

Retrieved context

1
2
3
4
5
6
7
8
9
10
11
12
13
Table:
analytics.sales_iceberg

Relevant columns:
region
transaction_date
revenue

Revenue definition:
Net recognized revenue.

Business rule:
Monthly revenue should be calculated using SUM(revenue).

The retrieved context is then passed to the SQL generation step.

Generating SQL with Amazon Bedrock

The foundation model receives three major inputs:

User question
Retrieved semantic context
SQL generation rules

A simplified prompt can look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
You are a SQL generation assistant.

Generate read-only Amazon Athena SQL.

Rules:

1. Only use tables provided in the retrieved metadata.
2. Only use columns provided in the retrieved metadata.
3. Do not invent tables or columns.
4. Do not generate INSERT, UPDATE, DELETE, DROP,
   ALTER, CREATE, or TRUNCATE statements.
5. Use Athena-compatible SQL.
6. Apply appropriate date filters.
7. Follow the provided business definitions.
8. Avoid SELECT * unless explicitly required.
9. Return only the SQL statement.

User question:

{question}

Retrieved metadata:

{metadata}

The important part is that the model is not asked to guess the data model.

It receives the relevant data model as context.

Example NL2SQL

For the question:

Show monthly revenue by region for Q2 2026.

The generated SQL could resemble:

1
2
3
4
5
6
7
8
9
10
11
12
13
SELECT
    region,
    date_trunc('month', transaction_date) AS month,
    SUM(revenue) AS monthly_revenue
FROM analytics.sales_iceberg
WHERE transaction_date >= DATE '2026-04-01'
  AND transaction_date < DATE '2026-07-01'
GROUP BY
    region,
    date_trunc('month', transaction_date)
ORDER BY
    month,
    region;

For a month-over-month comparison, a window function can be used:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
WITH monthly_revenue AS (
    SELECT
        region,
        date_trunc('month', transaction_date) AS month,
        SUM(revenue) AS revenue
    FROM analytics.sales_iceberg
    WHERE transaction_date >= DATE '2026-04-01'
      AND transaction_date < DATE '2026-07-01'
    GROUP BY
        region,
        date_trunc('month', transaction_date)
),

with_previous_month AS (
    SELECT
        region,
        month,
        revenue,
        LAG(revenue) OVER (
            PARTITION BY region
            ORDER BY month
        ) AS previous_month_revenue
    FROM monthly_revenue
)

SELECT
    region,
    month,
    revenue,
    previous_month_revenue,
    CASE
        WHEN previous_month_revenue IS NULL
             OR previous_month_revenue = 0
        THEN NULL
        ELSE
            ((revenue - previous_month_revenue)
              / previous_month_revenue) * 100
    END AS mom_change_percentage
FROM with_previous_month
ORDER BY
    month,
    region;

The exact SQL generated by a model should always be validated before execution.

SQL Validation: The Critical Security Boundary

One of the most important principles in an NL2SQL system is:

Never trust generated SQL simply because it came from an LLM.

Prompt instructions are useful.

They are not a security boundary.

The application should validate the generated SQL before sending it to Athena.

At minimum, the validation layer should detect operations such as:

1
2
3
4
5
6
7
INSERT
UPDATE
DELETE
DROP
ALTER
CREATE
TRUNCATE

The application should also restrict queries to approved catalogs, databases, schemas, tables, and columns.

A simplified educational validator could look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import re

FORBIDDEN_SQL = [
    "INSERT",
    "UPDATE",
    "DELETE",
    "DROP",
    "ALTER",
    "CREATE",
    "TRUNCATE"
]

def validate_sql(sql: str) -> None:
    normalized = sql.strip().upper()

    if not normalized.startswith("SELECT"):
        raise ValueError(
            "Only SELECT statements are permitted."
        )

    for keyword in FORBIDDEN_SQL:
        pattern = rf"\b{keyword}\b"

        if re.search(pattern, normalized):
            raise ValueError(
                f"Forbidden SQL operation detected: {keyword}"
            )

This example is intentionally simple.

Production recommendation

Do not treat regular expressions as your only SQL security mechanism.

A production implementation should use:

SQL parser / AST validation
allow-listed databases
allow-listed tables
allow-listed columns
read-only execution roles
Athena workgroups
query timeout controls
query result restrictions
Lake Formation permissions
IAM least privilege
application-level authorization

The validation layer should therefore look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Generated SQL
     │
     ▼
SQL Parser
     │
     ▼
AST Validation
     │
     ├── Forbidden operation? ──► Reject
     │
     ├── Unauthorized table? ───► Reject
     │
     ├── Unauthorized column? ──► Reject
     │
     ▼
Query Controls
     │
     ▼
Amazon Athena

Executing the Query with Amazon Athena

After validation, the application can submit the query to Athena.

A simplified Python example using boto3:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
import boto3
import time

athena = boto3.client("athena")

def execute_query(
    sql: str,
    database: str,
    output_location: str
):
    response = athena.start_query_execution(
        QueryString=sql,
        QueryExecutionContext={
            "Database": database
        },
        ResultConfiguration={
            "OutputLocation": output_location
        }
    )

    query_execution_id = response["QueryExecutionId"]

    while True:
        result = athena.get_query_execution(
            QueryExecutionId=query_execution_id
        )

        state = result["QueryExecution"]["Status"]["State"]

        if state in [
            "SUCCEEDED",
            "FAILED",
            "CANCELLED"
        ]:
            break

        time.sleep(1)

    if state != "SUCCEEDED":
        reason = result["QueryExecution"]["Status"].get(
            "StateChangeReason",
            "Unknown Athena error"
        )

        raise RuntimeError(reason)

    return query_execution_id

Athena uses the AWS Glue Data Catalog for metadata such as table and column definitions, while Iceberg provides the table format over data stored in S3.

In production, additional controls should be implemented around:

query timeout
retries
Athena workgroups
result encryption
output locations
query limits
bytes scanned
concurrency
observability

Returning Results to the User

Executing SQL is only part of the workflow.

The application can retrieve the Athena results and provide them back to the model for explanation.

For example:

1
2
3
4
5
6
7
8
9
10
Question:

Which regions declined more than 10% month over month?

Query result:

Region    Month       MoM Change
North     2026-06     -14.2%
West      2026-06     -11.7%
South     2026-06      -4.1%

The model can then generate:

1
2
3
4
5
North and West experienced month-over-month
revenue declines greater than 10% in June 2026.

North declined by 14.2%.
West declined by 11.7%.

The important distinction is that the explanation should be generated from the actual query result.

The model should not be allowed to invent business results.

A useful pattern is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
User Question
      │
      ▼
SQL Generation
      │
      ▼
SQL Validation
      │
      ▼
Athena
      │
      ▼
Actual Query Results
      │
      ▼
Bedrock Result Explanation
      │
      ▼
User

Security and Governance

A production implementation needs more than a good prompt.

The security architecture should use multiple layers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
                    Amazon Bedrock
                          │
                          ▼
                  Application Layer
                          │
                    SQL Validation
                          │
                          ▼
                       Athena
                          │
                ┌─────────┴─────────┐
                │                   │
                ▼                   ▼
         IAM Policies        Lake Formation
                │                   │
                └─────────┬─────────┘
                          ▼
                    Iceberg / S3

IAM

Use least-privilege IAM roles.

The execution role should only have access to the resources required by the application.

Avoid policies such as:

1
2
3
4
5
{
  "Effect": "Allow",
  "Action": "*",
  "Resource": "*"
}

for production workloads.

Separate roles where appropriate for:

application execution
metadata retrieval
Athena execution
data access
logging
administration

AWS Lake Formation

Where applicable, use Lake Formation to implement governed access to the underlying datasets.

The GenAI layer should not bypass organizational data access controls.

Lake Formation supports fine-grained access controls, including database-, table-, column-, row-, and cell-level controls. Athena engine version 3 supports Lake Formation fine-grained access control with Iceberg tables.

This creates an important architectural boundary:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
LLM
 │
 │ Generates SQL
 ▼
Application Validation
 │
 ▼
Athena
 │
 ▼
Lake Formation
 │
 ▼
Authorized Data

The model can request data.

It should not decide whether the user is authorized to access that data.

Amazon S3

The underlying S3 data should use appropriate:

encryption
bucket policies
IAM controls
access controls
logging
lifecycle policies
KMS controls where required

The GenAI layer should never receive broader S3 access simply because it needs to answer natural-language questions.

Handling Hallucinations

Consider the following scenario.

The actual table is:

1
analytics.sales_iceberg

But the model generates:

1
analytics.customer_revenue_2025

That is a hallucinated table reference.

RAG can reduce this problem by supplying relevant metadata, but it does not eliminate hallucinations.

A stronger architecture uses multiple controls:

1
2
3
4
5
6
7
8
9
10
11
12
13
RAG
 ↓
SQL Generation
 ↓
Schema Validation
 ↓
Allow-listed Tables
 ↓
SQL Parser
 ↓
Authorization
 ↓
Athena

Each layer provides another opportunity to reject invalid or unauthorized SQL.

This is an important principle:

Grounding improves model behavior; authorization enforces access.

They solve different problems.

Data Governance and Auditability

Every natural-language query should ideally produce an audit record.

For example:

1
2
3
4
5
6
7
8
9
10
11
12
{
  "request_id": "req-12345",
  "timestamp": "2026-09-22T10:15:00Z",
  "question": "Show revenue by region",
  "retrieved_tables": [
    "analytics.sales_iceberg"
  ],
  "generated_sql": "SELECT ...",
  "validation_status": "PASSED",
  "athena_query_id": "query-12345",
  "execution_status": "SUCCEEDED"
}

These records can be stored in:

CloudWatch Logs
an audit database
an Iceberg audit table
an operational data store

Useful audit information includes:

what users asked
which metadata was retrieved
what SQL was generated
whether validation passed
which queries were rejected
which queries succeeded
Athena query ID
execution time
bytes scanned
error information
model information

AWS Lake Formation also integrates with CloudTrail for auditing data-access activity.

Evaluating the System

A GenAI data application should not be evaluated only by asking whether the chatbot "sounds good."

Create a test dataset containing representative business questions.

For example:

Question	Expected Table	Expected Metric	Expected Filter
Revenue by region	sales_iceberg	revenue	region
Monthly revenue	sales_iceberg	revenue	month
Top products	sales_iceberg	revenue	product
Revenue decline	sales_iceberg	revenue	MoM

Evaluate:

table selection
column selection
SQL validity
SQL safety
query execution success
business correctness
hallucination rate
response latency
retrieval quality
Athena bytes scanned
model token usage

Do not assume that syntactically valid SQL is automatically business-correct.

For example:

1
SELECT SUM(revenue)

may be perfectly valid SQL while still answering the wrong business question.

A useful evaluation pipeline is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Test Question
     │
     ▼
Metadata Retrieval
     │
     ▼
SQL Generation
     │
     ▼
Validation
     │
     ▼
Athena
     │
     ▼
Expected Result Comparison
     │
     ▼
Evaluation Metrics

Performance and Cost Considerations

The architecture has several cost and latency components.

Amazon Athena

Athena query efficiency depends heavily on the amount of data processed.

Apache Iceberg table design therefore matters.

Consider:

partitioning
file sizes
compaction
predicate pushdown
partition pruning
column pruning
appropriate date filters
avoiding unnecessary SELECT *

The objective is not simply to make SQL execute.

The objective is to make it execute efficiently.

Amazon Bedrock

Generative AI introduces model inference costs based on the selected model and token usage.

Reducing unnecessary context can help control:

token consumption
latency
model cost

This is another reason semantic retrieval is useful.

Instead of sending an entire enterprise catalog to the model:

1
2
3
4
5
6
7
Enterprise Catalog
       │
       ▼
Relevant Metadata
       │
       ▼
Bedrock

Only the context relevant to the current question needs to be provided.

Retrieval Performance

The retrieval layer introduces its own operational considerations.

Measure:

retrieval latency
retrieval relevance
embedding/indexing cost where applicable
storage requirements
context size
retrieval failure rate

Do not optimize retrieval, model generation, and Athena independently.

Measure the complete request path.

Observability

A production implementation should monitor the complete lifecycle.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
User Question
      │
      ▼
Metadata Retrieval
      │
      ▼
LLM Generation
      │
      ▼
SQL Validation
      │
      ▼
Athena Execution
      │
      ▼
Result Generation
      │
      ▼
User Response

Useful metrics include:

retrieval latency
model latency
SQL generation latency
SQL validation failures
unauthorized query attempts
Athena execution time
Athena bytes scanned
failed queries
timeout rate
token usage
user feedback
business correctness
hallucination rate

CloudWatch can provide the foundation for application logging and operational monitoring.

Production Evolution

A prototype can start with a relatively simple architecture:

1
2
3
4
5
6
7
8
9
10
11
12
13
User
 │
 ▼
Application
 │
 ▼
Bedrock
 │
 ▼
SQL Validation
 │
 ▼
Athena

A production architecture can evolve into:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
                    User
                     │
                     ▼
              Application / UI
                     │
                     ▼
            Authentication Layer
                     │
                     ▼
             Orchestration Layer
                     │
          ┌──────────┴──────────┐
          │                     │
          ▼                     ▼
   Metadata Retrieval       Amazon Bedrock
          │                     │
          └──────────┬──────────┘
                     ▼
               SQL Validation
                     │
                     ▼
                  Athena
                     │
                     ▼
              Iceberg / S3
                     │
                     ▼
             Governed Results

Additional production capabilities can include:

authentication
authorization
rate limiting
query timeouts
retries
caching
cost controls
query quotas
monitoring
audit logging
evaluation pipelines
user feedback
prompt/version management
model evaluation
semantic-layer versioning

Optional AWS Managed Path

For teams that do not need a fully custom NL2SQL orchestration layer, Amazon Bedrock Knowledge Bases also provide a managed structured-data path.

AWS documentation currently describes support for structured data through Amazon Redshift and AWS Glue Data Catalog with Lake Formation, with natural-language queries converted into SQL for supported structured-data retrieval.

That managed capability can reduce the amount of custom orchestration required.

However, a custom architecture can still be useful when the application needs explicit control over:

metadata retrieval
SQL validation
table allow-lists
custom business rules
Athena-specific execution
application-specific authorization
custom audit records
evaluation pipelines

The right choice depends on the application's requirements.

When Not to Use GenAI NL2SQL

Natural-language querying is not a replacement for every analytics workload.

Traditional dashboards and deterministic SQL can remain preferable for:

regulatory reporting
fixed KPI dashboards
highly sensitive financial workflows
deterministic business calculations
compliance reporting
queries requiring guaranteed repeatability

GenAI can complement these systems by making exploratory analysis easier while the underlying governed data platform remains the source of truth.

Practical Implementation Checklist

Before moving a GenAI data-query application toward production, validate the following.

Data

Iceberg tables are properly structured
Glue Data Catalog metadata is accurate
Business definitions are documented
Sensitive columns are classified
Data quality is monitored

GenAI

Bedrock model access is configured
Prompts are version controlled
Semantic retrieval is implemented
Model output is treated as untrusted
Result explanations use actual query results

SQL

Only approved query types are allowed
SQL is parsed before execution
Tables are allow-listed
Columns are validated
Query limits are enforced
Query timeouts are configured

Security

IAM follows least privilege
Lake Formation permissions are configured
S3 access is restricted
Encryption is enabled where required
Query results are protected

Operations

CloudWatch logging is enabled
Audit records are stored
Athena query IDs are captured
Token usage is monitored
Bytes scanned are monitored
Evaluation datasets exist

Cleanup

If you build this architecture as a hands-on experiment, clean up resources that are no longer required.

Depending on your implementation, review:

Bedrock Knowledge Base resources
vector/indexing resources
Lambda functions
Athena workgroups
S3 test buckets and prefixes
Glue databases and tables
test Iceberg tables
audit tables
CloudWatch log groups
IAM roles created specifically for the experiment

Always verify dependencies before deleting shared or production resources.

Key Lessons

The most important lessons from this architecture are:

1. GenAI needs a strong data foundation

A language model cannot compensate for poor data quality, unclear definitions, or badly governed datasets.

2. Metadata is extremely important

The difference between:

1
revenue DECIMAL

and:

1
revenue = net recognized revenue

is significant when translating business questions into SQL.

3. RAG helps ground the model

Retrieving relevant metadata provides better context than passing an entire enterprise schema.

4. Generated SQL must be validated

LLM output should never be treated as a trusted execution boundary.

5. Governance must remain outside the model

IAM, Lake Formation, Athena, S3, and application-level controls should enforce access.

6. Managed and custom approaches can coexist

AWS provides managed structured-data capabilities through Amazon Bedrock Knowledge Bases, while custom RAG + NL2SQL architectures can provide additional application-level control.

7. Evaluation is essential

A system should be tested against representative business questions instead of being judged only by conversational quality.

Conclusion

A modern lakehouse provides the foundation for governed analytics, but the way users interact with that data can evolve.

Apache Iceberg can provide the analytical data layer.

AWS Glue can provide catalog and data-processing capabilities.

Amazon Athena can provide serverless SQL analytics.

Amazon Bedrock can provide the Generative AI interface.

Semantic metadata and RAG can provide relevant business context.

SQL validation can provide an additional execution safety layer.

IAM and Lake Formation can enforce data-access controls.

CloudWatch and audit logging can provide operational visibility.

Together, these components create a pattern where users can interact with governed analytical data using natural language without removing the controls of the underlying data platform.

The key idea is simple:

Data Engineering provides the foundation. GenAI becomes the interface. Governance remains the control plane.

This architecture also creates a natural path for future enhancements such as anomaly detection, predictive analytics, automated data-quality analysis, governed enterprise data agents, and richer semantic layers.

The foundation remains:

Trusted data → governed metadata → grounded AI → validated queries → observable results.

Next Steps

Possible extensions to this architecture include:

ML-based anomaly detection
predictive analytics
automated data-quality analysis
richer enterprise semantic layers
governed data agents
BI integration
automated chart generation
continuous NL2SQL evaluation
query optimization
natural-language data-quality investigation
conversational analytics across multiple data domains

The long-term objective is not simply to build a chatbot.

It is to build a governed AI interface for enterprise data.

References

Amazon Bedrock Knowledge Bases — structured data
Amazon Bedrock — GenerateQuery for structured data
Amazon Athena — Apache Iceberg
AWS Glue Data Catalog
AWS Lake Formation — fine-grained access control
Amazon Bedrock Agents / AgentCore

# rag # amazon-bedrock # generative-ai # data-analytics # artificial-intelligence

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

From Lakehouse to GenAI: Query Apache Iceberg Data Using Amazon Bedrock and Athena

Gunal K

From Lakehouse to GenAI: Query Apache Iceberg Data with Amazon Bedrock and Athena

Architecture Overview

Data foundation

GenAI interaction layer

From an Iceberg Lakehouse to a GenAI Interface

What Are We Building?

Why RAG + NL2SQL?

Managed Bedrock Knowledge Bases vs. Custom RAG

Designing the Semantic Layer

Building the RAG Layer

User question

Retrieved context

Generating SQL with Amazon Bedrock

Example NL2SQL

SQL Validation: The Critical Security Boundary

Production recommendation

Executing the Query with Amazon Athena

Returning Results to the User

Security and Governance

IAM

AWS Lake Formation

Amazon S3

Handling Hallucinations

Data Governance and Auditability

Evaluating the System

Performance and Cost Considerations

Amazon Athena

Amazon Bedrock

Retrieval Performance

Observability

Production Evolution

Optional AWS Managed Path

When Not to Use GenAI NL2SQL

Practical Implementation Checklist

Data

GenAI

SQL

Security

Operations

Cleanup

Key Lessons

1. GenAI needs a strong data foundation

2. Metadata is extremely important

3. RAG helps ground the model

4. Generated SQL must be validated

5. Governance must remain outside the model

6. Managed and custom approaches can coexist

7. Evaluation is essential

Conclusion

Next Steps

References