AWS Builder Center
Building a Semantic Layer for Data Platforms with Amazon S3 Vectors

Building a Semantic Layer for Data Platforms with Amazon S3 Vectors

How we use Amazon S3 Vectors to help AI agents discover the right tables across a medallion architecture—without loading the entire data catalog into every session

Principal Data & AI Architect

My Motivation

Natural language query agents need to find the right tables before generating SQL. As our lakehouse grows across countries and medallion layers, loading the entire data catalog into every session becomes inefficient.
We wanted the agent to retrieve only the table descriptions needed for each question, reducing input tokens and cost while keeping the same accuracy.

A real-world use case

We have a natural language query agent that helps business users consume lakehouse data through Amazon Athena. Before generating SQL, the agent must identify the tables that contain the requested information.
Our platform has hundreds of tables for each country across bronze, silver, and gold layers. Loading every table description for a simple question is expensive and unnecessary, especially as the company expands into new regions.

The solution: Building our semantic layer with Amazon S3 Vectors

We store each table description as a vector in Amazon S3 Vectors. Metadata such as country, business domain, and medallion layer remains available for filtering.
When the agent receives a question, it calls a search tool that first filters the catalog and then returns the Top-K semantic matches. The agent reads only those table descriptions instead of loading the complete catalog into its context. This provides a simple, serverless, and low-cost semantic layer that can grow with our data platform.

The impact on tokens and cost

We compared both approaches using 300 synthetic tables and nine business questions.
Strategy
Accuracy
Avg. input tokens
Avg. latency
Estimated cost
Full Markdown catalog
100%
29,888
1.39 s
$0.001048
Amazon S3 Vectors
100%
2,095
2.10 s
$0.000104
Amazon S3 Vectors kept the same accuracy while reducing input tokens by 93% and estimated variable cost by 90.1%. The trade-off was additional latency for the embedding, vector search, and second model call.
As our data platform grows, semantic retrieval allows the agent to find the right tables without loading the complete catalog into every session. Amazon S3 Vectors gives us a simple and serverless way to combine semantic search with metadata filters, reducing tokens and cost while keeping the same accuracy in our benchmark.

Talk is cheap! Run the complete benchmark yourself:

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article