AWS Data Lake Architecture: From Amazon S3 to Amazon Athena Using AWS Glue
This article explains how to build a Data Lake architecture using AWS services such as Amazon S3, AWS Glue, and Amazon Athena. It describes how data can be stored, processed, cataloged, and analyzed using SQL, as well as how the architecture can be integrated with Business Intelligence tools such as Power BI. The article also highlights its main advantages, performance considerations, and security aspects.
Introduction
In recent years, the growing amount of information generated by organizations has created a need for technologies capable of storing, processing, and analyzing large volumes of data. Cloud computing provides an alternative for building scalable data architectures without having to physically manage all the required infrastructure.
Amazon Web Services (AWS) provides several services designed for data storage, integration, and analytics. These services include Amazon Simple Storage Service (Amazon S3), AWS Glue, and Amazon Athena.
An architecture based on these services makes it possible to build a Data Lake, where data can be stored in Amazon S3 and then cataloged, transformed, and queried using AWS managed services.
A basic architecture can be represented as follows:
Data Sources → Amazon S3 → AWS Glue → Amazon Athena → Business Intelligence Tool
This architecture is particularly useful for projects involving data analytics, business intelligence, and large-scale data processing.
1. What Is a Data Lake?
A Data Lake is a centralized repository that allows organizations to store large amounts of data from different sources.
Unlike a traditional database, data in a Data Lake can be stored in different formats, such as:
- CSV
- JSON
- Parquet
- Avro
- Structured data
- Semi-structured data
- Unstructured data
Amazon S3 can be used as a storage layer for Data Lake architectures. AWS describes Amazon S3 as a storage service that can serve as a fundamental storage layer for analytics and machine learning architectures.
This allows organizations to centralize information from different systems without having to immediately convert all the data into the same format.
2. Amazon S3 as a Data Storage Layer
Amazon S3 (Simple Storage Service) is AWS's object storage service. Within a Data Lake architecture, it can be used to store files containing the organization's data.
For example, a company could receive daily CSV files containing sales information:
1
2
3
4
5
6
sales/
├── 2026/
│ ├── january/
│ ├── february/
│ ├── march/
│ └── ...
Different data sources could also be organized as follows:
1
2
3
4
5
6
7
s3://my-data-lake/
├── sales/
├── customers/
├── products/
├── inventory/
└── suppliers/
In this way, S3 functions as a centralized repository where data is stored before integration, processing, or analysis takes place.
An important advantage is that data can remain stored in S3 while different AWS services use it for processing and analytics.
3. AWS Glue for Data Integration and Preparation
After the data is stored in S3, it is necessary to understand its structure and, in many cases, clean or transform it.
AWS Glue can be used for this purpose.
AWS Glue is a serverless data integration service. AWS describes it as a service that helps users discover, prepare, move, and integrate data from different sources. It also provides tools for creating and running ETL processes.
ETL stands for:
Extract → Transform → Load
In other words:
Extract → Transform → Load
For example:
1
2
3
4
5
6
7
8
9
10
11
CSV File
↓
Extract data
↓
Clean information
↓
Transform columns
↓
Convert format
↓
Store processed data
AWS Glue can be used to perform data cleansing and transformation before the information is used for analytics.
4. AWS Glue Data Catalog
One of the important components of AWS Glue is the AWS Glue Data Catalog.
The Data Catalog acts as a metadata repository. In simple terms, it describes how the data stored in the Data Lake is organized.
For example, suppose an organization has the following file stored in S3:
1
sales.csv
The file could contain:
1
2
3
4
sale_id,date,product,quantity,price
001,2026-09-01,Laptop,2,15000
002,2026-09-02,Mouse,5,350
003,2026-09-03,Keyboard,3,700
The catalog could contain information describing its structure:
1
2
3
4
5
6
7
Table: sales
sale_id → string
date → date
product → string
quantity → integer
price → decimal
AWS Glue provides crawlers that can connect to data sources, infer information about their schemas, and create table definitions in the Data Catalog.
This is useful because services such as Amazon Athena can use this information to understand how the data should be interpreted.
5. Amazon Athena for Data Queries
Once the data is stored in S3 and its structure is available through the Data Catalog, Amazon Athena can be used.
Amazon Athena is an interactive query service that allows users to analyze data stored directly in Amazon S3 using standard SQL. It is also a serverless service, which means that users do not need to manage servers to execute queries.
For example, the following query could be used:
1
2
3
4
SELECT product, SUM(quantity) AS units_sold
FROM sales
GROUP BY product
ORDER BY units_sold DESC;
The purpose of this query would be to identify the products with the highest number of units sold.
Another example would be:
1
2
3
SELECT *
FROM sales
WHERE price > 10000;
This query would return sales involving products with a price greater than 10,000.
Athena supports different data formats, including CSV, JSON, Avro, and Parquet.
6. How Are S3, Glue, and Athena Related?
Each service has a different role within the architecture.
| Service | Main Function |
|---|---|
| Amazon S3 | Store data |
| AWS Glue | Discover, integrate, and transform data |
| Glue Data Catalog | Store metadata and schemas |
| Amazon Athena | Query and analyze data using SQL |
The relationship can be represented as follows:
Athena uses the AWS Glue Data Catalog to obtain information such as table names and column definitions for data stored in Amazon S3.
7. Practical Example
Suppose a company has sales information stored in CSV files.
A new file is generated every day:
1
2
3
sales_2026_10_01.csv
sales_2026_10_02.csv
sales_2026_10_03.csv
These files are stored in:
1
2
3
4
5
Amazon S3
│
├── sales_2026_10_01.csv
├── sales_2026_10_02.csv
└── sales_2026_10_03.csv
An AWS Glue Crawler can then analyze the files and register information about their structure in the Data Catalog.
Athena can use this catalog to query the data using SQL.
For example:
1
2
3
4
5
SELECT
product,
SUM(quantity) AS total_sold
FROM sales
GROUP BY product;
The results could then be used to generate business indicators such as:
- Best-selling product.
- Total sales.
- Sales by period.
- Revenue by product.
- Sales trends.
- Low-demand products.
8. Integration with Business Intelligence
Once the data can be queried through Athena, it can be used with compatible Business Intelligence tools.
Athena provides JDBC and ODBC drivers that allow it to connect with Business Intelligence tools and other applications.
Conceptually, the architecture could be:
1
2
3
4
5
6
7
8
9
10
11
Business Data
↓
S3
↓
Glue
↓
Athena
↓
Power BI
↓
Dashboards
In this way, end users do not necessarily need to work directly with the original data files.
Instead, they can work with processed information and visualize it through charts, indicators, and dashboards.
9. Advantages of This Architecture
Scalability
This architecture can be used to work with large amounts of information. Amazon S3 provides the storage layer, while Athena can query the data without requiring users to manage traditional database servers.
Reduced Infrastructure Management
AWS Glue and Athena are serverless services. This means that users do not have to directly manage servers to perform certain data integration or query operations.
Flexibility
Data can be stored in different formats and later processed according to the requirements of the project.
SQL Support
Athena allows users to use standard SQL to query data stored in S3. This makes the service easier to adopt for people who already have database knowledge.
Integration with Other Tools
Athena supports JDBC and ODBC connectivity, making it possible to integrate it with external applications and Business Intelligence tools.
10. Performance Considerations
Although this architecture is flexible, data should be organized properly.
AWS recommends using partitioning and columnar formats such as Apache Parquet or ORC to improve query performance and reduce the amount of data processed by Athena.
For example, instead of storing all information in a single file:
1
s3://company-data/sales.csv
the information can be organized by year and month:
1
2
3
4
s3://company-data/sales/
year=2026/
month=09/
month=10/
This organization can make queries more efficient when filters are applied to partition columns.
11. Security
Security is also a fundamental component of a data architecture in AWS.
Access to resources should be controlled through AWS identity and permission mechanisms. AWS Glue, for example, requires appropriate IAM permissions for certain use cases.
An enterprise data architecture should consider aspects such as:
- Access control using IAM.
- S3 bucket access policies.
- Data encryption.
- Data access permissions.
- Activity monitoring.
- Separation between development and production environments.
The specific implementation will depend on the security requirements of each organization.
12. Conclusion
The combination of Amazon S3, AWS Glue, and Amazon Athena provides a flexible architecture for building data analytics solutions in AWS.
S3 can serve as the storage layer where data from different sources is centralized. AWS Glue can be used to discover, catalog, integrate, and transform information, while the Glue Data Catalog provides the metadata needed to describe the data structures.
Finally, Amazon Athena allows users to query information directly from S3 using SQL.
The architecture can be summarized as:
Data Source → Amazon S3 → AWS Glue → Glue Data Catalog → Amazon Athena → Business Intelligence
This model is particularly relevant for projects involving data engineering, data analytics, Business Intelligence, and Big Data, because it makes it possible to build a cloud-based data platform without necessarily depending on traditional database server infrastructure.
For someone beginning to study AWS and data analytics, understanding this architecture provides a strong foundation for later learning services such as Amazon Redshift, AWS Lake Formation, Amazon QuickSight, Amazon EMR, and Amazon SageMaker.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article