Medallion Architecture
Medallion Architecture is a data architecture pattern that organizes data into Bronze, Silver, and Gold layers. Each layer represents a different stage of data processing, from raw data to cleaned and business-ready data.
Medallion Architecture
Medallion Architecture is a common approach to organizing and processing data. It consists of three layers: Bronze, Silver, and Gold, where each layer represents a different stage of data processing.
Bronze Layer
The Bronze layer stores the raw data. Data is kept in its original form after it is collected from sources such as databases, APIs, or applications.
For example, raw data can be stored in an Amazon S3 bucket.
Silver Layer
The Silver layer contains cleaned data. Here, we can remove duplicate records, handle missing values, fix data types, and apply data quality checks.
AWS Glue can be used to process the raw data and store the cleaned data in Amazon S3.
Gold Layer
The Gold layer contains data prepared for analytics. At this stage, data can be joined and aggregated to create useful datasets for reporting and analysis.
Amazon Athena can be used to query the Gold data, and the results can be connected to tools such as Power BI.
Simple Flow
Why Use It?
Keeping data in separate layers makes pipelines easier to manage. The original raw data remains available in Bronze, while Silver and Gold can be rebuilt when transformations change.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article