How Amazon DynamoDB scales and works without joins
The purpose of this blog is help developers – · understand the challenges in relational databases, · how Amazon DynamoDB overcomes those challenges and · limitations with Amazon DynamoDB, so that the developers can design their application accordingly
As more and more clients move towards modernization of applications, one of the most important questions which comes up in discussion is whether to move their relational databases to a non-relational database (for e.g. Amazon Dynamo DB) or not. There are lot of blogs published on internet, with regards too when to use relational database and when to use Amazon DynamoDB, therefore this blog doesn’t talk about that. For clarity I am considering Amazon DynamoDB as a non-relational database for this blog.
The purpose of this blog is help developers –
· understand the challenges in relational databases,
· how Amazon DynamoDB overcomes those challenges and
· limitations with Amazon DynamoDB, so that the developers can design their application accordingly
Before we start, let me put a disclaimer – Relational databases is a great technology and have been backbone of almost all applications and it will continue to be in the future applications as well. This blog is not to undermine relational databases.
Pre-requisites – I assume that as a reader you are familiar with the basis terminologies of Amazon DynamoDB like partition keys, sort keys, Global secondary indexes
Challenges with Relational databases
1. Performance of Relational database -
Have you ever faced situations when you created a new application and the database was returning query results within milliseconds, but as the time passed by the size of database kept on increasing and query response got decreasing (even after indexing). The same query which used to return results in milliseconds might take a couple of seconds to return results as the size of DB increases. Of course, you can do indexing but that too have its limitations. The performance of a relational database decreases as the size increases.
2. It’s difficult to horizontally scale Relational databases
It’s easy to vertically scale a relational database (i.e., increasing size of CPU, RAM of database server). But there is a limit on vertical scaling, you cannot infinitely keep on increasing CPU and RAM of a machine. And that’s where horizontal scaling (i.e., adding more machines to your database cluster) comes in to play. In horizontal scaling each machine holds a certain portion of the data.
With relational databases –
· there is no clear way of segregating the data between different machines in a cluster.
· SQL queries have multiple filters and conditions, due to which what data will be fetched is not known before the query is executed.
· In addition, SQL queries would become slower as network latency (across multiple machines) is added to the response time.
3. SQL joins are time consuming operations
SQL joins are CPU intensive operations and doesn’t scale well when the database size increases. When your database is small then the join operations might not consume CPU much. But as your database size increases SQL joins would start consuming more CPU and therefore becomes a time-consuming operation.
How DynamoDB overcomes these challenges?
Before we discuss how Amazon DynamoDB overcomes the above challenges, let’s have a quick look into how data is stored in relational databases.
In relational databases, we all know data is stored in normalized form. This helps you provide data integrity and eliminate data redundancy by ensuring data is stored logically.

Suppose you want names of employees who have logged in and logged out for less than 10 hours in a day, then you will use SQL joins and scan the entire table to get the results. You can very well imagine the response time of such queries if there are thousands of employees in a company.
- Performance in an Amazon DynamoDB The performance of a Amazon DynamoDB remains the same throughout, even if the same if the data is 1 GB or 100 GB as the records are stored and retrieved based upon Partition Keys
- How Amazon DynamoDB scales horizontally and provides same millisecond latency for database of any size (1 GB or 1 TB)? Amazon DynamoDB database allows to split data in smaller segments and perform queries in one segment only by using partition keys. Suppose you have millions of records, and it cannot be stored in one machine, then you can split the table across multiple machines in a database cluster and still you can retrieve data with the similar response time when the data was less and on one machine.
In the below example - suppose your application had to retrieve data EmpID 502, then it doesn’t need to scan the entire table spread across multiple nodes. Amazon DynamoDB exactly knows which node data for EmpID 502 is stored and it goes directly to that machine. Hence the performance of the query remains the same even if the table size is 5 TB.

1. How Amazon DynamoDB works without joins?
A join requires relational database to scan several tables and perform complex processing to return results. As DynamoDB is designed to provide single digit responses irrespective of the size of dataset, it doesn’t support joins.
Therefore, to retrieve data from DynamoDB, schema should be designed in such a way that it reduces query complexity. Which means
a) Identifying access patterns before hand – i.e., identifying the queries which an application would be running against the database retrieve the data. This information can be gathered by going through the user stories for the application, talking with stakeholders or analyzing the query logs for an existing application
For example – you are designing a DynamoDB table for an online books library – where in customers browse the books available in library and rent them. As a first step we need to identify the access patterns.
After completing your analysis, you might come up with a below list of access pattern for online books library. There can be more this list, this is to showcase what we mean by access patterns and how we can collect the required info
I. List of books currently available in the library
II. List of books currently with customers (rented out) along with start date and return date
III. Total list of books in the library (currently available + rented out)
IV. Search for a book by title
V. Search for a book by author name
VI. Search for books by genre (romantic, thriller, detective etc.)
VII. Books from same author
VIII. Books read by a customer
IX. Same book being taken by different customers
X. Books with no takers for more than 6 months
XI. Books which are more than 3 years old so that they can be removed from library and repurchased
b) Once the access patterns have been identified the schema designing can be done, by identifying the partition keys, sort keys and Global secondary indexes - to structure the data so that the application can retrieve whatever it needs for a given access pattern using a single query on the index.
Limitations of Amazon DynamoDB database
One of the limitations of Amazon DynamoDB database is with regards to data consistency. As data is stored in denormalized form in Amazon DynamoDB there is a possibility that same data can be present in multiple places in the DynamoDB table (refer e.g. below). If that data is changing on regular basis, then there will be possibility that the data which in one place will be update and, in another place it might not be updated.

Therefore, while designing the DynamoDB tables, this limitation needs to be considered and a decision needs to be taken with regards to how to handle such scenarios. Ideally, while designing the application only those fields should be allowed to be duplicated which don’t change to often (for e.g., Date of birth)
Indication Amazon DynamoDB database is a better fit for your applications
1. Database size is anticipated to increase over a period. When the application started the size of the database might be 10 GB but size is expected to increase to 500GB or 1TB after sometime in use
2. Is your application highly dependent on SQL join queries (for e.g., 70% of all your queries use SQL joins). As joins are time consuming operations, therefore it would be better to move to Amazon DynamoDB
3. Not much data duplication (refer to the above limitation)
4. Infinite scroll down functionality required
5. Clear understanding of data accessibility patterns for your application.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article