AWS Builder Center

20 Days of DynamoDB

A DynamoDB tip a day, keeps the ...?

For the next 20 days (don't ask me why I chose that number 😉), I will be publishing a DynamoDB quick tip per day with code snippets. The examples use the DynamoDB packages from  AWS SDK for Go V2 , but should be applicable to other languages as well.
logo


Day 20 - Converting between Go and DynamoDB types

Posted: 13/Feb/2024
The DynamoDB  attributevalue  package in the AWS SDK for Go package can save you a lot of time, thanks to the  Marshal  and  Unmarshal  family of utility functions that can be used to convert between Go types (including  struct s) and  AttributeValue s.
Here is an example using a Go  struct :
Go
type Customer struct {
Email string `dynamodbav:"email"`
Age int `dynamodbav:"age,omitempty"`
City string `dynamodbav:"city"`
}

customer := Customer{Email: "abhirockzz@gmail.com", City: "New Delhi"}

item, _ := attributevalue.MarshalMap(customer)

client.PutItem(context.Background(), &dynamodb.PutItemInput{
TableName: aws.String(tableName),
Item: item,
})

resp, _ := client.GetItem(context.Background(), &dynamodb.GetItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{"email": &types.AttributeValueMemberS{Value: "abhirockzz@gmail.com"}},
})

var cust Customer
attributevalue.UnmarshalMap(resp.Item, &cust)

log.Println("item info:", cust.Email, cust.City)
Notice how:
  •  MarshalMap  converts Customer  struct  into a  map[string]types.AttributeValue  that's required by  PutItem 
  •  UnmarshalMap  converts the  map[string]types.AttributeValue  returned by  GetItem  into a Customer  struct 
Recommended reading:

Day 19 - PartiQL Batch Operations

Posted: 12/Feb/2024
You can use batched operations with PartiQL as well, thanks to  BatchExecuteStatement . It allows you to batch reads as well as write requests.
Here is an example (note that you cannot mix both reads and writes in a single batch):
Go
//read statements
client.BatchExecuteStatement(context.Background(), &dynamodb.BatchExecuteStatementInput{
Statements: []types.BatchStatementRequest{
{
Statement: aws.String("SELECT * FROM url_metadata where shortcode=?"),
Parameters: []types.AttributeValue{
&types.AttributeValueMemberS{Value: "abcd1234"},
},
},
{
Statement: aws.String("SELECT * FROM url_metadata where shortcode=?"),
Parameters: []types.AttributeValue{
&types.AttributeValueMemberS{Value: "qwer4321"},
},
},
},
})

//separate batch for write statements
client.BatchExecuteStatement(context.Background(), &dynamodb.BatchExecuteStatementInput{
Statements: []types.BatchStatementRequest{
{
Statement: aws.String("INSERT INTO url_metadata value {'longurl':?,'shortcode':?, 'active': true}"),
Parameters: []types.AttributeValue{
&types.AttributeValueMemberS{Value: "https://github.com/abhirockzz"},
&types.AttributeValueMemberS{Value: uuid.New().String()[:8]},
},
},
{
Statement: aws.String("UPDATE url_metadata SET active=? where shortcode=?"),
Parameters: []types.AttributeValue{
&types.AttributeValueMemberBOOL{Value: false},
&types.AttributeValueMemberS{Value: "abcd1234"},
},
},
{
Statement: aws.String("DELETE FROM url_metadata where shortcode=?"),
Parameters: []types.AttributeValue{
&types.AttributeValueMemberS{Value: "qwer4321"},
},
},
},
})
Just like  BatchWriteItem ,  BatchExecuteStatement  is limited to 25 statements (operations) per batch.
Recommended reading:  BatchExecuteStatement  API docs

Day 18 - Using a SQL-compatible query language with DynamoDB

Posted: 6/Feb/2024
DynamoDB supports PartiQL to execute SQL-like select, insert, update, and delete operations.
Here is an example of how you would use PartiQL based queries for a simplified version of a URL shortener application. Notice how it uses a (generic)  ExecuteStatement  API to execute  INSERT ,  SELECT ,  UPDATE  and  DELETE :
Go
_, err := client.ExecuteStatement(context.Background(), &dynamodb.ExecuteStatementInput{
Statement: aws.String("INSERT INTO url_metadata value {'longurl':?,'shortcode':?, 'active': true}"),
Parameters: []types.AttributeValue{
&types.AttributeValueMemberS{Value: "https://github.com/abhirockzz"},
&types.AttributeValueMemberS{Value: uuid.New().String()[:8]},
},
})

_, err := client.ExecuteStatement(context.Background(), &dynamodb.ExecuteStatementInput{
Statement: aws.String("SELECT * FROM url_metadata where shortcode=? AND active=true"),
Parameters: []types.AttributeValue{
&types.AttributeValueMemberS{Value: "abcd1234"},
},
})

_, err := client.ExecuteStatement(context.Background(), &dynamodb.ExecuteStatementInput{
Statement: aws.String("UPDATE url_metadata SET active=? where shortcode=?"),
Parameters: []types.AttributeValue{
&types.AttributeValueMemberBOOL{Value: false},
&types.AttributeValueMemberS{Value: "abcd1234"},
},
})

_, err := client.ExecuteStatement(context.Background(), &dynamodb.ExecuteStatementInput{
Statement: aws.String("DELETE FROM url_metadata where shortcode=?"),
Parameters: []types.AttributeValue{
&types.AttributeValueMemberS{Value: "abcd1234"},
},
})
Recommended reading:

Day 17 - DynamoDB BatchGetItem operation

Posted: 5/Feb/2024
You can club multiple (up to 100)  GetItem  requests in a single  BatchGetItem  operation - this can be done across multiple tables.
Here is an example that fetches includes four  GetItem  calls across two different tables:
Go
resp, err := client.BatchGetItem(context.Background(), &dynamodb.BatchGetItemInput{
RequestItems: map[string]types.KeysAndAttributes{
"customer": types.KeysAndAttributes{
Keys: []map[string]types.AttributeValue{
{
"email": &types.AttributeValueMemberS{Value: "c1@foo.com"},
},
{
"email": &types.AttributeValueMemberS{Value: "c2@foo.com"},
},
},
},
"Thread": types.KeysAndAttributes{
Keys: []map[string]types.AttributeValue{
{
"ForumName": &types.AttributeValueMemberS{Value: "Amazon DynamoDB"},
"Subject": &types.AttributeValueMemberS{Value: "DynamoDB Thread 1"},
},
{
"ForumName": &types.AttributeValueMemberS{Value: "Amazon S3"},
"Subject": &types.AttributeValueMemberS{Value: "S3 Thread 1"},
},
},
ProjectionExpression: aws.String("Message"),
},
},
ReturnConsumedCapacity: types.ReturnConsumedCapacityTotal,
})
Just like an individual  GetItem  call, you can include Projection Expressions and return RCUs. Note that  BatchGetItem  can only retrieve up to 16 MB of data.
Recommended reading:  BatchGetItem  API doc

Day 16 - Enhancing Write Performance with Batching

Posted: 2/Feb/2024
The DynamoDB  BatchWriteItem  operation can provide a performance boost by allowing you to squeeze in 25 individual  PutItem  and  DeleteItem  requests in a single API call - this can be done across multiple tables.
Here is an example that combines  PutItem  and  DeleteItem  operations for two different tables ( customer ,  orders ):
Go
_, err := client.BatchWriteItem(context.Background(), &dynamodb.BatchWriteItemInput{
RequestItems: map[string][]types.WriteRequest{
"customer": []types.WriteRequest{
{
PutRequest: &types.PutRequest{
Item: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: "c3@foo.com"},
},
},
},
{
DeleteRequest: &types.DeleteRequest{
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: "c1@foo.com"},
},
},
},
},
"orders": []types.WriteRequest{
{
PutRequest: &types.PutRequest{
Item: map[string]types.AttributeValue{
"order_id": &types.AttributeValueMemberS{Value: "oid_1234"},
},
},
},
{
DeleteRequest: &types.DeleteRequest{
Key: map[string]types.AttributeValue{
"order_id": &types.AttributeValueMemberS{Value: "oid_4321"},
},
},
},
},
},
})
Be aware of the following constraints:
  • The total request size cannot exceed 16 MB
  •  BatchWriteItem  cannot update items
Recommended reading:  BatchWriteItem  API doc

Day 15 - Using the DynamoDB expression package to build Update expressions

Posted: 31/Jan/2024
The expression package in AWS Go SDK for DynamoDB supports programmatic creation of  Update  expressions.
Here is an example of how you can build an expression to include execute a  SET  operation of the  UpdateItem  API and combine it with a  Condition  expression (update criteria):
Go
updateExpressionBuilder := expression.Set(expression.Name("category"), expression.Value("standard"))
conditionExpressionBuilder := expression.AttributeNotExists(expression.Name("account_locked"))

expr, _ := expression.NewBuilder().
WithUpdate(updateExpressionBuilder).
WithCondition(conditionExpressionBuilder).
Build()

resp, err := client.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: "c1@foo.com"},
},
UpdateExpression: expr.Update(),
ConditionExpression: expr.Condition(),
ExpressionAttributeNames: expr.Names(),
ExpressionAttributeValues: expr.Values(),
ReturnValues: types.ReturnValueAllOld,
})
Recommended reading -  WithUpdate method  in the package API docs

Day 14 - Using the DynamoDB expression package to build Key Condition and Filter expressions

Posted: 30/Jan/2024
You can use  expression  package in the AWS Go SDK for DynamoDB to programmatically build key condition and filter expressions and use them with  Query  API.
Here is an example that queries for a specific thread based on the forum name (partition key) and subject (sort key):
Go
keyConditionBuilder := expression.Key("ForumName").Equal(expression.Value("Amazon DynamoDB"))
filterExpressionBuilder := expression.Name("Views").GreaterThanEqual(expression.Value(3))

expr, _ := expression.NewBuilder().
WithKeyCondition(keyConditionBuilder).
WithFilter(filterExpressionBuilder).
Build()

_, err := client.Query(context.Background(), &dynamodb.QueryInput{
TableName: aws.String("Thread"),
KeyConditionExpression: expr.KeyCondition(),
FilterExpression: expr.Filter(),
ExpressionAttributeNames: expr.Names(),
ExpressionAttributeValues: expr.Values(),
})
Recommended reading -  Key  and  NameBuilder  in the package API docs

Day 13 - Using the DynamoDB expression package to build Condition expressions

Posted: 25/Jan/2024
Thanks to the  expression  package in the AWS Go SDK for DynamoDB, you can programmatically build Condition expressions and use them with write operations. Here is an example with the  DeleteItem  API:
Go
conditionExpressionBuilder := expression.Name("inactive_days").GreaterThanEqual(expression.Value(20))
conditionExpression, _ := expression.NewBuilder().WithCondition(conditionExpressionBuilder).Build()

_, err := client.DeleteItem(context.Background(), &dynamodb.DeleteItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: email},
},
ConditionExpression: conditionExpression.Condition(),
ExpressionAttributeNames: conditionExpression.Names(),
ExpressionAttributeValues: conditionExpression.Values(),
})
Recommended reading -  WithCondition method  in the package API docs

Day 12 - Using the DynamoDB expression package to build Projection expressions

Posted: 24/Jan/2024
The  expression  package in the AWS Go SDK for DynamoDB provides a fluent builder API with types and functions to create expression strings programmatically along with corresponding expression attribute names and values.
Here is an example of how you would build a Projection Expression and use it with the  GetItem  API:
Go
projectionBuilder := expression.NamesList(expression.Name("first_name"), expression.Name("last_name"))
projectionExpression, _ := expression.NewBuilder().WithProjection(projectionBuilder).Build()

_, err := client.GetItem(context.Background(), &dynamodb.GetItemInput{
TableName: aws.String("customer"),
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: "c1@foo.com"},
},
ProjectionExpression: projectionExpression.Projection(),
ExpressionAttributeNames: projectionExpression.Names(),
})
Recommended reading:  expression  package API docs

Day 11 - Using pagination with Query API

Posted: 22/Jan/2024
The  Query  API returns the result set size to  1 MB . Use  ExclusiveStartKey  and  LastEvaluatedKey  elements to paginate over large result sets. You can also reduce page size by limiting the number of items in the result set, with the  Limit  parameter of the  Query  operation.
Go
func paginatedQuery(searchCriteria string, pageSize int32) {

currPage := 1
var exclusiveStartKey map[string]types.AttributeValue

for {
resp, _ := client.Query(context.Background(), &dynamodb.QueryInput{
TableName: aws.String(tableName),
KeyConditionExpression: aws.String("ForumName = :name"),
ExpressionAttributeValues: map[string]types.AttributeValue{
":name": &types.AttributeValueMemberS{Value: searchCriteria},
},
Limit: aws.Int32(pageSize),
ExclusiveStartKey: exclusiveStartKey,
})

if resp.LastEvaluatedKey == nil {
return
}
currPage++
exclusiveStartKey = resp.LastEvaluatedKey
}
}
Recommended reading -  Query Pagination 

Day 10 - Query API with Filter Expression

Posted: 19/Jan/2024
With the DynamoDB  Query  API, you can use Filter Expressions to discard specific query results based on a criteria. Note that the filter expression is applied after a Query finishes, but before the results are returned. Thus, it has no impact on the RCUs (read capacity unit) consumed by the query.
Here is an example that filters out forum discussion threads that have less than a specific number of views:
Go
resp, err := client.Query(context.Background(), &dynamodb.QueryInput{
TableName: aws.String(tableName),
KeyConditionExpression: aws.String("ForumName = :name"),
FilterExpression: aws.String("#v >= :num"),
ExpressionAttributeNames: map[string]string{
"#v": "Views",
},
ExpressionAttributeValues: map[string]types.AttributeValue{
":name": &types.AttributeValueMemberS{Value: forumName},
":num": &types.AttributeValueMemberN{Value: numViews},
},
})
Recommended reading:  Filter Expressions 

Day 9 - Query API

Posted: 18/Jan/2024
The  Query  API is used to model one-to-many relationships in DynamoDB. You can search for items based on (composite) primary key values using Key Condition Expressions. The value for partition key attribute is mandatory - the query returns all items with that partition key value. Additionally, you can also provide a sort key attribute and use a comparison operator to refine the search results.
Here is an example that queries for a specific thread based on the forum name (partition key) and subject (sort key). It only returns the  Message  attribute:
Go
resp, err = client.Query(context.Background(), &dynamodb.QueryInput{
TableName: aws.String(tableName),
KeyConditionExpression: aws.String("ForumName = :name and Subject = :sub"),
ExpressionAttributeValues: map[string]types.AttributeValue{
":name": &types.AttributeValueMemberS{Value: forumName},
":sub": &types.AttributeValueMemberS{Value: subject},
},
ReturnConsumedCapacity: types.ReturnConsumedCapacityTotal,
ConsistentRead: aws.Bool(true),
ProjectionExpression: aws.String("Message"),
})
With the  Query  API, you can also:
  • Switch to strongly consistent read (eventual consistent being the default)
  • Use a projection expression to return only some attributes
  • Return the consumed Read Capacity Units (RCU)

Day 8 - Conditional Delete operation

Posted: 17/Jan/2024
All the DynamoDB write APIs, including  DeleteItem  support criteria-based (conditional) execution. You can use  DeleteItem  operation with a condition expression - it must evaluate to  true  in order for the operation to succeed.
Here is an example that verifies the value of  inactive_days  attribute:
Go
resp, err := client.DeleteItem(context.Background(), &dynamodb.DeleteItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: email},
},
ConditionExpression: aws.String("inactive_days >= :val"),
ExpressionAttributeValues: map[string]types.AttributeValue{
":val": &types.AttributeValueMemberN{Value: "20"},
},
})

if err != nil {
if strings.Contains(err.Error(), "ConditionalCheckFailedException") {
return
} else {
log.Fatal(err)
}
}
Recommended reading -  Conditional deletes documentation 

Day 7 - DeleteItem API

Posted: 16/Jan/2024
The DynamoDB API does what it says - delete an item. But it can also:
  • Return the content of the old item (at no additional cost)
  • Return the consumed Write Capacity Units (WCU)
  • Return the item attributes for an operation that failed a condition check (again, no additional cost)
  • Retrieve statistics about item collections, if any, that were affected during the operation
Here is an example:
Go
resp, err := client.DeleteItem(context.Background(), &dynamodb.DeleteItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: email},
},
ReturnValues: types.ReturnValueAllOld,
ReturnConsumedCapacity: types.ReturnConsumedCapacityTotal,
ReturnValuesOnConditionCheckFailure: types.ReturnValuesOnConditionCheckFailureAllOld,
ReturnItemCollectionMetrics: types.ReturnItemCollectionMetricsSize,
})
Recommended reading -  DeleteItem API doc 

Day 6 - Atomic counters with UpdateItem

Posted: 15/Jan/2024
Need to implement atomic counter using DynamoDB? If you have a use-case that can tolerate over-counting or under-counting (for example, visitor count), use the API.
Here is an example that uses the operator in an update expression to increment attribute:
Go
resp, err := client.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: email},
},
UpdateExpression: aws.String("SET num_logins = num_logins + :num"),
ExpressionAttributeValues: map[string]types.AttributeValue{
":num": &types.AttributeValueMemberN{
Value: num,
},
},
ReturnConsumedCapacity: types.ReturnConsumedCapacityTotal,
})
Note that every invocation of will increment (or decrement) - hence it is not idempotent.
Recommended reading -  Atomic Counters 

Day 5 - Avoid overwrites when using DynamoDB UpdateItem API

Posted: 12/Jan/2024
The API creates a new item or modifies an existing item's attributes. If you want to avoid overwriting an existing attribute, make sure to use the operation with function.
Here is an example that sets the category of an item only if the item does not already have a category attribute:
Go
resp, err := client.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: email},
},
UpdateExpression: aws.String("SET category = if_not_exists(category, :category)"),
ExpressionAttributeValues: map[string]types.AttributeValue{
":category": &types.AttributeValueMemberS{
Value: category,
},
},
})
Note that the if_not_exists function can only be used in the SET action of an update expression.
Recommended reading -  DynamoDB documentation 

Day 4 - Conditional UpdateItem

Posted: 11/Jan/2024
Conditional operations are helpful in cases when you want a DynamoDB write operation (, or ) to be executed based on a certain criteria. To do so, use a condition expression - it must evaluate to in order for the operation to succeed.
Here is an example that demonstrates a conditional operation. It uses the function:
Go
resp, err := client.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: email},
},
UpdateExpression: aws.String("SET first_name = :fn"),
ExpressionAttributeValues: map[string]types.AttributeValue{
":fn": &types.AttributeValueMemberS{
Value: firstName,
},
},

ConditionExpression: aws.String("attribute_not_exists(account_locked)"),
ReturnConsumedCapacity: types.ReturnConsumedCapacityTotal,
})
Recommended reading:  ConditionExpressions 

Day 3 - UpdateItem add-on benefits

Posted: 10/Jan/2024
The DynamoDB operation is quite flexible. In addition to using many types of operations, you can:
  • Use multiple update expressions in a single statement
  • Get the item attributes as they appear before or after they are successfully updated
  • Understand which item attributes failed the condition check (no additional cost)
  • Retrieve the consumed Write Capacity Units (WCU)
Here is an example:
Go
resp, err = client.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: email},
},
UpdateExpression: aws.String("SET last_name = :ln REMOVE category"),
ExpressionAttributeValues: map[string]types.AttributeValue{
":ln": &types.AttributeValueMemberS{
Value: lastName,
},
},
ReturnValues: types.ReturnValueAllOld,
ReturnValuesOnConditionCheckFailure: types.ReturnValuesOnConditionCheckFailureAllOld,
ReturnConsumedCapacity: types.ReturnConsumedCapacityTotal,
}
Recommended reading:  UpdateItem API ,  Update Expressions 

Day 2 - GetItem operation

Posted: 9/Jan/2024
Did you know that the DynamoDB operation also gives you the ability to:
  • Switch to strongly consistent read (eventually consistent being the default)
  • Use a projection expression to return only some of the attributes
  • Return the consumed Read Capacity Units (RCU)
Here is an example:
Go
resp, err := client.GetItem(context.Background(), &dynamodb.GetItemInput{
TableName: aws.String(tableName),
Key: map[string]types.AttributeValue{
//email - partition key
"email": &types.AttributeValueMemberS{Value: email},
},
ConsistentRead: aws.Bool(true),
ProjectionExpression: aws.String("first_name, last_name"),
ReturnConsumedCapacity: types.ReturnConsumedCapacityTotal,
})

Day 1 - Conditional PutItem

Posted: 8/Jan/2024
The DynamoDB API overwrites the item in case an item with the same primary key already exists. To avoid (or work around) this behaviour, use with an additional condition.
Here is an example that uses the function:
Go
_, err := client.PutItem(context.Background(), &dynamodb.PutItemInput{
TableName: aws.String(tableName),
Item: map[string]types.AttributeValue{
"email": &types.AttributeValueMemberS{Value: email},
},
ConditionExpression: aws.String("attribute_not_exists(email)"),
ReturnConsumedCapacity: types.ReturnConsumedCapacityTotal,
ReturnValues: types.ReturnValueAllOld,
ReturnItemCollectionMetrics: types.ReturnItemCollectionMetricsSize,
})

if err != nil {
if strings.Contains(err.Error(), "ConditionalCheckFailedException") {
log.Println("failed pre-condition check")
return
} else {
log.Fatal(err)
}
}
With the operation, you can also:
  • Return the consumed Write Capacity Units (WCU)
  • Get the item attributes as they appeared before (in case they were updated during the operation)
  • Retrieve statistics about item collections, if any, that were modified during the operation
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article