AWS Builder Center

Builder Center

Testing DynamoDB Interactions using LocalStack and Testcontainers

Testing DynamoDB Interactions using LocalStack and Testcontainers

Write efficient integration tests to validate the applications interactions with AWS DynamoDB, by leveraging a disposable LocalStack container in Spring Boot.

Overview

Testing the data layer and datasource interactions is very crucial when developing a backend application. In this article, we’ll explore on how we can write Integration Tests for our Java Spring-boot application which interacts with AWS DynamoDB for storing and retrieving records.
We'll be creating an example data model and a repository class which performs basic CRUD operations on the provisioned DynamoDB Table. Then we'll be looking at how we can write Integration tests for our repository using LocalStack and Testcontainers where we'll be creating an In-memory DynamoDB Table inside a disposable docker container and making our application connect to it while running the tests.
With this setup, we'll be ensuring a controlled and isolated environment for testing our DynamoDB interactions. The code referenced in the article can be found in this  Repository .

Contextualizing Keywords

  • Spring Boot : is a popular framework for building Java-based, production-ready services with minimal configuration.
  •  AWS DynamoDB  : is a fully managed NoSQL database service provided by AWS and we'll be using it as the datasource that our backend application connects to.
  •  LocalStack  : is a cloud service emulator that enables development and testing of AWS services without connecting to a remote cloud provider. We'll be provisioning an in-memory DynamoDB table inside this emulator.
  •  Testcontainers  : is a library that provides lightweight, throwaway instances of Docker containers for integration testing.


AWS Configuration

In order to facilitate connection between the application and DynamoDB table, we'll need to define beans of  AmazonDynamoDB  and  DynamoDBMapper  in a configuration class.
When connecting to an actual provisioned DynamoDB table in AWS, we don't need to specify an endpoint URL since it automatically uses the default endpoint for each service in the configured AWS Region. But when connecting to the Table created inside the LocalStack container, we'll be needing to define an endpoint URL as well.
To solve the above issue, we'll be using  Spring Profiles  to define seperate beans of class  AmazonDynamoDB , that'll get injected into the application context depending on whether the application is running in test or non-test profiles.
Additionaly we'll make use of  @ConfigurationProperties  to map the values defined in the active  .yaml  or  .properties  file to a POJO that'll be referenced by our configuration class.
Java
@Getter
@Setter
@Validated
@ConfigurationProperties(prefix = "com.behl.aws")
public class AwsConfigurationProperties {

	@NotBlank(message = "AWS access key must be configured")
	private String accessKey;

	@NotBlank(message = "AWS secret access key must be configured")
	private String secretAccessKey;

	@NotBlank(message = "AWS region must be configured")
	private String region;

	private String endpoint;

}
As mentioned above, The  endpoint  field is optional and is needed to be defined only for testing and local development when using LocalStack.
Below is a snippet of our  application.yaml  file where we've defined the required properties which will be automatically mapped to the above defined class, along with the current active profile that we'll be using in our bean configuration class.
YAML
spring:
  profiles:
    active: ${ACTIVE_PROFILE}

com:
  behl:
    aws:
      access-key: ${AWS_ACCESS_KEY_ID}
      secret-access-key: ${AWS_SECRET_ACCESS_KEY}
      region: ${AWS_REGION}
      endpoint: ${AWS_ENDPOINT_URL}
Now let's create our  AwsConfiguration  class that defines the beans required for connecting to DynamoDB depending upon the active profile.
Java
@Configuration
@RequiredArgsConstructor
@EnableConfigurationProperties(value = AwsConfigurationProperties.class)
public class AwsConfiguration {

	private final AwsConfigurationProperties awsConfigurationProperties;

	@Bean
	@Profile("!test && !local")
	public AmazonDynamoDB amazonDynamoDB() {
		var credentials = constructCredentials();
		var regionName = awsConfigurationProperties.getRegion();
		return AmazonDynamoDBClientBuilder.standard()
				.withRegion(Regions.fromName(regionName))
				.withCredentials(credentials)
				.build();
	}
	
	@Bean
	@Profile("test | local")
	public AmazonDynamoDB testAmazonDynamoDb() {
		var credentials = constructCredentials();
		var regionName = awsConfigurationProperties.getRegion();
		var endpointUri = awsConfigurationProperties.getEndpoint();
		var endpointConfig = new EndpointConfiguration(endpointUri, regionName);
		return AmazonDynamoDBClientBuilder.standard()
				.withEndpointConfiguration(endpointConfig)
				.withCredentials(credentials)
				.build();
	}

	@Bean
	public DynamoDBMapper dynamoDBMapper(AmazonDynamoDB amazonDynamoDB) {
		return new DynamoDBMapper(amazonDynamoDB);
	}

	private AWSStaticCredentialsProvider constructCredentials() {
		var accessKey = awsConfigurationProperties.getAccessKey();
		var secretAccessKey = awsConfigurationProperties.getSecretAccessKey();
		var basicAwsCredentials = new BasicAWSCredentials(accessKey, secretAccessKey);
		return new AWSStaticCredentialsProvider(basicAwsCredentials);
	}
	
}

The bean specifying the endpoint URL for connecting to the LocalStack container will be created only if the active profile is  test  or  local . With this, we've successfully created the correct AWS DynamoDB configuration for our application.


Data Model and Repository Layer

We'll be creating an entity class  MedicalRecord  to represent the data stored in DynamoDB with the below defined attributes. We're leveraging lombok for generating getter and setter methods for the attribute to reduce boilerplate code.
Java
@Getter
@Setter
@DynamoDBTable(tableName = "MedicalRecords")
public class MedicalRecord {

	@DynamoDBHashKey
	@DynamoDBAttribute(attributeName = "Id")
	private String id;

	@DynamoDBAttribute(attributeName = "PatientName")
	private String patientName;

	@DynamoDBAttribute(attributeName = "Diagnosis")
	private String diagnosis;

	@DynamoDBAttribute(attributeName = "TreatmentPlan")
	private String treatmentPlan;

	@DynamoDBAttribute(attributeName = "AttendingPhysicianName")
	private String attendingPhysicianName;

}
To perform basic save and retrieve operations on the above defined entity, we'll be creating it's corresponding Respository class. The earlier defined bean of  DynamoDBMapper  will be autowired into this class via  Constructor Injection with Lombok .
Java
@Repository
@RequiredArgsConstructor
public class MedicalRecordRepository {

	private final DynamoDBMapper dynamoDBMapper;

	public void save(@NonNull MedicalRecord medicalRecord) {
		dynamoDBMapper.save(medicalRecord);
	}

	public Optional<MedicalRecord> findById(@NonNull String medicalRecordId) {
		var medicalRecord = dynamoDBMapper.load(MedicalRecord.class, medicalRecordId);
		return Optional.ofNullable(medicalRecord);
	}

}



Testing

Now that the context is established, we'll be moving on to the main agenda of this article: To test the data layer's interaction with DynamoDB.

Dependencies

Let’s start by declaring the required test dependencies in  pom.xml :
XML
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>localstack</artifactId>
    <scope>test</scope>
</dependency>

Prerequisite: Running Docker

The prerequisite for running a LocalStack container via Testcontainers is, as you've guessed it, an up and running Docker instance. We need to ensure this, when running the Test suite either locally or when using a CI/CD pipeline.
When using Github actions, The default Ubuntu runners come pre-equipped with Docker, eliminating the necessity for manual Docker setup procedures.
YAML
jobs:
  build:
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up JDK
        uses: actions/setup-java@v3
        with:
          java-version: 21
          distribution: adopt
          cache: maven

      - name: Run Integration Tests
        run: mvn integration-test verify

Creating DynamoDB Table using Init Hooks

Localstack gives us the ability to create resources when the container is started via  Initialization Hooks . We'll be creating our  init-dynamodb.sh  script in the  src/test/resources  folder which we want to be executed when the LocalStack container is started.
The script creates a DynamoDB table  MedicalRecords  for which we've already created a model class in our application.
sh
#!/bin/bash
table_name="MedicalRecords"
hash_key="Id"

awslocal dynamodb create-table \
    --table-name "$table_name" \
    --key-schema AttributeName="$hash_key",KeyType=HASH \
    --attribute-definitions AttributeName="$hash_key",AttributeType=S \
    --billing-mode PAY_PER_REQUEST

echo "DynamoDB table '$table_name' created successfully with hash key '$hash_key'"
echo "Executed init-dynamodb.sh"
Now in our Integration Test class, we need to do the following before we start to write any test cases:
  • Activate the  test  profile using  @ActiveProfiles 
  • Copy our  init-dynamodb.sh  script into the container to ensure DynamoDB table creation.
  • Configure a Wait strategy to wait for the log  Executed init-dynamodb.sh  to be printed, as defined in our init script.
  • Dynamically configure the AWS configuration properties needed by the application in order to create the required connection beans using  @DynamicPropertySource .
Java
@SpringBootTest
@ActiveProfiles("test")
class MedicalRecordRepositoryIT {

	@Autowired
	private MedicalRecordRepository medicalRecordRepository;

	private static LocalStackContainer localStackContainer;

	static {
		localStackContainer = new LocalStackContainer(DockerImageName.parse("localstack/localstack:3.2"))
				.withCopyFileToContainer(MountableFile.forClasspathResource("init-dynamodb.sh", 0744), "/etc/localstack/init/ready.d/init-dynamodb.sh")
				.withServices(Service.DYNAMODB)
				.waitingFor(Wait.forLogMessage(".*Executed init-dynamodb.sh.*", 1));
		localStackContainer.start();
	}

	@DynamicPropertySource
	static void properties(DynamicPropertyRegistry registry) {
		registry.add("com.behl.aws.access-key", localStackContainer::getAccessKey);
		registry.add("com.behl.aws.secret-access-key", localStackContainer::getSecretKey);
		registry.add("com.behl.aws.region", localStackContainer::getRegion);
		registry.add("com.behl.aws.endpoint", localStackContainer::getEndpoint);
	}

}
Now once we're done configuring the Localstack container, we can test the  MedicalRecordRepository  class with appropriate test cases:
Java
@Test
void shouldReturnEmptyOptionalForNonExistingMedicalRecord() {
    // generate random medical record Id
    var medicalRecordId = RandomString.make();

    // call method under test
    Optional<MedicalRecord> medicalRecord = medicalRecordRepository.findById(medicalRecordId);

    // assert response
    assertThat(medicalRecord).isEmpty();
}
Java
@Test
void shouldSaveMedicalRecordSuccessfully() {
    // create medical record
    var medicalRecord = new MedicalRecord();
    var medicalRecordId = RandomString.make();
    var patientName = RandomString.make();
    var diagnosis = RandomString.make();
    medicalRecord.setId(medicalRecordId);
    medicalRecord.setPatientName(patientName);
    medicalRecord.setDiagnosis(diagnosis);

    // call method under test
    medicalRecordRepository.save(medicalRecord);

    // assert record existence in datasource
    Optional<MedicalRecord> retrievedMedicalRecord = medicalRecordRepository.findById(medicalRecordId);
    assertThat(retrievedMedicalRecord).isPresent().get().satisfies(record -> {
        assertThat(record.getId()).isEqualTo(medicalRecordId);
        assertThat(record.getPatientName()).isEqualTo(patientName);
        assertThat(record.getDiagnosis()).isEqualTo(diagnosis);
    });
}
And with the above, we've successfully tested our repository class that interacts with DynamoDB table for storing and retrieving records.
The created LocalStack containers will be destroyed automatically once the Test Suite is completed, hence we do not need to worry about cleanup.


Pro-tip: Reduce Boilerplate Testcontainers Code

In our Integration Testing setup, we encounter repetitive boilerplate code for initializing and configuring the LocalStack container, as well as dynamically adding properties to it. These steps will be repeated in every Test class that tests a functionality related with the data layer.
This duplication clutters our Test classes and we'll be looking at a possible solution for this issue using the lifecycle callback extension available in JUnit 5 called  @BeforeEachCallback .
Inside our test folder itself, we'll be creating a class  DynamoDBInitializer  which implements the  BeforeEachCallback  interface and performs the boilerplate functionality.
Java
@Slf4j
public class DynamoDBInitializer implements BeforeAllCallback {

	private static final DockerImageName LOCALSTACK_IMAGE = DockerImageName.parse("localstack/localstack:3");
	
	private static final LocalStackContainer localStackContainer = new LocalStackContainer(LOCALSTACK_IMAGE)
			.withCopyFileToContainer(MountableFile.forClasspathResource("init-dynamodb.sh", 0744), "/etc/localstack/init/ready.d/init-dynamodb.sh")
			.withServices(Service.DYNAMODB)
			.waitingFor(Wait.forLogMessage(".*Executed init-dynamodb.sh.*", 1));

	@Override
	public void beforeAll(final ExtensionContext context) {
		log.info("Creating localstack container : {}", LOCALSTACK_IMAGE);
		
		localStackContainer.start();
		addConfigurationProperties();
		
		log.info("Successfully started localstack container : {}", LOCALSTACK_IMAGE);
	}

	private void addConfigurationProperties() {
		System.setProperty("com.behl.aws.access-key", localStackContainer.getAccessKey());
		System.setProperty("com.behl.aws.secret-access-key", localStackContainer.getSecretKey());
		System.setProperty("com.behl.aws.region", localStackContainer.getRegion());
		System.setProperty("com.behl.aws.endpoint", localStackContainer.getEndpoint().toString());
	}

}
Then, we'll be creating a custom Marker Annotation  @InitializeDynamoDB  that registers the above created Extension class using  @ExtendWith .
Java
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@ExtendWith(DynamoDBInitializer.class)
public @interface InitializeDynamoDB {
}
Now, we can annotate all of our Integration Test classes which require DynamoDB Table creation to test functionalities with our custom annotation  @InitializeDynamoDB .
Java
@SpringBootTest
@InitializeDynamoDB
@ActiveProfiles("test")
class MedicalRecordRepositoryIT {

	@Autowired
	private MedicalRecordRepository medicalRecordRepository;

	@Test
	void shouldReturnEmptyOptionalForNonExistingMedicalRecord() {
    	// test case
	}

    @Test
    void shouldSaveMedicalRecordSuccessfully(){
        // test case
    }

}
NOTE: To have a single shared container that is used by all Integration Test classes, we've implemented the BeforeAllCallback interface. This approach will reduce test execution time significantly and eliminate the need to start and stop containers for each test class. For cases where this behaviour is not needed, BeforeEachCallback interface can be implemented instead.


Conclusion

Through this article, we've explored on how we can create an isolated testing environment with LocalStack and Testcontainers to write Integration Tests for our data layer, specifically for one that interacts with AWS DynamoDB Database.
Using what we've learnt, we can confidently write integration tests without relying on external resources or impacting production data.  
# dynamodb# localstack# testcontainers# spring-boot# integration-testing
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article