
Guide: Integrating AWS APerf & async-profiler with Java Applications (Host & Container)
Learn how to integrate AWS APerf and async-profiler into your Java workflow - on the host or in containers - to uncover CPU hotspots, memory issues, and performance bottlenecks using reliable, low-overhead profiling techniques.
Introduction
This guide explains how to set up and use AWS APerf and async-profiler for profiling Java applications, both on a Linux host and inside a Docker container. The instructions are organized into two main sections: host-based profiling and container-based profiling.If you already have your own Java application or container image, steps for running a dummy Java app are optional. For containers, this guide includes the Dockerfile adjustments required to add APerf and async-profiler.
Purpose and Benefits
**Why use these profiling tools?**AWS APerf and async-profiler help you identify performance bottlenecks in your Java applications by:
- Detecting CPU-intensive methods and hotspots
- Analyzing memory allocation patterns and potential leaks
- Identifying thread contention and lock issues
- Providing flame graphs for intuitive visualization of call stacks
- Enabling data-driven optimization decisions
Using these tools can lead to significant performance improvements, reduced resource utilization, and lower operational costs for your Java applications.When to use which tool:
- APerf: AWS's performance analysis tool that provides comprehensive system-wide profiling with AWS-specific optimizations
- async-profiler: Lightweight, low-overhead Java profiler that uses HotSpot-specific APIs for accurate profiling without significant performance impact
Section A: Running APerf on the Host
1. Prerequisites
- This guide was tested on Ubuntu 22.04 LTS (x86_64). Commands and package names may differ on other distributions (e.g., Amazon Linux, CentOS, etc.).
- Java (OpenJDK 17+)
- Root or sudo privileges for installing packages and adjusting system settings
2. Install Java and System Tools
1
2
sudo apt-get update
sudo apt-get install -y openjdk-17-jdk wget curl linux-tools-commonThis installs Java and essential tools for performance profiling.
3. Install APerf
1
2
3
4
wget https://github.com/aws/aperf/releases/download/v0.1.15-alpha/aperf-v0.1.15-alpha-x86_64.tar.gz
tar xvfz aperf-v0.1.15-alpha-x86_64.tar.gz
sudo cp aperf-v0.1.15-alpha-x86_64/aperf /usr/local/bin/
sudo chmod +x /usr/local/bin/aperfThis downloads and installs the APerf binary system-wide.Verification: Confirm installation with
aperf --version, which should display the installed version.4. Install Async-Profiler 4.0
1
2
3
4
wget https://github.com/async-profiler/async-profiler/releases/download/v4.0/async-profiler-4.0-linux-x64.tar.gz
tar xvfz async-profiler-4.0-linux-x64.tar.gz
sudo mv async-profiler-4.0-linux-x64 /opt/async-profiler
sudo ln -sf /opt/async-profiler/bin/asprof /usr/local/bin/asprofThis installs async-profiler and makes its CLI (asprof) available in your PATH.Verification: Run
asprof -h to confirm the installation was successful.5. Adjust System Settings for Profiling
1
2
3
4
sudo sysctl kernel.perf_event_paranoid=0
sudo sysctl kernel.kptr_restrict=0
ulimit -n 4096perf_event_paranoid=0: Allows non-privileged users to access performance counterskptr_restrict=0: Enables access to kernel symbols for more detailed profiling- These settings are required for async-profiler to access kernel performance counters and symbol information.
Note: These sysctl settings do not persist across reboots.To make them persistent, add the following lines to
/etc/sysctl.conf:1
2
kernel.perf_event_paranoid=0
kernel.kptr_restrict=0Then run
sudo sysctl -p to apply.- The
ulimit -n 4096setting (open files) is per shell/session and does not affect containers unless set in the container's entrypoint or Dockerfile.
6. (Optional) Run a Java Application
If you already have a Java application running, skip this step. Otherwise, you can use a simple Java app for testing:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
cat > CPULoad.java << 'EOF'
public class CPULoad {
public static void main(String[] args) {
System.out.println("Starting CPU load test...");
int numThreads = Runtime.getRuntime().availableProcessors();
for (int i = 0; i < numThreads; i++) {
new Thread(() -> {
while (true) {
// Generate CPU load
for (long j = 0; j < 100_000_000; j++) {
Math.sqrt(j);
}
}
}).start();
}
// Keep the application running
try { Thread.sleep(Long.MAX_VALUE); } catch (Exception e) {}
}
}
EOF
javac CPULoad.java
java CPULoad &This creates a simple multi-threaded Java application that generates CPU load for testing.
7. Profile the Java App with APerf
- Find the Java PID:
1
jpsThis will list all Java processes. Identify your application's Process ID (PID).
- Profile with APerf and async-profiler:
1
aperf record --profile-java <PID> -r host_run -i 1 -p 10Replace
<PID> with your Java application's process ID.-i 1: Sets the sampling interval to 1 second-p 10: Sets the profiling duration to 10 seconds-r host_run: Specifies the output directory for raw profiling data- Generate a Report:
1
aperf report -r host_run -n host_reportThis creates
host_report.tar.gz (and/or a directory) in your current folder.8. Retrieve and View the Report
- The report (e.g.,
host_report.tar.gz) can be extracted and viewed locally:
1
2
tar -xzf host_report.tar.gz
cd host_report- Open the HTML report in your browser for analysis:
1
2
3
# If you have a GUI environment
xdg-open index.html # Linux
open index.html # macOSSection B: Running APerf in a Docker Container
1. Privileges Required
Profiling Java processes in containers requires extra privileges:
- Required Capabilities:
--cap-add SYS_ADMINand--security-opt seccomp=unconfinedare the most permissive and guarantee async-profiler will work.- For least privilege:
--cap-add CAP_PERFMONmay be sufficient on newer kernels, but some distributions and older kernels may still require SYS_ADMIN or additional capabilities.- Test with CAP_PERFMON first and escalate only if profiling fails.
Why these privileges are needed: The profiling tools need access to performance counters and kernel features that are restricted by default in containers for security reasons.
2. Dockerfile Adjustments for Existing Java Apps
Best Practice:Rather than altering production images directly, consider using a sidecar or ephemeral container approach for profiling.To add APerf and async-profiler to an existing image:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# Install async-profiler 4.0
RUN mkdir -p /opt/async-profiler && \
wget -O /tmp/async-profiler.tar.gz https://github.com/async-profiler/async-profiler/releases/download/v4.0/async-profiler-4.0-linux-x64.tar.gz && \
tar -xzf /tmp/async-profiler.tar.gz -C /opt/async-profiler --strip-components=1 && \
rm /tmp/async-profiler.tar.gz
# Install APerf
RUN mkdir -p /opt/aperf && \
wget -O /tmp/aperf.tar.gz https://github.com/aws/aperf/releases/download/v0.1.15-alpha/aperf-v0.1.15-alpha-x86_64.tar.gz && \
tar -xzf /tmp/aperf.tar.gz -C /opt/aperf --strip-components=1 && \
rm /tmp/aperf.tar.gz && \
chmod +x /opt/aperf/aperf
# Make async-profiler available in PATH
RUN ln -sf /opt/async-profiler/bin/asprof /usr/local/bin/asprofAdd these after your existing instructions so your Java app is still the container entrypoint.
3. Build and Run the Container
- Build the container (if you've modified a Dockerfile):
1
docker build -t my-java-app-with-profiler .- Run the container with appropriate privileges:
1
docker run --cap-add SYS_ADMIN --security-opt seccomp=unconfined -it --name my-java-app-profiled my-java-app-with-profilerOr try with
--cap-add CAP_PERFMON for least privilege:1
docker run --cap-add CAP_PERFMON -it --name my-java-app-profiled my-java-app-with-profiler- EKS/Kubernetes:
- Use a Kubernetes ephemeral container or a sidecar container to inject async-profiler and APerf into a running pod.
- For EKS, use
kubectl debugor add a sidecar in your deployment YAML. - Note: On newer Kubernetes clusters, Docker may not be present; use
kubectl cpandkubectl execinstead ofdocker cpanddocker exec.
4. Attaching to a Running Container
- If you cannot exec into the container (due to security policies or lack of shell):
- Use a Kubernetes ephemeral container or sidecar to run profiling tools in the same pod namespace.
- Copy async-profiler and APerf binaries into the running container or mount them via a shared volume.
- Use
kubectl execor the ephemeral container's shell to run profiling commands.
Example of using an ephemeral container in Kubernetes:
1
kubectl debug -it <pod-name> --image=ubuntu:20.04 --target=<container-name>5. Profile Your Java App in the Container
- Find the Java PID:
1
jps- Run APerf:
1
/opt/aperf/aperf record --profile-java <PID> -r container_run -i 1 -p 10- Generate the report:
1
/opt/aperf/aperf report -r container_run -n container_report6. Copy the Report to the Host
- Docker:
1
docker cp my-java-app-profiled:/workspace/container_report.tar.gz .- Kubernetes:
1
kubectl cp <pod-name>:/workspace/container_report.tar.gz .7. (Optional) Kubernetes/EKS Guidance
- For EKS and managed Kubernetes, prefer ephemeral containers or sidecars for profiling.
- Use
kubectl debug(Kubernetes 1.18+) to add a profiling container to a running pod:
1
kubectl debug -it <pod-name> --image=<profiling-image> --target=<container-name>- Use
kubectl cpandkubectl execfor copying files and running commands, as Docker CLI may not be available.
FAQ & Notes
- ulimit in containers:
ulimit settings must be configured in the container's entrypoint script or Dockerfile if needed inside the container. Host settings do not propagate automatically.
- sysctl persistence:
sysctl changes do not persist across reboot unless added to
/etc/sysctl.conf or /etc/sysctl.d/``.conf*.*Attaching to running containers:If docker exec or kubectl exec is not allowed, use sidecar or ephemeral containers for profiling.- Production safety:
Avoid modifying production images directly. Use sidecar containers, ephemeral containers, or overlay images for profiling tools.
- Kubernetes/containerd:
On clusters without Docker, use kubectl commands for file transfer and shell access.
Troubleshooting
Common Issues and Solutions
- Permission denied when running APerf
- Symptom: "Permission denied" or "Operation not permitted" errors
- Solution: Verify you've set the correct kernel parameters (
sysctl kernel.perf_event_paranoid=0)
- Missing symbols in profiling output
- Symptom: Stack traces show "unknown" methods
- Solution: Ensure
kernel.kptr_restrict=0is set and you're using debug symbols if available
- Container profiling not working
- Symptom: APerf fails to attach to Java process in container
- Solution: Try with
--cap-add SYS_ADMIN --security-opt seccomp=unconfinedifCAP_PERFMONis insufficient
- JVM crashes during profiling
- Symptom: JVM terminates unexpectedly during profiling
- Solution: Try reducing sampling frequency with
-iparameter (e.g.,-i 10for 10ms intervals)
- Empty or incomplete reports
- Symptom: Generated reports contain little or no data
- Solution: Ensure the profiling duration (
-pparameter) is sufficient for your application workload
Glossary
- APerf: AWS Performance Analysis Tool, designed for profiling applications on AWS infrastructure
- async-profiler: Low-overhead Java profiler that uses HotSpot-specific APIs
- CAP_PERFMON: Linux capability that grants access to performance monitoring subsystems
- CAP_SYS_ADMIN: Broad Linux capability that grants extensive system administration privileges
- Ephemeral container: Temporary container that runs in the same pod namespace as the target container
- Flame graph: Visualization that shows the distribution of time spent in call stacks
- JPS: Java Virtual Machine Process Status Tool, used to list Java processes
- Kubernetes pod: Group of one or more containers with shared storage and network resources
- Profiling: Process of collecting runtime information about a program's behavior
- seccomp: Secure computing mode, a Linux kernel feature that restricts the system calls available to a process
- Sidecar container: Container that runs alongside the main application container in the same pod
References
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article