AWS Builder Center

Builder Center

DeepRacer on AWS: Analyzing Your Training Logs - 2026 Edition

DeepRacer on AWS: Analyzing Your Training Logs - 2026 Edition

So you've been training an AWS DeepRacer model in the new DeepRacer on AWS Solution? How do you make sense of what the car actually learned? This guide walks you through setting up the deepracer-analysis notebook collection on a Windows machine and shows you the kinds of insights you can extract from your training logs.

Lars Lorentz Ludvigsen

Background

In the new DeepRacer on AWS  (DRoA) solution, regular racers do not have direct access to the training log files in an S3 bucket. Instead, you download a tar.gz archive from the models page in the DRoA console. With recent updates to the deepracer-utils  library, this downloaded archive can be loaded straight into the deepracer-analysis  notebooks — no Docker containers, AWS CLI configuration or S3 access required.

Prerequisites

Windows components

Starting from a clean Windows installation, you need three things:
  • VS Code — available from the Windows Store
  • Python — available from the Windows Store
  • Git — install via winget (winget install --id Git.Git -e --source winget) or download from Git 
You will also want to enable long path support (required because the Python virtual environment can create deeply nested paths). Run the following in an Administrator PowerShell:
1
2
New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" `
-Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force
And fix the script execution policy in a regular PowerShell:
1
Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned

VS Code extensions

Install the following extensions from within VS Code:
  • Python (ms-python.python)
  • Jupyter (ms-toolsai.jupyter)

Setting Up the Repository

Clone the repository

Open VS Code. Open the command palette (Ctrl+Shift+P), select Git: Clone, and clone:
1
https://github.com/aws-deepracer-community/deepracer-analysis.git
Select a local folder where the repository should be placed.

Create a Python environment

Open Training_analysis.ipynb. The Jupyter extension will load and will likely select the default Python kernel that we just installed. We need to install a custom environment to ensure all of our needed libraries can be installed.
Choose Select another kernel → Python Environments → Create Python Environment → Quick Create. This will create a .venv virtual environment and automatically install all dependencies listed in requirements.txt.

Loading Your Training Logs

Download the archive

On the models page in the DeepRacer on AWS console, go to the training section and click Download logs. This gives you a tar.gz archive. which will be downloaded as per your browser configuration.

Configure the notebook

The notebook supports three data sources via the MODE setting:
ModeDescription
MODE.TARA tar.gz archive downloaded from DeepRacer on AWS
MODE.FSA model folder extracted onto your local disk
MODE.S3A model folder stored in an S3 bucket
To be able to read in the downloaded archive update it to MODE.TAR.
1
my_mode = MODE.TAR
Then scroll down to the DeepRacer on AWS zipped archive section and point the path to your file:
1
ARCHIVE_PATH = 'logs\my-model-logs.tar.gz'
Click Execute above cells inside that section, then run the current cell. You should see an output like:
1
2
Folder type detected: DROA_SOLUTION_LOGS
Loaded training trace: 44247 steps, 560 episodes, 28 iterations
A following cell will attempt to print agent, network, and hyperparameter metadata; for DRoA archives this is not available and will print Metadata not available, which is expected.
Proceed further, stop at the cell that displays df.head() — if you see a table of simulation trace data, you have successfully loaded the log.
5 first steps in the trace log

Stability Analysis

Before diving into reward graphs, it's worth checking how smoothly the simulator was running. DeepRacer's simulator targets 15 frames per second, meaning each step should be roughly 66.6 ms apart. The stability summary flags two warning thresholds:
  • Mean step time > 70 ms — overall performance degradation
  • 95th-percentile step time > 90 ms — frequent spikes causing inconsistent physics
If either threshold is exceeded, the simulator was struggling during that run, which can explain unexpectedly noisy training results.
The summary also shows:
  • The real-time-factor (RTF) that shows how fast the training runs vs. the wall-clock. Numbers in the range from 65% to 75% is normal.
  • The training and policy update elapsed time per iteration, and the ratio between them.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
label steps avg_ms max_ms p95_ms std_ms rtf train_s policy_s ratio
----------------------------------------------------------------------------------------
0/0 1793 66.2 140.0 79.0 8.7 0.740 163.8 28.9 5.67
0/1 1458 66.1 134.0 78.0 8.1 0.749 133.8 43.5 3.08
0/2 1455 66.1 137.0 79.0 8.4 0.744 134.1 26.5 5.05
0/3 1278 66.2 141.0 78.0 8.3 0.743 117.2 47.8 2.45
0/4 1707 66.1 138.0 78.0 8.3 0.737 157.9 47.1 3.35
[...]
0/18 3785 66.2 150.0 79.0 8.5 0.762 332.3 52.8 6.29
0/19 4127 66.1 140.0 78.0 8.1 0.764 362.3 57.0 6.36
0/20 4032 66.1 138.0 79.0 8.2 0.763 352.9 55.1 6.41
0/21 3890 66.1 147.0 78.0 7.7 0.763 340.1 54.1 6.29
0/22 4018 66.1 138.0 78.0 8.2 0.755 355.0 n/a n/a
----------------------------------------------------------------------------------------
OVERALL 64166 66.1 150.0 78.3 8.1 0.754 248.3 46.5 5.29

Loading the Track

The notebook includes waypoint files for all official DeepRacer tracks. These define the center line, inside border, and outside border coordinates, and are used to overlay the car's path onto a visual representation of the track.
A searchable list lets you find your track by filename or official name:
Track Selector Widget
When loading from a tar.gz archive the notebook cannot auto-detect the track. You need to set it manually:
1
track_name = "reinvent_base" # replace with your track's filename (without .npy)
Run the cell to verify the track renders correctly.
reinvent_base waypoints

Analyzing Training

The analysis section is the heart of the notebook. Episodes are grouped into iterations, and a series of graphs show reward, progress, lap times, and completion rate over training time. These let you quickly judge whether the model is improving, plateauing, or regressing — and spot common issues like a learning rate that is too high (zig-zagging reward) or a reward function problem (sudden collapse to near-zero).
Beyond the per-iteration overview, the notebook also breaks down stats for all laps and for complete laps only, and offers a quintile view that splits the training into fifths so you can compare early and late behaviour side by side.]
SectionDescription
Training progressPer-iteration graphs of reward, progress, lap time, and completion rate show whether the model is improving, plateauing, or regressing. Stats views for all laps and complete laps only, plus a quintile split, round out the picture.
Data in tablesReady-made queries surface the fastest laps, best-rewarded episodes, or every step within a single episode. Useful for investigating anything the graphs flag as unusual.
Reward distributionA per-waypoint bar chart for a single episode reveals exactly how the reward function fired on that run, making disproportionate spikes or reward-sparse sections immediately visible.
Path analysisOverlays the car's driven route on the track outline and renders a reward heatmap across all training steps. Drill down to a single iteration or episode to understand where on track the car is succeeding or struggling.
Action breakdownShows which steering and speed combinations were used in each section of the track (discrete action spaces only), useful for spotting whether the car relies on a narrow subset of its action space.
You can visually see how the model converges as the iterations progress. (It is animated in the real notebook!)
You can visually see how the model converges as the iterations progress. (It is animated in the real notebook!)

Summary

The deepracer-analysis notebook gives you a structured way to move from a raw training archive to a detailed understanding of how your model learned. The key workflow is:
  1. Download the tar.gz archive from the DRoA console
  2. Point ARCHIVE_PATH at the file and set my_mode = MODE.TAR
  3. Run through the cells top to bottom, pausing to check the stability summary and track plot
  4. Use the training progress graphs to judge whether the model improved, plateaued, or regressed
  5. Drill into specific episodes with path plots and reward distribution charts when the aggregate graphs raise questions
Happy racing.
The deepracer-analysis notebooks and the deepracer-utils library are community projects maintained by the AWS DeepRacer Community   and have recently been updated to support the new DeepRacer on AWS Solution.
# aws-deepracer# jupyter-notebook# ai-ml
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article