Reachy Mini x Strands Agent on Jetson Thor - Part 13: Recording Interactions to S3 with Presigned URLs
Part 13 of the series: each interaction is recorded to a short MP4 by sampling the shared camera buffer, uploaded to Amazon S3, and a presigned download URL is attached to the reply — so whoever receives the event can watch what the robot saw. All swallowed on failure so video never breaks an interaction.
Series: Reachy Mini x Strands Agent on Jetson Thor (14 articles)
- …
- 13Reachy Mini x Strands Agent on Jetson Thor - Part 13: Recording Interactions to S3 with Presigned URLs This article
What would I like to build?
This is Part 13 of the series. The telemetry from Part 12 tells a subscriber what the robot did — but not what it saw. This post adds that: each interaction (and each presence detection) is recorded to a short MP4, uploaded to Amazon S3, and a presigned download URL is attached to the outgoing message, so whoever receives the event can watch the exact clip.
The crucial constraint is the camera. The device is owned by a single streaming loop that keeps
_latest_frame fresh for every consumer. Opening a second cv2.VideoCapture to record would collide with that owner — so the recorder never does. It samples the shared buffer instead.The second principle is that video must be invisible when it fails. The entire encode-and-upload path is wrapped so any error is swallowed and the message simply ships without a
video_url — a recording problem can never break or slow an interaction.Goals
- Record each wake interaction (and each presence detection) to a short MP4
- Sample the shared camera buffer — never open a second capture that would fight the owner
- Upload to Amazon S3 and attach a presigned GET URL to the reply / presence message
- Cap memory with a frame budget, and keep encode dependency-light (
mp4v, noimageio) - Swallow all failures — video must never break an interaction — and no-op when the camera or MQTT is off
The Overall System
For a wake,
handle_wake starts a recorder thread before the agent runs and stops-and-uploads after the reply. The recorder samples _latest_frame; on stop, the buffered frames are encoded to an MP4, uploaded to S3, and a presigned URL is attached to the reply message.System Components:
start_recording/_record_loop— a daemon thread that samples_latest_frameatVIDEO_FPS- Shared camera buffer —
_latest_frame, read under_cam_lock _encode_and_upload—cv2.VideoWriter(mp4v) →boto3upload → presigned URL- Amazon S3 — clips land under
videos/reachy_<timestamp>.mp4 - The reply / presence message — carries the
video_urlwhen present
Sequence Diagram
Follow a wake interaction — start recording, sample through the interaction, encode, upload, and attach the presigned URL to the reply.
Architecture
There are two entry points, both ending in
_encode_and_upload:| Path | Trigger | Capture | Duration |
|---|---|---|---|
start_recording() / stop_recording_and_upload() | wake interaction (handle_wake) | background thread spanning the whole interaction | up to VIDEO_MAX_SECONDS |
record_clip_and_upload() | presence detection (report_human_presence / report_cat_presence) | synchronous, inline | VIDEO_CLIP_SECONDS |
For a wake,
handle_wake calls start_recording() before building the agent and stop_recording_and_upload() after the reply is spoken; stop_recording() runs in finally so the capture thread is always torn down. An instantaneous event like presence has no interaction window, so record_clip_and_upload(seconds) records a fixed-length clip synchronously.How it works
Sampling the shared camera buffer
Opening a second
cv2.VideoCapture would collide with the camera owner, so the recorder samples _latest_frame under _cam_lock and copies each frame:1
2
3
4
5
with _cam_lock:
frame = _latest_frame
if frame is not None:
_rec_frames.append(frame.copy())
time.sleep(period)start_recording() spins a daemon thread (_record_loop) that samples at VIDEO_FPS until stop_recording() sets _rec_stop, or until VIDEO_FPS * VIDEO_MAX_SECONDS frames are buffered (a memory cap). It returns False and is a no-op when there's nowhere to send the result — MQTT off (_iot_conn is None) or the camera owner not streaming (_cam_active not set).Encoding and uploading to Amazon S3
_encode_and_upload(frames) handles both paths. It writes the buffered frames to a temp file with cv2.VideoWriter using the mp4v fourcc (no extra imageio dependency), uploads with boto3, then returns a presigned URL:1
2
3
4
5
6
7
8
9
10
11
12
writer = cv2.VideoWriter(tmp_path, cv2.VideoWriter_fourcc(*"mp4v"), VIDEO_FPS, (w, h))
for f in frames:
writer.write(f)
writer.release()
s3 = boto3.client("s3")
key = f"videos/reachy_{stamp}.mp4"
s3.upload_file(tmp_path, S3_BUCKET, key)
url = s3.generate_presigned_url(
"get_object",
Params={"Bucket": S3_BUCKET, "Key": key},
ExpiresIn=PRESIGNED_URL_EXPIRY,
)Objects land under
videos/ keyed by timestamp. The whole body is wrapped in a try/except that swallows any failure (returning None) and a finally that removes the temp file — encoding or upload must never break an interaction. Empty frame lists short-circuit to None.Presigned download URLs
The presigned GET URL expires after
PRESIGNED_URL_EXPIRY seconds and is attached to the outgoing message as video_url, included only when non-None:1
2
publish_state("reply", request=request, reply=reply,
**({"video_url": video_url} if video_url else {}))The same pattern attaches
video_url to the presence message.Configuration
| Variable | Default | Purpose |
|---|---|---|
S3_BUCKET | reachy-mini-051826725803 | target bucket for uploaded clips |
VIDEO_FPS | 15 | sample rate from the shared buffer and the MP4 frame rate |
VIDEO_MAX_SECONDS | 120 | memory cap on an interaction recording |
VIDEO_CLIP_SECONDS | 30 | length of a one-shot event clip |
PRESIGNED_URL_EXPIRY | 3600 | presigned URL lifetime in seconds |
Technical Challenges & Solutions
Challenge 1: Recording without a second camera capture
Problem: The camera is a single-opener device already held by the streaming owner. A recorder that opened its own capture would collide and stall both.
Solution: The recorder samples the shared
_latest_frame under _cam_lock and copies each frame — it never touches the device. Recording coexists with face tracking and Cosmos vision because all three read the one owner's buffer.Challenge 2: Video failures breaking interactions
Problem: Encoding or an S3 upload can fail (disk, network, credentials). If that propagated, a recording problem would break the actual interaction.
Solution: The whole encode-and-upload path is wrapped in
try/except that returns None on any failure, with a finally that cleans up the temp file. The reply just ships without a video_url — video is strictly best-effort.Challenge 3: Unbounded memory on a long interaction
Problem: Buffering frames for an open-ended interaction could grow without limit.
Solution: The recorder caps the buffer at
VIDEO_FPS * VIDEO_MAX_SECONDS frames and samples at a modest VIDEO_FPS (default 15), bounding memory while still capturing the interaction.Challenge 4: Two very different capture windows
Problem: A wake has a natural start/stop window; a presence detection is instantaneous with no window to wrap.
Solution: Two entry points share one encoder — a background thread spans the wake (
start_recording / stop_recording_and_upload), while record_clip_and_upload(seconds) grabs a fixed-length clip synchronously for instantaneous events.Getting Started
GitHub Repository: https://github.com/chiwaichan/chiwaichan-reachy-mini-strands-agent-jetson-thor
Enable clips
With the camera owner up and MQTT configured, set the bucket and run:
1
2
export S3_BUCKET=your-clip-bucket
./reachy_assistant.shEach wake reply and presence event that arrives on the MQTT topics now carries a
video_url — a presigned link to the clip the robot recorded. When the camera or MQTT is off, both paths are no-ops and messages simply ship without a video_url.What's Next
In Part 14 - The Camera/Mic Media Bus , I generalise the single-owner idea into a media bus: one broker owns each single-opener device and fans the live stream out over Unix sockets, so any number of processes — including new ones you add — can read the same camera and mic at once.
Summary
This post covered interaction clip recording to S3:
- Sample, don't re-open — the recorder copies the shared
_latest_frameunder_cam_lock, so it never collides with the camera owner - Encode + upload —
cv2.VideoWriter(mp4v, noimageio) →boto3upload tovideos/reachy_<ts>.mp4→ presigned GET URL - Best-effort by design — the path swallows all failures (returns
None) and is a no-op when the camera or MQTT is off, so video never breaks an interaction - Two paths, one encoder — a background thread spans a wake;
record_clip_and_uploadgrabs a fixed clip for instantaneous presence events - Attached to the event — the
video_urlrides on thereply/presencemessage, included only when present, expiring afterPRESIGNED_URL_EXPIRYMore in this series - Previous - Part 12: Robot-State Telemetry to AWS IoT Core
- Next - Part 14: The Media Bus - One Owner per Device
Series: Reachy Mini x Strands Agent on Jetson Thor (14 articles)
- …
- 13Reachy Mini x Strands Agent on Jetson Thor - Part 13: Recording Interactions to S3 with Presigned URLs This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article