AWS Builder Center
Getting Started with AWS Elemental Inference: A Python Sample for Ingest and Metadata Retrieval

Getting Started with AWS Elemental Inference: A Python Sample for Ingest and Metadata Retrieval

A Python sample (aws-samples) demonstrating the complete AWS Elemental Inference integration: create feeds, upload CMAF media via SigV4-signed data-plane calls, and retrieve AI metadata for smart cropping, event clipping (EventBridge), and smart subtitles. Covers two-plane API architecture, pull vs. push metadata delivery, and practical integration details. For SAs, developers building outside MediaLive, or anyone exploring the service.

We've published  sample-elemental-inference-ingest-metadata  on GitHub. It's a minimal, self-contained Python project that walks through the complete AWS Elemental Inference flow: segment a video, create a feed, upload media to it, and retrieve the AI-generated metadata. No frameworks, no production boilerplate, just clean, readable code that shows exactly how the pieces fit together.

Why this sample exists

In a production environment, AWS Elemental MediaLive handles Elemental Inference integration natively. You configure a feed, attach it to a channel, and the encoder manages ingest and metadata retrieval for you. But if you're building a custom integration, working on a proof-of-concept, or simply want to understand what's happening under the hood, you need to interact with Elemental Inference directly.
That direct interaction involves a few distinct concepts worth understanding up front:
  • The API has two distinct planes (control and data) with different calling conventions
  • The data plane is a SigV4 CMAF Ingest destination
  • Metadata delivery differs by feature: some results are pulled, others are pushed via EventBridge
  • There are specific constraints around resolution, segment duration, and end-of-stream signalling that aren't immediately obvious from the API reference alone
This sample demonstrates all of these in working code, so you can see the complete integration rather than assembling it from scattered documentation.

What you'll learn

The two-plane architecture

Like many AWS services, Elemental Inference separates control-plane and data-plane operations. The control plane (CreateFeed, AssociateFeed, GetFeed, DeleteFeed) uses the standard  elementalinference  boto3 client. The data plane (PutMedia, GetMetadata) uses SigV4-signed REST calls against a per-feed data endpoint returned by CreateFeed.
The sample demonstrates how to sign and issue these data-plane calls using  botocore.auth.SigV4Auth , the same signer boto3 uses internally, so no extra dependencies are needed. If you've worked with services like Kinesis Video Streams or IoT Data, the pattern will feel familiar.

Pull vs. push metadata delivery

The sample covers three Elemental Inference features, each demonstrating a different metadata delivery model:
Feature
Delivery model
What you get
Smart crop
Pull (GetMetadata)
Per-frame crop center coordinates as normalized x/y values
Event clipping
Push (EventBridge)
 Clip Metadata Generated  events with timestamps and quality scores
Smart subtitles
Pull (GetMetadata)
TTML subtitle documents with cue timing
Understanding this distinction is critical for integration planning. Smart crop and subtitles require you to poll for results over PTS windows. Event clipping requires you to set up an EventBridge rule and a downstream consumer (Lambda, SQS, CloudWatch Logs) to receive events as they're detected. The sample walks through both patterns.

Integration pitfalls (and how to avoid them)

The sample documents several practical integration details that are useful to know before you start building:
  • SigV4 payload signing: The data plane verifies  X-Amz-Content-SHA256  against the request body. Omitting this header before signing results in a  403 SignatureDoesNotMatch  for every non-empty request.
  • Segment duration constraints: Segments must be exactly 1 second. The service validates this and rejects anything else.
  • End-of-stream signalling: The AI features maintain a lookahead window. Without a proper end-of-stream signal (the  lmsg  brand in the last segment's  styp  box), the final metadata windows never flush.
  • Eventual metadata readiness: Inference runs asynchronously after media upload, so metadata for a given time window isn't available immediately. The sample demonstrates polling with backoff.
Each of these is documented alongside a working solution in the sample, so you can integrate with confidence from the start.

How it differs from production

It's worth being explicit: this is a teaching tool, not a production template. A real integration (like MediaLive's native one) streams CMAF fragments in real time, pulls metadata at the leading edge under a latency deadline, manages long-lived feeds, and handles credential refresh over extended sessions.
The sample trades all of that for legibility. It segments to disk, uploads in sequence, retrieves after the fact, and cleans up the feed when done. The API calls are identical to what a production system would make; the orchestration around them is deliberately simplified so you can focus on understanding the interface rather than the infrastructure.

Who should use this

  • Solutions architects evaluating Elemental Inference for a customer workload, who want to understand the integration model before recommending an architecture
  • Developers building custom integrations outside of MediaLive (for example, a file-based VOD workflow or a non-AWS encoder)
  • Anyone exploring the service who wants a working baseline to experiment with before writing their own code

Getting started

The project requires Python 3.9+, ffmpeg, and AWS credentials with Elemental Inference permissions. Clone the repo, install dependencies, and point it at any MP4 file. The  examples/  directory includes real captured outputs from actual service runs, so you can inspect the metadata shape without running anything.
The README includes the full IAM policy, CLI options, output format documentation, and detailed EventBridge rule setup for event clipping.
We welcome contributions and feedback via GitHub issues.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article