AWS Builder Center

Builder Center

Watching Claude Code Work: Streaming Agent Telemetry to CloudWatch and Rebuilding the Conversation

Watching Claude Code Work: Streaming Agent Telemetry to CloudWatch and Rebuilding the Conversation

When Claude Code runs in GitHub Actions, the agent works where you can't see it. Here's how to stream its OpenTelemetry logs and metrics to CloudWatch with a bearer token — no collector — and rebuild a readable conversation transcript from the raw events.

Walt

Once you move Claude Code off your laptop and into GitHub Actions, you gain a lot — parallel branches, no local dependency, agents scoped to exactly the credentials they need. But you lose one thing: you can't see what the agent is doing. It's working on an ephemeral runner somewhere, and when the job ends, the runner is gone. If a run does something surprising, all you have is the final PR comment and whatever scrolled past in the Actions log.
Claude Code has built-in  OpenTelemetry  support, and Amazon CloudWatch now accepts OpenTelemetry data through native  OTLP endpoints . Put those together and you can stream every prompt, tool call, and token count out of the runner and into a durable place you control — in real time.
This post walks through two things:
  1. Getting the telemetry out — metrics and log events to CloudWatch with a bearer token, no collector required.
  1. Making it readable — turning the raw OTLP log events back into a conversation transcript you can actually follow.
If you haven't set up Claude Code in GitHub Actions with Bedrock yet, I covered that foundation in  @claude in Your PRs, on Your Terms . This picks up where that leaves off.

The auth wrinkle that shapes everything

CloudWatch's OTLP endpoints support two authentication methods:  AWS Signature Version 4 (SigV4)  and bearer token (API key). AWS recommends SigV4 for workloads running on AWS — but SigV4 requires signing each request, and Claude Code's built-in OTel exporter only sends static headers. It can't sign.
That leaves two options:
  • Run a collector (the standard OpenTelemetry Collector with the  sigv4auth  extension) as a sidecar that signs and forwards. More moving parts.
  • Use a bearer token — a long-lived API key you pass as a plain  Authorization: Bearer <key>  header. Since that's just a static header, Claude Code's exporter can set it directly. No collector.
For a CI use case, the bearer token wins on simplicity. It's exactly the scenario AWS scopes it to:  "workloads running outside AWS or environments that do not support the AWS SDK."  A GitHub-hosted runner qualifies. The trade-off is that it's a long-lived credential, so you scope it tightly and rotate it — more on that below.

Metrics: the easy half

Metrics are the clean case. Generate a CloudWatch Metrics API key, and point Claude Code's exporter at the metrics endpoint:
YAML
- uses: anthropics/claude-code-action@v1
  env:
    CLAUDE_CODE_ENABLE_TELEMETRY: "1"
    OTEL_METRICS_EXPORTER: otlp
    OTEL_EXPORTER_OTLP_PROTOCOL: http/protobuf
    OTEL_EXPORTER_OTLP_METRICS_ENDPOINT: https://monitoring.us-east-1.amazonaws.com/v1/metrics
    OTEL_EXPORTER_OTLP_METRICS_HEADERS: "Authorization=Bearer ${{ secrets.CW_METRICS_API_KEY }}"
    OTEL_METRIC_EXPORT_INTERVAL: "10000"
Two details worth internalizing, because they trip people up:
  • Protocol must be  http/protobuf , never gRPC. The CloudWatch OTLP endpoint is  HTTP 1.1 only  — it explicitly does not support gRPC. (The gRPC  4317  port only applies when you're talking to a local collector.)
  • You generate the API key with  iam create-service-specific-credential  against a dedicated IAM user, using the managed   CloudWatchAPIKeyAccess   policy. That key can only call the metrics ingestion endpoint — nothing else in CloudWatch. Full setup is in the  bearer token docs .
Now for the gotcha that cost me an afternoon: these metrics never show up in  aws cloudwatch list-metrics . OTLP-ingested metrics don't land in the classic metrics store — they go into a  Prometheus-compatible store queryable only with PromQL . Query them in CloudWatch Query Studio's PromQL mode, through Amazon Managed Grafana, or via the Prometheus-compatible HTTP API:
Bash
awscurl --region us-east-1 --service monitoring -X POST \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d 'query={"claude_code.token.usage"}&start=<epoch>&end=<epoch>&step=300' \
  "https://monitoring.us-east-1.amazonaws.com/api/v1/query_range"
Claude Code emits  claude_code.token.usage ,  claude_code.cost.usage ,  claude_code.session.count , and  claude_code.active_time.total . An empty  list-metrics  result proves nothing — check PromQL before you conclude it isn't working.

Logs: three things instead of one

Log events are where the interesting story lives — the prompts, the tool decisions, the responses. Getting them flowing takes more than metrics did, because logs bearer auth has three requirements the metrics path doesn't:
  1. A separate API key. The logs endpoint uses a different service ( logs.amazonaws.com ) and a different managed policy (  CloudWatchLogsAPIKeyAccess  ). The metrics key does not work on the logs endpoint.
  1. A pre-existing log group and stream. The OTLP endpoint won't create them for you.
  1. Bearer-token auth explicitly enabled on the log group via  aws logs put-bearer-token-authentication . Without this, the endpoint rejects the token outright.
Once those exist, the exporter config mirrors the metrics one, with routing headers that name the target group and stream:
YAML
    OTEL_LOGS_EXPORTER: otlp
    OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: https://logs.us-east-1.amazonaws.com/v1/logs
    OTEL_EXPORTER_OTLP_LOGS_HEADERS: >-
      Authorization=Bearer ${{ secrets.CW_LOGS_API_KEY }},
      x-aws-log-group=/github-actions/claude-code,
      x-aws-log-stream=${{ github.repository }}/${{ github.run_id }}
Using  ${{ github.run_id }}  in the stream name gives every run its own log stream — which becomes important when you want to isolate a single agent session later.
Unlike the metrics, these land in normal CloudWatch Logs, so you query them with Logs Insights or  get-log-events  like anything else.

Push the resources into CloudFormation

The IAM users and the log group belong in infrastructure-as-code, so they go into a CloudFormation template — the two machine users with their managed policies, and the log group itself. The bearer tokens stay in the deploy script: they're short-lived credentials you'll want to rotate on a schedule, so minting them with an SDK/CLI call after the stack converges keeps rotation independent of stack updates.
The split is a useful rule of thumb: declare the durable resources, script the things that rotate.

Content verbosity: opt in deliberately

By default, the log events are metadata only — a  user_prompt  event with no prompt text, a  tool_result  with a byte count instead of output. Several flags unlock the actual content, and they're worth understanding individually because they differ in sensitivity:
Flag
Unlocks
Lands in logs?
 OTEL_LOG_USER_PROMPTS=1 
prompt text
✅
 OTEL_LOG_ASSISTANT_RESPONSES=1 
assistant reply text
✅
 OTEL_LOG_TOOL_DETAILS=1 
tool input arguments (the bash command, file paths)
✅
 OTEL_LOG_RAW_API_BODIES=1 
full Messages API request + response JSON
✅
 OTEL_LOG_TOOL_CONTENT=1 
full tool input and output bodies
❌ — trace spans only
That last row is a real trap.  OTEL_LOG_TOOL_CONTENT  sounds like exactly what you want — the tool output — but I tested it empirically and it adds nothing to the logs pipeline. It emits to trace spans (X-Ray), which the OTLP logs endpoint doesn't receive. The  monitoring docs  hint at this; the runner confirmed it.
One note on  OTEL_LOG_RAW_API_BODIES : it logs the entire conversation — every prompt, tool result, and response — on every request, so it's the most verbose flag by far. The data lands in your CloudWatch log group, governed by the same IAM and encryption controls as any other AWS resource, so this is a volume-and-access decision, not a data-exposure one: scope read access to the log group appropriately and mind the storage cost.

The part that isn't obvious: rebuilding the conversation

Here's the problem. Even with the content flags on, the log stream is a flat sequence of events —  api_request ,  mcp_server_connection ,  tool_decision ,  tool_result , interleaved. It's data, not a conversation. And the single most useful thing — the output of a tool call — isn't on the  tool_result  event at all.
I went looking for a first-party shortcut. Claude Code stores session transcripts locally as JSONL and has an interactive  /export  command that renders a readable transcript. But both need the runner's local session file and an interactive terminal — neither of which survives a finished GitHub Actions job. There's  no headless command that renders a transcript  from captured output. So reconstructing from the CloudWatch events is the durable path, and it's worth doing well.

Where the tool output actually lives

The  tool_result  event records  tool_result_size_bytes  — a byte count, not the content. But the output does exist in the log stream: it's inside the  api_request_body  events, because the result gets fed back to the model on its next turn as a  tool_result  content block. Every tool event carries a  tool_use_id , and that same ID appears in the API body. That's the join key.
So the reconstruction is: parse the events, build a map of  tool_use_id → output  by scanning the API bodies, and stitch each tool call to its result. A  jq  filter can do the whole thing. A few hard-won details:
  •  tool_use_id  values contain underscores ( toolu_bdrk_01Ntyf... ). A regex character class of  [A-Za-z0-9]+  silently matches nothing — you need  [A-Za-z0-9_]+ . This kind of bug is invisible until you test against real data.
  • Tool output often contains quotes. Extracting a JSON string value with  "content":"([^"]*)"  truncates at the first  \"  inside the payload (a  cat package.json  stops at  { ). The pattern has to consume escaped characters:  "content":"((?:[^"\\]|\\.)*)" .
  • Raw API bodies truncate at 60 KB inline. For very large outputs, the join is best-effort. Acceptable for normal commands; worth knowing.

Rendering it as YAML

Escaped JSON blobs are unreadable in a terminal ( "{\"body\":\"### Tasks\\n- [x]... ). The fix is to decode the nested JSON and emit real YAML — multi-line strings become block scalars, objects nest properly. A small recursive  jq  function does it, and the result reads like a transcript:
YAML
- time:  "17:33:33.702"
  event: tool_result
  tool:   Bash
  id:     toolu_bdrk_01B1JBU5nPvvnz5Qt5XJw1rm
  input:
    command: "ls"
    description: "List repo root files"
  output: |
    CLAUDE.md
    LICENSE
    README.md
    amplify
    ...
Two more moves make it genuinely followable:
  • Filter to conversation-relevant events. Drop  api_request  (token/cost noise),  mcp_server_connection ,  tool_decision  (redundant with  tool_result ), and internal tool-search calls. Keep prompts, responses, and meaningful tool results. A ~23-event stream collapses to ~7 you actually care about.
  • Use  start-live-tail , not  tail --follow . My first instinct —  aws logs tail --follow | jq  — returned nothing in my terminal. The AWS CLI v2 is a frozen binary that block-buffers stdout when its output is a pipe rather than a terminal, so a low-volume stream never fills the buffer and nothing reaches  jq . Rather than fight it with pty tricks, reach for the   start-live-tail   streaming API, which is built for this. Its  --mode print-only  emits one raw event JSON per line, unbuffered, made for piping — no  --follow , no poll loop, no dedup:
Bash
aws logs start-live-tail --region us-east-1 \
  --log-group-identifiers "arn:aws:logs:us-east-1:<account>:log-group:<group>" \
  --log-stream-names "<repo>/<run-id>" --mode print-only \
  | jq -Rr --unbuffered -f transcript.jq
One behavioral note: Live Tail streams events ingested after you connect — it has no history — so you start it while the run is going. That also means the  jq  becomes a small streaming state machine ( foreach inputs ): a  tool_result  whose output hasn't arrived yet is held until the API-body event carrying its output streams in, then released with the output attached.
The payoff: drop a collapsible one-liner into the PR comment, and anyone with read access to the account can watch the agent's conversation stream into their terminal as YAML — prompts, tool calls with arguments, and the outputs, live.

What I'd tell you to keep

  • Bearer tokens for CI telemetry — no collector, just env vars and a scoped, rotated key.
  •  http/protobuf , not gRPC, to the CloudWatch OTLP endpoint.
  • Metrics live in PromQL, logs live in Logs Insights — don't debug the wrong store.
  • Tool output isn't on the  tool_result  event — join to the API bodies by  tool_use_id .
  • For a live pipe, use  start-live-tail --mode print-only , not  tail --follow  — the streaming API pipes cleanly;  --follow  block-buffers.
  • Test regex assumptions against real data. The underscore-in- tool_use_id  bug was invisible until a real run proved it.
The broader point: an agent you can't observe is an agent you can't trust at scale. A durable, queryable telemetry trail — one that doesn't depend on an ephemeral runner or a version-fragile local file — is what makes it safe to let  @claude  loose across your repos.

Related reading

# ai# ai-dlc# agentic-ide# agentic-interactions
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article