On Behavioural Observability: The Missing Layer for Continuously Improving AI Agents
How to build a flywheel that continuously improves your agents
Series: Building Agents (1 article)
- 1On Behavioural Observability: The Missing Layer for Continuously Improving AI Agents This article
Building agents that perform useful work is only the first step. Choosing an agent framework, defining tools and skills, implementing evaluation, observability, and deployment pipelines are all necessary, but they don't answer the most important question:
How does the agent improve after it reaches production?
Every invocation generates evidence: reasoning traces, tool selections, outputs, failures, and outcomes. Modern agent platforms such as Amazon Bedrock AgentCore already emit valuable operational observability signals such as latency, errors, traces, and execution metrics. These are essential for operating agents reliably in production.
They are not, however, sufficient to improve the agent.
Continuous improvement requires operational observability to be extended into behavioural observability. Behavioural observability captures not only what happened, but why, what decisions the agent made, and whether those decisions were correct. It records reasoning traces, tool selection, intermediate conclusions, produced outcomes, and their subsequent evaluation. Only when this richer behavioural context is available does telemetry become feedback. That feedback is what powers the agentic flywheel.
Every production agent should already emit foundational operational telemetry. The flywheel extends this foundation by generating and capturing additional behavioural observability data that can be evaluated, judged, and fed back into the agent. This additional instrumentation is what enables continuous improvement.
Built-in operational observability tells you whether the agent executed correctly. Behavioural observability tells us whether it made the right decisions. A useful agentic flywheel can take various forms. After iterating over alternatives, we landed on the following two-part loop structure.
The first enriches operational telemetry with behavioural evidence that captures what the agent decided and how well those decisions performed. The second converts that evidence into improvements through calibration, evaluation, scoring, and deployment gates.
TL;DR
The key architectural idea is simple: operational observability tells us whether an agent executed correctly; behavioural observability tells us whether it made correct decisions. The flywheel is built by extending the former into the latter. The reference architecture uses an example agent to illustrate the flywheel implementation, which shows how behavioural observability data enables the flywheel that improves the agent with every turn.
The key architectural idea is simple: operational observability tells us whether an agent executed correctly; behavioural observability tells us whether it made correct decisions. The flywheel is built by extending the former into the latter. The reference architecture uses an example agent to illustrate the flywheel implementation, which shows how behavioural observability data enables the flywheel that improves the agent with every turn.
In our implementation example, we're using an agent built with the Strands SDK , hosted on Amazon Bedrock AgentCore components, using built-in Amazon CloudWatch GenAI Observability as a foundation. Then we extend the 'signal' with additional custom Amazon CloudWatch Metrics , and we start streaming agent reasoning/response/tool-call content to Amazon CloudWatch Logs . Then we define a new dedicated Amazon DynamoDB repository that persists agent activity records and self-critique scores. The 'capture' stage then preserves agents' published decision content as input→output pairs on Amazon S3 . In aggregate, for each agent invocation session, we see operational metrics, the corresponding agent invocation prompt, the reasoning traces and the tool calls it made, as well as its outcome and conclusions: All the things we need to improve the agent's behaviour.
Note: The agentic flywheel concept builds on previous posts: Building on Amazon Bedrock AgentCore Runtime and Instrumenting Runtime Agents .
Another note: Use Kiro or your favourite coding assistant to generate additional implementation details. The provided code snippets aim at highlighting design decisions, not implementation guidance.
Building out an agentic flywheel consists of two distinct phases. First, we extend operational observability into behavioural observability. The goal is to capture enough evidence to understand not only what the agent did, but whether it made the right decisions.
Second, we use that evidence to improve the agent. Behavioural data becomes the input to calibration, evaluation, scoring, and deployment gates that continuously raise agent quality.
The following diagram encapsulates the idea:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
**Operational Observability**
EMF
Logs
Metrics
Traces
│ (1)
▼
**Behavioural Observability**
Activity Record
Decision Corpus
Self-Critique
Score
│ (2)
▼
**Flywheel**
Calibration
Replay
Eval+Judge
CILet's go.
Part 1: Build Behavioural Observability
Extending the Signal
We extend the built-in OpenTelemetry (OTel) formatted agent observability of AgentCore in three distinct ways.
We extend the built-in OpenTelemetry (OTel) formatted agent observability of AgentCore in three distinct ways.
1/ The first extension is the cheapest, and the one most teams skip: create a workflow catalogue and know which of your agentic workflows actually fired.
Our agent trigger architecture is based on AWS Lambda functions that are invoked by different trigger sources, such as S3 EventBridge events , Amazon Managed Grafana (AMG) alerts via SNS subscriptions , and scheduled triggers with Amazon EventBridge scheduler : All of them are dispatched through a shared Lambda function module rather than each calling `InvokeAgentRuntime ` themselves directly. That module defines the centralised 'registered/available workflows' and owns the payload contract, the timeouts, and CloudWatch metrics emitted per agent dispatch.
The metrics are emitted as Embedded Metric Format (EMF) , a structured JSON line on stdout that CloudWatch extracts into a metric: No `PutMetricData ` call, no extra AWS IAM permissions , no client to initialise. For a Lambda function that already logs, EMF is nearly free. The following snippet highlights the simplicity.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
def _emit_dispatch_metric(result: DispatchResult) -> None:
if not _METRICS_ENABLED:
return
emf = {
"_aws": {
"Timestamp": int(time.time() * 1000),
"CloudWatchMetrics": [
{
"Namespace": _METRIC_NAMESPACE,
"Dimensions": [
["trigger_type", "status"],
["status"],
["trigger_type"],
],
"Metrics": [{"Name": "Dispatch", "Unit": "Count"}],
}
],
},
"trigger_type": result.trigger_type,
"status": result.status,
"cluster": result.cluster or "none",
"session_id": result.session_id,
"Dispatch": 1,
}
print(json.dumps(emf)) # standalone JSON line on stdout — CloudWatch extracts itThe EMF metrics are consumed in three places.
- A CloudWatch alarm on `status=failed` catches the failure mode
- A CloudWatch dashboard shows dispatches by outcome and by trigger
- A scheduled rollup Lambda that builds a periodic activity report from it
Sidebar: Supporting a `no_action` status is important: A trigger that results in finding nothing actionable, and therefore declines to invoke the agent, is working correctly. But if that were silent, a no-op and a broken S3 event notification would look identical. Emitting a `no_action` status makes “nothing is arriving at all, check the wiring” a distinguishable state.
2/ The EMF metrics are for counting and alarming; alongside these metrics, each meaningful agent action writes metadata as a durable activity record item to DynamoDB. Importantly, DynamoDB holds the itemised history that the metric aggregates away.
Then, a scheduled rollup function aggregates over both (including the built-in AgentCore Runtime metrics for application and the usage logs) and writes a single JSON consolidated activity report to S3. The activity report consolidates across CloudWatch Metrics, Logs Insights, and DynamoDB activity record items. Only the finished JSON is accessed by a consumer, avoiding direct access to any of the agent's metrics or logs.
The following snippet shows an example rollup activity report.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
{
"schema_version": "1",
"generated_at": "2026-08-01T21:10:21+00:00",
"window": {
"from": "2026-07-25T21:10:21+00:00",
"to": "2026-08-01T21:10:21+00:00",
"label": "7d"
},
"config": {
"workflows": [
{
"id": "workflow_1",
"trigger": "s3_event_trigger",
"state": "live"
},
[...]
]
},
"value": {
"totals": {
"mr_opened": 17,
"mr_exists": 54,
"investigated": 36,
"no_op": 184,
"error": 0
},
"by_cluster": {
"c1-prod": {
"mr_opened": 1
},
[...]
},
"recent": [
{
"ts": "2026-08-01T11:15:16+00:00",
"cluster": "c1",
"action": "investigated",
"symptom": "",
"summary": "Posted to Slack: \ud83d\udea8 Alert: c1 \u2014 Application Availability at 93.8%",
"mr_url": "",
"session_id": "98275e02-6118-44dd-927e-375864932898",
"critique_score": 0.9,
"needs_review": false,
"artifact_kind": "diagnosis",
"corpus_key": "agent-decisions/2026-08-01/workflow_id/1785582916-98275e02-6118-44dd-927e-375864932898.json"
},
[...]
]
},
"activity": {
"by_trigger": {
"workflow_1": {
"dispatched": 21
[...]
}
},
"cost": {
"vcpu_hours": 13.055,
"gb_hours": 1358.565
}
}Notice the 'state' field. The rollup builds the workflow list from CloudWatch ListMetrics , so a workflow that never fired within a given time window wouldn't appear at all. Hence, we built an EventBridge scheduled heartbeat that pings each trigger, which emits a WorkflowEnabled datapoint per registered workflow (1 enabled, 0 denylisted or globally off) and returns without invoking the agent. That gives us three distinguishable states: live (ran, including no_action), idle (armed but quiet), dark (dry-run or disabled). This provides us with an option to "ship dark, then flip live" toggle.
The 'mr_opened' and 'mr_exists' fields count the number of merge requests the agent has created, which is a particular tool we build for the agent. Providing an agent a tool to create GitOps merge requests directly can be a low-risk option for an agent to propose remediations. The 'mr_exists' field represents a de-duplication method that avoids creating multiple MRs for the same issue: A high ratio means the agent keeps re-finding problems whose MRs nobody merged.
Also notice the costs field, which is read from the AgentCore usage logs . These are not Amazon Bedrock model inference costs , but the AgentCore compute costs.
3/ Extend the built-in agent logging with an additional CloudWatch log group. The AgentCore application-log group carries the agent invocation envelope with a null response payload, and native stdout is grouped per microVM, so neither gives us a readable invocation. We use a handler that groups by session_id (one stream per invocation) and a Strands callback handler that routes reasoning, tool calls, and streamed response text into it. Reasoning text arrives token-by-token with no completion event, so it's buffered and flushed on the next non-reasoning event. That's the concrete "operational → behavioural" move to augment our data set.
Behavioural observability now knows the workflow executed.
Capturing Behaviour
The signal tells us what decisions happened, and the capture preserves what the decision was. Every time the agent completes a workflow, it publishes something, and we write one JSON object to S3. The following snippet provides an example.
The signal tells us what decisions happened, and the capture preserves what the decision was. Every time the agent completes a workflow, it publishes something, and we write one JSON object to S3. The following snippet provides an example.
1
2
3
4
5
6
7
8
9
10
{
"schema_version": "1",
"ts": "2026-07-12T20:03:11Z",
"session_id": "...",
"trigger_type": "workflow_1",
"action": "investigated",
"cluster": "...",
"card": { "title": "...", "body": "...", "facts": { } },
"prompt": "..."
}Two fields carry the weight. `card` is the rendered artefact as the operator saw it — the title, body and structured facts of the Teams message. `prompt` is the input the agent was handed.
The input prompt and the output card together form an input→output pair, which is the smallest unit that can be judged. A card on its own cannot be judged: you cannot tell a shallow diagnosis from a correct one if we don’t know what evidence was in scope.
Three additional design choices proved load-bearing.
- The agent decision corpus is a separate store:
The agent activity records (in DDB) are one-line summary items, because they back the live 'hot' activity feed and a rollup that range-queries them on a schedule. The corpus objects, however, are fat and 'cold', stored on Amazon S3, and are read only when an eval harness runs a playback or a human goes looking at them. We landed on building a hot and a cold data store for the same event, because the access patterns have nothing in common. - The observability layer must never break the thing it observes:
If the bucket is unset or S3 errors emerge, the write returns `False` and never raises. The same posture applies to the activity record and the critique. An agent that fails to post because its corpus write timed out is strictly worse than an agent with a gap in its corpus. - The session_id is the connecting link between the stores:
We use the session_id within the various observability data stores. It's the spine: EMF metric property → agent activity item → reasoning, tool calls and response text → corpus object → log stream → trace.
The data stores also use timestamps. Date/trigger partitioning is not decoration: It is what makes review scopeable. "Show me every post from last week for a given workflow" is an efficient prefix listing, not a full scan.
Behavioural observability now preserves what the agent decided
Review and self-critique
This stage implements an online 'adversarial' self-critique and scoring method of what the agent decided on each invocation. An automated review process is essential, as every outcome the agent publishes is captured, and that volume makes a human review to capture issues unscalable.
This stage implements an online 'adversarial' self-critique and scoring method of what the agent decided on each invocation. An automated review process is essential, as every outcome the agent publishes is captured, and that volume makes a human review to capture issues unscalable.
Therefore, immediately after each agent invocation completes, we re-read the decision with an adversarial reviewer prompt (a senior reviewer told to find the weakest point, not to be helpful) and score it on three axes:
- Accuracy: Does the stated root cause follow from the evidence shown?
- Completeness: Were the signals that matter examined before concluding?
- Actionability: Are the recommendations specific, safe, executable?
We get all this from one Amazon Bedrock Converse call, with temperature 0, 500 max tokens, 30-second read timeout, and a JSON output response: No additional agent, no tools, no loop. The mean of the three axes becomes the `overall` score, and `overall <= threshold` sets a `needs_review` rating, which is the only field anything acts on.
Sidebar: Both the score and the ‘needs review’ result land on the DynamoDB activity item, which can be used conveniently by a consumer or a frontend can offer a “Needs review” filter and a human’s attention goes to the flagged minority.
The scoring rubrics, however, need to be specific to the agent workflow being reviewed:
- Per-artifact rubrics
Agent can publish structurally different things by workflow, and each is now judged on what it is supposed to contain. For example, a remediation workflow is graded on whether the target and field are right and whether the magnitude is proportionate. A workflow that produces a status summary digest is graded on internal consistency and instruction-following, because the underlying report was fetched by a tool the reviewer cannot see. - Explicit scope rules
The reviewer is told not to penalise the absence of detail that lives somewhere it cannot see, and not to reward fluency, length, or confident tone. Confident tone can be a specific trap: the wrong answer might be very well written. - Numeric anchors
An unanchored version can compress everything into a low-scoring band, and a ranking that doesn't spread cannot order a queue. Each band now has a definition (0.9–1.0 is "no defect you can name", 0.3–0.4 is "a defect that could mislead an operator") with an instruction not to cluster.
The critique score results are written as triples: the overall_mean as float, the needs_review as bool, and artifact_kind as string
diagnosis|remediation_notice|status_digest, depending on which rubric graded it.Notice that scores are only comparable within an artefact kind, so a sorted queue should be created by workflow. Queues across kinds mix rubrics and rank badly: therefore segment first, then rank.
The scoring earns its keep as one thing only: a key to sort a review queue by, so the human starts with the decisions most likely to be wrong instead of reading the corpus front to back. The guardrail is written into the data contract: a critique score can triage, but it must never promote. Nothing may become an eval scenario because the agent liked its own work.
Behavioural observability now records if the decision appears correct.
Part 2: Turn the Flywheel
Everything above runs continuously. What follows is the part a human turns.
Calibrate
The calibration is driven by human review of low-scoring self-critique assessments. The first question when a low-scoring decision looks wrong is not "how do I fix the prompt". It is "what actually decided this". We have several decision layers:
The calibration is driven by human review of low-scoring self-critique assessments. The first question when a low-scoring decision looks wrong is not "how do I fix the prompt". It is "what actually decided this". We have several decision layers:
- deterministic configuration
- agent invocation filters
- agent system prompts
- agent skills and knowledge base
- workflow invocation prompts
Each of these layers influences the agent behaviour.
Sidebar: Being clear about where decisions happen is key to efficiently calibrating the agent: Get that wrong, and you may spend time on prompt engineering to change a threshold number that is defined in a configuration file.
A calibration matrix can be a useful mental model. The columns provide a four-step sequence across: What was observed • Who decided • Where to fix • How to verify. See the snippet below.
1
2
3
4
5
CALIBRATION = {
ThresholdError: Config,
RoutingError: SupervisorPrompt,
AnalysisError: SpecialistPrompt,
}The rule of thumb:
- thresholds and verdicts are deterministic; they can be fixed quickly and followed up with a unit test. The flywheel turns quickly.
- Depth, consistency and tone are prompts; those need the corpus and an eval to prove a given change resolved the issue.
From experience, our working estimate is that about half of what initially read as "the agent is inconsistent" actually sits in this deterministic layer, resulting in "check who computed this" now being the first question, not the last. The layer that does need evals is prompts, and that is exactly the layer where we cannot write an assertion. There is no unit test for "good analysis".
Behavioural observability now identifies which layer owns the defect.
Evals
As we cannot write assertions, the solution is an evaluation scenario: a frozen issue (e.g. a specific kind of incident or alert the SRE agent needs to analyse) with a human-written expected answer. We define human-labelled eval scenarios, e.g., per agentic workflow, that encode the expected answer, which is then compared to the answer generated by the agent.
As we cannot write assertions, the solution is an evaluation scenario: a frozen issue (e.g. a specific kind of incident or alert the SRE agent needs to analyse) with a human-written expected answer. We define human-labelled eval scenarios, e.g., per agentic workflow, that encode the expected answer, which is then compared to the answer generated by the agent.
Each eval is a JSON file carrying a scenario context. For example, a particular alert payload, per-service availability, workload states, Kubernetes events, a timeline — plus an `expected_root_cause` and `expected_actions` written by a human labeller that knows what the right answer was.
Sidebar: Critically, they also carry an `excluded_data` list: facts that only became knowable after the incident, filtered out before replay, so the model is judged on the evidence that was actually available at the time. Without that filter, every scenario leaks its own answer.
With the eval scenarios in hand, we can now run human-in-the-loop offline eval replay runs. We use a real model over injected context, and an LLM judge performs the scoring. Every evaluation run produces a fresh answer, which an LLM judges against the human-written (labelled) expected answer.
Practically, the scoring replays the scenario against the real model, but not the full agentic loop; just a single structured model call. Our trade-off was deliberate:
- The multi-turn tool loop varies run to run in which tools are selected, which can make scores noisy and regressions unattributable.
- The scenario's actual 'live cluster state' is long gone, so live tool calls cannot work anyway.
- A single call still exercises the surface we care about: the system prompt of the specialist agents, the skill markdown , and the** knowledge base** documents go into the prompt, so degrading either measurably drops the score. That is the regression we want to catch.
The model returns `{root_cause, evidence, recommended_actions}`, and an LLM judge grades it against the human-written expectation on three dimensions: root-cause match, evidence quality, action relevance.
One hard-won detail on the judge: We started with a smaller model, and it was unreliable. It scored a verbatim correct root cause at 0.0 and varied widely between runs. Nuanced RCA grading turned out to need a strong model. Both the eval replay generator and the judge need strong models, which can come from different model providers. Amazon Bedrock is helpful here, as pointing the judge at a strong model from a different model provider is a one-variable change.
Behavioural observability now accumulates reusable expected behaviour
Gate
Reviewing, fixing, and validating low-scoring agent decisions feeds the continuous improvement loop of the flywheel. Defining CI gates can help lock in improvement gains and prevent regressions from shipping.
Reviewing, fixing, and validating low-scoring agent decisions feeds the continuous improvement loop of the flywheel. Defining CI gates can help lock in improvement gains and prevent regressions from shipping.
1/ Overnight run:
The overnight pipeline run is how we safely find out whether CI can invoke Bedrock with the right model permissions. The pipeline also executes the real replays plus judge at three runs per scenario: Using multiple runs per scenario helps with stability and reduces the impact of outlier results. This allows us to implement a progression: run the overnight pipeline in non-blocking mode. Detection first, enforcement second.
The overnight pipeline run is how we safely find out whether CI can invoke Bedrock with the right model permissions. The pipeline also executes the real replays plus judge at three runs per scenario: Using multiple runs per scenario helps with stability and reduces the impact of outlier results. This allows us to implement a progression: run the overnight pipeline in non-blocking mode. Detection first, enforcement second.
2/ Smoke test on push/merge:
On a normal push, run a cheap deterministic structural smoke test that catches broken scenario files and harness breakage: No Bedrock calls, no additional cost.
On a normal push, run a cheap deterministic structural smoke test that catches broken scenario files and harness breakage: No Bedrock calls, no additional cost.
3/ Knowledge base and documentation:
For an agent whose steering docs, skill markdown, and knowledge base documentation are runtime context, doc drift is a behavioural defect, not a tidiness issue. Which is why we use a checker that fails when a doc's file:line citation stops resolving, and a test asserting the always-injected docs match reality.
For an agent whose steering docs, skill markdown, and knowledge base documentation are runtime context, doc drift is a behavioural defect, not a tidiness issue. Which is why we use a checker that fails when a doc's file:line citation stops resolving, and a test asserting the always-injected docs match reality.
Quality gates don't improve the agent, but they prevent regressions while the flywheel improves it.
Behavioural observability now protects future behaviour from regression
Conclusions
The opening question was: "How does an agent improve after it reaches production?" The answer isn't a bigger model, another framework, or more prompt engineering.
It is a behavioural observability layer that captures the evidence needed to evaluate decisions, and a flywheel that systematically converts that evidence into improvements.
Every production agent emits foundational operational observability, but to achieve continuous improvement, we require an additional behavioural observability layer that captures the agent's decisions and the evidence needed to evaluate them: Runtime agents become continuously improvable only after operational observability is extended into behavioural observability.
The flywheel is the mechanism that continuously converts that richer evidence into better behaviour. Continuous improvement needs that second layer: capturing behavioural observability data is what enables turning the flywheel. The distinction that matters: Operational observability tells us the agent executed correctly. Behavioural observability tells us it decided correctly. Only the second one turns the wheel.
Concretely, for us, that is a dispatch metric per trigger, a durable itemised activity record, an input→output decision corpus, an adversarial review pass over each decision, a calibration surface that names which layer owns which class of defect, human-labelled eval scenarios, offline replay with an LLM judge, and a CI gate.
The flywheel turns with a human at the Review and Eval stages. That is deliberate and gives us control over the improvement loop. The Calibrate stage is driven by someone accountable for reviewing low-scoring decisions and implementing a fix at the correct layer. The Eval stage then encodes an expected answer that has to come from someone who knows the right answer.
The order matters more than the individual technologies. Build the agent. Extend its observability into behavioural observability. Then build the flywheel that uses that evidence to improve it.
The act of instrumenting an agent changes how you build the agent! What's next: The next layer is making the agent close its own improvement loop; going from human-controlled flywheel turns to increasingly self-directed improvement.
Series: Building Agents (1 article)
- 1On Behavioural Observability: The Missing Layer for Continuously Improving AI Agents This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article