Amazon CloudWatch Omni: An Architect's Read on Observability for the AI Era
The problem Omni is actually built for
An AI agent can hallucinate, pick the wrong tool, or answer from a stale knowledge base while p99 latency and error rate stay perfectly green. That single fact is why CloudWatch Omni exists, and it is the lens I use for the rest of this post.
In 17 years of running production platforms, the monitoring question was always binary enough: is it up, is it fast, is it throwing errors. Agents break that model. The same input can take a different reasoning path, call different tools and produce a different answer on every invocation.
The second problem is older and more familiar. Debugging a bad agent response today means hopping between an agent-tracing tool, an APM console and an infrastructure dashboard that share no data. The root cause might be the prompt, a retrieval call, a downstream API, or a database connection pool three layers down.
AWS's answer, announced GA on 23 September 2026, is to treat these as one problem: agent quality, application health and infrastructure signals in one data layer, queried from one place. Below is how it works, what it costs, and where I think it earns its place in an enterprise architecture.
What CloudWatch Omni is
CloudWatch Omni is a new, AI-driven experience layered on the existing CloudWatch telemetry store, not a separate observability product with its own database. AWS calls it the next generation of CloudWatch: one place to observe applications and the AI agents running beside them.
Three design choices define it:
- Off-console by design. Omni runs outside the AWS Console, with its own web UI, enterprise SSO and access model, plus an IDE extension for VS Code, Cursor and Kiro. End users do not need AWS Console access.
- Open standards at the edge. Telemetry comes in over OpenTelemetry (OTLP). Existing CloudWatch logs, metrics and traces show up with no reconfiguration.
- AI woven through the workflow. Natural-language questions, generated SQL and PromQL, and root-cause investigation driven by AWS DevOps Agent.
| Fact | Detail |
|---|---|
| GA announced | 23 September 2026 |
| Regions at GA | US East (N. Virginia), US West (Oregon), Europe (Ireland) |
| Surfaces | Omni web UI (SSO), IDE extension for VS Code, Cursor, Kiro |
| Non-AWS sources | Azure workloads today; other clouds announced as coming |
| Agent frameworks | LangChain, LangGraph, CrewAI, Strands, OpenAI Agents SDK, Vercel AI SDK |
| Native languages | Python and TypeScript; any runtime via OTLP |
| Evaluators | AgentCore built-ins, Braintrust, DeepEval, Ragas, custom LLM-as-judge |
Sources: What's New announcement , product page .
How it works
Omni is a layer over one store: telemetry lands in CloudWatch once, and every Omni surface reads the same correlated data. That is the architectural decision that makes the rest possible.
Read the diagram top down. Sources send telemetry over OTLP, or keep using the CloudWatch ingest they already have. Evaluators write quality scores into the same store as traces and metrics.
Spaces are the organising unit. You create a space in a central account, scoped to an application, team or set of workloads, and it pulls telemetry from across AWS accounts and Regions. The first centralized copy of data is free, so a hub-and-spoke observability account is the natural pattern.
The intelligence layer sits on top: auto-discovered topology, the natural-language assistant, AWS DevOps Agent for investigations, and alerting. None of these keep their own copy of data, so a trace an operator flags in the web UI is the same trace a developer opens in the IDE.
For existing CloudWatch users, onboarding is enabling Omni and creating a space. For new teams, it is pointing an OTLP exporter at CloudWatch; topology and health signals appear once data flows.
Feature deep dive
Six capabilities do the real work. I've grouped them by the question each one answers for an on-call engineer.
1. Auto-discovered topology: "what depends on what?"
Omni builds the service map across accounts and Regions with no tagging or manual config. Dashboards are organised around services, user journeys and business transactions rather than individual resources. Agents appear on the same map as the APIs, knowledge bases and compute they depend on.
For anyone who has maintained hand-built CloudWatch dashboards per account, this is the most immediately useful change. The starting point of an investigation becomes the application, not a resource ID.
2. AI-guided investigation: "why did this happen?"
You can query logs, metrics and traces with SQL, PromQL or plain English. Ask something like "why did latency spike on the checkout service?" and the assistant pulls the relevant signals and walks the analysis step by step. AWS DevOps Agent is the investigator underneath.
Two details matter operationally. Investigations are shared sessions, so the next engineer joins with context already in place. And interaction is flexible: chat, a guided point-and-click path, or your own tooling through Agent Toolkit for AWS.
3. Agent observability and evaluations: "did the agent reason correctly?"
Every agent invocation can be scored for correctness, groundedness, hallucination, toxicity and tool-selection accuracy. Scores are stored as first-class signals next to traces and metrics. That means you alert on an eval regression the same way you alert on p99 latency.
Evaluations run in two modes:
- Online against live production traffic, to catch quality drift.
- Offline against versioned datasets that capture edge cases, to validate a fix before release.
Sony's quote on the product page highlights one-click dataset creation from live traces. In my experience that is exactly where eval programmes stall: nobody owns building the golden dataset.
4. IDE extension: "what did my agent just do locally?"
The extension for VS Code, Cursor and Kiro shows the full execution path — every LLM call, tool invocation and reasoning step — inside the editor. It is free and needs no AWS account for local development. Because it shares the data layer with the web UI, an operator can flag a trace and a developer can open that exact trace in the IDE.
5. Bring your own stack: "do I have to rewrite instrumentation?"
No. Instrument once with OTLP and keep your agent framework and evaluator of choice. The same evaluator interface works in development and production. Teams on Amazon Bedrock AgentCore get Omni observability automatically, and existing AgentCore Observability instrumentation carries forward.
6. Native alerting: "who gets told?"
Alerts fire on conditions across any signal type, including eval scores. Routing targets are Amazon SNS and Slack, with a GitHub integration also mentioned in the launch blog. Omni alerts and dashboards carry no extra charge within service quotas; classic CloudWatch alarms stay separately priced.
Sources: product page , launch blog .
The eval-driven loop, and a worked incident
The core agent workflow is a four-step loop that runs entirely inside Omni, each step feeding the next:
- Diagnose. Open the flagged trace in the playground and try a different prompt or model config against real data.
- Score. Run evaluators for correctness, coherence, retrieval quality and tool selection.
- Compare and ship. Put the new variant side by side with the current one against a dataset built from production traffic. Ship the one that scores higher with no regressions.
- Watch. Leave those evaluators running continuously on live traffic so the next degradation surfaces as an alert.
Walking one incident through it
The scenario below is illustrative, built on the flow AWS describes, not a production case I have run. A support agent starts giving confidently wrong refund answers. Latency is normal and error rate is flat.
- Signal. The online groundedness score for the refund intent drops below its alert threshold. That alert fires to Slack even though every infrastructure metric is green.
- Agent layer. The trace shows the agent called the policy-lookup tool and got an empty result, then answered from model priors.
- Application layer. Following the same trace, the policy API returned errors because its backing service hit a capacity limit.
- Infrastructure layer. One hop further, the database connection pool behind that service was exhausted.
That top-to-bottom thread — eval score, tool call, API, database — is the whole pitch. Today most teams would need three tools and a war-room call to assemble it. In Omni it is one investigation on one data layer.
The fix then goes back through the loop. The agent gets a guard for empty tool results, the variant is scored offline against a dataset captured from the bad traces, and the online evaluator stays on as the regression net.
Pricing: what you actually pay for
Omni bills on data volume, not hosts or seats: standard CloudWatch ingest and storage rates, plus a query charge only above a free allowance of 5× your monthly log and span ingestion. Existing CloudWatch customers do not pay twice for storage, because Omni reads the same telemetry.
Key US East (N. Virginia) rates from the pricing page :
| Dimension | Price | Note |
|---|---|---|
| Application and custom logs ingest | $0.50/GB | Standard CloudWatch rate |
| OpenTelemetry metrics ingest | $0.50/GB | $0.08/GB for EKS Container Insights OTel metrics |
| Vended logs ingest | $0.50 → $0.05/GB | Tiered per payer account, per Region, per month |
| Spans ingest | $0.35 → $0.15/GB | Same rate as Application Signals |
| Storage (Standard) | $0.030/GB-month | Compressed bytes; $0.018 IA, $0.006 archive with Intelligent Tiering |
| Log and trace queries | $0.005/GB scanned | Free up to 5× monthly log + span ingest |
| PromQL queries | $0.01/million samples | Standard CloudWatch PromQL rate |
| Centralization | First copy free; $0.05/GB after | Data transfer charges may apply |
| Agent evaluations | AgentCore Evaluations rates | Billed separately |
| Dashboards, Omni alerts, topology, investigation agent | Included | Within service quotas |
A quick worked example
Take a team ingesting 4 TB of application logs and 1 TB of spans a month. The free query allowance is (4 + 1) × 5 = 25 TB of scans. Ingest alone costs 4,000 × $0.50 + 1,000 × $0.35 = $2,350, and compressed storage for a month is small change next to it.
The practical takeaway: in Omni, ingest is the bill. Query overage and storage are rounding errors for most shapes; AWS's own multi-account example puts storage at $126 against $12,050 of ingest.
Cost levers an architect should pull
- Filter at the collector. Drop debug logs and sample spans in the OTel Collector before they reach CloudWatch, since every GB in is billed.
- Sample evaluations. AWS's agent example evaluates 2% of traces. Evaluate a sample online and run full coverage offline.
- Watch dashboard refresh. Dashboard queries count against scan volume; AWS suggests refresh intervals of 5 minutes or more.
- Centralize once. The first centralized copy is free; a DR copy in another Region costs $0.05/GB plus transfer.
- Use the trial deliberately. $1,000 in credits for 30 days covers OTel ingestion only, for up to 10 accounts per Organization. Point it at a real workload, not a toy.
The architect's view: where it pays off
Omni's biggest benefit is not the AI chat; it is collapsing agent quality, application health and infrastructure into one correlated data layer with an open ingest path. Everything else builds on that.
| Who | Concrete benefit | Why it matters |
|---|---|---|
| Agent developers | Traces, evals and experiments inside VS Code, Cursor or Kiro | Quality work moves left, before production |
| SRE and on-call | One topology map and cross-signal investigation | Shorter time from alert to root cause |
| Platform teams | Off-console access with SSO and role-scoped data | Observability for people who should never hold Console credentials |
| Security and governance | Telemetry stays in CloudWatch; OTLP avoids proprietary agents | Data ownership and a cleaner exit path |
| FinOps | Volume-based bill; dashboards and alerts included | Adding users or dashboards does not change cost |
Where I'd adopt it first
- Teams already shipping agents on Bedrock AgentCore. Instrumentation carries forward automatically, so the cost of trying is close to zero.
- Multi-account AWS estates with console sprawl. Topology across accounts and Regions without tagging projects is a strong win on its own.
- Organisations that already standardised on OpenTelemetry. Pointing an OTLP exporter at a new endpoint is a config change, not a migration.
A sensible adoption path
- Enable Omni in a central observability account and create one space for a single application.
- Wire SSO through your identity provider and scope access by team.
- Let existing CloudWatch telemetry populate the topology; confirm the service map matches reality.
- Add OTLP instrumentation for one production agent and turn on sampled online evaluations.
- Move one on-call rotation's alerts to Omni alerts routed to Slack; compare triage time with the old flow.
- Only then expand spaces and accounts, with centralization and cost filters designed up front.
Limitations and open questions
Omni is a strong GA, but a few gaps matter for enterprise rollout, and some claims can only be proven in your own estate.
- Three Regions at launch. N. Virginia, Oregon and Ireland only. Teams with data-residency needs in Frankfurt, Mumbai or elsewhere should check whether centralizing telemetry into those Regions is acceptable.
- Multi-cloud is partial. Azure ingestion works today; GCP and others are described as coming soon. It is not yet a single pane for a three-cloud estate.
- Evaluation cost is a separate line. Evals bill at AgentCore Evaluations rates, and LLM-as-judge on every invocation gets expensive fast. Budget for sampling.
- Ingest-heavy pricing. Volume pricing is predictable, but chatty OTel instrumentation will show up on the bill at $0.50/GB. Collector-side filtering is not optional.
- AI investigation needs proof. Root-cause quality from DevOps Agent depends on telemetry completeness. Treat its conclusions as hypotheses to verify, especially early on.
- Two alerting models. Omni alerts are included, classic CloudWatch alarms are billed separately. Decide which one owns paging before both start firing.
- Air-gapped and regulated environments. A standalone web UI with SSO is great for access, but review how it fits network isolation, identity federation and audit requirements before rollout.
The open question I'd take into any pilot: does one correlated view actually shorten mean time to resolution for agent incidents, or does it mostly move the same work into a nicer UI? Measure it on one rotation before you commit.
Getting started, and the bottom line
The fastest way in costs nothing: install the free extension and point a local agent at it.
- Agent developers: install the CloudWatch Omni extension for VS Code, or from Open VSX for Cursor and Kiro. Instrument with OTLP, run the agent, and explore traces, evals and experiments locally. No AWS account is needed.
- Application teams: create an Omni space from the CloudWatch console, configure SSO, and sign in to the web UI. Existing CloudWatch telemetry appears without reconfiguration; new workloads send OTLP.
My bottom line as an architect: CloudWatch Omni is the first AWS observability release that treats "did the agent reason correctly?" as an operational signal rather than an ML-team side project. Pair that with off-console access and OTLP-first ingest, and it is worth a structured pilot for any team running agents in production on AWS. Go in with collector filtering and eval sampling designed from day one, and measure MTTR honestly.
Sources
- Amazon CloudWatch Omni product page
- What's New: Amazon CloudWatch Omni GA (23 Sep 2026)
- Introducing Amazon CloudWatch Omni: Observability for the AI Era, AWS Cloud Operations Blog
- Amazon CloudWatch Omni pricing
- CloudWatch Omni documentation
About the author
Ashish Kasaudhan is a DevSecOps Architect at Giesecke+Devrient and a member of the AWS Community Builders program. He has spent 17 years in DevOps and platform engineering, working across cloud infrastructure, containerization, CI/CD, and enterprise automation, with earlier roles at PwC Acceleration Centers, Thales, and NTT DATA. He is the creator of SkyArch AI, an open-source multi-cloud architecture diagramming tool, and writes practitioner-focused pieces on cloud, DevOps, AIOps, and MLOps.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article