Multi-Tenant Observability on Amazon Bedrock and AgentCore: Track It, Cap It, Alert on It
A field guide to isolating visibility, tracking usage at the right grain, enforcing limits, and alerting — per tenant — across the Bedrock model layer and the AgentCore agent layer.
If you run a SaaS on Amazon Bedrock and AgentCore, four questions eventually land on your desk at the same time: Which tenant drove last month's spike? Is this tenant profitable at their contract price? Can I stop a tenant before they blow their tier? Whose traces do I pull when their agent is slow? These are the four pillars of multi-tenant operations — isolated observability, granular tracking, granular limits, and alerting — and the honest answer is that the platform gives you strong native parts for three of them and a deliberate build-it-yourself gap on the fourth.
This post walks the native capabilities of both layers, lays out exactly what grain you can track and cap usage at, compares the attribution mechanisms (and when to reach for each), and maps it all to a reference architecture. Every claim links to an AWS source. Where a number would be illustrative, it's marked as such.
One correction worth stating up front, because it changes a common design assumption: it is widely repeated that Bedrock's CloudWatch metrics are "per model only, so per-tenant metrics require parsing logs." That's only true for raw model invocations. Invoke through a per-tenant application inference profile and Bedrock emits token and latency metrics natively per profile — i.e. per tenant — with no log parsing. We'll come back to this; it's the single biggest lever for cheap per-tenant observability.
--------------------------------------------------------------------------------------------------------------------
1. What the native observability layers actually give you
Bedrock native observability
There are three distinct planes, and conflating them is the most common design mistake.
| Plane | What it emits | Finest native grain | Source |
|---|---|---|---|
| CloudWatch metrics (AWS/Bedrock namespace) | Invocations, InputTokenCount, OutputTokenCount, InvocationLatency, InvocationThrottles, TimeToFirstToken, CacheReadInputTokens, CacheWriteInputTokens, EstimatedTPMQuotaUsage | Per ModelId — or per inference-profile ID | Runtime metrics doc |
| Model invocation logging → CloudWatch Logs and/or S3 | Full request + response + metadata, per-call token counts, IAM identity, requestMetadata | Per request | Logging doc |
| Cost Explorer / CUR 2.0 | Billed dollars by tag / identity | Per usage-type per day | AIP cost doc |
The nuance that decides your design: for raw model invocations, the AWS/Bedrock metric stream carries the ModelId dimension only — not tenant, user, or app. But when you invoke via a user-defined (application) inference profile, Bedrock automatically collects metrics for "the input tokens sent to it, the output tokens it produces, and the latency of the invocation" — per profile (user-defined inference profile metrics , inference profiles overview ). So if one profile maps to one tenant, you get native per-tenant token and latency metrics without touching the logs.
Two caveats straight from the docs:
- Dollar cost is not a native metric. The native per-profile signals are tokens and latency. To get per-tenant dollars you either compute tokens × your rate card, or read billed dollars from Cost Explorer/CUR via cost-allocation tags (which aggregate per usage-type per day).
- EstimatedTPMQuotaUsage is explicitly "an approximation… do not use as the sole indicator for quota use or capacity planning," and model invocation logging is off by default.
AgentCore native observability
AgentCore adds an agent-shaped hierarchy on top: Sessions → Traces → Spans (concepts doc ).
- Session = the full user↔agent interaction; Trace = one request-response cycle (including tool calls and LLM calls); Span = one unit of work inside a trace.
- Metrics publish under the bedrock-agentcore namespace. Session-level metrics are default for Runtime-hosted agents, plus default metrics for Memory, Gateway, and built-in tools (view-data doc ).
- Traces and spans require ADOT instrumentation plus a one-time enablement of CloudWatch Transaction Search per account (get-started doc ). Runtime-hosted agents get automatic instrumentation; agents hosted elsewhere add the ADOT library.
- Runtime isolation is real and per-session: each session runs in its own microVM with isolated compute, memory, and filesystem, up to 8 hours (isolated sessions doc ).
The critical fact: per-tenant cost attribution at the agent layer "requires application-level instrumentation that emits tenant-tagged metrics… for every agent invocation, capturing I/O tokens, tool invocations, and execution duration" (AgentCore multi-tenant blog, §9 ). Tenant context flows in via custom HTTP headers, not an automatic AgentCore field.
----------------------------------------------------------------------------------------------------------------
2. The granularity ladder — what you can track, and how
From coarsest to finest, with the mechanism that unlocks each level:
| Granularity | Mechanism | Native or custom | Dollars or estimate |
|---|---|---|---|
| Account / Region | Cost Explorer, service quotas | Native | Billed $ |
| Model (raw ModelId) | CloudWatch AWS/Bedrock metrics | Native | Token / latency metrics |
| Inference profile (= per-tenant if one profile per tenant) | Same AWS/Bedrock metrics, ModelId dimension = profile ID | Native | Per-profile token / latency metrics |
| Application / team / tenant ($ chargeback) | Application Inference Profiles (cost-allocation tags) or Projects/Workspaces | Native resource | Billed $, per usage-type/day |
| IAM principal (per-user, per-role) | IAM principal attribution (auto) + principal/session tags | Native, no code change | Billed $, per usage-type/day |
| Per request / per prompt (any per-call dimension) | Per-request metadata (requestMetadata / X-Amzn-Bedrock-Request-Metadata) → invocation logs | Custom (log parsing) | Token-count estimate (you compute) |
| Agent / session / trace / span / tool | AgentCore Sessions→Traces→Spans + app-level tenant-tagged metrics | Native (session) + custom (tenant) | Metrics; $ computed from tokens |
Read it as a trade: the higher you climb, the closer to real billed dollars but the coarser the time grain; the lower you climb, the closer to per-prompt truth but the more it's a computed estimate you maintain.
-------------------------------------------------------------------------------------------------------------
3. The three Bedrock attribution methods — when to use which
These are orthogonal axes, not competitors — the AWS docs explicitly recommend combining them. Think WHO / WHAT / WHICH-request.
A. Application Inference Profiles (AIP) — the "WHAT" axis
Create a model-specific resource, attach cost-allocation tags (e.g. TenantID, business-unit), and call the profile ARN in place of the model ID in InvokeModel/Converse. Tags flow to Cost Explorer + CUR 2.0 (doc ).
- Pros: Real billed dollars (not estimates); native per-profile CloudWatch token and latency metrics with no log parsing; works in standard FinOps tooling; AWS Budgets can set tag-based budgets.
- Cons / limits: For dollar attribution the finest grain is per usage-type per day — never per request; one profile per model × tag-set, so profiles proliferate as you add models/tenants (check the Service Quotas console for your account's per-Region profile limit before assuming headroom — don't rely on a remembered number); tags take ≤24h and are not retroactive; bedrock-runtime only.
- Use when: you need defensible per-tenant/per-app chargeback in dollars plus native per-tenant usage metrics, and tenant count is bounded. Note: AIP is bedrock-runtime only — if your workload calls the OpenAI-/Anthropic-compatible APIs (bedrock-mantle), use Projects instead (see §D), which also scale better across many models.
B. IAM principal attribution — the "WHO" axis
Bedrock automatically records the calling IAM user/role identity on every inference request — zero code change. Optional principal tags add team/department; session tags (passed at sts:AssumeRole) distinguish users/tenants behind a shared role (doc ).
- Pros: No code changes; per-user and per-role for free; session tags are the clean multi-tenant pattern for an LLM gateway that assumes a Bedrock role per tenant; tags appear in CUR under the iamPrincipal/ prefix.
- Cons / limits: Aggregated per usage-type per day, not per request; requires roles structured per-tenant or session-tag plumbing (sts:TagSession in the trust policy); bedrock-runtime only (mantle "coming soon").
- Use when: tenants/users map cleanly to IAM identities, or you front Bedrock with a gateway that can assume-role-per-tenant.
C. Per-request metadata — the "WHICH-request" axis
Attach up to 16 key-value tags to each call (requestMetadata on Converse, the X-Amzn-Bedrock-Request-Metadata header on InvokeModel). Tags land in model invocation logs alongside per-call token counts (doc ).
- Pros: Finest grain — per prompt; any dimension can vary per call (tenant, user email, feature, environment); no resource to pre-create; near real-time via Logs Insights / Athena.
- Cons / limits: Cost is computed from token counts × your rate card — an estimate, not the bill; requires model invocation logging enabled in that Region; keys/values ≤256 chars; SigV4 must include the header in SignedHeaders (SDKs handle this); bedrock-runtime only.
- Use when: you need true per-tenant/per-request token detail for granular dashboards, anomaly detection, and near-real-time enforcement.
D. Projects / Workspaces — cost attribution for the bedrock-mantle endpoint
The bedrock-mantle endpoint serves the OpenAI-compatible Responses and Chat Completions APIs and the Anthropic-compatible Messages API. Its cost-attribution model is different from bedrock-runtime, so treat it as its own track.
Projects attribute cost at the application/workload level for the OpenAI-compatible APIs; Workspaces are the same underlying resource for the Anthropic-compatible Messages API. You set the identifier once on the client and every call made with it is attributed; untagged calls fall to the account's default project. (Projects doc )
What the bedrock-mantle endpoint gives you today (per axis):
| Axis | On bedrock-mantle |
|---|---|
| WHAT (app / team / workload) | Projects (OpenAI APIs) / Workspaces (Messages API) — tagged, dollars to Cost Explorer + CUR 2.0 |
| WHO (identity / user / role) | Coming soon — IAM principal attribution support for bedrock-mantle is announced but not yet GA (IAM principal doc) |
| WHICH request (per prompt) | Not available — per-request metadata is unsupported, and model invocation logging is bedrock-runtime-only, so there is no per-prompt token/cost detail on mantle |
- Key advantage over AIP: a project is not tied to a model — the model is a per-request parameter — so one project spans many models, avoiding AIP's per-model resource growth. This is the main reason Projects scale better when one application uses many models.
- Pros: Real billed dollars by application / team / env / cost-center in Cost Explorer + CUR 2.0; set once on the client, nothing per call; clean scaling across models.
- Cons / limits: bedrock-mantle only (not Converse/InvokeModel); aggregated per usage-type per day, never per request; no per-prompt detail (no request metadata, no invocation logging on mantle); no per-user dollars yet (IAM principal is coming soon, not GA); tags take ≤24h and aren't retroactive.
Projects vs AIP — the decision is endpoint-first, then scale (cost FAQ ): if you call Converse/InvokeModel (bedrock-runtime), use AIP and you can layer IAM principal (WHO) + per-request metadata (WHICH) on top. If you call the OpenAI-/Anthropic-compatible APIs (bedrock-mantle), use Projects/Workspaces — but understand the trade-off: today mantle gives you the WHAT axis only. WHO is coming soon and per-prompt detail isn't available, so a mantle workload that needs per-user chargeback or per-prompt token tracing should either wait for IAM-principal-on-mantle GA or route that traffic through bedrock-runtime. So "build the solution on Projects" is correct only if your workload uses the mantle APIs and application/workload-level dollar attribution is sufficient.
----------------------------------------------------------------------------------------------------------------
4. Mapping to the four principles
Principle 1 — Completely isolated observability per tenant
Bedrock — three ways to get isolation, strongest first:
- Account-per-tenant (Silo) — true blast-radius, cost, and observability isolation; highest overhead.
- One inference profile per tenant — invoking through a per-tenant AIP gives you native per-profile token and latency metrics (no log parsing). Add per-profile CloudWatch dashboards and alarms; get dollars from Cost Explorer tags or compute from tokens. This is the highest-leverage option for most pooled SaaS.
- Per-request metadata + Logs Insights — on shared infrastructure, filter the log stream by TenantID for per-prompt detail; you parse and aggregate.
A deployable AWS reference for options 2–3 lives in the multi-tenant Bedrock cost blog and its sample repo . Note the sample also runs a Lambda to compute dollar cost from tokens, since dollars aren't a native metric.
AgentCore — Silo = dedicated Runtime + Gateway + Memory per tenant behind separate IAM → fully isolated observability. Pool = shared infrastructure + tenant-tagged metrics/structured logging surfaced in GenAI Observability. Per-session microVM isolation holds either way.
Principle 2 — Granular usage tracking
Use the ladder in §2. For native per-tenant usage, route each tenant through its own inference profile (row 2b). For per-prompt token truth, per-request metadata in invocation logs (row 5) is the only source. AgentCore adds session/trace/span/tool granularity (row 6) via ADOT.
Principle 3 — Limits at a granular level (the honest gap)
Bedrock has no native per-tenant or per-application hard usage limit. Native service quotas (TPM/RPM) are per account/Region/model. To enforce per-tenant ceilings you build it:
- AWS Budgets with tag-based budgets + budget actions — can apply an IAM policy at a threshold, but it's coarse and not real-time token-gating.
- CloudWatch alarm on a per-tenant custom (or per-profile) metric → SNS → Lambda → app/gateway block or rate-limit. The multi-tenant blog explicitly notes alerts can trigger "immediate rate-limiting actions."
- AgentCore — the session limit (8 hours) is native; per-iteration/per-token caps live in your agent code or framework, keyed off the tenant header.
Principle 4 — Alert on meet/exceed
- AWS Budgets tag-based alerts on AIP tags or IAM-principal tags.
- CloudWatch alarms on per-tenant/per-profile metrics. The AWS sample ships three per profile — BedrockTokenCostAlarm, BedrockTokensPerMinuteAlarm, BedrockRequestsPerMinuteAlarm — and recommends a graduated green→yellow→red model over binary alerts.
- CloudWatch anomaly detection to separate "healthy growth" from "sudden spike."
5. Recommended pattern by tenant profile
| Tenant profile | Isolation pattern | Bedrock attribution stack | AgentCore stack | Enforcement |
|---|---|---|---|---|
| Regulated / large enterprise | Silo (account or dedicated resources per tenant) | Account-level cost + AIP per model (native per-profile metrics) | Dedicated Runtime / Gateway / Memory + separate IAM | Account quotas + per-tenant alarms |
| Many small/medium tenants | Pool (shared infra) | One AIP per tenant (WHAT, native metrics) + IAM session tags (WHO) + per-request metadata (WHICH) | Shared Runtime, tenant via HTTP header, tenant-tagged metrics | App/gateway token metering + CloudWatch alarm → Lambda rate-limit |
| Mixed tiers | Bridge (shared base + tenant-specific where needed) | Combine all axes; tier-specific models | Shared base skills + tenant-specific skills | Graduated alerts + tier-aware caps |
The one-line takeaway
Native observability gives you more than the conventional wisdom admits: per-model and per-inference-profile metrics, per-request logs, and per-tag/per-identity dollars. Route each tenant through its own inference profile and per-tenant token/latency observability is native. What still isn't native — and what you must assemble — is per-tenant dollar cost, per-tenant limits, and per-tenant alerting. Combine IAM principal (WHO) + Application Inference Profiles or Projects (WHAT) + per-request metadata (WHICH request) for tracking, then layer AWS Budgets + CloudWatch alarms + an app/gateway enforcement point on top — because Bedrock has no built-in per-tenant hard cap. And carry a TenantID through every layer: no tenant ID, no multi-tenancy.
Sources
Bedrock Quotas reference: Quotas for Amazon Bedrock — check the Service Quotas console for current per-account limits rather than relying on remembered values
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article