Bedrock Meter — a self-hosted governance gateway for Amazon Bedrock

Overview and context

• In-region inference — model calls are made to a Bedrock endpoint in your chosen region, so prompts and completions are processed within that region.
• In-region state and audit — the data Bedrock Meter stores (users, keys, budgets, usage, and the optional prompt/completion audit archive) is configured to reside in the same region.
• Your account, your controls — data stays in your AWS account, under your IAM, encryption (KMS), logging (CloudTrail), and retention settings.
• Reduced credential exposure — end users are not issued AWS credentials; they reach only your in-region gateway over a controlled network path.

Challenges this addresses

• Credential sprawl — handing each developer AWS keys or a long-lived token spreads credentials across workstations and scripts: hard to rotate and easy to leak.
• Limited per-user cost control — Bedrock billing is pooled at the account level. Native options can approximate per-user limits, but they are operationally heavy and not a simple real-time hard cap on an individual's spend.
• Limited usage visibility — native billing does not easily answer "who used what, on which model, this period?" — the attribution teams want.
• Data residency needs — regulated workloads often require requests and usage data to remain within a specific region.
• Onboarding & self-service — adding developers, issuing and rotating their keys, and letting each person see their own usage should be simple admin and self-service actions, not a ticket for every change.
• Limit-increase workflow — when someone reaches their cap, they need a clear way to request more capacity and for an admin to review and raise it, without a service restart or redeploy.

How it works

• Per-user API keys — each developer receives their own bm_ bearer token, independent of anyone else's.
• Configurable budgets — requests are blocked when a user's configured budget for the period is exhausted.
• Usage metering — per-user token and cost tracking, with admin drill-down.
• In-region by configuration — state and inference are configured to stay in the chosen region.
• Admin & self-service consoles — admins onboard users and manage budgets; each user has a self-service console to view their own usage and rotate their key.
• Budget-increase requests — users request more capacity from their console; an admin reviews and raises the cap for the current period without a restart.

Architecture

Design approach

1. Authenticate — the gateway hashes the presented bm_ key and resolves the owning user. Unknown or inactive keys are rejected first.
2. Budget check — the user's remaining budget for the current period is evaluated; if the cap is reached, the request is refused with a clear error and nothing is billed.
3. Translate — the OpenAI /v1/responses request is mapped to Bedrock's Converse API (streaming or non-streaming).
4. Invoke — Bedrock is called over SigV4 using the Fargate task role; no static token is in the request path.
5. Meter — token usage from the response is converted to integer micro-dollar amounts and recorded against the user; budget warnings fire at the configured threshold.
6. Audit — prompt/completion records are archived to a private S3 bucket for traceability.

• Application Load Balancer — restricts reachability to an allowed CIDR.
• ECS Fargate (container) — the gateway: auth, budgets, metering, translation.
• DynamoDB (7 tables) — users, keys, budgets, usage.
• Secrets Manager (4 secrets) — peppers, session, bootstrap.
• S3 (private) — prompt/completion audit archive.
• Amazon Bedrock — model calls via the SigV4 task role.

Authentication and access controls

• Each user is issued a high-entropy, URL-safe key with a recognizable bm_ prefix.
• The raw key is shown once at issuance and is not retained in the system afterwards.
• What is stored is a keyed hash (HMAC-SHA-256) under a server-side pepper held in Secrets Manager — not the raw key. A datastore compromise alone does not reveal the high-entropy keys without also obtaining the pepper.
• Keys are per-user and revocable: disabling a key cuts off that user without affecting others and without an AWS change.
• Console passwords are hashed with the scrypt KDF; sessions use stateless HMAC-signed cookies.

• The gateway calls Bedrock using the ECS Fargate task role via the AWS SDK's default credential provider chain. The container assumes the role automatically; the credentials are short-lived and rotated by AWS.
• There is no code path that accepts a static Bedrock bearer token, so the gateway is not designed to store or distribute one.
• The task role is configured as invoke-only: bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream, bedrock:Converse, bedrock:ConverseStream. It does not include model management, iam:*, or sts:AssumeRole escalation. The Bedrock statement uses a wildcard resource because geographic inference profiles require it; the constraint comes from the limited action set, not a resource ARN. DynamoDB, Secrets Manager, and S3 permissions are each resource-scoped to this stack's own resources.

• Network — the load balancer is restricted to a specified CIDR; the service accepts traffic only from the load balancer.
• Secrets — four generated secrets reside in Secrets Manager and are injected into the task as ECS secrets rather than written to the environment in plaintext.
• Data residency — DynamoDB, S3 audit, Secrets, and inference are configured to the chosen region.

User onboarding (admin console)

• An admin creates a user in the console (/admin). If no password is given, the system generates a temporary password and sets the user to must-change-on-first-login (24-hour expiry).
• A welcome email is sent with the username and temporary password (optional SES; the gateway degrades gracefully to no email if SES is not configured).
• On first login, the user is redirected to change their password; the temp flags are then cleared.
• The admin issues the user's bm_ API key, which is shown once in the console. (The key is handed over through the console, not emailed.)
• Users can rotate their own key and submit budget-increase requests from their self-service console; admins review and approve, raising the cap without a restart.

Installation steps

Prerequisites: an AWS account with Bedrock model access enabled; an existing VPC with appropriate subnets and connectivity for your chosen ALB scheme; AWS CLI with CloudFormation deploy permissions; the CDK CLI (npm install -g aws-cdk), Docker, and Node.js 20+.

1. Clone and install dependencies

2. Run the tests (recommended)

3. Build and push the container image

4. Bootstrap CDK (once per account/region)

5. Deploy the stack

6. First-run setup

7. Create a user and issue an API key

• Sign in at /login with your admin credentials.
• In /admin, create a user (username + temporary password; a welcome email is sent if SES is configured).
• Set that user's budget (amount per period).
• Issue an API key — copy the bm_ key immediately; it is shown only once.

8. Configure the client (Codex)

9. Verify

Production network access

Optional: TLS / HTTPS

Summary

• Reduce credential sprawl — users get revocable bm_ keys with no direct AWS access; the AWS credential stays in the task role.
• Per-user cost control — configurable budgets block requests when a cap is reached.
• Usage visibility — per-user token and cost metering with admin drill-down.
• Bedrock access control — SigV4 via an invoke-only task role (no model management or IAM/STS escalation); no static tokens.
• Data residency — state and inference configured to the chosen region.
• Onboarding & self-service — admin consoles to onboard users and manage budgets; a per-user console for usage and key rotation.
• Capacity requests — users request a budget increase; admins review and raise the cap without a restart.