AWS Builder Center

Builder Center

From idea to release in a day: reviewing who can call Amazon Bedrock in your AWS accounts

Learn how a read-only, open-source tool finds the long-lived IAM credentials in your AWS Organization that can call Amazon Bedrock, ranks them by risk, and suggests next steps. It also shows how spec-driven development with Kiro took it from idea to release in a day.

Hoi Wing

AI for operations has two halves

When people talk about AI for operations, they usually mean agents that solve operational problems: triaging events, investigating incidents, summarizing cases. That work matters.
There is a second half that gets less attention: how quickly we can use AI to respond to the new operational questions that AI adoption itself creates. Generative AI is spreading across organizations faster than most governance processes were designed to follow. The teams that adopt it safely and quickly are the ones that can review, adjust, and tighten their configuration at the same pace.
This article walks through one example: a small, read-only open-source tool called Bedrock Permissions Audit , and how an AI-Driven Development Life Cycle (AI-DLC) approach took it from idea to release in a single day.

The question: which long-lived credentials can call Amazon Bedrock?

Here is a situation that applies even to teams that have never built on generative AI.
An AWS account has IAM users with long-lived access keys. Some of those keys were created years ago for a CI job or a reporting script. Their policies may be broad, for example AdministratorAccess. A policy like that also allows calls to Amazon Bedrock models and subscribing to models through AWS Marketplace, whether or not anyone intended it and whether or not the team uses Bedrock.
Often nobody made a decision to grant that access, so it rarely gets reviewed. This is a credential-hygiene and governance question, not an AWS service issue, and it is straightforward to close. The first step is knowing where you stand.
Answering the question by hand is slow. You have to read inline policies, managed policies, group policies, and permissions boundaries for every user, in every account, and work out whether the combined result allows model invocation.

What the tool does

Bedrock Permissions Audit is a command-line tool that answers that question for one account or for every account in an AWS Organization.
  1. It lists every IAM user that has an access key or a long-term Amazon Bedrock API key, plus the root account's access keys.
  2. It asks the IAM policy simulator whether each credential can invoke Amazon Bedrock models, enable additional models, or create new API keys. The simulator takes inline policies, managed policies, group policies, and permissions boundaries into account.
  3. It ranks the results into Critical, High, Medium, Low, and Info tiers, with dormant credentials first inside each tier, and gives each one a recommended action.
  4. It writes a terminal report, findings.csv, findings.json, and suggested guard-rail policies.
A few design choices are worth calling out.
It is read-only. It never changes, deletes, or creates anything in an account, never invokes a model, and never sends data anywhere. A test in the repository fails the build if any code or shipped IAM policy could write. In an organization, it assumes a dedicated read-only role that is deployed to member accounts with a CloudFormation StackSet.
It does not stop at the first account that fails. An account without the audit role is reported as failed with a hint, and the rest are still audited. An interrupted run can continue with --resume.
It treats inactive keys with care. Deactivating an access key does not invalidate its secret. Anyone who can call iam:UpdateAccessKey on the user can switch it back on. So an inactive key that would rate Critical or High if it were active is reported as Low, with the tier it would receive. That keeps it visible without extra flags. The recommended action is to delete it.
Here is an abbreviated example of the report. Account IDs and key IDs are placeholders.
TIERACCOUNTPRINCIPALCREDENTIALAGELAST USEDMFAGRANTRECOMMENDATION
High111111111111ci-botaccess key400dnevernobroadDelete access key ...MPLE (never used). Recreate it only if a workload breaks.
Medium111111111111reportingaccess key20d1d agoyesbroadDelete access key ...MPLE (never used). Recreate it only if a workload breaks.

What to do with the findings

The preferred outcome for each finding is to replace the long-lived credential with a short-lived one: IAM Identity Center for people, IAM roles for workloads, and OIDC federation or IAM Roles Anywhere for systems outside AWS.
Where a long-lived key has to stay for now, these controls reduce what it can do and make its use visible:
  • Scope its permissions to the Amazon Bedrock actions and models it needs.
  • Add IAM condition keys that limit where it can be used from, such as source IP or VPC endpoint.
  • Set AWS Budgets and AWS Cost Anomaly Detection alerts on Amazon Bedrock spend.
  • Keep AWS CloudTrail on, and turn on Amazon Bedrock model invocation logging where you need request detail.
  • Enable Amazon GuardDuty AI Protection .
The tool also writes suggested policies, such as a service control policy that denies model invocation to IAM users, and one that limits Amazon Bedrock to approved Regions and models. See the AWS Security Blog post Implementing least privilege access for Amazon Bedrock  for related patterns. Test every service control policy on a sandbox organizational unit first, because a policy that denies IAM users will stop any workload still running on user keys.
After you make a change, run the tool again and confirm that the tier has dropped.

Know the limits

The tool is a point-in-time review of IAM configuration. It does not monitor usage or enforce anything. The most important limits:
  • The IAM policy simulator does not evaluate service control policies or resource control policies. A principal shown as able to invoke Amazon Bedrock may be blocked at the organization level.
  • Only long-lived credentials are in scope. Roles are not audited, and sts:AssumeRole chains are not followed.
  • The simulator has no request context, so a grant limited by source IP, VPC, or tags can be over-reported. The results flag the condition keys it had no value for.
  • "Last used" reflects any AWS service, not Amazon Bedrock specifically.
  • It covers the commercial AWS partition only.
Use it as a starting point for a review, and pair it with detection and logging services for ongoing coverage.

How it was built in a day

The part I find most interesting is the process. Our team used AI-DLC methodology and Kiro's spec-driven development to build a framework of three phase agents: ideation, build and release. We then used that framework to build this tool. The work started with a specification of what the tool should do and why, then moved to a design, then to implementation and tests. The specification stayed the source of truth for intent, so when a review turned up a gap, we updated the spec first and the code second. That discipline is what kept a one-day build from turning into a one-day mess.
This is what I mean by the second half of AI for operations. The question "who can call Bedrock in our accounts?" appeared because AI adoption is accelerating. Being able to respond with a reviewed, tested, documented tool on the same timescale is itself an operational capability.

Get started

The source, README, and the read-only role template are in the AWS sample-ai-for-operations  repository.
Requires Python 3.12 or later and AWS credentials with the read-only permissions listed in the repository.
The repository holds several samples. Get only this one with a sparse clone (Git 2.25 or later):
1
2
3
4
git clone --depth 1 --filter=blob:none --sparse https://github.com/aws-samples/sample-ai-for-operations.git
cd sample-ai-for-operations
git sparse-checkout set security/bedrock-permissions-audit
cd security/bedrock-permissions-audit
Or clone the whole repository:
1
2
git clone https://github.com/aws-samples/sample-ai-for-operations.git
cd sample-ai-for-operations/security/bedrock-permissions-audit
Then install and run it:
1
2
3
4
5
6
7
8
python3 -m venv .venv && source .venv/bin/activate
pip install .

# Audit only the account your credentials belong to
bedrock-audit --single-account

# Audit every account in your organization
bedrock-audit
IAM API calls, including the policy simulator, have no charge. The output names accounts and users, so treat the results directory as confidential and delete it when you are done.
Governance is what gives leadership the confidence to say yes to scaling AI. A short review of which credentials can reach Amazon Bedrock is a good place to start.

Feedback

Tried it on your accounts, found a case it misses, or want a different report format? Open an issue on the repository , or leave a comment below.
The sample is provided under the MIT-0 license. It is a starting point to review, test, and adapt in your own environment.
# kiro# cloud-operations# ai-security# ai-dlc# amazon-bedrock
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article