Managing an AWS Organization With AI Generated Code
A new take on the AWS Control Tower concept built with AWS Kiro CLI. Tracking costs, mistakes, and progress and finding ways to improve results when writing AI generated code.
This is part of my series of blog posts on creating an AWS Bootstrap Script to set up secure AI agent infrastructure.
In the last post I wrote about Tracking AI Success Rate On An AWS Infrastructure Project
Overview
I set out to write a real-word program with AI and actually finish it. It started out great. I wrote a whole bunch of code with Kiro CLI that did a lot of things in 2.5 weeks . Cool.
But then you want to get it “done.” And that’s where it diverges from the initial cool factor into trying to build something maintainable and truly useful. Because I’ve had a lot of fits and starts with AI projects I started tracking my project here to see if I can actually achieve my objective.
AWS CloudFormation would probably solve some of this but I have reasons for not using it in the full post. I’m not saying you shouldn’t use CloudFormation or Terraform, I’m just going to explain why I didn’t on this project. You probably should. And you can use it with this solution if you want. Check out how resource action files work in the full post. But CloudFormation alone doesn’t solve the problems I’m solving with this code. Neither does AWS Organizations or Control Tower or Stacksets or SSM.
With my solution, I can configure my organization and resources any way I want and still ensure governance. I am not using AI to deploy resources which would lead to numerous difficult to govern variations in implementation as shown throughout the mistakes I’ve experienced over the course of this project.
I use AI to generate deterministic code that will not do things like try to delete all the accounts in my AWS organization. Again, see the mistakes section. In this post I’m going to give a high-level overview of what I’m building and how it works and I’ll explain more of the details in follow on posts.
I’m finally at the point where I think I have a program that’s going to work. Not done, but working to some degree. It’s a bit complicated because it allows you to completely manage your AWS organization and all resources in it. Easy, right?
I’ve written an application architecture and the guts of each of the components which allow me to configure the types of things I can deploy, a concrete definition of the organization, environments, and resources I want to deploy, deploy that definition, and then manage drift.
The Objective
When I started using AI, I quickly landed on the concept of objectives, which now everyone is calling “goals.” I wrote that in my post on getting started with AI using a simple README . I define an objective for each project I start.
The goals as have been from the start:
- Quickly spin up and tear down environments and accounts in an AWS Organization.
- Configure the organization, OUs, accounts, and resources any way I want (easily) without any structural limitations beyond what is allowed in AWS.
- No Service Linked roles because they are not governed by Service Control Policies.
- Deployments which require MFA when using the AWS CLI.
- Maintain a secure and well-governed AWS Infrastructure.
- Track costs, security, and get alerts when problems occur.
- Deploy a jobs that require MFA to kick off.
- Segregate all the things.
Segregating all the things
One of the main goals for this deployment is to be able to segregate things that are allowed to operate in different trust boundaries. Here are some examples of things I segregate when I deploy AWS resources:
- Each pentest gets its own environment. Other resources and principles cannot access my pentest environment outside the resources for that pentest. Resources for that pentest cannot access things outside of its own environment.
- AI agents get a separate environment with a lower level of trust and are locked down to specific resources.
- Public and private networks for different resources which are or are not allowed to access the Internet.
- A development environment with users allowed to access development resources. In production resources are more segregated and separate users are allowed to access those resources.
- Different admins for different resource types is entirely possible, such as a KMS admin, a network admin, and an IAM admin - the big three pillars of any security control architecture.
- Different accounts are used for AWS Organizations Delegated administrators for different job functions.
- Different jobs can only access their own resources in their own environment.
- I also lock down agents within an operating system currently but am exploring some new services on AWS to see if I can do that in another way which I hope to write about later (if it works the way I want).
Those are just some examples. The way my program is designed you can configure whatever trust boundaries you want any which way you want. You are not locked into a particular account and OU configuration.
The Bootstrap Orchestrator
I put things into different projects when writing AI code. That way a single project agent cannot mess up the code for the entire application, only the code it is allowed to change. That doesn’t mean the agents still don’t mess up a lot. You can see the mistakes link on my GitHub AI tracker project for examples, like trying to sneak a delete all accounts script into another script sourced by another script…hmm.
But anyway, the blast radius is somewhat contained.
To run the code in each project, I have an orchestrator project that facilitates running the code for each of the other projects.
Giving the AI agents a more focused objective helps them stay more on track to a degree. Less variation and less paths to take helps reign in the scope a bit.
In the rest of the post I provide an overview of the architecture and data model designed to meet the objectives while trying to reduce mistakes, cost, and time to complete the project.
Read the full article:
-- Teri Radichel
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article