Tracking an AI Project: Menu complications

I've been tracking my progress on a real world project I need to complete for my work in a series of blog posts on secure infrastructure for AI agents . Here's the latest.

Stayed up way too long trying to restore this code to a working state with Kiro CLI and Opus 4.6 or 4.8. I’ve already burned over 10% of a $200 plan in like two days. I’ll add that to the repo in a bit and monthly cost for prior months.

The thing that took the longest time for the model was getting all the menus correct throughout the application. It almost felt like the model was nerfed again. It was spinning and spinning its wheels and not following instructions for menus which was to create a common file to handle steps repeated at the end of the menu and a wrapper file for the steps above it that can vary. Then use source those files as appropriate everywhere instead of duplicating that logic.

I have menus that can get long so I have functionality that lets the user select a number. If they select a letter then it filters the menu by that so can have a shorter list to select from. The numbers are generated dynamically as explained in a one of my blogs.

For whatever reason the model just kept spinning and not implementing this consistently. It also implemented it in a more complicated way than I wanted and added way too many variables. At some point I just took a chunk of what it was working on because the menus are not really a secret.

I dumped the output into google aimode and it dumped out a solution SO FAST. Of course it did not follow my rules and didn’t have as much context was hiding errors all over the place so would have been hard to troubleshoot. But I dumped that back into the Anthropic model and said, will this work?

I didn’t care if the Gemini code did or did not work but it seemed to kick the Anthropic model into gear. And it finished. It wasn’t super immediate but it seemed to help. (Do not put sensitive data into Internet models, please - this is code I'm practically giving away and it's already not implemented in the greatest way. I'm trying to fix it.)

💫 Also interesting that Google just announced a new model that is supposedly faster and cheaper.  I wish we could get all the models in Bedrock in Kiro. The price of Kiro wouldn't change but hopefully we'd get more done with less tokens so the total cost to complete a block of work would decrease. Wish: https://builder.aws.com/wishlist/3EdZnhuYbi5bCR6vBvzE4THvE8C_w/i-wish-we-could-select-from-all-the-bedrock-models-in-kiro-cli-and-others  💫

One other thing I did was expand consistent file naming for resources. List, add, delete, verify. I’m glad I put the whole network deployment configuration in a separate project as none of that got messed up as I made all these changes. Same for bucket deployments. I may eventually move everything to a separate project to reduce my context when working on menus and maybe that would speed up completing all of this but I keep feeling like I'm almost done...you know how that goes.

The model struggles with the RAM shares and EIP in IPAM CIDRs when have cross account resources. Are none of y’all doing that so no training data? I’ll explain why I do certain things in another post. I already have a post on easier EIP allocation and managing prefix lists for remote access IPs . It should make your life easier until you need to delete it or handle automated deployments apparently. The RDP and SSH security group rules that were working before now can't find the prefix list used to track the admin IPs allowed in by those rules.

I had some ideas while working on that about how to improve RAM and EIP deployments but there are pros and cons to any approach. See related wish list items in GitHub but not posted here yet. May or may not post here. I feel like I'm posting too many wishes and ideas. 😀 My brain just spins with ideas like that. At least I can try them all out in my own code!

Once that was done finishing the rest was reasonably easy with a few road blocks. I have to move a domain registration or register a new domain to redeploy the Yubikey auth. Thinking about how to handle that.

Time tracking, progress, issues with a few lines of code deleted notes when I fixed something, and my AWS Wishlist  items while completing this in the related GitHub repository. I added some additional objectives and tracking what I've deployed and tested so far.

Answering the question of whether AI can build real world projects and how much faster is it really? How much does it cost? If I can get this done I'll tell you what it was for me on this project, though I'm sure I can do some things better. Pushing towards done as top priority.