AWS Builder Center

Builder Center

I Made My LLM Fix Its Own CloudFormation Errors: A 3-Retry Self-Repair Loop

LLM-generated CloudFormation often fails validation on the first try. Here is how I feed the exact errors back to the model, cap retries, and hand off to a human when the loop can't fix it.

LLMs can turn plain English into CloudFormation in seconds. But the first draft can be broken: an invented property, a wrong type, a missing required field. In my AI-powered AWS Deployment Assistant, I stopped treating the first output as final and added a self-repair loop.

The problem

Ask an LLM for "an S3 bucket with versioning and a Lambda that reads from it" and the template may look perfect but fail validation. One failure is fine. Showing the user a raw validation error is not.

The idea

Validation tools already produce precise errors. So I send them back to the model:
  1. Generate the template from the user's prompt.
  2. Validate it (cfn-lint plus a CloudFormation ChangeSet check).
  3. If it fails, send the original prompt, the broken template, and the exact error back to the LLM.
  4. Repeat, with a maximum of 3 retries.
  5. If it still fails, stop and hand off to a human with the full error trail.

The loop

1
2
3
4
5
6
7
8
9
10
11
12
MAX_RETRIES = 3

def generate_with_repair(prompt: str) -> dict:
template = llm_generate(prompt)
for attempt in range(MAX_RETRIES + 1):
errors = validate(template) # cfn-lint + ChangeSet check
if not errors:
return {"status": "ok", "template": template, "attempts": attempt}
if attempt == MAX_RETRIES:
break
template = llm_repair(prompt, template, errors)
return {"status": "failed", "template": template, "errors": errors}

The repair prompt

1
2
3
4
5
6
7
8
9
10
11
12
13
The CloudFormation template below failed validation.

ORIGINAL REQUEST:
{prompt}

TEMPLATE:
{template}

VALIDATION ERRORS:
{errors}

Fix ONLY what the errors point to. Do not change anything else.
Return the full corrected template as YAML, nothing else.
The line "fix ONLY what the errors point to" is deliberate. A model asked to "fix this" can rewrite parts that already work and introduce new problems. Narrowing the instruction keeps each repair small and easy to review.

Example: before and after

This is an illustrative example of the kind of error the loop handles.
Error:
1
E3002 Invalid Property Resources/MyBucket/Properties/VersioningEnabled
Before:
1
2
3
4
MyBucket:
Type: AWS::S3::Bucket
Properties:
VersioningEnabled: true
After:
1
2
3
4
5
MyBucket:
Type: AWS::S3::Bucket
Properties:
VersioningConfiguration:
Status: Enabled
The model invented a flat property. The error names the exact path, which is usually enough context to find the real nested property.

Design decisions

  • Cap the retries. An unbounded loop can burn tokens and time forever on an error the model can't fix. A hard cap of 3 keeps cost and latency predictable.
  • Pass exact errors, not summaries. The validator's message already contains the resource, property path, and error code. Summarizing it only throws information away.
  • Constrain the repair prompt. Small, targeted fixes are easier to trust and to diff.
  • Fail loudly. When the loop gives up, the assistant shows the full history (each attempt and each error) instead of a vague "something went wrong".
  • Keep the other guardrails after the loop. A template that passes validation can still be insecure or expensive. Security and cost checks run on the final output, not on the first draft.

What the loop does not do

It only fixes what the validator can detect. If a template is valid but wrong for what the user wanted, validation won't catch it. That's why the loop is one layer in the pipeline, not a replacement for the others.

Conclusion

A self-repair loop turns validation errors from a dead end into feedback. It doesn't make the LLM perfect, but it means a human only sees templates that have already survived validation or come with a clear error trail.
This is part of my series on building a safe AI-powered AWS Deployment Assistant. See my earlier posts on guardrails, testing, and cost checks.
# cloudformation-resources# infrastructure-as-code# generative-ai# devops
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article