How to Think About AI Agents That Write and Deploy Code with Amazon Bedrock
When autonomous AI agents generate code that runs unsupervised, the key architectural decision isn't what model to use, it's how much the agent should be allowed to decide. This article introduces a mental model for scoping agency in code generation with Amazon Bedrock, backed by patterns from a production system.
AWS Solutions Architect
This Isn't a Prompting Problem
This isn't about AI helping developers write code faster. It's about AI systems that produce, validate, and deploy code autonomously — where the generated output runs in production without a human ever touching it. Once you cross that line, the core engineering challenge changes.
It's no longer "how do I prompt this?" It's "how much should this agent be allowed to decide on its own?"
We call this the scope of agency problem. Getting it right is the difference between AI-generated code that demos well and AI-generated code that runs in production unsupervised.
We learned this building a platform where AI agents generate, validate, and deploy data transformation code. Users configure a pipeline through a dashboard — source data, target destination, field mappings — and a Strands Agent on Amazon Bedrock produces the PySpark code that runs as an AWS Glue job. No human writes code. The agent does.
After months in production, we've arrived at a mental model for how to scope agency in code generation systems. The principles apply whether you're generating Glue jobs, Lambda handlers, CDK constructs, SQL queries, or any other executable artifact.
The Spectrum of Agency
Think of AI code generation as a spectrum:
Plain text
Zero Agency Full Agency│ │▼ ▼Template Fill-in Bounded Architectvariable completion creative zone from scratch│ │ │ │"Insert the "Write the "Write the "Design andbucket name SQL WHERE transform implement thehere" clause for logic for entire system"this query" these fieldmappings"
Most teams start at the right end. The model is impressive, it can write entire systems — why not let it? You quickly discover that full agency works for exploration and prototyping but breaks in production:
- The agent decides to initialize the Spark session differently across runs
- The agent invents its own error handling pattern (sometimes good, sometimes catastrophic)
- The agent makes an interpretation of ambiguous mappings that silently drops data
- The agent uses a deprecated API because it appeared in training data
The instinct is to move left — reduce the AI to a glorified template variable. But that throws away the thing that makes agents valuable: judgment in ambiguous situations. Field mapping logic genuinely requires semantic understanding. "Map
cust_nm to full_name , trim whitespace, lowercase" is a decision that benefits from AI reasoning.The right position on this spectrum is the minimum scope where AI judgment adds value. Everything outside that scope should be deterministic.
The Core Principle: Identify the Creative Zone
Every code generation task has two zones:
The deterministic zone — code that must be correct, must be consistent, and doesn't benefit from AI judgment. Session initialization, import statements, error handling patterns, logging configuration, output writing, catalog setup. This code is identical across hundreds of generated artifacts.
The creative zone — code that requires understanding semantics, making judgment calls, and translating human intent into logic. Field mappings, business rules, conditional transformations. This varies per artifact and genuinely benefits from AI reasoning.
The foundational architectural decision is: give the agent authority over the creative zone only. Everything in the deterministic zone is handled by templates, configuration, or other deterministic systems.
In our system, the creative zone is a PySpark transform block — typically 5-30 lines of
.withColumn() , .select() , .filter() , and .cast() calls. The deterministic zone is everything else: ~200 lines of Spark session setup, Iceberg catalog registration, write-mode handling, error boundaries, and record count validation.The agent writes 15 lines. Templates write 235. The agent's scope of agency is the transform block and nothing else.
Here's what the creative zone looks like in practice. Given these field mappings from the user:
Plain text
customer_name (string) → full_name (string), trim and lowercaseorder_total (string) → amount_cents (integer), multiply by 100status (string) → is_active (boolean), "active" = true
The agent produces this — and only this:
Python
df_transformed = df.withColumn("full_name", lower(trim(col("customer_name"))))df_transformed = df_transformed.withColumn("amount_cents",(col("order_total").cast(DoubleType()) * 100).cast(IntegerType()))df_transformed = df_transformed.withColumn("is_active",when(col("status") == "active", lit(True)).otherwise(lit(False)))df_transformed = df_transformed.drop("customer_name", "order_total", "status")
Notice the judgment calls: interpreting "multiply by 100" as a cast-then-multiply chain, choosing
when().otherwise() for boolean conversion, deciding to drop the original columns after renaming. These are semantic decisions that benefit from AI reasoning. Everything else — reading the source data, configuring the Iceberg catalog, handling write modes, validating record counts — is handled by the template without any AI involvement.
Why This Isn't Just "Better Prompting"
You might think: "Just tell the agent to only generate the transform block in your prompt." We do. But constraining via prompt alone is insufficient for three reasons:
1. Prompts are suggestions, not guarantees.
A prompt that says "Output ONLY the transform code, no imports" works 95% of the time. The other 5%, the agent includes imports anyway — maybe because the mappings are complex and it "felt" like the code needed a helper function. Prompt constraints are probabilistic. Architectural constraints are deterministic.
2. Agency leaks through tool use.
An agent with access to MCP tools can make decisions beyond what the prompt covers. If your codegen MCP server exposes a
generate_full_script tool alongside a generate_transform_block tool, the agent might call either based on its reasoning. The architecture must constrain which decisions flow through the agent at all.3. Validation requires understanding scope.
If the agent can generate anything, you need to validate everything. If the agent can only generate a transform block, you only need to validate transform logic. Bounded scope enables bounded validation — which enables automation rather than mandatory human review.
Architectural Containment: Four Layers
Once you've identified your creative zone, you need containment around it. We use four layers, each catching a different class of failure:
Layer 1: Constrain the Agent's Decision Space
The agent receives a prompt that explicitly defines what it can and cannot decide:
Plain text
Generate ONLY the PySpark transformation code.Input DataFrame: `df`. Output DataFrame: `df_transformed`.Use only: .select(), .withColumn(), .withColumnRenamed(),.cast(), .filter(), .drop()CRITICAL RULES:- Use lower(col("field")) NOT col("field").lower()- Use trim(col("field")) NOT col("field").trim()- CSV source data has ALL columns as strings — you MUST cast explicitlyDo NOT create classes, helper functions, or imports.
Three constraints define the scope:
- Fixed interface contract. Input is
df, output isdf_transformed. The agent cannot rename these, introduce intermediates, or change the interface between its code and the surrounding template.
- Allowed operations whitelist. Only
.select(),.withColumn(), etc. Not.repartition(), not.write(), not.createOrReplaceTempView(). The agent handles transformation, not orchestration.
- Explicit anti-patterns.
col("field").lower()is valid Python but wrong PySpark. The model defaults to method-call style (pandas dominates training data). You must name these failures in the prompt because the agent has no way to discover they're wrong — the code parses, it just doesn't run.
Layer 2: Deterministic Assembly Around the Creative Zone
The agent's output is injected into a Jinja2 template that provides everything outside the creative zone:
jinja2
# ... 100 lines of session init, catalog config, source reading ...# === AI-Generated Transform Block ==={{ transform_block | indent(4, first=False) }}# === End Transform Block ===# ... 100 lines of auto-casting, data quality, write logic ...
This isn't just an optimization — it's a containment boundary. If the agent produces garbage, the blast radius is limited to the creative zone. The Spark session still initializes. Error handling still catches failures. The job fails loudly in the transform block rather than silently corrupting data through a bad write pattern.
We maintain one template per target type (S3 Tables/Iceberg, Glue Catalog, JDBC, S3 Parquet). Each encodes target-specific infrastructure that the agent shouldn't be reasoning about — Iceberg catalog configuration, merge-vs-append write semantics, partition column handling.
A note on template injection security. When you inject AI-generated content into a template, the natural question is: can the generated code break out of its containment? With Jinja2, the answer is no — the
{{ transform_block }} substitution inserts a literal string value. It doesn't re-parse the result for template directives. If the model outputs text containing {{ }} or {% %} , those render as literal characters in the final script, not as Jinja2 instructions.The actual security surface is different: the generated code executes inside a Spark cluster with IAM permissions. A hallucinated or adversarial transform block could theoretically import
boto3 and access resources, write data to unintended locations, or exfiltrate information to external endpoints. Our mitigation is layered: the template controls the import block (the agent's code has no import statements), the Glue job runs with a scoped IAM role that only permits access to the specific source and target paths, and network egress from the Spark cluster is restricted via VPC configuration. The agent can't grant itself permissions it doesn't have.Layer 3: Post-Agent Structural Validation
Here's the pattern that convinced us prompts alone aren't enough:
Python
df_transformed = df.withColumn("name", col("name").lower())
This is syntactically valid Python. It passes linting. It passes type checking. But it's wrong — PySpark
Column objects don't have a .lower() method. The correct form is lower(col("name")) . This fails at runtime on a Spark cluster after potentially minutes of compute.The agent produces this despite explicit prompt instructions because the pattern exists in training data (it's valid in pandas) and is reinforced by how similar it looks to correct code.
We catch it with an AST (Abstract Syntax Tree) validator — a program that parses the generated code into its structural representation and walks it looking for specific patterns:
Python
def validate_pyspark_antipatterns(code: str) -> list[str]:import astissues = []COLUMN_METHODS = {"lower", "upper", "trim", "substring", "length"}tree = ast.parse(code)for node in ast.walk(tree):if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute):if node.func.attr in COLUMN_METHODS:issues.append(f"Line {node.lineno}: .{node.func.attr}() called as method"f" — use pyspark.sql.functions.{node.func.attr}() instead")return issues
This takes microseconds. It catches the exact class of bug where the agent's output is valid code with wrong runtime behavior. No Spark cluster needed, no execution, no waiting.
The key insight: the agent does not validate its own output. A deterministic system validates the agent's output. This is deliberate — asking an agent to validate its own code is asking the same probabilistic system to catch its own probabilistic errors. Separation of concerns applies to AI systems too.
Layer 4: Quality Gates — Retry or Escalate
When validation catches issues, the system responds proportionally:
Python
antipattern_issues = validate_pyspark_antipatterns(generated_code)if antipattern_issues:generated_code = generate_code(pipeline_metadata, mapping_item) # One retryretry_issues = validate_pyspark_antipatterns(generated_code)if retry_issues:# Escalate to human review
Exactly one retry. If the agent produces the same anti-pattern twice, a third attempt won't help — the prompt needs updating or the case requires human judgment. Infinite retries waste compute and mask systematic failures.
Beyond anti-pattern detection, a quality score drives routing:
- Score >= 70 + has error handling + has logging → auto-approved (no human)
- Below threshold or regenerated → human review required
In practice, ~85% of generations auto-approve. The remaining 15% get human eyes. This is the critical operating point: enough automation to be valuable, enough oversight to be safe.
The Closed Loop: Human Review Feeds Back Into the System
Escalating to a human isn't a dead end — it's a learning signal. Every correction a reviewer makes represents a gap in the system that should be closed:
- A reviewer catches a new anti-pattern the AST validator doesn't know about → it becomes a new check in Layer 3 and a new "CRITICAL RULES" entry in the Layer 1 prompt.
- A reviewer rewrites the same type of cast logic three times → that's a systematic prompt gap, not a one-off. The prompt template gets updated.
- A reviewer approves code that later fails in production → the failure analysis becomes a new test case in the PySpark-to-pandas converter (Part 2).
We track reviewer corrections categorized by which layer should have caught them. If Layer 3 (structural validation) is letting through bugs that humans consistently catch, that's a signal to expand the validator — not to add more human review. The goal is to shrink the 15% over time by absorbing human judgment into the automated layers. Each review makes the system slightly better; escalation is the mechanism through which the architecture evolves.
When to Expand Agency
Bounded agency is the starting point, not the end state. Once your containment layers are solid, you can selectively expand what agents decide — with guardrails growing alongside agency.
Our system expands agency in two directions:
Lateral expansion: more agents in the pipeline.
Beyond the code generation agent, we run:
- A schema mapping agent that decides what to transform (upstream)
- A testing agent that validates the generated code in a sandbox (downstream)
- A self-healing agent that detects runtime failures and regenerates code autonomously
Each agent follows the same containment architecture described above — it's not unique to code generation. The testing agent, for example, has its own creative zone (interpreting code quality and producing a structured review), its own containment (a fixed report format it must output), its own structural validation (we parse the report for numeric scores and pass/fail verdicts), and its own quality gate (if confidence is low, escalate rather than auto-pass). The agents communicate through well-defined artifact contracts (DynamoDB records with fixed schemas), not free-form messages. No agent passes unstructured text to another — every handoff is a typed record with required fields.
Vertical expansion: increasing autonomy for proven patterns.
For pipelines that have run successfully 10+ times with the same mapping shape, we relax the approval gate — auto-approve even regenerated code. The agent has "earned" expanded authority through demonstrated reliability in that specific context.
The principle: agency expands only as fast as guardrails expand. Never grant more autonomy than you can validate.
The Mental Model, Summarized
Plain text
┌──────────────────────────────────────────────────────────────────┐│ ││ 1. Identify the creative zone ││ What decisions genuinely benefit from AI judgment? ││ Everything else is deterministic. ││ ││ 2. Constrain the agent to that zone ││ Fixed interface contracts. Operation whitelists. ││ Explicit anti-patterns. No structural authority. ││ ││ 3. Contain the agent's output ││ Templates as blast-radius limiters. ││ Agent code is injected, not assembled. ││ ││ 4. Validate with a separate deterministic system ││ The agent does not check its own work. ││ AST analysis, structural checks, quality scoring. ││ ││ 5. Gate deployment proportionally ││ Auto-approve when confident. Retry once. Then escalate. ││ Never retry infinitely. Never skip review for edge cases. ││ ││ 6. Expand agency only with matching guardrails ││ More autonomy requires more validation. ││ Proven patterns can earn relaxed gates. ││ │└──────────────────────────────────────────────────────────────────┘
Applying This Beyond ETL
The specific examples in this article use PySpark and AWS Glue, but the mental model maps directly to other domains:
Lambda handler generation:
- Creative zone: business logic inside the handler
- Deterministic zone: middleware chain, error responses, metrics, tracing
- Anti-patterns to validate: responses after return, missing error handling, hardcoded secrets
CDK/Infrastructure-as-Code generation:
- Creative zone: resource configuration based on requirements
- Deterministic zone: stack structure, naming conventions, tagging
- Anti-patterns to validate: public S3 buckets, missing encryption, overly permissive IAM
SQL query generation:
- Creative zone: WHERE clauses, JOINs, aggregations from natural language
- Deterministic zone: query structure, parameterization, result formatting
- Anti-patterns to validate: SELECT *, missing WHERE on DELETE/UPDATE, unparameterized user input
Test generation:
- Creative zone: test logic, assertions, edge case identification
- Deterministic zone: test framework boilerplate, setup/teardown, import structure
- Anti-patterns to validate: tests that always pass, mocking the thing being tested, missing assertions
In every case, the questions are the same: What's the creative zone? What's the containment boundary? What can go wrong that looks correct? How do I validate without executing?
Getting Started
You don't need a full pipeline to apply this thinking. Start with wherever you're currently using Bedrock for code generation:
- Audit your last 10 failures. What did the agent decide that it shouldn't have? That's agency that needs to be revoked or constrained.
- Draw the line between your deterministic and creative zones. If you can't identify a clear boundary, your agent has too much scope.
- Build one structural validator for your top failure pattern. Python's
astmodule is an afternoon of work. For other languages, tree-sitter gives you the same structural analysis capability.
- Separate generation from validation. The agent produces. A different system checks. Never ask the agent to validate its own output.
The goal isn't perfect generation — it's predictable generation with known failure modes and automated containment. That's what makes agentic code generation production-ready.
Built with Amazon Bedrock, Strands Agents, AWS Glue, and AWS Step Functions. Part 2 covers testing AI-generated code without its native runtime — extending the guardrails into execution validation. You can read that here .
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article