AWS Builder Center

Builder Center

Engineering the SDLC for coding agents: decisions, evidence, and autonomy

Coding agents can ship working features faster than teams can verify and own them. This article models the records that authorize and trace a software change (intent, decision, spec, change, evidence, policy, incident) so a release can be tied to the exact requirements it satisfies. It then uses that model to set agent autonomy per action and environment, covers protecting checks from bypass, maps the approach to Kiro and AgentCore capabilities, and reviews what productivity evidence supports.

Luca Bianchi

CTO

Key takeaways

 
  • Coding agents need a delivery process that connects business intent, architectural decisions, implementation, verification, and production feedback.
  • Version those relationships. Evidence from an earlier commit or specification must not silently authorize a different change.
  • Run inexpensive automated checks early, evaluate agent reviewers against known cases, and reserve human review for decisions and implementation risks that need judgment.
  • Protect the checks themselves. An agent that can weaken its own release criteria can produce a green pipeline without satisfying the requirement.
  • Grant autonomy for specific actions within a domain. Business differentiation, operational risk, and reversibility all matter; a generic capability such as payments can still require strict controls.
 
A coding agent can deliver a working feature while leaving the team with unanswered questions. Why was this design chosen? Which failure cases were tested? Who approved the data model change? What happens when the service has to evolve?
A successful demo answers very few of those questions. A green pipeline answers only the ones its checks were designed to ask.
For a CTO, the practical concern is the cost of owning what the team ships. Faster implementation helps when review, deployment, and maintenance can keep up. If every generated change needs hours of reconstruction before an engineer can assess it, some of that speed has already been spent.
Luca Mezzalira  makes a related point in The prototype is the easy part: engineers still have to understand the behavior they are accepting, including mistakes that look plausible on first inspection. [1] 
This article develops a model for making that work explicit. It connects decisions to specifications, changes and evidence, then uses those connections to decide how much autonomy an agent should have. The AWS tooling discussed later is one way to implement parts of it.

Design the delivery process around the agent

Harness engineering describes the controls and feedback around a model. Birgitta Böckeler distinguishes between guides, which help an agent produce an acceptable result, and sensors, which detect problems after it acts. Repository instructions and examples are guides; tests and review tools are sensors. Both need maintenance as the system changes. [2] 
OpenAI's account of building an internal product with Codex gives a concrete example. The team started with three engineers and later grew to seven. Over five months, the repository reached roughly a million lines across application code, infrastructure, tooling and documentation. Engineers invested in structured repository knowledge, architectural checks and feedback the agents could use. This is an experience report from a particular environment, not a controlled productivity estimate. [3] 
The repository is a useful place to start. It is also only part of the delivery process. A coding agent may receive a carefully written specification whose business assumptions were never agreed. It may implement a decision that was superseded in a meeting. It may pass every check and still deploy into an environment those checks do not represent.
I would therefore design the process from the point where someone requests an outcome through to the point where the team observes that outcome in production. That includes who can approve a specification, which evidence permits a release, and how an incident changes the next implementation. The agent needs a defined role within that process, with permissions that match it.

Model the records and their dependencies

Domain-Driven Design provides useful language for this work. A bounded context defines where a model and its terminology apply. It does not imply that an organization's entire development process belongs in one system. For this proposal, the context is narrower: the records and rules used to authorize and trace software changes. Product planning, source control and incident management can retain their own models. [4] 
 
 
 
I would start with seven kinds of record:
 
  • Intent. The desired outcome, constraints and reason for doing the work. Accountable: a named product or engineering owner; an agent may propose or clarify it.
  • Decision. A significant choice, its rationale, status and scope. Accountable: the decision owner, including when an agent acts under delegated authority.
  • Spec. Expected behavior and acceptance criteria, linked to applicable decisions. Accountable: the owner who accepts the specification.
  • Change. A proposed implementation, identified by an immutable revision. Accountable: the contributor and the team accepting it.
  • Evidence. Check results, observations, and reviews for identified inputs. Accountable: the producing system or reviewer, with its identity and configuration recorded.
  • Policy. Allowed actions, approval requirements and release conditions. Accountable: the owner of the relevant platform, service or security control.
  • Incident. Observed production failure, impact, investigation and corrective work. Accountable: the service owner and incident responders.
 
These are modeling choices, not seven mandatory document templates. A ticket may hold the intent. A pull request may hold the change and its review. The requirement is that their identities and relationships survive the tools used to create them.
Decisions are central to the graph, but that does not make a Decision the aggregate root of the whole SDLC. In DDD, an aggregate is a consistency boundary. Specifications, builds and incidents usually have independent lifecycles and cannot sensibly be updated in one transaction. Keep their local invariants explicit and coordinate across records through versioned references and events. [4] 
Events such as SpecApproved, ChangeProposed, EvidenceProduced, Deployed and IncidentOpened make those transitions visible. EventStorming helps discover them with the people doing the work: it exposes missing decisions and handoffs before anyone builds an integration. It does not oblige the team to adopt event sourcing or a new event bus. [5] 
The rule I would enforce is straightforward: a release must use evidence that applies to the exact change and the requirements currently governing it.
Suppose a decision requires a service to retain customer records for 30 days. A specification references that decision, and a change passes tests against it. If the retention decision changes to seven days, the old test result remains a valid historical observation. It no longer establishes compliance with the new requirement. The dependent specification needs review, and the affected checks need to run again.
In practice, record the specification revision, applicable decision revisions, source commit, check configuration, and relevant environment alongside each result. A change to one of those inputs should trigger an impact assessment. Some evidence can remain applicable; some must be regenerated. Do not erase history or rerun the entire delivery pipeline because someone corrected a spelling mistake in an ADR.
There is a concurrency problem here too. A specification can change after CI passes and before a release starts. The release authorization step needs to check current revisions and reject stale approval. An asynchronous notification alone cannot enforce that rule.

Put inexpensive checks early, and protect them

For implementation feedback, start with cheap, repeatable checks: formatting, type checking, unit tests, contract tests, and architectural dependency rules. Run expensive integration or performance checks where their cost is justified. An architectural fitness function measures a characteristic the team wants to preserve, such as permitted dependencies or a latency budget; it need not be a binary check executed on every commit. [6] 
A passing test establishes that a particular assertion held under particular conditions. It does not establish that the specification is complete. Tests can share a mistaken assumption with the implementation, especially when both are generated from the same prompt. Critical acceptance cases need review against the business requirement, and useful test suites need failure cases the implementation author did not simply copy from their own code.
Agent reviewers can cover a different kind of question: does this change contradict an accepted decision, bypass a domain rule, or introduce an unexplained dependency? Give them a narrow remit. Ask each finding to identify the relevant code, the rule it conflicts with and a reproducible example where possible.
Before allowing such a reviewer to block delivery, evaluate it against labelled examples. Measure missed defects as well as false alarms, and distinguish a minor style complaint from a missed authorization defect. Repeated runs can reveal instability, but ten agreeing responses from the same model are not ten independent reviews. A confidence score is not a measured probability of correctness.
Human involvement begins with requirements and design. Later, reviewers need enough implementation detail to assess remaining risks. Authorization logic, migrations, and concurrency changes may require close code inspection. A 3,000-line pull request is a reason to reduce the batch size or allocate proper review time; it is not a reason to declare that humans no longer review diffs.
The integrity of the checks deserves the same attention as the checks themselves. Consider a simple failure: a change relaxes input validation and removes the failing test from the pipeline. CI is green because the relevant check no longer runs.
Preventing that requires controls outside the contributor's discretion:
 
  • Require independent approval for changes to release checks, test exclusions and approval rules.
  • Run mandatory checks from a trusted configuration that the proposed change cannot silently replace.
  • Bind results to the revision being released, and prevent the implementation agent from rewriting those results or approving its own exceptions.
  • Restrict credentials and tool access so an agent cannot bypass the protected path through another command or API.
 
Repository instructions can explain these rules. Enforced permissions and release controls make them hold. A hook is only as trustworthy as the process that runs it and the actor allowed to edit it.

Grant autonomy for specific actions

Mezzalira relates developer involvement to the core, supporting and generic subdomains of strategic DDD. It is a useful starting point for deciding where engineers should spend their attention. [1] 
It needs a separate risk assessment. Payments may be generic to a streaming business and still move real money. An authentication service may offer no competitive differentiation and still control access to every customer account. Conversely, a documentation update in a core domain may be safe to automate.
Set autonomy by domain, action, and environment. Consider the consequences of an error, the ability to detect it, and the cost of recovery. Keep the policy stable enough to operate, with reassessment when the service, agent, or controls change materially.
The following scale proposes making permissions explicit. It is not an industry standard, and the levels are not a maturity score.
Level 0 — Retrieve information and explain it. Required boundary: approved data access, attributable sources, no persistent changes.
Level 1 — Create drafts and local artefacts. Required boundary: isolated workspace; no writes to shared systems.
Level 2 — Propose changes through pull requests. Required boundary: scoped repository access, required checks and independent acceptance; no merge or deployment authority.
Level 3 — Execute approved actions in non-production environments. Required boundary: limited credentials, audit records and tested recovery; no production access.
Level 4 — Execute named, pre-authorized production actions. Required boundary: explicit action limits, monitoring, revocation and a tested recovery procedure.
An agent can be level 2 for application code and level 0 for infrastructure in the same repository. Level 4 might cover a particular deployment or rollback procedure; it should not confer general production administration.
To expand those permissions, the owner should be able to show which cases were evaluated, which failure modes remain, what threshold is acceptable, and who accepted it. Evaluate the agent with the tools, context, and permissions it will actually use. A model upgrade or a new tool can change the result.
A small collection of past incidents helps find obvious weaknesses. It cannot establish a low failure rate for rare, serious events. Size the evaluation to the claim being made, add adversarial and boundary cases, and keep monitoring after rollout.
Revocation also needs testing. Record who can stop new actions, what happens to work already in flight, and how credentials are withdrawn. A deterministic controller can suspend an agent when it exceeds a defined limit. Even suspension needs operational care if the agent is in the middle of a recovery procedure. Permission increases should go through a separate approval path.

Preserve the reason for a change

Michael Nygard's ADR format records context, decision, status and consequences. It also preserves superseded decisions so future engineers can understand why the system took its current shape. [7] 
For agent-assisted work, add the proposer, the accepting owner and the authority under which the decision was made. Distinguish a human decision, an agent suggestion accepted by a human, and an agent decision made under an explicit delegation. All still need an accountable owner. Review cadence should reflect impact and uncertainty as well as origin.
An agent can extract candidate ADRs from a working session. The extraction must remain a proposal: a conversation includes abandoned ideas, tentative assumptions and instructions that were later corrected. Saving a transcript is useful, but it does not turn every sentence into an accepted decision.
For each release, keep a compact evidence manifest linking the intent and specification to the source revision, build artifact, check results, and approvals. Include the agent configuration, model identifier as exposed by the provider, relevant tool versions and policy revision when they affected the work. Capture enough to investigate the result without assuming a future model invocation will reproduce it exactly.
This complements software supply-chain provenance. SLSA describes verifiable information about where, when, and how an artifact was produced. Build provenance can connect a deployed artifact to its source; the decision and specification links explain why that source was accepted. Neither substitutes for the other. [8] 
A Markdown manifest may be sufficient initially. Once automation relies on it, generate factual fields from trusted systems and protect the records from contributor alteration. Commit trailers are convenient references, but rewriting or squashing commits can lose them. The release record should carry the authoritative links. Keep sensitive transcripts under appropriate access and retention controls rather than copying every session into the repository.

Where AWS tooling fits

The mapping below connects documented capabilities to the proposed workflow. It is a design mapping, not an out-of-the-box integration. Product documentation was checked on 1 October 2026.
Turn intent into implementation work. Kiro Specs structures requirements, design, and tasks. [9]  The team still owns acceptance criteria, ownership, and approval policy.
Supply project knowledge. Kiro Steering provides persistent guidance; Powers packages context and tools for on-demand use. [10]  [11]  The team still owns correct, current context and explicit domain scope.
Trigger checks during agent work. Kiro Hooks automates actions around workflow events. [12]  The team still owns the check implementation and protection against bypass.
Run asynchronous work. Kiro Crew supports persistent sessions, scheduled work, and event triggers. [13]  The team still owns task eligibility, credentials, limits, and escalation.
Host platform agents. AgentCore Runtime hosts agents and tools; Observability provides operational visibility. [14]  The team still owns the agent's behavior and its integration with delivery systems.
Authorize tool access. AgentCore Policy evaluates rules at the Gateway. [15]  The team still owns policy design, enforcement mode, and closure of alternate access paths.
Evaluate agent behavior. AgentCore Evaluations supports online and on-demand assessment. [16]  The team still owns representative cases, calibrated evaluators and acceptance thresholds.
A Power could package the guidance for a bounded context. That is an organizational convention, not a security boundary: loading the right context does not prevent access to the wrong system.
AgentCore Policy enforces calls that pass through the configured Gateway. AWS's documentation distinguishes ENFORCE, which blocks denied calls, from LOG_ONLY, which records decisions without blocking them. Permission to change the Gateway configuration must also be restricted: it can disable enforcement. A direct credential or alternate network path can sit outside that boundary; those paths need their own restrictions. [15]  [17] 
AgentCore Evaluations measures agent behavior, including tool use and task completion. It can contribute evidence about a review or triage agent. It does not replace tests of the application code that the agent produces. [16] 
A practical first platform agent would assist incident investigation. Given an incident, it retrieves the deployed artifact, related changes, and available specifications, then proposes an explanation with links to the evidence. It must also be able to say that it has not identified a causal change. Incidents can involve dependencies, configuration, capacity, or several interacting releases.
Keep that agent's initial role limited to retrieval and proposals. Read access still needs scope: logs and traces can contain customer data, and an unrestricted query can be expensive. Once an engineer confirms the cause, the agent can propose a regression test or an update to an existing architectural check. Use an event handler or a conventional service where the task is deterministic; an agent is useful where interpreting incomplete evidence adds value.

Measure delivery, including the work after generation

Published productivity evidence does not support a single multiplier a CTO can put into a staffing plan.
In a controlled experiment published in 2023, Sida Peng and colleagues found that developers with GitHub Copilot completed a JavaScript HTTP-server task 55.8% faster. That result reflects a single bounded implementation exercise. It does not measure the cost of evolving an established production system. [18] 
METR's early-2025 randomized study followed 16 experienced open-source developers working on 246 tasks in repositories they knew well. Allowing AI tools increased completion time by 19%. In February 2026, METR reported that its follow-up data was affected by selection and measurement problems, including developers' reluctance to work without AI. The authors considered greater speedups plausible but cautioned that the new data did not reliably establish their magnitude. The original slowdown should not be presented as a current estimate for all coding agents. [19]  [20] 
DORA's findings have also changed. Its 2024 report associated AI adoption with lower delivery throughput and stability. The 2025 research found a positive relationship with throughput and product performance, while the negative relationship with stability remained. These are observational findings, not proof that AI alone caused the outcomes. [21]  [22] 
These studies examine different tasks, populations, and tools. They justify measuring local results; they do not establish that a particular harness caused a productivity gain.
For a team adopting agents, I would track elapsed time to an accepted change, engineering time spent reviewing and correcting it, and post-release failures. Include the cost of maintaining the checks and context the agents depend on. Compare similar classes of work and record significant changes to the tools or process. Generated lines and pull request counts can describe activity, but they cannot tell you whether the team delivered something useful or made the next change harder.

Start with one service

Choose a service with a clear owner and enough delivery history to establish a baseline. Link its changes to specifications and decisions, and record the deployed artifact. Protect mandatory checks before asking an agent to work unattended.
Then run a narrowly scoped reviewer without blocking releases. Compare its findings with engineer assessments and known defects. Use the disagreements to improve the instructions, evaluation cases or conventional checks. Only after that should the team consider delegating a specific action, with a named owner and a tested way to stop it.
The platform team can provide shared release controls, evidence formats, and access rules. Service teams supply the domain constraints. This avoids rebuilding the same machinery in every repository while leaving room for different risk profiles.
The decision to increase autonomy should be easy to explain in an engineering review: this agent may perform this action, in this environment, because these checks cover the relevant failures and this person owns recovery. If that sentence needs several paragraphs of qualifications, narrow the action before granting the permission.
 
 

References

  1. Luca Mezzalira. The prototype is the easy part . 24 July 2026.
  2. Birgitta Böckeler. Harness engineering for coding agent users . 2 April 2026.
  3. Ryan Lopopolo. Harness engineering: leveraging Codex in an agent-first world . OpenAI, 11 February 2026.
  4. Eric Evans. Domain-Driven Design Reference: Definitions and Pattern Summaries . March 2015. See Aggregates, Bounded Context and Core Domain.
  5. Alberto Brandolini. EventStorming . Official method overview.
  6. Neal Ford, Rebecca Parsons and Patrick Kua. Building Evolutionary Architectures: précis . See Fitness Functions.
  7. Michael Nygard. Documenting Architecture Decisions . 15 November 2011.
  8. SLSA. Provenance, specification v1.2 .
  9. Kiro. Specs .
  10. Kiro. Steering .
  11. Kiro. Powers .
  12. Kiro. Hooks .
  13. Kiro. Kiro Crew .
  14. AWS. Amazon Bedrock AgentCore overview .
  15. AWS. Policy in Amazon Bedrock AgentCore .
  16. AWS. Amazon Bedrock AgentCore Evaluations is now generally available . 31 March 2026.
  17. AWS. Policy enforcement modes .
  18. Sida Peng, Eirini Kalliamvakou, Peter Cihon and Mert Demirer. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot . February 2023.
  19. METR. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity . 10 July 2025.
  20. Joel Becker, Nate Rush, Tom Cunningham, David Rein and Khalid Mahamud. We are Changing our Developer Productivity Experiment Design . METR, 24 February 2026.
  21. DORA. Accelerate State of DevOps Report 2024 .
  22. Nathen Harvey and Derek DeBellis. Announcing the 2025 DORA Report: State of AI-Assisted Software Development . 23 September 2025. Full report .
 
Online sources accessed 1 October 2026. This article proposes the seven-record model and autonomy scale; the cited sources support the underlying concepts and reported findings
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article