AWS Builder Center

Builder Center

Build a self-improving development environment with Trace and Kiro

Turn your corrections into fixes for Kiro's steering files: a hands-on build of Amazon's Trace research with Kiro hooks, an MCP server, and git.

Faris Haddad - AWS

Data and Ai lead at AWS
Building a self-improving development environment with Trace and Kiro starts with a simple observation: your corrections are already diagnostic data.
Every developer who uses an AI coding agent has the same experience: the agent generates code, you fix it, and a week later it makes the same mistake in a different file. Many coding agents now keep some memory of your corrections. What none of them does is trace a failure back to the specific rule file that caused it, confirm the gap, and propose a reviewable fix to that file. So the context source stays broken.
A new paper from Amazon, Trace: TRajectory Attribution for Automated Context Engineering (Zhao, Misra, and Pandey, KDD 2026; styled TRACE in the paper), proposes exactly that loop. It mines implicit dissatisfaction signals such as your corrections, traces each failure back to the context source that caused it, and proposes a targeted fix.
This post explains the paper's core ideas and shows how to implement them in  Kiro , the agentic development environment  built and operated by AWS , using Kiro's v1 hooks, MCP server support, and steering files. A companion post,  Why AI coding agents forget: memory, handoffs, and latent context,  compares how other coding agents handle corrections and looks at a second, shorter-lived kind of context loss: what happens when one agent hands work to another.
Prerequisites: Kiro IDE 1.0+ or Kiro CLI V3, Node.js 22+, and an  Amazon Bedrock -enabled AWS account.
Target audience: developers using AI coding agents who want to reduce rework. Learning level: 300 (advanced).


The problem: context rot

AI coding agents run on context: steering files that define conventions, specs that describe features, tool descriptions that guide function calling, and knowledge bases that supply domain facts. When an agent like Kiro produces wrong output, the root cause is often in one of these context sources: a stale convention, a missing constraint, an ambiguous spec.
The challenge is diagnosis. When Kiro generates an  Amazon SQS  queue without the FIFO configuration you need, is that because:
  • Your  aws-conventions.md  steering file doesn't mention FIFO queue rules?
  • Your spec's acceptance criteria omitted the constraint?
  • The AWS CDK construct documentation in your MCP server is outdated?
Today you fix the generated code and move on. The context source stays broken. Next week, Kiro makes the same mistake on a different file. This is context rot: the slow accumulation of gaps and staleness in agent context sources that degrades output quality over time.


What Trace does

Trace is an automated feedback loop with three stages.

1. Dissatisfaction mining

Every time you correct an agent's output, you leave an implicit signal: "this was wrong." Trace captures these signals systematically. It doesn't rely on thumbs-up/down buttons or feedback forms; it detects the corrections themselves. User corrections, rephrasing, and abandonment cues all count.
Trace builds on  DRIFT  (Wang et al., 2025), which found that dissatisfaction signals occur roughly twice as often as explicit satisfaction signals and carry richer information. They're also free to collect, because they already exist in your workflow.

2. Root cause attribution

Once a failure is detected, Trace works backward through the agent's execution trace — the sequence of context lookups, tool calls, and decisions that led to the output — to find the node most responsible.
The key design choice is holistic attribution: rather than examining each node in isolation, Trace presents the whole trace to the LLM at once and asks which source best explains the gap between expected and actual output. This finds root causes even when several downstream nodes show symptoms. In the paper's ablation on complex traces, holistic attribution found the root cause 40% of the time, compared with 20% for an iterative, TextGrad-style approach that examines one node at a time.
On the paper's benchmark of 60 synthetic dissatisfaction traces (up to 16 execution nodes each), Trace identifies the root-cause node 72.7% of the time (95% confidence interval: 59–85%), and 85% of the time within its top three candidates. For a sense of how hard attribution is: on the  Who&When  benchmark of real multi-agent logs, the best method found the responsible agent 53.5% of the time and the failing step only 14.2% of the time. The benchmarks differ, so the numbers aren't directly comparable.

3. Exploratory verification and recommendation

The Recommender doesn't blindly accept the attribution. It reads the implicated context source, searches authoritative references, and cross-references them to confirm the diagnosis. Only then does it propose a fix: create, update, or delete content (or take no action), with the exact text change, supporting evidence, and a confidence score.
This verification step matters. In the paper's ablation, exploration raised accuracy on knowledge-base create-versus-update decisions from 33% to 83%. Overall, Trace chose the correct operation 96% of the time, and got both the operation and the target file right 82% of the time.


How Trace differs from agent memory

Several coding agents now learn from your corrections.  Claude Code ,  GitHub Copilot , and  Windsurf / Devin (Cascade)  record memories automatically, usually in a separate store or notes file. Claude Code's prompt audit comes closest to Trace: it produces a report of proposed edits to your instruction files, and nothing changes until you approve them.
Trace adds the steps in between. It ties each failure to the specific source file that caused it, classifies the problem as missing or stale content, verifies the diagnosis against authoritative references, and patches the original rule file, so the fix is reviewable and version-controlled. Kiro has no built-in equivalent today, which is the gap this post fills. The companion post compares these agents in more detail.
Trace also sits within a growing body of research:
  •  ACE  (Agentic Context Engineering, ICLR 2026) treats contexts as evolving playbooks that are refined through generation, reflection, and curation. It identified two failure modes of repeatedly rewritten contexts: brevity bias, which drops domain detail in favor of concise summaries, and context collapse, where iterative rewriting erodes detail over time. Trace borrows ACE's small add, update, and delete operations for exactly this reason.
  •  PRELUDE and CIPHER  (Gao et al.) learn a user's latent preferences from their edit history. They're the closest prior work to using edits as feedback, and they suggest a way to keep personal style edits out of shared rules (see Constraints).


Mapping Trace to Kiro's architecture

Kiro's project structure already separates context into distinct, editable files:
Trace context source
Kiro equivalent
System prompt
Always-included steering files in  .kiro/steering/  ( product.md ,  tech.md ,  structure.md ), plus  AGENTS.md 
Knowledge base
Conditionally included steering files ( fileMatch  or  auto  inclusion), and global steering in  ~/.kiro/steering/ 
Procedural skills
Manually included steering files, which run as  /<filename>  slash commands
Tool descriptions
MCP server tool definitions, configured in  .kiro/settings/mcp.json 
(Kiro-specific) Feature context
Specs in  .kiro/specs/<feature>/  (requirements, design, and tasks)
Two details matter for attribution:
  • Inclusion mode decides whether a file was in context. Kiro supports  four inclusion modes : always, fileMatch, manual, and auto. An always-included file is present in every generation; a  fileMatch  file loads only for matching files. Attribution has to know the difference.
  • Custom agents don't load steering automatically. You list steering files in the agent's resources. A missing entry is itself a context failure that Trace can surface.
Because Kiro also reads  AGENTS.md  — an open format now stewarded by the Linux Foundation's  Agentic AI Foundation  and read by many coding agents — patching AGENTS.md lets a fix carry over to other tools your team uses.
Every Trace attribution can point to a specific file and section that Kiro loaded during generation. The fix is a patch to that file: reviewable, testable, and version-controlled.


Implementation: hooks, MCP, and git

 Kiro hooks  provide the event triggers. Each hook is a JSON file in  .kiro/hooks/ , using the v1 schema introduced in Kiro IDE 1.0 and CLI V3. MCP server support provides the tool interface, and git provides correction detection. Hooks run in the Kiro IDE and CLI V3; web sessions can store hook configuration but don't run it.

The core challenge: detecting user corrections

Kiro hooks fire on agent events, not user edits.  PostFileSave  triggers when the agent saves or modifies a file;  no trigger fires  when you edit one. Correction detection therefore compares what the agent last wrote with what's in the working tree later, in two phases.
Phase 1 — Record a baseline and a snapshot. A  PreTaskExec  hook records the working-tree state before a spec task runs, so later diffs include only files the task touched. A  PostTaskExec  hook snapshots what the agent produced. Save both in  .kiro/hooks/trace-snapshot.json :
Plain text
{
  "version": "v1",
  "hooks": [
    {
      "name": "trace-baseline",
      "trigger": "PreTaskExec",
      "action": {
        "type": "command",
        "command": "node mcp-servers/trace-server/dist/cli/snapshot.js --baseline"
      },
      "timeout": 10
    },
    {
      "name": "trace-snapshot",
      "trigger": "PostTaskExec",
      "action": {
        "type": "command",
        "command": "node mcp-servers/trace-server/dist/cli/snapshot.js"
      },
      "timeout": 10
    }
  ]
}
The snapshot script stores a content hash for each file the agent touched in  .kiro/trace/.snapshots/<task-id>/ . Run the same script from  PostFileSave , and from a  PostToolUse  hook that matches the  write  tool category, so the snapshot always reflects the agent's latest output. Otherwise an agent edit made after the task finished would later look like your correction.
Phase 2 — Compare after the agent finishes. A  Stop  hook compares the working tree against the stored snapshot. Substantive differences — beyond whitespace or import reordering — are candidate corrections. Save it in  .kiro/hooks/trace-detect.json :
Plain text
{
  "version": "v1",
  "hooks": [{
    "name": "trace-detect-corrections",
    "trigger": "Stop",
    "action": {
      "type": "command",
      "command": "node mcp-servers/trace-server/dist/cli/detect.js"
    },
    "timeout": 15
  }]
}
Three practical details:
  •  Stop  fires every turn. It runs each time the agent finishes responding, not once per session, so the detector must be cheap and idempotent: it compares hashes and processes only files that changed since its last run. On Kiro CLI V3, a  SessionEnd  hook is a cleaner point for the comparison.
  • Ignore git operations. Skip files changed by merges, pulls, or rebases since the snapshot, so upstream changes aren't mistaken for your corrections.
  • Ask when it's unclear. A  Stop  command hook can ask you a confirmation question before it runs. Used sparingly ("Was that change a correction?"), this gives you an explicit label for one click and removes guesswork about refactors and new features.
Each correction event records which spec was active, which context sources were loaded, the unified diff, and an LLM-classified signal type (factual error, missing constraint, style violation, logic error, or dependency error).
Log what the agent actually loaded. Attribution depends on knowing which context sources were in context. Rather than inferring this, add a  PostToolUse  hook that matches the  read  tool category and MCP calls ( @mcp , or  @<server>/<tool>  for a single tool) and appends each file or tool the agent used to the session log. Combined with each steering file's inclusion mode, this gives attribution a record of the agent's real context — a lightweight version of the execution trace that Trace analyzes. For agents you also deploy to production,  Amazon Bedrock AgentCore Observability  provides the same kind of record as sessions, traces, and spans; locally, OpenTelemetry GenAI spans exported to a tool such as Langfuse or Arize Phoenix do the same job.

Exposing Trace as an MCP server

Rather than running Trace as an external process, expose it as a local MCP server that Kiro's agent can call natively. Register it in  .kiro/settings/mcp.json :
Plain text
{
  "mcpServers": {
    "trace": {
      "command": "node",
      "args": ["mcp-servers/trace-server/dist/index.js"],
      "autoApprove": ["trace_review_proposals"]
    }
  }
}
The server exposes four tools:
  •  trace_detect_corrections  runs correction detection on demand.
  •  trace_attribute  runs attribution on accumulated corrections.
  •  trace_recommend  generates fix proposals.
  •  trace_review_proposals  lists pending proposals with their evidence. It's read-only, so it's safe to auto-approve.
The server also exposes a  trace_weekly_review  prompt, which you invoke from Kiro chat through the  #  mention menu. For a native slash command, add a manually included steering file,  .kiro/steering/trace-weekly-review.md , that asks the agent to run the review; Kiro exposes it as  /trace-weekly-review .

The attribution algorithm

Trace's own attribution is a single holistic LLM pass over one session's trace. This implementation adds a cross-session layer, which the paper lists as future work: it waits until corrections accumulate against the same source before acting. The default threshold of 3 corrections in 7 days is a starting heuristic; tune it to your team's volume.
When the threshold is crossed:
  1. Blame scoring. For each context source, divide the number of corrections where the source was in context and the diff relates to its content by the number of generations where the source was in context. Decide "relates" with an LLM judgment or an embedding-similarity threshold, and document which you use. Compare scores within the same inclusion mode, because always-included files appear in almost every generation, which dilutes their ratio.
  1. Gap vs. stale classification. Using an LLM, compare each correction diff against the implicated source:
  • If the diff adds information not present in the source → GAP (CREATE)
  • If the diff contradicts what the source says → STALE (UPDATE)
  • If the diff adds specificity to a vague statement → UNDERSPECIFIED (UPDATE with detail)
  1. Exploratory verification. Before generating a proposal, the Recommender reads the implicated file, searches authoritative references (AWS documentation via MCP,  package.json ,  tsconfig.json ), and cross-references them to confirm the gap or staleness is real — not a one-off personal preference.
  1. Proposal generation. A  .patch  file is written to  .kiro/trace/proposed_updates/  with the text change, evidence trail, and confidence score. Keep each patch a small, itemized delta that adds, updates, or deletes specific lines; never regenerate a whole steering file. A proposal can also narrow scope, for example by moving a rule into a  fileMatch  steering file so it loads only where it applies. Proposals are never auto-applied.
Optionally, before a proposal reaches you, replay the corrections that triggered it against the patched context and check that the agent now produces the corrected output. Optimizers such as  GEPA , which ships as a DSPy optimizer, automate this kind of feedback-driven evaluation of prompts and other text.

Surfacing proposals

A  SessionStart  hook checks whether any proposals are pending and injects context into the agent:
Plain text
{
  "version": "v1",
  "hooks": [{
    "name": "trace-alert",
    "trigger": "SessionStart",
    "action": {
      "type": "agent",
      "prompt": "TRACE ALERT: Steering files have pending improvement proposals. Run trace_review_proposals to see recommended fixes."
    }
  }]
}
This means every time you open Kiro, if your steering files have accumulated enough corrections to warrant a fix, you're told about it without interrupting your flow.


What this looks like in practice

Week 1: You use Kiro normally. It generates CDK code. Three times this week, you add  fifo: true  to SQS queue configurations that Kiro missed. Each correction is logged automatically.
Week 2: The attribution engine fires. It identifies  aws-conventions.md  as the source with the highest blame score for SQS-related corrections. It reads the file, confirms there's no FIFO guidance, searches AWS CDK documentation via MCP, and generates a proposal:
Plain text
Operation: CREATE (new section)
File: .kiro/steering/aws-conventions.md
Confidence: 0.91
Evidence: 3 corrections in 5 days, all adding fifo: true

Proposed addition:
## SQS Queue Configuration
- When a queue name ends with `.fifo`, set `fifo: true` and
  `contentBasedDeduplication: true` on the construct.
- FIFO queues require FIFO dead-letter queues. Match the DLQ type.
- Standard-to-FIFO fan-out requires an SNS FIFO topic.
You review the proposal in Kiro chat, approve it, and the steering file is updated. From that point on, Kiro consistently produces correct FIFO configurations. Because the rule only matters for infrastructure code, an equally good proposal would put it in a  fileMatch  steering file scoped to your CDK stacks.


Constraints and limitations

Where hooks run. The design needs the Kiro IDE or CLI V3. Web sessions don't run hooks.
Latency. Correction detection is asynchronous: it runs after the agent finishes, not while you type. If you need immediate feedback, invoke the  trace_detect_corrections  tool manually.
Volume. The attribution engine uses LLM calls to classify signal types and compare diffs against source content. For teams generating hundreds of corrections per week, batch processing with a scheduled job is more practical than real-time attribution. Batching also helps performance: model providers cache unchanged prompt prefixes, and every steering-file edit invalidates that cache, so applying approved patches weekly rather than one at a time keeps your context stable. For team-wide correction history, a shared store such as Amazon Bedrock AgentCore Memory (whose episodic strategy records goals, reasoning steps, actions, outcomes, and reflections) can replace local  .kiro/trace/  files.
False positives. Not every user edit is a correction. Refactoring, feature additions, and personal style preferences can be misclassified. The threshold, the optional "was this a correction?" prompt, and exploratory verification filter most of this noise. Keep personal style preferences out of shared steering files: store them per user, in the spirit of CIPHER, or in global steering under  ~/.kiro/steering/ .
Steering-file drift. Repeated patching can bloat steering files, and repeated LLM rewriting can erode them (ACE's "context collapse"). Keep patches small and itemized, run a periodic merge pass to remove duplicates and contradictions, set a size budget for each steering file, and use delete operations as readily as additions.
Evidence base. Trace's published numbers come from 60 synthetic dissatisfaction traces, not production deployments. Track your own proposal acceptance rate and repeat-correction rate to see how well it works on your codebase.
Scope. This implementation targets steering files and specs. Attributing failures to MCP server tool implementations (code bugs rather than context gaps) requires deeper integration with test results and is left as future work.


Getting started

There's no package to install: you build the pieces described in this post, and Kiro can do much of the scaffolding for you.
Start from a spec. Kiro's  spec workflow  turns a feature description into three files:  requirements.md ,  design.md , and  tasks.md . Create a feature spec (for example, in  .kiro/specs/trace-integration/ ) and give Kiro the "Implementation" and "The attribution algorithm" sections of this post as the description. Review the generated requirements and design before you run any tasks. That's where you decide your threshold, how "relates" is judged, and which files Trace may propose changes to.
Build in stages. Each stage is useful on its own, so run the tasks one stage at a time and check the output before moving on:
  1. Detection. Write the snapshot and detection scripts and add the  PreTaskExec ,  PostTaskExec ,  PostFileSave , and  Stop  hooks. Use Kiro normally for a week, then read the correction log. If it's full of refactors and new features rather than real corrections, tune the filtering before going further.
  1. Context logging. Add the  PostToolUse  hook that records which steering files and MCP tools the agent actually used.
  1. The MCP server. Start with  trace_detect_corrections  and  trace_review_proposals , which need no LLM calls.
  1. Attribution and proposals. Add  trace_attribute  and  trace_recommend , which call the LLM endpoint, and the  SessionStart  alert.
  1. Optional extras. Add the  Stop  confirmation prompt and replaying corrections against a proposed patch.
Use this layout. The paths in the hook and MCP examples assume this structure, with the TypeScript sources compiled to  dist/ :
Plain text
your-repo/
├── .kiro/
│   ├── hooks/              # trace-snapshot.json, trace-detect.json, trace-alert.json
│   ├── settings/mcp.json   # registers the Trace MCP server
│   ├── steering/           # the files Trace proposes changes to
│   └── trace/              # .snapshots/, the correction log, proposed_updates/
└── mcp-servers/trace-server/
    └── src/
        ├── index.ts        # MCP server and tool definitions
        └── cli/            # snapshot.ts and detect.ts, called by the hooks
You can also create the hooks by describing them to Kiro in chat instead of writing the JSON by hand. On Kiro CLI V3, hooks in  ~/.kiro/hooks/  run in every workspace; if you install Trace's hooks there, point their commands at an absolute path to the server rather than the repo-relative paths shown above.
Key dependencies (test before upgrading an existing project):
  •  @modelcontextprotocol/sdk  ^1.32 — the v1 maintenance line. The current stable  v2 SDK  ships as separate  @modelcontextprotocol/server  and  @modelcontextprotocol/client  packages.
  •  zod  ^3.25 or 4.x — the MCP SDK requires Zod 3.25 or later.
  •  simple-git  ^4.0 — check the v4 release notes if you're upgrading from 3.x.
  • An LLM endpoint for classification: a current Claude Sonnet model on Amazon Bedrock (for example, Claude Sonnet 5.5).
The hooks are small command scripts, and attribution and recommendation run only when thresholds are crossed. Proposals always require your approval before any context source changes.
Useful resources:


Conclusion

On a benchmark of 60 synthetic failure traces, Trace chose the right fix operation and the right file to change 82% of the time. The key insight is that user corrections are free diagnostic data: they already exist in your git history, waiting to be used.
Kiro's architecture — with its clean separation of steering files, specs, hooks, and MCP servers — maps directly onto Trace's attribution framework. Every correction points to a file. Every file can be patched. Every patch can be reviewed.
The result is a development environment where your context improves every time you use it, where the same mistake doesn't happen twice, and where the gap between "what Kiro knows" and "what you need Kiro to know" closes over time.
Trace deals with context that goes stale across sessions. Context can also be lost within a session, when one agent hands work to another. The companion post,  Why AI coding agents forget: memory, handoffs, and latent context , looks at what production systems and recent research do about that.

References

  • Zhao, Y., Misra, P. K., & Pandey, S. (2026). TRACE: TRajectory Attribution for Automated Context Engineering. KDD '26, August 9–13, 2026, Jeju, South Korea.  arXiv:2608.09153  ·  Amazon Science 
  • Wang, Y. et al. (2025). DRIFT: Learning from Abundant User Dissatisfaction in Real-World Preference Learning.  arXiv:2510.02341 
  • Zhang, Q. et al. (2026). Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. ICLR 2026.  arXiv:2510.04618 
  • Gao, G. et al. (2024). Aligning LLM Agents by Learning Latent Preference from User Edits (PRELUDE and CIPHER).  arXiv:2404.15269 
  • Zhang et al. (2025). Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems (Who&When benchmark). ICML 2025.  arXiv:2505.00212 
  • Agrawal et al. (2025). GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning.  arXiv:2507.19457 
# kiro# amazon-bedrock# ai-agents# developer-tools# mcp
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article