From Free AI Agent to Governed SRE — Building Harness Engineering with Kiro CLI
How to turn an AI coding agent into a disciplined SRE operating system using execution ladders, safety gate hooks, trace audit logs, and skills-based knowledge management — with a real-world security red team case study.
An SRE shares how to turn an AI coding agent from "a chatbot that runs commands" into a disciplined operating system: skills, safety gates, trace audit, and red teaming.
Introduction: AI Agents Don't Lack Intelligence — They Lack Discipline
The first time I used an AI agent for SRE work, I was thrilled. Ask it to investigate an incident — it greps logs, checks pods, suggests fixes. But after a few weeks, I noticed a pattern: the agent works fast, but not consistently. One day it checks monitoring before fixing, the next it jumps straight to
kubectl delete without asking. One day it remembers to use the correct AWS profile, next week it forgets.The problem isn't that the model lacks capability. The problem is nothing forces it to follow the right process every time.
This is when I started thinking about Harness Engineering — not "add more agents for speed", but enforce discipline into AI workflows, the same way CI/CD enforces discipline into deployments.
Part 1: Skills — Teach the Agent What You Know
The Problem
Every incident that happens, I investigate, resolve, then... forget. Next time something similar occurs, I start from scratch. The agent is the same — it doesn't "remember" across sessions.
The Solution
I created Skills — markdown files containing structured knowledge, one file per domain:
1
2
3
4
5
6
7
8
~/.kiro/skills/
├── incident-response/SKILL.md # Runbook, severity, rollback
├── ci-templates/SKILL.md # CI/CD patterns + common issues
├── slo-sli-error-budget/SKILL.md # Error budget, deploy freeze
├── observability-standards/SKILL.md # RED/USE, naming, dashboards
├── security-and-hardening/SKILL.md # OWASP Top 10, auth patterns
├── deployment-safety/SKILL.md # Freeze windows, pre-deploy checklist
└── ... (22 skills total)Each skill has frontmatter so the agent knows when to load it:
1
2
3
4
name: incident-response
description: Incident response runbook. Use when investigating or resolving production incidents.
The agent only loads metadata at initialization (lightweight). Full content loads on-demand when the task matches the description. 22 skills but no context wasted on irrelevant ones.
Accumulation Workflow
1
Incident happens → Resolve → Ask "update skill?" → Record knowledgeAfter 2 months, skills become the team's long-term memory — not one person's, but the entire system's.
Part 2: Agent — From Chatbot to SRE Specialist
Kiro CLI allows creating custom agents — each agent is a "persona" with its own prompt, tools, skills, and hooks.
1
2
3
4
5
6
7
8
{
"name": "sre",
"prompt": "You are an SRE expert...",
"tools": ["fs_read", "fs_write", "execute_bash", "use_aws", "@atlassian"],
"resources": ["skill://~/.kiro/skills/**/SKILL.md"],
"hooks": { ... },
"keyboardShortcut": "ctrl+shift+s"
}Ctrl+Shift+S — enter SRE mode. Ctrl+Shift+H — enter Harness mode (stricter). Toggle between them with shortcuts.The SRE agent reads Jira tickets directly (Atlassian MCP), queries AWS, runs kubectl, reads source code — all in one session, with context from 22 skills.
Part 3: Harness — Enforce Discipline, Don't Replace Engineers
This is the most important part. Skills give the agent "know-how", but nothing guarantees it will use them correctly every time. Harness solves this with 3 layers:
L1: Execution Ladder (Prompt-enforced)
Every change task must go through 7 steps:
1
SCOPE → VERIFY → PLAN → CONFIRM → EXECUTE → VERIFY → CAPTURE- SCOPE: State what changes and what does NOT change
- CONFIRM: Destructive tasks must NEVER skip this step
- CAPTURE: End of task, ask "update skill?"
The ladder isn't code — it's a prompt instruction. But with a good enough model, it follows very consistently.
L2: Safety Gate (Hook — deterministic)
This is the hard part. A bash script blocks dangerous commands before execution:
1
2
3
4
5
6
7
8
9
# ~/.kiro/hooks/block-destructive.sh
EVENT=$(cat)
CMD=$(python3 -c "import json,sys; print(json.load(sys.stdin).get('tool_input',{}).get('command',''))" <<< "$EVENT")
if echo "$CMD" | grep -qE "rm -rf|push --force|terraform destroy|kubectl delete namespace"; then
echo "BLOCKED: $CMD" >&2
exit 2 # Exit 2 = block tool execution
fi
exit 0Agent wants to run
terraform destroy? Hook blocks it, returns error, forces agent to ask the user. Not "AI promises to be careful" — but code that doesn't allow it.L3: Trace Audit (Hook — postToolUse)
Every tool call gets logged:
1
2
3
4
5
# ~/.kiro/hooks/trace-log.sh
EVENT=$(cat)
mkdir -p ~/.kiro/traces/$(date +%Y-%m-%d)
echo "$EVENT" >> ~/.kiro/traces/$(date +%Y-%m-%d)/$(date +%H%M%S)-tool.jsonlAfter a session, I have a full audit trail: which files the agent read, which commands it ran, what it modified. Don't trust "AI says it did X" — check the log.
Result
| Free Agent | Governed Agent (Harness) | |
|---|---|---|
| Workflow | Ad-hoc, different each time | Fixed 7-step ladder |
| Destructive ops | Prompt instruction (can be ignored) | Hook blocks hard |
| Audit | None | Every tool call traced |
| Auto-merge | Possible | Explicitly banned |
Part 4: Red Team — Using the Agent to Pentest Internal Tools
Another powerful use case for agent + skills: security assessment.
I had the SRE agent scan an internal workflow portal — result: 28 vulnerabilities, including 6 Critical.
Methodology
1
2
3
4
1. Reconnaissance: Swagger exposed → 137 endpoints mapped
2. Auth testing: webhook endpoints, no signature → bypass confirmed
3. Source code review: delegated to subagent analyzing each module
4. Exploit verification: curl directly against productionKey Findings
| # | Issue | Severity |
|---|---|---|
| 1 | Webhook endpoint — no auth required | 🔴 Critical |
| 2 | Webhook — no signature verification | 🔴 Critical |
| 3 | CLI runner with --trust-all-tools + full process.env | 🔴 Critical (RCE) |
| 4 | Templates exposed without auth (system prompts + emails) | 🟠 High |
Worst case attack chain:
1
2
Forge webhook (unauthenticated) → Trigger workflow → CLI agent spawns
→ Attacker-controlled prompt → Full server compromiseThe entire assessment ran in one session: agent reads source, finds vulns, suggests exploits, verifies on production. Skills
security-and-hardening + incident-response gave the agent context to assess severity correctly.Part 5: Lessons Learned
1. Skills > Memory
AI agents don't have long-term memory. But skill files on disk = durable memory. Write once, reference forever.
2. Hooks > Promises
"AI will be careful" is a promise.
exit 2 in a preToolUse hook is a guarantee. Use code where you need hard rules, use the model where you need judgment.3. Trace > Trust
Don't know what the agent did during an hour of unattended work? Trace log tells you. Especially important when debugging "why did the service restart at 3 AM".
4. Do It 3 Times Before You Automate
See a task repeat once → want to write a script immediately. Wrong. Do it at least 3 times to know which part is a real pattern and which is coincidence. Automate too early = automate the wrong thing.
5. Invariants Start Empty
Don't borrow rules from another project. Each project starts with humility. Rules only get added after paying the price with a real bug or a clear architecture decision record.
Part 6: Full Stack
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
~/.kiro/
├── agents/
│ ├── sre.json # Free agent (quick tasks)
│ └── sre-harness.json # Governed agent (changes, incidents)
├── hooks/
│ ├── block-destructive.sh # L2: Hard gate
│ └── trace-log.sh # L3: Audit trail
├── skills/ # 22 skills (on-demand loaded)
│ ├── incident-response/
│ ├── slo-sli-error-budget/
│ ├── security-and-hardening/
│ ├── sre-harness/ # Meta-skill: harness conventions
│ └── ...
└── traces/ # Auto-generated audit logs
└── YYYY-MM-DD/*.jsonlMaturity ladder:
1
2
3
4
5
6
L0: Free agent (no discipline)
L1: Ladder in prompt (workflow steps) ← applied
L2: Hard gate (hooks block destructive) ← applied
L3: Trace audit (hooks log tool calls) ← applied
L4: Per-project invariants (future)
L5: Walk-away mode (future)Conclusion
Harness Engineering isn't about "adding more AI agents". It's about designing a system that forces you (and AI) to follow the right process — even when you're in a rush, it's 3 AM, and you want to skip steps.
The value isn't in which model you use. The value is in the ladder you force every execution to climb. Models change, editors change, stacks change — but discipline is portable.
Are you using AI agents for SRE/DevOps? Ask yourself: in the ladder from idea to merge, which rung do you skip most often? That's exactly the rung that needs a harness.
Tools: Kiro CLI, MCP (Model Context Protocol), AI Gateway
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article