Harness Engineering: building an ops agent with AgentCore & Strands

One approach that has grown fast in popularity is agents combined with something called harness engineering.

So what is harness engineering, really? The goal for us here is to learn the basics of yet another concept in AI but also build a proof of concept for correctness and audit trace.

A language model on its own is like a bare CPU lying on a desk. Has a lot of computational power, but by itself, unable to do a single useful thing for you. Elements of AI makes the same point about computers in general: a processor only becomes a computer inside a system. Memory that holds what it is working on. Input and output that connect it to the world. An operating system that decides what it is allowed to do.

The harness is that system, built around the model. Harness engineering is the discipline of designing it - so that raw capability becomes something you can trust in your daily work.

When you use an agentic tool like Kiro, most of what impresses me is not the model. It is the harness. Concretely, a harness is made up of five things:

Tools. The agent's connection to reality: the ability to run a command, query a database, call an API, search a metadata catalog. An AI model without tools can only tell you what it remembers from training. A model with tools can go and look. This single difference is the largest hallucination reducer that exists - most hallucinations are not lies, they are answers given from assumptions, the closest neighbor, bayes thereom, or any other algorithm the AI was built on.

Context management. What the agent gets to see, and when. Instruction files, skills, retrieved documents, the schema of the system it is working in. A well-engineered harness feeds the agent current, authoritative context instead of letting it lean on stale training data. This is where files like AGENTS.md and reusable skills are defined.

Guardrails. The rules the agent cannot break regardless of what it generates: which directories it may write to, which actions require approval, which claims require evidence. Guardrails are enforced by the harness, not requested politely in a prompt. There is a world of difference between "please do not delete files" and an execution environment where the delete call does not exist.

Output contracts. Schemas that the agent's answers must conform to. When an agent must return a defined structure, it can be a certain schema, types or format. Downstream systems can consume the output safely, and you can compare answers mechanically.

Observability. A record of what the agent actually did: which tools it called, in what order, with what results. This trajectory is the harness's audit log, and it is the key to the trust question. It is a way to understand how agents came to choices.

Imagine you're working for a company where someone asks your AI tool "where does our revenue data from October 2025 live and what does it tell us?" - with no harness the answer will confidently say that a table exists in a completely fabricated place. If you apply harness to a model with a lookup tool with guardrails the answers will be tied to a certain warehouse or finding nothing and output says so.

As a platform engineer in a big company a frequent question could be "why did our Bedrock bill spike?" "what can we save most on in terms of infrastructure?"

We are building a harnessed coding agent to build a harnessed ops agent that focuses on exactly this. The answers our engine will produce are audit logs with references to AWS API calls.

Quick check by running node -v && terraform -v && docker ps && aws sts get-caller-identity

The chain stops at the first missing piece, which tells you exactly what to install. On Arch Linux (my machine), the gaps were filled with:

For macOS: brew tap hashicorp/tap && brew install hashicorp/tap/terraform

For AWS login

When the command aws sts get-caller-identity returns your account JSON, you're ready. Create the project:

Amazon Bedrock AgentCore gives us the platform for deployment. It is a container based runtime. AgentCore has all ingredients for enterprise-ready harness engineering (Identity, policies, storage, observability, evaluations).

Strands Agents is AWS's open-source, model-agnostic agent framework. When in early release Python was the main choice. We will use Typescript. Why? Mainly because we want to do structured output with Zod schemas with validation retry on fail. The evidence contract is therefore enforced by the SDK. Fun fact: the TypeScript SDK recently moved into a monorepo literally named harness-sdk.

This will be the agent's grounded answer instead of free text. The answer can't be verified unless we have evidence reference.

It is a TypeScript-first schema validation with static type inference. With Zod you can define schemas you can use to validate data, from a simple string to a complex nested object.

Every time the code runs verified responses must point at evidence that actually exists in the output.

If answer.claims is marked as verified, it must reference real evidence that exists in answer.evidence.

The enterprise data intelligence need to be stored somewhere where the agent can correctly query. What we want to store is a model where the agent can reason over. I considered a few storage option. But came to S3, much because of versioning. Every read gives a VersionId, so an audit log is created that lets us trace every claim back to an exact version and time

The EDM has four entity types:

The full implementation of the IAM you will see in the deploy section. In an enterprise the JSON would probably have its source from a CMDB or Backstage. We're going to skip the pipeline CI/CD so you can deploy the S3 (assuming you have added the above code in an infra directory) with:

Next we are creating the edmLookup method. This is the agents connection to reality:

So in order for the agent to be able to use the context we just created through the edmLookup we need the method costLookup. The costLookup enables us to fetch data from Cost Explorer with daily or monthly granularity. It is also able to check for services. This method is deterministic.

This function is unlike deterministic. It has an interface with a schema, with an API call JSON is returned. There are no algorithms with nearest neighbour, vectors, bayesian learning. In this way we can run unit tests on it. The agent is what loops around the model that decides when to call which tool, with what parameters, how many times and it now has the contract-bound answer with our two tools. This is essentially harness design, tools stay stupid and we can audit them. Agent performs the reasoning and the contract is the expert at keeping the history of minimum info of if it is validated or not and when. This means we also can keep track if a question has different answers the same day. Remember we wanted to measure correctness and accuracy.

With structuredOutputSchema Strands validates the response against the contract and automatically retries with the validation errors as feedback. If it still doesn't validate, a StructuredOutputError is thrown, that is a harness catch instead of an assumption.

So to answer the question "Why did our Bedrock bill spike last month?":

The agent calls costLookup with variable monthly, groups it by service to check for the spike, also checks for daily Bedrock-filtered cost to dates, uses catalog to attribute it and owners for seeing department of spend.

The agent pulls monthly spend grouped by service to confirm the spike, checks daily Bedrock-filtered costs to date it, then switches to the catalog to attribute it, which services and owners that spend belongs to. Cost claims references GetCostAndUsage, ownership references the EDM with a version. Great, now we have accuracy and audit trail.

If someone in Slack asks the question of "Who owns quantum-db?", the code runs the lookup, returns nothing the contract fulfillment response is not-found, that is "I don't know".

Mine has domain, teams, services and datastores. But what would your job role and colleagues care about? Tell me in the comments!

Find the complete code for this Proof of Concept in my GitHub: caroline-cah.

Thanks for reading, would love your feedback<3