Agent Harness: Where Engineers Add Value When Models Keep Getting Smarter
As AI models keep getting smarter, the real engineering leverage is moving to the scaffolding around them and two directions worth committing to: outward toward customers or inward toward builders.
Based on my lightning talk at AWS Summit Hong Kong 2026.
If you spent 2025 building AI agents, you can feel the mood shift.
Last year agents broke out — everywhere, every industry, a shiny new demo every week. The question was "can we build an agent that does X?" and the answer kept coming back yes.
This year the question is harder. Anyone can spin up an agent now. What separates a weekend demo from something a company actually runs comes down to one thing:
How do you get an agent into production?
That's what I want to talk about. And the answer finally has a name: the agent harness. As Aakash Gupta put it , "2025 was agents; 2026 is agent harnesses."
The formula: Agent = Model + Harness
Plain text
Agent = Model + Harness
If you're not the model, you're the harness. Tools, memory, prompts, orchestration, observability, guardrails, routing, deployment — all harness. Engineering it on purpose (and turning every mistake into a permanent fix , the way Mitchell Hashimoto describes) is what harness engineering means.
A car analogy makes it click for me.
The engine generates power. The harness turns it into a useful, safe vehicle.
Think of your agent as a product you ship — a car. The model is the engine: it gives the car power, the ability to reason, plan, actually drive. But you don't hand someone a bare engine sitting on the ground. A real vehicle also needs a chassis, steering, brakes, a dashboard, seatbelts, lights.
The engine makes power. The harness turns it into a car you'd put your family in. A great model no more guarantees a great agent than a great engine guarantees a great car.
We built harnesses for years without a word for them — writing the prompts, gluing the tools, patching the failures. Now there's a name, and a name lets you be deliberate about it.
A harness has two halves
When I look at what a harness actually does, it splits cleanly in two.
Development extends what the model can do: memory across sessions, tools and MCP, retrieval and context, system prompts, planning, orchestration.
Operations makes it work reliably: observability, evaluation, guardrails, routing, monitoring for drift and cost, deployment, scaling. This half feels a lot like classic cloud ops — because an agent is a product, and products need that care. Martin Fowler's team calls it harness engineering.
Tag every capability "model" or "harness" and the split is lopsided: a handful of rows are the model; nearly everything else is the harness. That asymmetry is the whole story.
The harness matters more than we think
The pushback I hear most: "Sure, the harness helps — but if I just wait for a smarter model, most of this work goes away." So here's the evidence, all cases where the model stayed fixed and only the harness changed.
In each case the model was held constant. Only the harness changed.
Vercel's v0 — less is more. A big context window only helps if you manage it; stuff everything in and it rots — stale context, lower accuracy, slower answers. The v0 team went the other way and cut tools from 15 down to 2 (basically bash + filesystem), a lesson the Manus team hit independently. Same model, better numbers: accuracy 80% → 100%, token cost down ~37%, responses 3.5× faster. Removing capability made it better.
LangChain on Terminal Bench 2.0 — same model, better plumbing. They added a self-verify loop, a pre-completion checklist, and loop detection . Score went 52.8% → 66.5%, roughly top 30 → top 5 — a bigger jump than most model upgrades hand you.
And it goes further: a smaller model in a better harness can beat a bigger one in a worse harness.
- Hebia ran GPT-5.4 mini in a harness-first setup for finance and legal documents — higher pass rates and better source attribution than the flagship, at lower cost.
- MIT CSAIL's Recursive Language Models wrapped GPT-5-mini so it calls itself on chunks of context. No fine-tuning — the harness is the architecture. It scored 64.9% vs 30.3% against a much larger model.
Models are commoditizing; the harness is the moat. And you don't have to wait for your vendor's next release to ship a better agent — harness work keeps you a step ahead of the curve.
"But won't the model absorb all of this?"
Fair question. If I pour months into the harness and the next model is smarter, does my work just evaporate?
Partly — yes, and pretending otherwise would be dishonest. Context management is the clearest case: we used to babysit the window, trimming and summarizing and resetting, and longer, smarter windows retired most of that. Planning is another — Anthropic reports a newer model "ran coherently for over two hours" without the sprint decomposition an older one needed, and Addy Osmani says the upgrade "largely killed the context-anxiety failure mode." Self-correction moved into the model column too.
I hit a fourth case myself. Last month I was building a guardrail to block unsafe input and output. To test it, I asked Claude to generate some toxic samples — and it refused. The model had already absorbed part of the job my guardrail was built for. Useful, and a little humbling.
So the objection isn't wrong — it's incomplete. The harness doesn't die. It moves up the stack.
The model absorbs low-level jobs; engineering value migrates to the operations layer.
Most of what looks like "the harness shrinking" is really one crude pattern dying so a better one can take its place. And an entire half — Operations — sits where the model simply can't reach.
Take routing. A serious shop runs many models — Claude here, GPT there, a cheap local one for easy tasks — behind a single endpoint that sends each request to the right backend. A model can't do that. A model is a wonderful model; it is not a router. That's infrastructure, and it lives in the harness — today and after the next three releases.
A model is a wonderful model — but it is not a router.
Same for observability, evaluation, monitoring, guardrail enforcement, deployment, scaling. The model has no role in tracing your production runs or paging you when cost drifts. That work is yours.
Here's how I think about what shrinks and what grows:
Component | Shrinks (absorbed / obsoleted) | Grows (your leverage) |
|---|---|---|
Memory | Per-session scratchpads | Multi-session persistence, cross-agent shared memory, knowledge graphs |
Tools / MCP | Hand-narrowed tool wrappers | Atomic primitives, MCP ecosystems, just-in-time tool assembly |
Planning | Sprint decomposition, context-anxiety scaffolding | Multi-day planning, long-horizon decomposition, self-revising plans |
Context | Prompt-stuffing, manual RAG tuning | Context-quality strategy, progressive disclosure, recursive contexts |
Prompts | Monolithic prompt blobs | Rules-as-code, per-failure AGENTS.md entries, hierarchical rule files |
Observability | Print-statement debugging | Structured trace pipelines, trace mining, semantic search over runs |
Evaluation | One-off prompt tests | Outer eval loops, domain-specific evals, LLM-as-judge frameworks |
Guardrails | Soft prompts ("please don't…") | Deterministic hooks, permission gates, CI-layer enforcement |
Routing | Single-model deployments | Model routing, cost/quality tradeoffs, big-brain + mini-worker splits |
Deployment | Single-process agents | Parallel agent fleets, sandboxes, crash recovery, lifecycle management |
Read it top to bottom and nothing on the right gets easier because the model got smarter. If anything, a smarter model unlocks harder problems that need more harness.
So where should you go next?
If the harness matters — and keeps mattering — the question turns personal: what do you actually do about it? I see two directions. Both win. The trap is doing both halfway.
Direction A — Outward, toward customers
Use AI as leverage to build products that solve real problems — faster, cheaper, more personal than was possible before. You win by understanding the user. A good harness plus a frontier model gives a tiny team what used to take 50 engineers. I keep seeing it: students and designers, people who "can't code," shipping genuinely great products with an agent doing the heavy lifting. As someone who came up through engineering and can't design my way out of a paper bag, I find that thrilling.
Direction B — Inward, toward builders
Build the harness, tools, and infrastructure that make every other builder faster. You win by understanding the workflow. The nice part: you ride two curves at once. Models improve fast, but the harness improves faster — and it's the part you own. Hooks, middleware, eval loops, trace miners, routers: they compound across every product and outlive any single model. The model you build on is obsolete in a year; a good eval loop isn't.
Closing thought
Both directions win. Pick one and commit. Most teams stall trying to build a great product and great platform at the same time, and do neither. Vercel, LangChain, Hebia, MIT — they all point the same way: as models commoditize, the harness is where durable engineering value collects. So don't wait for the next model to save you.
The engine keeps getting better. The car is still yours to build.
P.S. — Just as I wrapped this up, AWS shipped Amazon Bedrock AgentCore Harness — a managed harness for the production plumbing, landing squarely on the Operations half the model can't absorb. Haven't tried it yet, but I'm curious to see how far it takes us ;-)
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article