From Spec to Rewrite: Can an Agent Rebuild a Monolith from Its Documentation?
A follow-up experiment — I took the business-rules-extraction spec, handed it plus the original test suite to Kiro, and asked it to recreate a domain from scratch. Here's what passed, what broke, and what it teaches you about spec-driven rewrites.
My last post, From Monolith to Spec: Auto-Extracting Business Rules with AWS Transform custom , walked through AWS Transform's
business-rules-extraction — pointing an agent at 160,000 lines of legacy Java and getting back the documentation the codebase never had: domains, business rules, a reconstructed schema, and a decomposition plan.Then a reader left a comment that was really a dare:
"Have you taken the next step — taken your specs and the existing test suite to see if [an agent] could recreate it? Spec + full test suite + good agent → rewrite your code. I think this is the logical extension of what's possible."
They're right that it's the logical extension. The spec describes what the system does; the test suite describes how you'd know if you broke it. Put those together with a capable coding agent, and in theory you should be able to regenerate the implementation and prove it works.
So I ran it. Not as a thought experiment — as a real, measured one. This post is the result.
The setup
I kept the experiment small, honest, and reproducible.
| Codebase | OpenMRS Core — the same open-source EMR monolith from the first post |
| Target | The cohort domain (its smallest: 2 entities — Cohort and CohortMembership) |
| Spec | The business-rules-extraction blueprint I generated last time |
| Rewrite agent | Kiro |
| Language | Kept the same (Java), so the original JUnit tests run unmodified as the yardstick |
| Toolchain | Real compile + test on Amazon Corretto 21 and Maven — no mocks, actual pass/fail |
One rule, or the whole thing is meaningless
Kiro was firewalled from the original source. It was never allowed to read the original
Cohort.java or CohortMembership.java. Its only inputs were the spec and the tests. The original code's only job was to compile the tests that serve as the scorecard.If the agent can peek at the answer, you're measuring copy-paste, not whether the spec is good enough to rebuild from. The firewall is what makes the result mean something.
The yardstick
The measure of success is simple and brutal: do the original tests pass against the regenerated code?
OpenMRS ships a real test suite for cohort — 15 + 21 pure unit tests, plus 44 + 7 + 2 integration tests that boot Spring and hit an in-memory database. I confirmed the baseline first: all green on the original code. Now the question is whether Kiro can produce code that's also green — working only from the documentation.
The first attempt: the obvious starting point
business-rules-extraction writes one folder per domain, so my first instinct was the obvious one: point Kiro at the cohort domain folder — overview, features, business-rules, validations, workflows, integrations, and the diagrams — plus the domain's test files. That's the "spec for this domain," right there in one place. It's exactly what most people would grab first.Kiro read the tests as a contract and the spec as intent, and wrote two fresh Java classes. The results:
| Test class | Tests | Result |
|---|---|---|
CohortTest | 15 | ✅ |
CohortMembershipTest | 21 | ✅ |
| Unit total | 36 | ✅ 36 / 36 |
CohortServiceTest | 44 | ❌ 44 errors |
CohortValidatorTest | 7 | ❌ 7 errors |
The good news is genuinely impressive. Kiro reconstructed behavior the spec never explicitly described — set operations (
union / intersect / subtract), seven different constructors including comma-separated-ID parsing, "active membership as of a given date" logic, and the full compareTo / equals / hashCode contracts — all correctly inferred from how the tests exercised them. The 36 unit tests passed in the real Maven build.Then the integration tests all failed with the same error:
1
org.dbunit.dataset.NoSuchTableException: cohortThe
cohort table didn't exist. Kiro's entity had zero ORM mapping — no @Entity, @Table, @Column, @Id. Hibernate had nothing to create a table from, so the moment a test tried to persist a cohort, it blew up.And here's the important part: this was my mistake, not Kiro's. The domain folder describes what a cohort does, not how it's stored — the physical schema lives in a different part of the blueprint that I hadn't handed over. The unit tests, being pure POJO tests, never persist anything, so nothing in Kiro's inputs so much as hinted that a database mapping was required. The agent produced exactly what its inputs implied — the inputs were just incomplete.
The lesson: grabbing the per-domain folder feels like grabbing "the spec," but it's only half of it. A behavioral spec plus unit tests gets you correct logic — and a class that can't talk to the database. Worse, the 36 green unit tests made it look finished.
The fix: the full blueprint + the full test suite
The fix follows directly from the diagnosis. The persistence contract wasn't missing from the blueprint — it was missing from the inputs I gave Kiro.
business-rules-extraction had produced a complete schema file all along, database-documentation/00-global-schema.sql, with the real table definitions:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
CREATE TABLE cohort (
cohort_id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
name VARCHAR(255) NOT NULL,
description VARCHAR(1000),
...
uuid CHAR(38) NOT NULL UNIQUE
);
CREATE TABLE cohort_member (
cohort_member_id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
cohort_id INT NOT NULL,
patient_id INT NOT NULL,
start_date DATETIME DEFAULT NULL,
end_date DATETIME DEFAULT NULL,
...
);So this time I gave Kiro the entire blueprint — spec and schema — and the full cohort test suite, not just the two unit files. Still firewalled from the original source.
Iteration 1. Kiro produced fully ORM-mapped entities this time —
@Entity, @Table(name="cohort"), @Column mappings straight from the schema, and it correctly mapped the CohortMembership class to the differently-named cohort_member table. Unit tests: still 36 / 36. But now the integration tests wouldn't even compile — CohortServiceTest called two method overloads (getActiveMemberships() with no argument, and getMemberships(boolean)) that weren't present in the handful of test files Kiro had seen the first time.Iteration 2. I gave Kiro the remaining test files as contract. It added the two missing overloads — purely additive, ORM mappings untouched.
This time everything ran:
| Test class | Tests | Type | Result |
|---|---|---|---|
CohortTest | 15 | unit | ✅ |
CohortMembershipTest | 21 | unit | ✅ |
CohortServiceTest | 44 | Spring + DB | ✅ |
CohortValidatorTest | 7 | Spring + DB | ✅ |
PatientDataVoidHandlerTest | 2 | Spring + DB | ✅ |
| Total | 89 | ✅ 89 / 89 |
The
cohort domain's entities were recreated from the documentation alone — never the original source — and passed the entire suite, unit and integration and persistence. The regenerated files even came out close to the originals in size (405 vs 419 lines; 264 vs 223).
What this actually teaches you
It works. But the two failures along the way are more instructive than the final green checkmark, because each one is a trap you'd hit on a real project.
1. "Spec" has to mean the whole blueprint, not just the prose. Behavior came from the behavioral spec and the tests. The persistence contract came from the schema document. These are different artifacts describing different layers, and the rewrite needed both. Hand an agent only the "what it does" docs and you get code that can't be stored.
2. The test suite is the yardstick — and its coverage is the hard ceiling. This bit twice. First, unit-only tests hid a missing database mapping until an integration test tried to run. Then a partial set of test files hid missing methods until the compiler complained. A behavior that no test exercises is a behavior nothing will verify — and a green-but-incomplete suite gives false confidence, which is more dangerous than an obviously red one.
3. It's a loop, not a one-shot. Nobody types a perfect prompt and gets a finished rewrite. It took two cheap, mechanical iterations — add the schema → get the mappings; add the full tests → get the missing methods — to go from 36/89 to 89/89. That's exactly how agent-driven rewrites actually run: generate, compile, test, read the failure, fix, repeat. Kiro is good at this loop precisely because each failure is a concrete, machine-checkable signal.
The catch nobody can skip: you need the tests
Here's the uncomfortable part. This entire approach rests on inheriting a real, comprehensive test suite — including integration and persistence tests. OpenMRS has one, which is the only reason we caught the mapping bug at all. Most legacy code is the exact opposite: thin unit coverage, no integration tests, and the behavior that matters most living in exactly the untested corners.
If your codebase is like that, the rewrite isn't the hard part — building the yardstick is. Before you can safely regenerate anything, you have to pin down what the current system actually does, by writing characterization tests: record the existing system's real behavior (its API responses, its database state after an operation) and freeze those recordings as the contract the rewrite must match. That's a whole project in itself, and it's the honest cost of a spec-driven rewrite on untested code.
Cross-language rewrites: move the tests to the boundary
Our experiment kept the language the same (Java → Java), so the JUnit tests ran unchanged. A true port — Java → Python, say — loses that: JUnit tests can't validate Python code, so the yardstick itself has to move. That sounds like it reintroduces the whole fidelity problem, but there's a clean way through: stop testing in the language, and start testing at the boundary.
Instead of translating unit tests class-by-class (which just relocates the risk), pin the behavior at a language-neutral contract that both the old and new systems must satisfy:
- Test the API, not the code. Record real request → response pairs from the running old system (HTTP endpoints, message payloads, CLI in/out) and replay them against the new one. A JSON request and its expected JSON response don't care whether the server behind them is Java or Python. This is the same golden-master / characterization idea, captured at the interface.
- Test the data, not the objects. For the persistence layer, assert on database state: seed a known dataset, run an operation, and compare the resulting rows. The schema is shared across both implementations, so "after
addPatientToCohort, thecohort_membertable contains exactly these rows" is a language-agnostic check. - Run old and new side by side. The strongest safety net is a shadow run: feed the same inputs to both systems and diff the outputs. Every divergence is a finding — and you don't have to have predicted it in advance.
And if you do translate any in-language tests, add one non-negotiable step: run the translated tests against the original system first. They must go green on the old code before you trust them to judge the new. A translated test that passes on neither is measuring nothing.
One more honest boundary
This was one small, same-language domain, with the service and DAO layers left in place and only the entities regenerated. Bigger domains, the full service layer, and genuine cross-language ports are the real frontier — and the next experiments.
The bottom line
Spec + full test suite + a capable agent → a working rewrite. Confirmed — 89 of 89 tests, on a real build, from documentation alone.
The refined recipe, with the caveats earned the hard way:
Behavioral spec + schema / data-model spec + the complete test suite (including integration tests) + an agent that iterates against the compiler and the failures. And if those tests don't already exist, building them is the project.
The first post showed you can turn a monolith into a spec. This one shows the spec can turn back into working code — as long as you respect what the spec covers, and what only the tests can.
Experiment run against the open-source OpenMRS Core repository. The rewrite agent never read the original source, and the repository was restored to its original state after testing.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article