AWS Builder Center

Builder Center

Docket: why I built an exam-marking agent on Strands Agents and AWS

Docket: why I built an exam-marking agent on Strands Agents and AWS

A Strands Graph stops for a teacher when two models disagree, AWS KMS signs every exam grade, and EventBridge with Step Functions re-marks when rules change.

Docket marks photos of handwritten exam scripts, and every grade it gives carries a signature anyone can check in a browser. When the marking scheme changes, it finds the grades the change made wrong and re-issues them. I dropped a new scheme file into S3, and 3.6 seconds later EventBridge, Step Functions and Lambda had re-marked the class set, re-issued the 2 grades that changed and re-signed the revocation list. The project post, with every measurement, is  here .
This article is about the tools. Strands Agents gave me routing in code, a pause that waits for a teacher, a guard in front of signing, typed readings and a free choice of models. AWS gave me a signature nobody else can make, a page anyone can open, a batch that re-marks a class when one file lands, and an audit line for every signature. Each section below says how I set it up and what it returned.

The rule both had to hold



The model reads, code decides. A vision model transcribes what the student wrote. Plain code checks it against an answer key, applies the scheme and routes the script. If two readers agree, the grade is signed. If they disagree on any criterion, a teacher decides that criterion first. A model never sets a grade and never picks the route.
My first design broke that rule. It let the models judge each criterion and signed when they agreed. It signed 6 of 10 scripts without a teacher, and only 2 of them matched the official grade. Moving the judgement into code took the transcribing reader to 16 of 21 exact on task 13, where the dataset authors' best published result is 12 of 21 (McNemar p = 0.29, not significant at this size). On tasks 15, 16 and 18 it matched 17 of 26, below their 19 of 26. Switching designs took one new reader prompt and the answer keys. The routing, the guard and the signing stayed the same.

Why Strands Agents

Docket runs on strands-agents 1.63.0.

GraphBuilder with conditional edges: routing a model cannot override



The  Strands Graph  runs nodes and lets each edge carry a condition. Docket has five nodes. Both readers are entry points, so they read in parallel. Two edges with  condition=both_read  join them at the comparator. From there,  routed("signer", ...)  and  routed("teacher")  read one field the comparator wrote and pick the next node.
Python
builder.add_edge("reader_a", "comparator", condition=both_read)
builder.add_edge("reader_b", "comparator", condition=both_read)
builder.add_edge("comparator", "signer", condition=routed("signer", "unread"))
builder.add_edge("comparator", "teacher", condition=routed("teacher"))
builder.add_edge("teacher", "signer")
builder.set_hook_providers([TeacherInterrupt(), SigningGuard()])
Before
With GraphBuilder
A prompt that asks a model whether to escalate
A Python condition: the same comparison always takes the same edge
Parallel calls and a join I write myself
Two entry points and a conditional join
On the AWS class set
6 of 9 scripts went straight to the signer, 3 stopped for the teacher

Interrupts: a pause that waits for a teacher

 Interrupts in multi-agent systems  let a node stop the Graph and hand a question to a person. A  BeforeNodeCallEvent  hook on Docket's teacher node calls  event.interrupt("teacher", reason=...) . The Graph returns  Status.INTERRUPTED , and the reason holds every contested criterion with both readings and the evidence each reader saw. The teacher's answer goes back as an  interruptResponse , and the Graph resumes at that node.
The teacher answers later, from another process. Strands'  S3SessionManager  saves the Graph's state, interrupt included, under a session id per script and scheme version. A script interrupted in one handler invocation was resumed in a separate one, with its own  /tmp , and signed.
Appeals use the same pause. When a student asks for a grade to be checked again, Docket runs the Graph on the cached readings with the reader order swapped and sends the appealed criteria to the teacher through the same interrupt. On 13.3.3 the appeal was upheld, 0 to 1 of 2, and Docket signed a new credential and superseded the old one. On 13.5.1 the grade stood at 1 of 2, and Docket signed an appeal record that says so. Neither called a model. The same appeal runs on AWS Lambda. On 13.1.1 it took 3.6 seconds and left the grade at 2 of 2.
Before
With interrupts
A queue, a status column and a resume handler to write
 Status.INTERRUPTED , then  interruptResponse 
State that lives in one process
State in S3 through the SDK's own session manager
A disagreement settled by a third model
A disagreement settled by the teacher, with the note signed into the credential

HookProvider: a guard in front of the signer

 Hooks  run my code at points in the run.  SigningGuard  is a  HookProvider  on  BeforeNodeCallEvent . Before the signer node runs, it checks six things. Both readings exist, neither reader declined, every criterion has a reading, every reading was made against the current criterion definition, the scheme version is current, and every contested criterion has a teacher decision. If any fails, it sets  event.cancel_node  and logs why. KMS is never called.
Before
With the hook
A check I must remember at every place that signs
One guard on the only path to the signer
Measured
50 of 50 injected bad readings blocked, 10 of each of 5 kinds; 10 of 10 clean controls signed

Structured output: readings, not prose

 Structured output  makes an agent return a typed object. Each reader is an  Agent  called with a Pydantic output model. In design 2 that model holds, per part, whether work is present and legible, the root families as fractions of pi, the roots, and whether an error is only computational. For tasks 15, 16 and 18 the model holds the answer as intervals and points, or as one number with its unit. Code compares those fields with the answer key. There is no regular expression over a model's answer.
Typed output also made the reader's limits measurable. With the student's pages removed, reader B said no student work was visible on 2 of 10 requests. On the other 8 it filled the typed fields with a student answer solved from the question. Because each reading is a typed record that also says whether a student page was sent, code can discard every reading made without one. Re-scored from the same cached readings, the pipeline refuses all 10 and grades none.
Before
With structured output
Parse a grade out of a paragraph
Read root families from a typed object and check them in code
A grade with no trace of how it was reached
Every criterion carries the transcript token it was decided on

Strands Evals: a test of the route itself

 Strands Evals  1.8.0 gave me  Case ,  Experiment ,  StateEquals  and  exact_match_scorer . A  BeforeNodeCallEvent  hook records every node the Graph calls, and the eval replays 22 cached reader pairs and compares each trail with the one the routing rule names. The library's  TrajectoryEvaluator  wants an LLM judge on Bedrock, so I wrapped its exact-match scorer in a deterministic  Evaluator  subclass instead.
Before
With Strands Evals
Trust that a refactor kept the teacher in the loop
22 of 22 cases follow the routing rule, 0 model calls
Hope a broken route gets noticed
Every replayed case is checked against the routing rule, node by node

Model-agnostic providers: one environment variable per reader

Strands is model-agnostic. Reader A uses the  OpenAI-compatible provider ,  OpenAIModel , pointed at Groq's  qwen/qwen3.8-27b . Reader B uses  GeminiModel  with  gemini-3.5-flash-lite . Each is set as  provider:model  in  DOCKET_READER_A  or  DOCKET_READER_B .
I needed that mid-build. Gemini's free tier refused  gemini-3-flash  for the rest of the day, quota 20 requests per model, retry in 23 hours 47 minutes. Reader B moved to  gemini-3.5-flash-lite  by changing one variable, and the change is recorded in the preregistration before the readings it affected.
Before
With Strands providers
Changing a model means changing code
Changing a model means changing one environment variable
One vendor for both readers
Qwen and Gemini, two families

Why AWS

AWS KMS: a grade anyone can verify, with no account

How I set it up. One asymmetric key in  ap-south-1 . The signer writes the credential as canonical JSON (sorted keys, no spaces, whole-number floats as integers) without its  signature  field, takes its SHA-256 and calls   Sign   with  MessageType=DIGEST . The key policy denies  kms:Sign  to every principal except the Lambda role and the build role.
What it returned.  DescribeKey  on the key:
JSON
{
  "KeyId": "3e29a662-62e2-4526-aacf-28f7b0722921",
  "Description": "Docket exam credential signing key",
  "KeyUsage": "SIGN_VERIFY",
  "KeySpec": "ECC_NIST_P256",
  "SigningAlgorithms": ["ECDSA_SHA_256"],
  "KeyState": "Enabled",
  "Origin": "AWS_KMS"
}
A  Sign  call from another IAM user in the same account returned  AccessDeniedException ... with an explicit deny in a resource-based policy .
What it made possible. The private key never leaves KMS, and only two roles may use it. The public key is a file anyone can download,  docket-public.pem , and the page verifies every credential in the browser with  WebCrypto . A parent needs no AWS account and no login. I changed one character at every position of 12 credentials and the revocation list, and the page rejected 10,258 of 10,258 tampered copies.
Before
With KMS
A grade in a database anyone with access can edit
A grade whose signature fails if one character changes
A private key in a file I have to guard
A key that never leaves KMS, usable by two roles
"Trust the school's system"
Anyone verifies with a public key, in a browser

Amazon S3 and Amazon CloudFront: a verify page with no login

How I set it up. Two private buckets. The site bucket holds the page. The data bucket holds scans, the readings cache, Strands sessions, pending interrupts, credentials,  revocations.json  and the index the page reads. One CloudFront distribution serves both through  Origin Access Control , the data at  /data/* . Neither bucket is public.
What it returned. Distribution  E128IQWEBE8CGT  serves  the live verify page . Its   /data/index.json   lists 12 credentials, and   /data/revocations.json   lists 3 superseded ones, 2 from the re-mark and 1 from an upheld appeal.
What it made possible. Each credential is an object that never changes. A corrected grade is a new object with  supersedes  pointing at the old one, and the signed revocation list says why. The page reads both. 9 credentials say valid, and 3 say superseded with the reason and a link to the new one. The same data feeds the Moderation tab, where a board samples signed grades against the official ones.
Before
With S3 and CloudFront
Overwrite the grade and lose the old one
Keep both, linked, with the reason
Ask the school to confirm a grade
Open one link

Amazon EventBridge and AWS Step Functions Map: re-marking a whole class

How I set it up. The data bucket sends its events to EventBridge. A rule matches a new  schemes/*.json  object and starts the re-mark state machine. It reads the scheme and bumps the version, then a  Map state  runs one  remark  Lambda call per script, then one step re-signs the revocation list, then the page index is rebuilt. Only that one step writes the list, so parallel iterations never race on it. A second state machine runs a Map of  mark  over a class set of scans.


What it returned.
text
s3://<data bucket>/schemes/v2.json uploaded
  -> EventBridge rule
  -> RemarkRemarkStateMachine execution 4d4bdc28-f2bf-d83a-6c4a-23d880efb3ad_daa9eda0-...
     SUCCEEDED, 15:10:25.99 to 15:10:29.56 IST (3.6 s)
     bump v1 -> v2, Map re-marked 9
     re-issued: dkt_13.2.3_v2_1 (1 -> 0), dkt_13.3.3_v2_1 (1 -> 0); 7 unchanged
     revocation list re-signed with 2 superseded entries; index rebuilt, 11 credentials
What it made possible. One file landing in a bucket re-marks every script. Each script is its own Map iteration, so a failure stops one script and the rest finish. No model was called. The readings were cached and the criteria text did not change, so the re-mark is code and signatures.
Before
With EventBridge and Step Functions
Re-read every script by hand after a correction
Upload the new scheme file
A loop that stops at the first error
A Map where each script succeeds or fails alone
Nobody knows which grades a correction changed
A count, and a signed revocation for each

AWS Lambda: the same agent, in the cloud

How I set it up.  RemarkScript  runs the same Python package as the CLI,  docket.aws_handler.handler : Python 3.12 on arm64, 1,024 MB, a 900-second timeout. Its events are  mark ,  teacher ,  bump ,  remark ,  publish-revocations ,  verify  and  appeal . Upload from my laptop ran at a few KB a second, so a small builder Lambda downloads the arm64 wheels from PyPI inside AWS and writes the package to S3.
What it returned. All 12  Sign  calls of the AWS run were made by the Lambda's own role. An appeal on script 13.1.1 sent to the  appeal  event (request  b13bb1d0-2ffc-4d11-875e-b22a92dd211d ) loaded the cached readings from S3, swapped the readers, sent all three criteria to the teacher step, and returned "appeal reviewed, grade unchanged" at 2 of 2 in 3.6 seconds, with 0 model calls. CloudTrail shows its one  Sign  call, event  ce5b4d9d-c140-44ec-ade1-65e05772dc42 , made by the Lambda. The appeal record it signed verifies as valid (  appeal_lambda.json  ).
What it made possible. The Strands Graph, both readers, the guard and the signer run on AWS with no server to keep up, and a pending interrupt waits in S3 while the teacher decides.
Before
With Lambda
A laptop that has to stay on during a batch
A function each Map iteration calls

AWS Systems Manager Parameter Store: reader keys with no storage charge

How I set it up. The Groq and Google API keys are SecureString parameters,  /docket/GROQ_API_KEY  and  /docket/GOOGLE_API_KEY , on the standard tier. The Lambda's bootstrap loads them into its environment before the handler runs.
What it made possible. No key is in the code, the package or the repo, and standard-tier parameters carry no storage charge.
Before
With Parameter Store
A key in an environment file shipped with the function
An encrypted parameter the function reads at start

AWS CloudTrail: a second record of every signature

How I set it up. KMS  logs every  Sign  call to CloudTrail , and CloudTrail event history keeps them with no trail configured.
What it returned. 12  Sign  events between 09:39:45Z and 09:40:28Z, all by the Lambda role, for 11 credentials and 1 revocation list. Here is one.
JSON
{
  "eventTime": "2026-11-04T09:40:28Z",
  "eventSource": "kms.amazonaws.com",
  "eventName": "Sign",
  "awsRegion": "ap-south-1",
  "requestParameters": {"keyId": "alias/docket-credentials", "signingAlgorithm": "ECDSA_SHA_256", "messageType": "DIGEST"},
  "resources": [{"type": "AWS::KMS::Key", "ARN": "arn:aws:kms:ap-south-1:960149837193:key/3e29a662-62e2-4526-aacf-28f7b0722921"}]
}
The denied attempt from the other IAM user is there too, as event  9b18bfbd-76b5-49a4-902b-3cabf66dc11a .
What it made possible. Every signed grade has a record outside my code that says when it was signed, by which role and with which key. The AWS run made 12 signatures, and the log holds 12 events.
Before
With CloudTrail
My log says it signed
AWS's log says when, with which key, and which role asked

What I would tell another builder

  • Put routing in code and let the Graph carry it. Five  add_edge  lines hold every routing decision in Docket, and they did not change when the readers did.
  • Use an interrupt wherever a person must decide, and give it a session manager so the pause survives the process.
  • Test your own verifier the way an attacker would. Our tamper eval found that the page accepted 4 of 909 tampered credentials because it skipped one DER length check, while Python rejected all 909. After the fix the page rejected 10,258 of 10,258.
  • Do not count two models agreeing as evidence. Check the answer in code where you can.
  • Make each model one environment variable. When a free tier ran out mid-run, the switch cost one line.
Docket's code is MIT and public at  github.com/ashutosh887/docket . The project post with every measurement is  here , the walkthrough is on  YouTube , and you can check a real signed grade on  the live verify page .
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article