
Weekend Creative Agent Challenge: Personal Study Evaluator Agent
An autonomous, always-on serverless agent on AWS that reads your daily study notes from Notion, evaluates your spoken explanations using Amazon Bedrock (Amazon Nova), and autonomously schedules targeted follow-up reviews.
1. Vision & What It Does
The Problem: The Passive Learning Trap
When studying for cloud certifications, software architecture, or distributed systems, most of us diligently take notes. However, simply re-reading notes is one of the least effective ways to learn. True knowledge retention requires active recall and the Feynman technique—explaining concepts out loud in your own words without reading a script.
The friction is that creating daily quizzes, setting up review schedules, and grading open-ended oral answers takes significant effort. Most study tools require you to manually open an app, configure prompts, and grade yourself. When a system requires too much friction, you stop using it after the first week.
The Solution: An Always-On, Zero-Touch Agent
Personal Study Evaluator is an autonomous, "always-on" agent deployed on AWS that operates while you sleep or go about your daily routine. The best tool is the one you never have to open.
Here is the autonomous daily lifecycle:
- Autonomous Preparation (21:00 Nightly): Without any manual intervention, an Amazon EventBridge Scheduler triggers the agent. It connects to your Notion database, extracts the notes you wrote that day, uses Amazon Bedrock (Amazon Nova Lite) via Converse Tool Calling to craft 5 open-ended conceptual questions, and sends a clean email directly to your inbox with a secure recording link.
- Mobile-First Voice Response: You tap the link from your phone, see the 5 questions, press record, and speak your answers out loud. The audio is uploaded directly to Amazon S3 using a 5-minute on-demand presigned URL.
- Evidence-Based Evaluation: An event-driven AWS Step Functions pipeline triggers Amazon Transcribe and Amazon Bedrock (Amazon Nova Pro) to evaluate your spoken answers strictly against the raw content of your notes.
- The Agentic Loop (Adaptive Memory): The agent does not just grade answers and stop. It inspects a 14-day history of concepts you have struggled with. If it identifies recurring weaknesses, the LLM autonomously decides to branch into a review planning state (
PlanRepaso) and seeds targeted follow-up questions into tomorrow's quiz.
- Morning Performance Report: You wake up with an evaluation email delivered via Amazon SES, breaking down each question with direct quotes from your notes and transcript.
2. End-to-End Walkthrough (Screenshots & Flow)
Below is the step-by-step walkthrough showing the agent in action with real screenshots from the application:
Step 1: Study Notes in Notion
The day begins normally: writing notes in Notion covering AWS architectural topics (e.g., Shared Responsibility Model, IAM Roles vs. Users, and the Principle of Least Privilege).

Figure 1: Study notes recorded in Notion with the Date property used by the agent.
Step 2: Nightly Quiz Arrives via Email (21:00)
At 21:00, EventBridge Scheduler invokes the Preparer Lambda. It queries Notion, calls Amazon Nova Lite to synthesize 5 conceptual questions, and delivers an email to the inbox with a direct link.

Figure 2: Automated nightly email containing the 5 open questions generated by Amazon Nova Lite.
Step 3: Voice Response on Mobile
Tapping the email link opens a lightweight web interface on the phone. You press "Record", explain your answers out loud, and press "Send".
(In our test run, Question 2 regarding OS patching responsibilities in EC2 vs. Lambda was deliberately answered ambiguously to verify the evaluator's strictness).

Figure 3: Web recorder uploading voice answers directly to S3 via an on-demand presigned URL.
Step 4: Visual Agentic Execution in AWS Step Functions
Once the audio file lands in S3, EventBridge triggers the Step Functions state machine. In the execution graph below, you can see where the AI agency lives: Bedrock evaluates the transcript and executes the decision branch in the state machine.

Figure 4: Visual state machine execution completing transcription, evaluation with Nova Pro, and report delivery.
Step 5: Morning Evaluation Report with Evidence Citations
The evaluation report arrives via SES. The agent accurately flagged the incorrect answer on Question 2, citing the exact note evidence (
evidencia_apuntes ) and the spoken transcript ( evidencia_transcripcion ).
Figure 5: Evaluation report highlighting evidence from notes, transcript citations, and study suggestions.
3. Architecture & AWS Services Used
The solution is 100% serverless, event-driven, cost-effective, and provisioned using AWS CDK in Python.
EventBridge Scheduler (21:00 America/Guayaquil) │ ▼ Lambda: Preparer ──► Notion API (Today's Notes) ──► Amazon Bedrock (Amazon Nova Lite + Tool Use) ──► Amazon DynamoDB & Amazon SES (Email with quiz link) │ ▼ (User answers out loud on mobile) Lambda: Gateway ──► Serves record.html / Signs on-demand Presigned URL │ ▼ Amazon S3 (Media Bucket) │ (S3 Object Created Event via EventBridge) ▼ AWS Step Functions: StudyPipeline │ ├─► StartTranscriptionJob (Amazon Transcribe + Custom Vocabulary) ├─► Wait & Poll Transcription Status ├─► Lambda: Evaluate (Amazon Bedrock Amazon Nova Pro via Tool Use) │ │ │ ▼ │ ┌── AgenteDecide (Choice State based on Model's Output) ──┐ │ │ │ │ ▼ ▼ │ Lambda: PlanRepaso Lambda: CloseDay │ (Seeds recurring questions into DynamoDB) (Closes day) │ └────────────────────────┬────────────────────────────────┘ │ ▼ └───────────────────► Lambda: SendReport ──► Amazon SES (Morning Report)AWS Services Breakdown:
- Amazon Bedrock (Converse API + Tool Calling):
- Amazon Nova Lite (
us.amazon.nova-lite-v1:0): High-speed, ultra-low-cost extraction of quiz questions from daily notes.
- Amazon Nova Pro (
us.amazon.nova-pro-v1:0): High-reasoning evaluation and autonomous recurrence reasoning.
- AWS Step Functions: Orchestrates the long-running transcription and evaluation workflow. The agent's autonomous branching is explicitly visible in the state machine graph as a
Choicestate (AgenteDecide).
- Amazon Transcribe: Converts spoken audio into text with a Custom Vocabulary (
estudio-aws) ensuring technical acronyms like S3, EC2, IAM, VPC, and Bedrock are accurately transcribed in Spanish.
- AWS Lambda (Python 3.12): Executes isolated, single-responsibility functions (Preparer, Gateway, Evaluator, PlanRepaso, CloseDay, SendReport).
- Amazon DynamoDB: Single-table design (
PK=user_id,SK=date) storing daily statuses, transcripts, quiz schemas, and historical error counts with automated TTL and point-in-time recovery.
- Amazon S3: Dual-bucket architecture (
MediaBucketfor incoming voice recordings andArtifactsBucketfor transcripts) with a 7-day automatic lifecycle expiration rule.
- Amazon EventBridge & Scheduler: Manages timezone-aware cron executions (
America/Guayaquil) and routes S3 object-creation events directly into Step Functions without glue code.
- Amazon SES: Sends daily questions and rich evaluation reports.
- AWS Secrets Manager & SSM Parameter Store: Manages Notion integration tokens, HMAC signing secrets, and runtime configuration.
- Amazon Cognito & HTTP API Gateway (Switchable Auth): Supports zero-friction HMAC one-time token links for personal use or switchable Cognito User Pool authentication with JWT authorizers.
4. Real-Time CloudWatch Logs & Observability
To demonstrate operational rigor and production readiness, here are actual structured CloudWatch logs from a live run:
1. Preparer Lambda (Quiz Synthesis via Amazon Nova Lite)
{ "timestamp": "2026-08-21T22:04:10.512-05:00", "level": "INFO", "msg": "preparer.start", "date": "2026-08-21", "user_id": "solo"}{ "timestamp": "2026-08-21T22:04:11.230-05:00", "level": "INFO", "msg": "notion.query", "date": "2026-08-21", "pages": 1}{ "timestamp": "2026-08-21T22:04:12.890-05:00", "level": "INFO", "msg": "bedrock.converse", "model": "us.amazon.nova-lite-v1:0", "tool": "registrar_quiz", "stop_reason": "tool_use", "input_tokens": 1308, "output_tokens": 539, "date": "2026-08-21"}{ "timestamp": "2026-08-21T22:04:13.415-05:00", "level": "INFO", "msg": "ses.sent", "message_id": "010001a0276d0d32-e9da4687-27a7-4677-a2a8-b4a9696be20b-000000", "subject": "Tus preguntas de hoy · 2026-08-21", "date": "2026-08-21"}{ "timestamp": "2026-08-21T22:04:13.450-05:00", "level": "INFO", "msg": "preparer.done", "date": "2026-08-21", "preguntas": 5}2. Evaluator Lambda (Evaluation & Decision via Amazon Nova Pro)
{ "timestamp": "2026-08-21T22:31:05.120-05:00", "level": "INFO", "msg": "evaluator.start", "date": "2026-08-21", "user_id": "solo", "job": "eval-2026-08-21-079b6732-f206-42e4-a4eb-50f62954dc7c"}{ "timestamp": "2026-08-21T22:31:05.340-05:00", "level": "INFO", "msg": "evaluator.transcript_read", "date": "2026-08-21", "chars": 2100}{ "timestamp": "2026-08-21T22:31:11.890-05:00", "level": "INFO", "msg": "bedrock.converse", "model": "us.amazon.nova-pro-v1:0", "tool": "registrar_evaluacion", "stop_reason": "tool_use", "input_tokens": 3144, "output_tokens": 764, "date": "2026-08-21"}{ "timestamp": "2026-08-21T22:31:12.110-05:00", "level": "INFO", "msg": "evaluator.done", "date": "2026-08-21", "decision": "cerrar_dia", "fallados": ["shared-responsibility-model"]}3. SendReport Lambda (Delivery to User)
{ "timestamp": "2026-08-21T22:31:13.150-05:00", "level": "INFO", "msg": "ses.sent", "message_id": "010001a02785c477-d974c290-35cc-4730-a3a0-ae131bd1da87-000000", "subject": "Tu informe · 2026-08-21", "date": "2026-08-21"}4. DynamoDB Stored State (Live Record)
{ "user_id": "solo", "date": "2026-08-21", "status": "completado", "evaluation": { "conceptos_fallados": ["shared-responsibility-model"], "decision_repaso": { "accion": "cerrar_dia", "justificacion": "El concepto fallado 'shared-responsibility-model' no muestra un patron recurrente en el historial." }, "evaluacion_preguntas": [ { "numero_pregunta": 1, "puntuacion": "correcto", "comentario": "La respuesta es consistente con los apuntes.", "evidencia_apuntes": "AWS es responsable de la seguridad DE la nube...", "evidencia_transcripcion": "la responsabilidad de la seguridad entre AWS y el cliente..." }, { "numero_pregunta": 2, "puntuacion": "incorrecto", "comentario": "La respuesta menciona 'S2' en lugar de 'EC2' y no aclara correctamente la diferencia en responsabilidades entre EC2 y Lambda.", "evidencia_apuntes": "En EC2 el cliente parchea el sistema operativo. En Lambda no hay sistema operativo que parchear...", "evidencia_transcripcion": "la responsabilidad de parcheo del sistema operativo en S2 en S2..." } ] }}
5. Additional Deployment Evidence
To provide full transparency, here is additional proof of deployment across AWS:

Notion Integration Connection
Figure 6: Notion database permissions connected to the integration token.

Deployed AWS Lambda Functions
Figure 7: Deployed AWS Lambda functions (Preparer, Gateway, Evaluate, PlanRepaso, CloseDay, SendReport).
6. How I Built It: Key Decisions & Challenges
1. The Hierarchy of Evidence
A critical flaw in most AI evaluators is treating the model's improvised response as absolute truth. If a reference answer contains a hallucination, the student gets penalized unfairly.
I established a strict Hierarchy of Evidence:
- Ground Truth: The raw text of your own notes (
raw_text).
- Advisory Guide: The reference answer generated during quiz creation (unverified by humans).
- Evaluated Target: The literal speech transcript.
In the evaluation prompt, every grade requires two mandatory citations:
-
evidencia_apuntes: The exact verbatim quote from your notes used to evaluate.
-
evidencia_transcripcion: The exact snippet of what you spoke.
If an explanation is consistent with your notes, it is marked correct, even if phrased differently from the reference answer.
2. Genuine Agency vs. Hardcoded If-Statements
An agent is not just an automated script; it must possess memory and the ability to decide its own next actions.
- The evaluator Lambda analyzes the last 14 days of DynamoDB history for recurring concept failures.
- The model returns a structured
decision_repasoobject containing an action (cerrar_diavsgenerar_repreguntas).
- Step Functions executes an explicit
AgenteDecideChoice state based on what the model determined. If a concept has failed repeatedly,PlanRepasoseeds review questions into tomorrow's DynamoDB record. The next evening, the Preparer Lambda automatically includes these seeded questions in the new quiz. The loop is closed by the agent itself.
3. Tool Use over Raw JSON Parsing
Prompting LLMs to "Respond only with JSON" frequently causes runtime failures due to markdown code blocks, conversational filler, or truncated responses.
By utilizing Bedrock's Converse API with forced Tool Calling (
toolChoice ) and temperature=0 , the model is strictly constrained to output valid JSON conforming to our schema ( registrar_evaluacion ). If the response hits the token limit, Bedrock reports stopReason=max_tokens , enabling immediate detection of truncated outputs.4. Zero-Friction Mobile Voice UX
To ensure the habit sticks, answering questions must take less than 3 taps on mobile:
- Mobile browsers (particularly iOS Safari) record audio in
.m4aformat rather than.webm. The Gateway Lambda inspects the client MIME type and configures the S3 key extension dynamically so Amazon Transcribe receives the correct format.
- Presigned URLs are generated on-demand with a 5-minute expiration when the user begins recording, resolving execution role STS token expiration limits.
7. What I Learned
- Model Tiering with Amazon Nova: Pairing Amazon Nova Lite for high-throughput question generation with Amazon Nova Pro for deep semantic evaluation provided outstanding reasoning quality at a fraction of the cost.
- Domain Vocabularies Are Non-Negotiable: Without Transcribe's Custom Vocabulary, Spanish speech explaining cloud concepts transcribes "IAM" as "i am" or "EC2" as "S2". Custom vocabularies bridge the gap between spoken audio and accurate evaluation.
- Storage Lifecycle Economics: Setting up an S3 7-day expiration rule cost $0 and eliminated manual cleanup, whereas archiving to S3 Glacier would have incurred a 90-day minimum storage fee per object.
- Model Availability & Access Friction (Claude vs. Amazon Nova): Initially exploring third-party models like Claude on Amazon Bedrock required extra administrative procedures, including filling out Anthropic use-case forms and AWS Marketplace subscription validations. Pivoting to Amazon Nova (Nova Lite and Nova Pro) eliminated onboarding friction entirely—they are instantly available, natively integrated, and proved to be just as capable in strict JSON Schema adherence, Converse Tool Calling, and deep semantic evaluation.
8. Summary & Links
The Personal Study Evaluator transforms passive study notes into an active, intelligent learning loop powered by Amazon Bedrock (Amazon Nova), Step Functions, and AWS Serverless.
Note on Live App URL & Access: Because this agent is purpose-built for personal study and triggers automated pipelines invoking Amazon Bedrock (Nova Pro) and Amazon Transcribe, a public live URL is intentionally not exposed to avoid unintended traffic, unauthorized requests, and API cost spikes. Instead, the full Infrastructure as Code (AWS CDK) is completely open-sourced in the repository below, allowing anyone to deploy their own private, secure instance in their own AWS account in minutes.
- GitHub Repository: https://github.com/JostinAlvaradoS/agentic-personal-evaluator
- Architecture Decisions (ADRs): Documented in
/docs/adr/in the repository.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article