AWS Builder Center
ElderLink: turning a caregiver's voice note into a record you can trust

ElderLink: turning a caregiver's voice note into a record you can trust

How we built a serverless AWS pipeline (S3, EventBridge, Lambda, DynamoDB, HTTP API, SNS) that turns caregiver voice notes into evidence-linked care events, why the AI never gets to mark its own work as verified, and the bugs that fought back along the way.

Built for the WeMakeDevs "First Commit" hackathon, Ship It track. Live at  https://main.daq0fog5qwn9u.amplifyapp.com   · Code:  https://github.com/shivaansh0610-LUFFY/Elder\_Link 

The message that gets lost

Picture a family looking after an elderly parent. Three people share the work: one covers mornings, one covers evenings, one lives in another city and calls every night. Almost all the important information moves through voice notes and chat messages.
"He barely touched lunch." "My sister says he might have skipped his tablet, I'm not sure." "Knee was hurting again by evening."
That information is valuable and it is also fragile. It sits in a thread, gets buried under forwarded videos, and is gone by the next shift. Someone doubles a dose. Someone else misses one. The evening caregiver starts from zero.
The obvious fix is a form: date, medication, dose, notes. Caregivers won't fill it in. They are tired, holding a spoon, or sitting in a hospital corridor. What they will do is talk. So ElderLink starts from the voice note and does the structuring for them.

The part I actually care about: not lying

Taking a voice note and producing a tidy summary is the easy half. Any weekend project can do it. The hard half is that in caregiving, a confident summary that is slightly wrong is worse than no summary.
Take this sentence: "My sister thinks Dad may have missed his medication."
A naive summariser writes "Dad missed his medication", which is wrong. The sister wasn't sure, and she wasn't even the one who saw it. Two separate pieces of doubt collapsed into a fact. So we designed the data model first, before touching any AWS console. A Care Event in ElderLink is one claim, and it carries three independent questions:
  • claim_stance: is this asserted, uncertain, or negated? ("He did not miss his pills" is real information, not silence.)
  • source_type: did the speaker see it, or are they relaying someone else's account?
  • review_state: has any human actually checked it?
Every event must also point at the exact transcript sentence it came from. If it can't cite one, it can't be saved. The code enforces this, so it isn't just a guideline.
One rule we held very firmly: the AI never grants trust. Extraction can describe a claim, but whether it needs review is decided by a small deterministic policy written in plain code. Uncertain, secondhand, conflicting or thinly-evidenced claims are always flagged. An event can only become "verified" when a person clicks the button. If a model could mark its own output as verified, it would confuse "I transcribed this cleanly" with "this happened", and that confusion is exactly what the schema exists to prevent.

How it works

Browser → presigned PUT → S3 (audio/) → EventBridge → process_audio (Lambda)
→ S3 (transcripts/) → EventBridge → extract_events (Lambda) → DynamoDB
→ HTTP API → React app on Amplify Hosting
  1. Record. The browser records audio and asks a small Lambda for a presigned S3 URL, then uploads straight to S3. No AWS credentials ever reach the browser, and the audio never passes through Lambda or API Gateway.
  1. Transcribe. S3 emits an event to EventBridge, which invokes  process_audio . It calls a  TranscriptionProvider  interface and writes a transcript JSON under  transcripts/ .
  1. Extract. That write triggers a second event and a second Lambda.  extract_events  turns the transcript into typed Care Events, applies the review policy, and writes only validated events to DynamoDB.
  1. Read and decide. An HTTP API serves the timeline. A caregiver reviews an event's evidence and marks it verified or keeps it uncertain. That decision goes through a PATCH Lambda whose IAM role can only run  dynamodb:UpdateItem .
  1. Tell the family. When someone flags an event, a Lambda emails the rest of that patient's Care Circle through SNS, one subscription per person.
The stack in  us-east-1  is one private S3 bucket, two EventBridge rules, six Lambdas, one on-demand DynamoDB table, one HTTP API, two SNS topics and two CloudWatch alarms, all defined in a single SAM template. The frontend is React and Vite on Amplify Hosting. We deliberately did not add services to pad the diagram. There is no Step Functions, no OpenSearch and no Cognito, because none of them earn their place in a demo this size.
Two design choices are worth a sentence each. Both AI stages sit behind small interfaces, so swapping a provider is one environment variable, not a rewrite. And the pipeline is event-driven end to end, so nothing runs when nobody is talking.

What fought back

This is the section I would have wanted to read before starting.
The Lambda that ran twice. Our first deployment worked. Then a CloudWatch check showed two invocations and two log streams for a single upload. The template wired the S3 trigger two ways: an explicit EventBridge rule and a SAM  Events:  block, which quietly generates its own rule with an identical pattern. Every upload was processed twice. The fix was deleting one of them. We also added an existence check on the transcript, because S3 and EventBridge deliver at-least-once, so duplicates are legitimate and can't be ruled out by config alone.
A 403 that means "not found". That existence check uses  HeadObject . For a missing key, S3 returns 403, not 404, when the caller lacks  s3:ListBucket , which is deliberate so it doesn't leak whether objects exist. Our role intentionally has no list permission, so we had to treat 403 as "doesn't exist" and document why, because it looks like a bug to anyone reading it later.
A format the model would have rejected. Our Voxtral provider allowed  aiff . We checked it against the installed  botocore  service model and it isn't in Bedrock Converse's audio enum, so it would have failed on the first real call.  m4a  was valid but missing. And there is a bigger version of the same problem we haven't solved: browsers record  audio/webm , which Voxtral doesn't accept. It doesn't matter on the deployed path today, but it would need a transcode step before Bedrock could be switched on.
Bedrock said no, and it kept saying no. Our plan was Amazon Bedrock for everything: Mistral's Voxtral for speech-to-text, a Claude model for extraction. On the first real call, every model returned  ValidationException: Operation not allowed . The account was flagged as "currently being verified". We requested verification, and the approval never came. We confirmed the block with a direct API test instead of guessing at IAM, because the message had nothing to do with our permissions, and then we made the call to shift instead of waiting on a queue we could not control.
That is where the interface we built on day one paid off. Both AI stages sit behind a small provider interface, so we did not rewrite anything. We wrote Groq (Whisper), Deepgram and OpenAI transcription providers alongside Voxtral, kept each one loud about failure with key-scrubbed errors, and left the mock provider as a safe default. Groq's  whisper-large-v3-turbo  transcribed real recorded speech end to end in roughly 10 to 15 seconds in our earlier deployment, with timed segments that let each event point at the exact sentence it came from.
Two more lessons hid inside that detour. First, a model ID that worked in the docs can be retired: the Claude 3.5 model we had pinned for extraction has since reached end of life, so a stale ID fails just as loudly as a missing permission. Second, in a hackathon it is tempting to quietly fall back to a mock when the real thing breaks. We refused. There is no silent fallback anywhere; a failed transcription is recorded as  status: failed  and never becomes an invented event.
The stack we are submitting still runs the mock providers, and we say so plainly: the plumbing is real and running; the AI stages are stand-ins in the submitted deployment. Switching to a real provider is one environment variable,  TRANSCRIPTION_PROVIDER  or  EXTRACTION_PROVIDER , and no code change.
CORS and a chicken-and-egg problem. CORS is where deployed apps quietly break. The browser has to be allowed by three separate things: the S3 bucket (for the upload PUT), the HTTP API, and the Lambda response headers. The frontend URL isn't known until the frontend is hosted, and the frontend build needs the API URL. We broke the loop by creating the Amplify app first, which gives a known domain, then deploying the backend with that origin as a template parameter, then building and shipping the frontend. Earlier, a Vite dev server silently hopping from port 5173 to 5174 was enough to make every request fail CORS, so we pinned the port with  strictPort .
Trusting our own tests. The evaluation harness scores extractors against 16 hand-written golden cases: negation, secondhand claims, contradictions, and claims the transcript doesn't support. On the current rule-based extractor:
  • Event recall: 92.3%
  • Event precision: 85.7%
  • Field accuracy: 95.2%
  • Claim stance accuracy: 92.9%
  • Evidence validity: 100%
  • Unsupported events: 0%
The 0% unsupported rate and 100% evidence validity are the numbers that matter most to us. The failures are informative too: it misses a mid-note retraction ("I'm no longer sure he took it") and reads a hedged "wasn't feeling like himself" as a flat assertion. Those are the cases an LLM extractor is meant to improve, and we would rather show the harness catching them than hide them.

Known limitations

  • No authentication. Anyone with the URL can read or write the demo care recipient. That is a hackathon shortcut, not an accident. CORS restricts browsers, not other clients.
  • Care Circle and patients live in the browser. They are per-device and not shared. Only the email delivery is backend-side.
  • Mock providers for transcription and extraction in the submitted stack, as described above.
  • Hindi voice notes produce no events with the rule-based extractor.
  • The second demo patient only has data in mock mode.

What I'd do next

  1. Get Bedrock access verified and run the Voxtral and extraction path on real speech, with a webm to wav transcode in front.
  1. Add Cognito and per-family authorisation before anyone puts real data in.
  1. Move Care Circle to DynamoDB so the whole family sees the same thing.
  1. Show the audio timestamps in the evidence panel, so a caregiver can hear the exact words that produced an event.

What I took away

The lesson I keep coming back to isn't about a service. It's that in a domain like this, the safest feature is the one that says "I'm not sure". Most of the effort went into what the system refuses to do: refusing to save an event without evidence, refusing to let a model mark its own work as true, and refusing to smooth over a contradiction. AWS made the boring parts fast: a bucket, two rules and a handful of small Lambdas, all defined in one template. That left us time to spend on the part that matters.
If you want to poke at it, open the live app, record a note, and open the resulting event. Look at the evidence panel, then decide for yourself whether to trust it.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article