
Day 2 -Giving CareRelay an AI Brain
On Day 2 of Bharat Builds, we took CareRelay a step further by adding AI capabilities. We worked on OCR with Amazon Textract, RAG-based document question answering, and a frontend UI to make medical documents easier to understand and access.
Series: Bharat Builds Journey (4 articles)
- 2Day 2 -Giving CareRelay an AI Brain This article
Day 1 of my Bharat Builds Tour was about creating the foundation for CareRelay β uploading and securely storing medical documents using AWS.
On Day 2, I wanted to go one step further.
What if CareRelay could actually understand those documents?
That question became the focus of today's build.
π₯ The Problem
A medical document is useful only when the information inside it can be easily accessed and understood.
Patients may have multiple prescriptions, lab reports, scans, and other medical records. Manually opening every document to find one specific piece of information can be time-consuming.
For example:
βWhat was my latest hemoglobin value?β
Instead of searching through multiple files manually, the goal of CareRelay is to allow users to ask questions about their uploaded medical documents and retrieve relevant information.
This led me to explore OCR + RAG + AI-powered question answering.
ο»Ώ
π Step 1 β Making Documents Readable
The first challenge was that uploaded documents aren't always simple text files.
They can be:
- π PDFs
- π§Ύ Scanned prescriptions
- πΈ Images
- π Laboratory reports
To extract useful information from these documents, I worked with Amazon Textract.
The basic workflow is:
Document β Amazon S3 β Amazon Textract β Extracted Text
Textract helps convert information from supported documents and images into machine-readable text that can be processed further.
This becomes the foundation for the AI pipeline.
ο»Ώ
π§ Step 2 β Bringing RAG Into CareRelay
Once the text is extracted, another question appears:
How can the AI answer questions using the patient's actual documents?
This is where Retrieval-Augmented Generation (RAG) comes into the picture.
Instead of directly asking an AI model to answer a question, the system first searches the available document information for relevant content.
The relevant context is then provided to the AI model to generate a response.
The concept looks like:
Documents β OCR β Text Processing β Retrieval β Relevant Context β AI β Answer
For example:
User asks:
βWhat medicines are mentioned in my prescription?β
The system can retrieve the relevant information from the uploaded document and use that context to generate the response.
The objective is to make CareRelay document-aware, rather than simply creating a general-purpose chatbot.
ο»Ώ
AWS Services Behind the Build
Day 2 expanded the AWS architecture that I started on Day 1.
Amazon S3 β Document Storage
S3 continues to act as the storage layer for uploaded medical documents.
Document β S3
ο»Ώ
Amazon Textract β OCR
Textract is used to extract text and information from supported documents.
Document β Textract β Extracted Content
ο»Ώ
AWS Lambda β Processing
Lambda handles backend processing and helps connect the different services in the workflow.
Role: Serverless application logic
ο»Ώ
Amazon API Gateway β API Layer
API Gateway provides endpoints for communication between the frontend and backend.
Role: Application API layer
ο»Ώ
Amazon DynamoDB β Metadata
DynamoDB can maintain information associated with documents and application workflows.
Role: Metadata and application data
ο»Ώ
The Day 2 Architecture
The overall workflow I worked on can be represented as:
CareRelay β βΌ Frontend UI β βΌ API Gateway β βΌ Lambda β ββββββββββ΄βββββββββ βΌ βΌ Amazon S3 Amazon DynamoDB β βΌ Amazon Textract β βΌ Extracted Text β βΌ RAG / Retrieval β βΌ AI Model β βΌ User's Answerο»Ώ
The architecture is still evolving, but this gives CareRelay a path from document storage to document understanding.
ο»Ώ
Building the Frontend
Alongside the backend and AI workflow, I also started working on the frontend UI.
The goal was to keep the experience simple.
Instead of exposing all the technical complexity to the user, the interface should eventually provide a straightforward experience:
π My Documents
View uploaded medical documents.
π Document Details
Access information associated with individual documents.
π¬ Ask CareRelay
Ask natural-language questions about the available medical information.
The idea is simple:
Upload β Understand β Ask β Retrieve
ο»Ώ
π§© Why RAG?
One of the main reasons I'm exploring RAG for CareRelay is that the application needs to work with user-provided information.
A general AI model may know a lot about medicine, but it does not automatically know the contents of a particular patient's uploaded documents.
RAG provides a mechanism for retrieving relevant information from the application's knowledge source and using that information as context for generation.
This makes the system more suitable for document-based question answering.
However, AI-generated responses still need to be treated carefully, especially in healthcare.
CareRelay is intended to help users access and understand their documents β not replace medical professionals or provide a medical diagnosis.
ο»Ώ
Day 2 Wasn't Just About AI
Adding AI also introduced a new set of engineering challenges.
OCR output isn't always perfectly structured.
Different documents have different layouts.
Some reports contain tables, images, headings, and handwritten or scanned content.
Then comes the RAG side:
Did we retrieve the right information?
Because even if the AI model is capable of generating a good response, the quality of the answer depends heavily on the context retrieved from the documents.
So Day 2 involved more than simply connecting an AI model.
It involved thinking about:
Extraction β Processing β Retrieval β Context β Generation
ο»Ώ
πSecurity Matters
CareRelay deals with sensitive medical information, so security cannot be treated as an afterthought.
As the project develops, I need to pay attention to:
- π Authentication and authorization
- πͺͺ Patient-specific access control
- βοΈ Secure S3 configuration
- π Least-privilege IAM permissions
- π‘οΈ Encryption
- π Secure API design
- π« Avoiding sensitive information in application logs
Building an AI application is exciting, but building one responsibly is even more important.
ο»Ώ
π₯ Learning With My Teammates
Day 2 involved several areas at once β AWS services, OCR, RAG, backend integration, and frontend development.
Whenever something didn't work as expected, discussing the issue and debugging it together helped us move forward.
A hackathon isn't just about writing code.
It's also about learning faster, solving problems together, and continuously improving the idea.
ο»Ώ
π Day 1 β Day 2
The progress from the first two days has been interesting.
Day 1
Upload β Store β Connect
CareRelay got its document-storage foundation.
Day 2
Extract β Retrieve β Ask β Answer
CareRelay started moving toward document intelligence.
The project is gradually evolving from a simple document-management application into an AI-assisted medical document platform.
ο»Ώ
π― Day 2 Takeaway
Today's biggest lesson was that adding AI to an application isn't simply about connecting an AI model.
There is an entire pipeline behind a useful AI experience:
Real-world problem
β
Documents
β
OCR
β
Text processing
β
Retrieval
β
Relevant context
β
AI generation
β
Useful response
Understanding that complete flow helped me see how cloud services and AI technologies can work together to solve a practical problem.
Day 2 complete. π
The foundation is getting stronger, the AI layer is taking shape, and there is still plenty left to build.
Day 3 β let's see what breaks next. π
Series: Bharat Builds Journey (4 articles)
- 2Day 2 -Giving CareRelay an AI Brain This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article