The Bag Was the System — Why We Built Potli PocketEHR

My father was 85 years old when we lost her. He grew up speaking Bengali in Hooghly District, West Bengal, India, and he spent the last years of my mother's life carrying everything the healthcare system needed to know about her in a small cloth bag — a potli.

Prescriptions from three specialists. Lab reports from two diagnostic labs. A dozen medicine strips, some expired, some duplicated across prescriptions. Every appointment, every hospital visit, every pharmacy run — he carried the bag from their home in Kolkata, from doctor to doctor, across a city that was never designed for people like him.

He is not unusual. Across India, this is simply how families manage. The bag is the filing system. The bag is the history. The bag is the continuity of care that no institution provides.

The potli was the system. It still is. The question was never whether families needed a record — they had one. The question was how to give it a spine that didn't depend on a single cloth bag making it to every appointment.

India has no patient-side electronic health record in everyday use. The government's ABHA system is a promising credentialing layer. But what travels with the patient — what a doctor in an emergency needs to know at midnight — still lives in a bag, in memory, in the hands of the person who loves them.

The families who carry potlis are not underserved by technology because they lack intelligence or initiative. They are underserved because every digital health solution built for India has been designed for someone else: the English-literate, smartphone-native, urban professional who has time to log in and type.

My father was 85 when we lost Ma. He speaks Bengali. He has never used an app. He reads prescriptions the way millions of Indian elders read prescriptions — carefully, holding the paper close, doing his best. He needed something that met him where he was. A camera. A voice. A result.

Before Potli, I was building a different project. CareMap  — a clinical AI assistant submitted to the Kaggle MedGemma Impact Challenge  — was an attempt to use a medical-grade language model to help caregivers navigate clinical information. During that work, I spoke with Dr. Manini Moudgal, a physician practising in India.

What she told me changed the design of everything that came after.

Dr. Moudgal sees over a hundred patients a day. Not because she wants to — because India has one of the world's most severe doctor shortages, and she, like every physician in a public system, absorbs the volume that the system cannot otherwise contain. She is not slow. She is not negligent. She is doing the work of three doctors with the hours of one.

When I described building an EHR, she was unequivocal: another system that required a doctor to type, to log in, to review structured data entry — however well-intentioned — would be abandoned the first week. Not because doctors don't care about records. Because a doctor who types during a consultation is a doctor who is not listening to the patient. And for someone seeing a hundred patients a day, every minute of administrative friction has a cost measured in people not seen.

"Don't build a system that adds to the burden. Build one that removes it."

That single sentence — combined with the memory of my father and his bag — became the complete design brief for Potli. Zero doctor friction. Zero typing. The caregiver takes a photograph. The doctor reads a card. Nobody changes their workflow. Nobody logs in. Nobody is asked to do more.

I asked my father, who was visiting relatives in Hooghly District, West Bengal, to collect some test cases — just photograph whatever medications people had at home. He asked a relative, gathered four strips from her medicine bag, and sent me a single picture. Everything at once, the way you actually find medications — not neatly separated, but together in a pile. Here, this is all of it.

The pipeline processed all four strips from that single photograph. It returned a complete medication list — brand names, generic names, doses, batch numbers, expiry dates — for every strip in the frame. One entry had a red flag: Rosuvastatin, a statin prescribed for her heart, had expired eight months earlier.

Nobody had caught it. The strip looked identical to every other strip she had taken home from that pharmacy. Expiry dates on Indian blister packs are printed in small type, in English — a language that millions of elderly patients in rural West Bengal do not read fluently. There is no one checking. The pharmacist dispenses. The caregiver administers. The patient trusts. This is not negligence — it is the ordinary reality of a system that puts clinical information on packaging and assumes someone in the chain can read it. Often, nobody can.

When I shared the result with my father, he passed it along. The expired strips were discarded. But the larger point stayed with me: this is almost certainly happening in every medicine bag in every village. Silently. Routinely. Without anyone meaning harm.

This is the promise of camera-first design. You do not need to read the strip. You do not need to type anything. You point a phone at a table, and the system tells you what you have — and what you should not still have.

That moment — one photograph, one scan, one finding that could have mattered enormously — is why Potli is built around a camera rather than a keyboard.

The design is built around one non-negotiable: the caregiver has a smartphone and ten minutes. Everything else follows.

Medicine strip photo — photograph any number of strips, laid flat on a table. Potli extracts brand name, generic name, dose, batch number, and expiry date for every strip in the frame — even if the labels are in English and the caregiver reads Bengali.

Aadhaar card — photograph the ID to create the patient profile. Name, date of birth, gender. No Aadhaar number is stored.

Voice vitals — speak the patient's vitals in Hindi, Hinglish, or English. "BP ek so chalis over nabbe, weight saath kilo" becomes structured clinical data.

Lab PDF — upload the lab report. Potli extracts every test value, flags abnormals, and preserves them for the next visit.

A single AI model reading a medicine strip is not safe for clinical use. Strip printing fades. Photographs are imperfect. A wrong expiry date is a patient safety event. Potli uses a three-agent consensus pipeline on Amazon Nova: two models read the same strip independently, and an AI Judge reconciles their outputs field by field — flagging any disagreement for human review rather than silently choosing.

Null is safer than wrong. Every prompt instructs: if a field is not clearly visible, return null. Do not guess. A wrong drug name stored silently is not a safe state.

The user is not the patient. It is the person who loves them — a daughter managing her parents' medicines over a video call from another city; a son sitting in a waiting room with a bag full of paper; an ASHA worker doing her monthly rounds in a village in Hooghly; a neighbour who checks in on the elderly couple next door.

Potli asks only this: ten minutes, and a camera. In return, it gives the caregiver something they have never had — a record that travels, that is legible, and that a doctor anywhere can read in thirty seconds.

ASHA and ANM workers — Community health workers already visit homes. Potli turns every home visit into a medication audit. No literacy required. No typing required.

District hospitals and PHCs — A tablet at the registration desk running Potli replaces the paper form for every new elderly patient.

Old age homes — A scan of every resident's medicines at admission. Monthly rescans flag new prescriptions, expired drugs, and duplicates automatically.

Pharmacy counters — A pharmacist who can see the full medication list before dispensing catches dangerous interactions that no prescription-only view reveals.

Potli PocketEHR  is half of something larger. CareMap  — built during the Kaggle MedGemma Impact Challenge  — is a clinical AI companion that helps caregivers understand medical information, prepare for appointments, and navigate a system that was not designed for them. Where Potli digitizes the record, CareMap helps make sense of it.

Together, they close the loop that currently exists only in a cloth bag: the record that travels with the patient, and the understanding that helps the caregiver use it.

Every conversation about AI in healthcare eventually arrives at the same question: who pays for it? The answer, for Potli, is almost embarrassingly small.

The entire pipeline — two Nova models reading a strip, an AI Judge reconciling them, an RxNorm lookup confirming the drug exists — costs less than a quarter of a US cent per scan.

Running Potli for an entire village of 500 patients, with monthly medication rescans, costs approximately $116 a year in AI inference. One preventable hospital admission from an undetected drug interaction costs ten times that.

The infrastructure underneath is equally lean. A single EC2 t3.medium instance — $42 a month — runs the full application for an entire district. No GPU required. No specialised hardware. No ongoing licensing. The application is stateless; the data lives in DynamoDB. The source images live in S3. Both cost pennies at the scale we are describing.

This is not a technology that requires a hospital budget or a government programme to deploy. An NGO, a district health authority, an old age home trust, a community pharmacy chain — any of these could run Potli for their entire population for less than a staff member's monthly mobile phone bill.

I want to be honest about where Potli stands. The framework is real. The pipeline works. The expired drug was caught. The lab report was parsed. The doctor chart rendered. That is more than I expected to have at the end of a hackathon week.

But there is much more testing and iteration ahead. I am still wrestling with prompt edge cases — strips with unusual formatting, handwritten batch numbers, low-light photographs. I have not yet run a rigorous evaluation across Nova Lite, Nova Pro, and different judge configurations to know which combination performs best in the real world. The consensus pipeline I built makes a strong structural argument, but the model-level evidence is still being gathered.

What this hackathon gave me is not a finished product. It is a working hypothesis — and the early outcomes are genuinely exciting. What makes me hopeful is not just that it works, but why it works: because foundation models like Amazon Nova have compressed what would have taken months of fine-tuning into a structured prompt. A three-agent consensus pipeline that cross-verifies extraction, scores confidence, and flags human review — built in days, without a labelled dataset, without a custom model. That is the real breakthrough here.

The construct and the framework are established. The work of finding the best model, the sharpest prompt, the most reliable pipeline — that work continues. But the direction is clear, and I am more hopeful than I have been at any point in this project.

My mother had Alzheimer's, diabetes, hypertension, hyperlipidaemia, and acute anaemia. My father managed her medication across three specialists and two diagnostic labs, carrying everything in a cloth bag, speaking Bengali in a system that largely communicated in English. When she was hospitalised, he would lay the prescriptions out on the bed so the doctors could read them. Sometimes they did. Often they wrote new ones on top.

We lost her in 2024.

I cannot give her back the years she was unwell. I cannot give my father back the weight he carried. But I can make sure the next family — in Hooghly, in a village in Bihar, in an old age home in Chennai — has a tool that sees what the bag holds, checks what is expired, and hands a doctor a readable record in thirty seconds.

Try the live demo: Watch the AI pipeline live 

Read the full story: Potli PocketEHR 

GitHub: codekunoichi 

#AmazonNova