Paper, Plastic or E-Waste: One WhatsApp and Voice Agent on Serverless AWS
EcoLink connects households to India's unorganized recycling sector with a photo on WhatsApp, a phone call to a kabadiwala and e-waste recycling centers with a Strands agent running on Amazon Bedrock. This is how we got there, how it is built and what pushed back.
The drawer problem
Almost every home has one drawer. It holds dead phones, tangled chargers, an old router and a few batteries that nobody wants to throw away and nobody knows how to dispose of. Paper and cardboard are easy, since the neighbourhood kabadiwala already takes them. E-waste is different. It carries metals that have value and materials that are dangerous when burnt or broken in the open, yet the household has no clear way to find out where it should go.
Three things made this hard for us to ignore:
- The unorganized sector does most of the work. Kabadiwalas and local scrap dealers already collect, sort and resell a large share of recyclable material, and e-waste handled informally is commonly quoted at around 90 percent. They are efficient, but fragmented. There is no common directory, no standard price and no way for a household to know who accepts what.
- Authorised centres exist but are lesser known. Registered collection points are listed on regulator portals and producer take-back pages that very few people would ever search for. A household that wants to do the right thing usually ends up handing the item to whoever arrives first.
- Nobody explains the correct way. Batteries and hazardous e-waste must never be mixed into a normal pickup. A person who is never told this cannot be expected to follow it, thus the system itself has to carry that knowledge.
EcoLink started as an attempt to connect these fragments: the household, the scrap dealer and the authorised drop-off point, through an interface that anyone with a phone already knows how to use.
How we got to this solution
Our first idea was an app. It was dropped quickly, because an app needs a download, a sign-up and a user who is comfortable with menus, which would filter out the exact people we wanted to reach. The channel that almost every household already opens every day is WhatsApp, and the channel that works for everyone, including those who prefer not to type, is a phone call.
This decided the shape of the product:
- WhatsApp first. A household sends a photo of the waste, or describes an e-waste item in words. Nothing needs to be installed.
- Bilingual from the first message. Hindi and English are both supported, and the reply follows the language of the user.
- A voice agent on the same brain. The phone agent shares the same agent core and conversation state as WhatsApp, with Amazon Polly voices for Indian English and Hindi. Users who find typing slow can simply call.
- The recycler is reached by phone, not by app. A kabadiwala receives a call, hears a short offer and presses 1 to accept or 2 to decline. No account, no download and no training. This was the most important decision, since the supply side of the market is the hardest one to onboard.
- Honest prices. Every price is labelled an estimate, as the final value depends on weight and condition that only the dealer can check.
- Safety in code. Hazardous items are routed to authorized
- drop-off points and never booked as a pickup, and the household address reaches a recycler only after that recycler accepts.
-
The architecture
The whole backend is a single AWS CDK stack in Python, deployed in one region and one account. Twilio provides the WhatsApp and voice channels, and everything else is serverless AWS.
EcoLink on AWS. The agent calls the same API as every other client, so the API Gateway authorizers and the pickup state machine apply to it too.
Why each service is there
Service | Role in EcoLink | Why this choice |
|---|---|---|
Amazon Bedrock (Nova Pro) | Language model for the Strands agent and first-choice vision classifier | One managed model for text and images, no GPU to operate, and IAM scoped to a single model |
AWS Lambda (Python 3.12, arm64) | About a dozen functions: webhook, agent, voice agent, pickups, voice offers, matching, pricing, uploads, auth, dashboard, authorizer | Traffic is bursty and small, so paying per request fits far better than an idle server |
Amazon API Gateway | One public entry point, 16 routes, two authorizers, throttling at 50 requests per second | The agent, Twilio and the dashboard share one API with different authentication per route |
Amazon DynamoDB | Five on-demand tables: users, recyclers, pickups, rate table, conversation state | Access is by key plus two status indexes. Conversation state uses a 7 day TTL so it cleans itself |
Amazon S3 | Private images bucket with 7 day expiry, versioned models bucket | Household photos are sensitive, thus private and short lived. Model weights can be rolled back |
Amazon SNS | An event on every pickup status change, carrying ids and status but never the address | Notifications stay decoupled from pickup logic, and email or a queue can subscribe later |
Amazon Cognito | Dashboard sign-in with admins and viewers groups | Role based access to addresses without writing our own auth |
AWS Secrets Manager | API key and Twilio credentials | Nothing sensitive sits in the repository, the template or the Lambda settings |
Amazon Polly | Indian English and Hindi voices for the phone agent | Natural regional voices matter most for a voice product in India |
Amazon Location Service | Map tiles for the live dispatch map, drawn with MapLibre GL JS | Keeps the map inside the same AWS account and billing |
AWS CDK (Python) | The whole stack as code | One command to deploy, one to destroy, same language as the backend |
The agent and its guardrails
The conversation is handled by an agent built with the Strands Agents SDK. It has seven tools: classify an image, get a price, quote e-waste, find recyclers, create a pickup, update a pickup and offer a pickup to a recycler by phone.
A language model alone is not enough for a system that books real people and shares real addresses. For this reason the rules that matter live in Python and not in the prompt:
- Confirmation gate.
create_pickuprefuses to run unless an explicit yes, in English or Hindi, was detected.
- Hazardous block. Items labelled
batteryorewasteare rejected by the pickup tool itself, and the agent intercepts such requests before the model is called.
- Estimate formatting. Every price is normalised to "estimated Rs X to Y", even if the model returns a raw number.
- The address rule. Recycler-facing payloads are built from a whitelist in one module, and an assertion re-checks the payload before a call is placed. The address is included only once the pickup is
confirmed.
The waste path
When the classifier or the user points to e-waste or a battery, the agent does not offer a pickup. It explains that the item needs special handling, and on a yes it lists the nearest authorised drop-off points with distance, along with a reminder to remove personal data from devices. This is the part of the product that answers the lesser known centres problem directly: the household does not need to know that such a centre exists, since the agent brings it to the household.
What fought back
1. The model that would not fit in a Lambda.
The trained YOLOv8n checkpoint needs PyTorch and ultralytics, which are hundreds of megabytes and do not fit a Lambda zip. The fix was to export the model to ONNX and ship it with onnxruntime, numpy and Pillow in an arm64 layer of about 120 MB. The model file itself is small enough to be pulled from S3 on a cold start and cached for warm invocations.
2. A resize that quietly changed the answers.
Pillow's resize applies a low pass filter, which shifted the detector's confidences compared with the OpenCV resize used in training. Since a threshold decides what is returned, small shifts were enough to change results. The fallback now uses an OpenCV-equivalent bilinear resize written in numpy.
3. Training size must equal inference size.
The shipped weights were trained at 416 pixels, and running inference at 640 roughly halved recall without raising any error. The inference config now carries the training size, and a test checks it against the checkpoint.
4. Training data does not look like WhatsApp photos.
Part of the battery data is clean product photography, and some of its labels were machine generated. A model trained on this would be too sure of itself on blurry, badly lit household photos. For that reason hazardous classes need an extra confirmation when confidence is low, and Bedrock is the primary classifier while the local model acts as the safety net.
5. WhatsApp is impatient and a model call is slow.
Twilio expects a webhook reply within about 15 seconds, while a full agent turn with tools can take longer. Because the reply is sent through Twilio's REST API, the user still gets the answer, but the synchronous design would not survive heavy load. The better pattern is to acknowledge first and process through a queue.
6. The same message can arrive twice.
Duplicate deliveries are normal on messaging platforms, and two Lambdas writing the same conversation can corrupt its state. Conversation state therefore uses a version counter with a conditional write, so a concurrent invocation fails cleanly and retries once.
7. A model cannot be the only safety layer.
A system prompt can be argued with, rephrased around or simply ignored by a model on a bad turn. Thus the rules that matter were placed in the tool layer and the API, where a booking without a confirmation or an address leak cannot happen whatever the model says. Tests fail if the address rule is weakened.
What we upgraded along the way
- Leaflet to MapLibre GL JS with Amazon Location Service, with the map key injected at serve time from
.envso it is never committed.
- Bedrock only to Bedrock plus a local ONNX fallback, together with a rate-table fallback in S3 so pricing still answers when DynamoDB is empty or unavailable.
- WhatsApp only to WhatsApp plus voice. The inbound phone agent shares the agent core and conversation table, with voice users prefixed so they never collide with WhatsApp users.
- A WhatsApp users view on the dashboard, joining households with their conversation state and pickup count, with the session restored after a refresh.
- A smoke test against the deployed stack. Emulated AWS tests cover handlers, matching and the address rule, but Bedrock, Twilio media and live calls need real accounts, so
scripts/smoke_test.py --callchecks them end to end.
Limitations we are honest about
- Prices come from a rule table. The rate values are placeholders and there is no trained pricing model, so every number stays an estimate.
- Matching scans the recyclers table. This is fine for tens of recyclers, but a geohash index would be needed for a city.
- An unanswered call leaves a pickup in
offereduntil an admin returns it torequested.
- One open security gap. The image fetch follows HTTP redirects without re-validating the host. It is documented as an expected failure in the tests and must be closed before production.
- The frontends are static files and are not hosted by the stack.
What comes next
The future work would focus on reliability and reach. Moving the webhook to an acknowledge-and-queue pattern with SQS would remove the timeout risk and allow de-duplication. A verified, searchable directory of authorised e-waste centers can turn the drop-off answer into a real address with opening hours. More Indian languages would help the system reach regions where it is needed most, and a pricing model trained on real dealer transactions would replace the placeholder rate table. A short follow-up after each pickup to collect the actual weight and price would create that data. Hosting the dashboard on S3 and CloudFront would complete the deployment.
Closing thoughts
The technology was not the hardest part of EcoLink. The harder part was choosing the right interface for each person in the chain: a chat for the household, a phone call for the scrap dealer and a dashboard for the operator. Once these were fixed, serverless AWS made the rest manageable, since every piece could be built, tested and replaced on its own. The unorganized sector does not need to be replaced. It needs to be connected, and a message on WhatsApp is a good place to start.
Code and how to run it- Try it here
Bash
cp .env.example .env && pip install -r infra/requirements.txt # add AWS + Twilio valuespython scripts/put_secrets.py # Twilio credentials to Secrets Managercd infra && cdk bootstrap && cdk deploy && cd .. # deploys the whole stackpython scripts/seed.py && python scripts/smoke_test.py # demo data + end to end checkpython scripts/serve_dashboard.py # dashboard at localhost:8000/dashboard
Paste the
TwilioMessagingWebhookUrl output into the Twilio WhatsApp sandbox and send "hi". Full steps are in backend/README.md .EcoLink was built by Team (OffBeat). The CDK stack, the agent and the full infrastructure notes are in the project repository.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article