AWS Builder Center

Matsya AI: why we built an on-phone fishing assistant on AWS

Small-scale Indian fishermen still guess species and price by hand. Matsya AI turns a photo into species, weight, and market advice — shipped on Amplify, Lambda, Cognito, DynamoDB, S3, and EC2.

Why we made Matsya AI

India’s small-scale fishers often cannot name a species with certainty, weigh catch without a scale, or check live market and weather before they sell. That gap is money lost at the landing centre.
We built Matsya AI so a fisher can open a phone or the web app, photograph the catch, and get species, quality, a weight estimate, and advice in a regional language.

What the product does

Sign-in is Amazon Cognito. After login, the home screen is an agent: upload catch, open the map, view analytics, or type a question in a local language.
Catch photos go to a private S3 bucket through a short-lived presigned URL issued by our Lambda. A vision model returns species, health, estimated weight, and a rupee value. In a real run we identified Mullet, marked it healthy, and estimated ~0.05 kg with a market band in ₹.
The same agent answers in Hindi, Bengali, and other Indian languages, and can pin the fisher’s location on a map inside the chat. History and earnings (catches, groups, threads) are stored in DynamoDB. Analytics showed, for example, 10 catches logged, top species Mullet, 2 kg total.
Voice uses Amazon Polly (en-IN / hi-IN) with browser TTS for other languages. There is also an Expo Android client for camera and on-device TFLite when the network is poor.

How we shipped it on AWS (us-east-1)

We had about $100 in credits and roughly 20 days. A stack with API Gateway, an ALB, ECS, and SageMaker would have spent most of that on idle networking. This is what we actually run.
Web — AWS Amplify Hosting. Next.js 15 SSR, app name  Matsya-AI , branch  main , status Deployed.  NEXT_PUBLIC_*  URLs are baked in at build time.
API — one Lambda + Function URL. Function  Matsya-AI-Backend  in us-east-1, Node.js Express wrapped with  serverless-http , timeout 180 seconds. No API Gateway. The Function URL is public HTTPS. IAM on the execution role covers DynamoDB, S3, and Polly. Lambda overwrites  AWS_ACCESS_KEY_ID , so IAM user keys in function environment do not work.
Data — DynamoDB on-demand. Seven tables, all prefixed  matsya-ai-* : users, images, groups, chats, conversations, messages, memory. Partition keys are  userId ,  imageId ,  groupId ,  conversationId .
Photos — S3. Bucket  matsya-ai-fish-images  in us-east-1. The bucket stays private. The browser uploads with presigned PUT; the Lambda later reads the object for analysis.
Auth — Amazon Cognito. User pool in us-east-1 ( us-east-1_tMjqlAf1Q ). Web and mobile send the access token. The same JWT is verified on the Lambda and on the agent.
Agent — EC2 t2.micro. Chat streaming and fishing-spot scan use SSE. That cannot sit behind API Gateway or an ALB idle timeout. The agent is Python FastAPI + LangGraph in Docker on port 8080, with nginx on 80/443 and Let’s Encrypt on  {public-ip}.sslip.io . Security group opens 80, 443, 8080, and 22. Health check:  https://3.91.152.168.sslip.io/health  returns  {"status":"ok","service":"matsya-ai-agent"} . The Amplify site is HTTPS, so the agent must be HTTPS too — a plain  http://  agent URL is blocked as mixed content.
Vision models run on a Hugging Face Space. The LLM is Gemini on that EC2 instance. Both are called from AWS, not from SageMaker or Bedrock in the live deploy.
Typical 20-day cost at demo traffic is about $8–12, almost all  t2.micro  plus a public IPv4 address. Lambda, DynamoDB, S3, Cognito, Polly, and Amplify stayed near the free tier.

What broke (and what we learned)

Function URL CORS stacked with Express  cors()  sent two  Access-Control-Allow-Origin  headers. The browser rejected the response. Pick one CORS layer, not both.
The EC2 instance has no Elastic IP. Stop/start assigns a new public IP. The sslip.io name and certificate go stale, and Amplify must rebuild  NEXT_PUBLIC_AGENT_URL .
Changing Amplify environment variables without a new build does nothing, because Next.js inlines  NEXT_PUBLIC_*  at compile time. “Redeploy this version” is not enough.
Matsya AI is for the person on the boat: from a photo of the catch to a better selling decision.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article