Matsya AI: why we built an on-phone fishing assistant on AWS
Small-scale Indian fishermen still guess species and price by hand. Matsya AI turns a photo into species, weight, and market advice — shipped on Amplify, Lambda, Cognito, DynamoDB, S3, and EC2.
Why we made Matsya AI
India’s small-scale fishers often cannot name a species with certainty, weigh catch without a scale, or check live market and weather before they sell. That gap is money lost at the landing centre.
We built Matsya AI so a fisher can open a phone or the web app, photograph the catch, and get species, quality, a weight estimate, and advice in a regional language.
What the product does
Sign-in is Amazon Cognito. After login, the home screen is an agent: upload catch, open the map, view analytics, or type a question in a local language.
Catch photos go to a private S3 bucket through a short-lived presigned URL issued by our Lambda. A vision model returns species, health, estimated weight, and a rupee value. In a real run we identified Mullet, marked it healthy, and estimated ~0.05 kg with a market band in ₹.
The same agent answers in Hindi, Bengali, and other Indian languages, and can pin the fisher’s location on a map inside the chat. History and earnings (catches, groups, threads) are stored in DynamoDB. Analytics showed, for example, 10 catches logged, top species Mullet, 2 kg total.
Voice uses Amazon Polly (en-IN / hi-IN) with browser TTS for other languages. There is also an Expo Android client for camera and on-device TFLite when the network is poor.
How we shipped it on AWS (us-east-1)
We had about $100 in credits and roughly 20 days. A stack with API Gateway, an ALB, ECS, and SageMaker would have spent most of that on idle networking. This is what we actually run.
Web — AWS Amplify Hosting. Next.js 15 SSR, app name
Matsya-AI , branch main , status Deployed. NEXT_PUBLIC_* URLs are baked in at build time.API — one Lambda + Function URL. Function
Matsya-AI-Backend in us-east-1, Node.js Express wrapped with serverless-http , timeout 180 seconds. No API Gateway. The Function URL is public HTTPS. IAM on the execution role covers DynamoDB, S3, and Polly. Lambda overwrites AWS_ACCESS_KEY_ID , so IAM user keys in function environment do not work.Data — DynamoDB on-demand. Seven tables, all prefixed
matsya-ai-* : users, images, groups, chats, conversations, messages, memory. Partition keys are userId , imageId , groupId , conversationId .Photos — S3. Bucket
matsya-ai-fish-images in us-east-1. The bucket stays private. The browser uploads with presigned PUT; the Lambda later reads the object for analysis.Auth — Amazon Cognito. User pool in us-east-1 (
us-east-1_tMjqlAf1Q ). Web and mobile send the access token. The same JWT is verified on the Lambda and on the agent.Agent — EC2 t2.micro. Chat streaming and fishing-spot scan use SSE. That cannot sit behind API Gateway or an ALB idle timeout. The agent is Python FastAPI + LangGraph in Docker on port 8080, with nginx on 80/443 and Let’s Encrypt on
{public-ip}.sslip.io . Security group opens 80, 443, 8080, and 22. Health check: https://3.91.152.168.sslip.io/health returns {"status":"ok","service":"matsya-ai-agent"} . The Amplify site is HTTPS, so the agent must be HTTPS too — a plain http:// agent URL is blocked as mixed content.Vision models run on a Hugging Face Space. The LLM is Gemini on that EC2 instance. Both are called from AWS, not from SageMaker or Bedrock in the live deploy.
Typical 20-day cost at demo traffic is about $8–12, almost all
t2.micro plus a public IPv4 address. Lambda, DynamoDB, S3, Cognito, Polly, and Amplify stayed near the free tier.What broke (and what we learned)
Function URL CORS stacked with Express
cors() sent two Access-Control-Allow-Origin headers. The browser rejected the response. Pick one CORS layer, not both.The EC2 instance has no Elastic IP. Stop/start assigns a new public IP. The sslip.io name and certificate go stale, and Amplify must rebuild
NEXT_PUBLIC_AGENT_URL .Changing Amplify environment variables without a new build does nothing, because Next.js inlines
NEXT_PUBLIC_* at compile time. “Redeploy this version” is not enough.Matsya AI is for the person on the boat: from a photo of the catch to a better selling decision.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article