Building a voice weather agent on AWS — and what we did when Bedrock said no
Weather Buddy is a voice-first, AI-powered weather dashboard that lets you ask about the weather using natural language — by speaking or typing — in any language. It combines a conversational AI agent with a rich, real-time weather dashboard that adapts its entire interface to the language you speak.
We started the First Commit hackathon with the idea of building a weather assistant. We eventually came back to that idea with one change that made it worth building: instead of a fixed pipeline, we put an agent in the middle.
What it does
wx-agent is a voice-first, multilingual weather assistant. You speak a question in your language, it answers out loud in the same language.
The interesting part isn't the weather data — anyone can hit OpenWeatherMap. It's that there's no hardcoded flow. Most weather apps parse your query into a slot ("city: Chennai, intent: current"), call one endpoint, and render a card. Ours hands the question to a Strands agent with four tools — current weather, forecast, alerts, and an activity advisor — and lets the model decide which to call and in what order.
That sounds like a small distinction until you ask it something a fixed pipeline can't handle. "Compare Mumbai and Bengaluru" triggers two separate lookups and a synthesised answer. "Is it a good evening for a run?" pulls current conditions and reasons about them rather than reciting numbers. Follow-ups work because conversation history travels with each request.
The flow
Browser (Web Speech API) → API Gateway → Lambda (agent-handler) → Comprehend: detect language → Translate: → English → Strands agent (LLM) ⇄ tools ⇄ OpenWeatherMap → Translate: → user's language → Lambda (tts-handler) → Polly → MP3 → Browser plays audio + renders weather cardThe agent always reasons in English. Translation happens at the edges, in and out. That keeps the tool descriptions and system prompt in one language and stops multilingual support from multiplying the prompt-engineering surface.
Two Lambdas, not one: the agent calls the TTS function directly via boto3 rather than over HTTP. It's internal, so it doesn't need to sit behind API Gateway, and it doesn't need CORS headers — a small thing that's easy to get wrong when you copy-paste a handler.
AWS services in play: Lambda, API Gateway, Comprehend, Translate, Polly, Amplify for the frontend, SAM to define all of it, and — not by choice — EC2.
Where it went wrong
The plan was Bedrock with Claude Haiku. That is not what happened.
First invoke came back with
ResourceNotFoundException: Model use case details have not been submitted for this account . Fine, fill out the form. Then Nova Lite threw ThrottlingException: Too many tokens per day after a single test call. So we went digging through Service Quotas, and the picture was bleak:On-demand model inference requests per minute for Amazon Nova Lite 0.0On-demand model inference tokens per minute for Amazon Nova Lite 0.0Model invocation max tokens per day for Amazon Nova Lite 0.0Zero across the board. We eventually listed every Bedrock quota on the account with a non-zero value and found exactly two usable models — both AI21 Jamba, at 1 request per minute.
The root cause turned out to be the payment method. The account was set up with UPI autopay, and some AWS services only unlock when a card is on file. Nobody tells you this; you just get quotas of zero and error messages that point in other directions.
We lost most of an evening to that. The lesson isn't really "use a card" — it's that when three different services fail in three different ways, stop debugging the third one and go look at the account.
The pivot
With Bedrock closed, we self-hosted. A
g4dn.xlarge with the Deep Learning AMI, Ollama installed, Qwen2.5-7B-Instruct pulled down. Ollama exposes an OpenAI-compatible endpoint, so nothing upstream had to care where the model lived.Launching the instance hit the same class of wall —
vCPU limit of 0 for the G instance family — but that one is a normal quota request and came through.Two gotchas worth writing down. Ollama's systemd service binds to
127.0.0.1 by default, so it's invisible from outside the box until you override OLLAMA_HOST=0.0.0.0:11434 . And once we wired it through LiteLLM, the config took three tries to get right:LLM_MODEL_ID=openai/qwen2.5:7b-instruct # provider prefix is requiredLLM_BASE_URL=http://<ip>:11434/v1 # stop at /v1, not /chat/completionsLLM_API_KEY=ollama # ignored by the server, required by the clientEvery one of those produced a different unhelpful stack trace.
Going through LiteLLM rather than a Bedrock client turned out to be the accidental win. The model is three environment variables. We can point at Ollama today and Bedrock tomorrow, if AWS ever lets us in, without touching a line of agent code.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article