AWS Builder Center
I built a voice-first civic agent on Amazon Bedrock that refuses to guess when a farmer asks for help

I built a voice-first civic agent on Amazon Bedrock that refuses to guess when a farmer asks for help

There are two kinds of "AI for public welfare," and both fail the people who need them most. The first is the hallucinatory chatbot. It reads a question, guesses an eligibility criterion from its pre-training weights, and confidently tells a 62-year-old widow she qualifies for a scheme she doesn't. She spends two days' wages traveling to a Mandal office only to be turned away at the desk. Production is where confident reasoning goes to die; in civic tech, confident hallucinations cause financial harm.

AWS Certified Cloud Practitioner | Aspiring Cloud Support Engineer
Built for the Bharat Builds Tour (First Commit).
There are two kinds of "AI for public welfare," and both fail the people who need them most.
The first is the hallucinatory chatbot. It reads a question, guesses an eligibility criterion from its pre-training weights, and confidently tells a 62-year-old widow she qualifies for a scheme she doesn't. She spends two days' wages traveling to a Mandal office only to be turned away at the desk. Production is where confident reasoning goes to die; in civic tech, confident hallucinations cause financial harm.
The second is the digital brochure bot. It refuses to risk a mistake, so it dumps a wall of text: 40 points, five nested criteria, and a PDF link. It is legally safe, and completely useless to someone holding a phone in a field who cannot read administrative English or formal Hindi.
I wanted to see if an assistant could have the first one's conversational speed with absolute, provable grounding. Not by begging the model with  "Please be accurate"  in a system prompt—you cannot prompt your way to a legal guarantee—but by anchoring the reasoning to strict vector boundaries in Amazon Bedrock Knowledge Bases and enforcing persona logic before the model ever drafts an answer.
That is NitiMitra.
  • Live system:  [https://main.d1lxt02jso6bg5.amplifyapp.com/](https://main.d1lxt02jso6bg5.amplifyapp.com/) 
  • Code:  [Insert GitHub Repo URL] 
  • Walkthrough:  [Insert YouTube Demo URL] 

What it actually does

A user taps the mic and speaks in their regional language (Telugu, Hindi, Kannada, Tamil, or English). NitiMitra runs a constrained cycle:
  1. Ingests vernacular audio locally: The browser handles speech transcription using native BCP-47 locale matching, passing raw vernacular text over HTTPS without audio-streaming overhead.
  1. Normalizes colloquial terms: Before calling vector search, it intercepts code-mixed dialect and maps informal loanwords and spoken acronyms to canonical scheme identifiers.
  1. Retrieves strictly bounded gazette chunks: Amazon Bedrock executes vector retrieval against official scheme circulars indexed in OpenSearch Serverless via Titan Text Embeddings v2.
  1. Applies persona evaluation: It checks the user's demographic state (age, landholding, occupation) against legal boundaries (e.g., minor exclusion rules, cultivable land requirements).
  1. Synthesizes audio on the client: It speaks the result back in the user's chosen language using client-side speech engines, then provides a zero-data checklist.

The part that matters: The grounding is structural, not polite

Anyone can tell an LLM: "Only answer from the documents." Anyone who has deployed an LLM knows it will still extrapolate when pressed.
In NitiMitra, that failure mode is blocked at the orchestration layer:
Plaintext
[Citizen Voice Input (English/Telugu/Hindi/etc.)]
│
▼ (Web Speech API)
[Amplify React Client] ── demographic state: {age, land_acres, category}
│
▼ HTTPS POST /chat
[AWS Lambda (Python 3.12)]
│
▼ boto3: retrieve_and_generate
┌───────────────────────────────────────────────────────────┐
│ Amazon Bedrock Knowledge Bases │
│ │
│ [ S3 Gazettes ] ──► [ Titan Text Embeddings v2 ] │
│ │ │
│ ▼ │
│ [ OpenSearch Serverless ] │
│ │ │
│ ▼ │
│ [ Claude 3.5 Sonnet ] │
│ (Strict Citation Only) │
└───────────────────────────────────────────────────────── ─┘

The Bedrock Knowledge Base configuration runs with explicit search boundaries:
Python
response = bedrock_agent_runtime.retrieve_and_generate(
input={'text': normalized_prompt},
retrieveAndGenerateConfiguration={
'type': 'KNOWLEDGE_BASE',
'knowledgeBaseConfiguration': {
'knowledgeBaseId': KB_ID,
'modelArn': 'arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20240620-v1:0',
'generationConfiguration': {
'guardrailConfiguration': {
'guardrailId': GUARDRAIL_ID,
'guardrailVersion': 'DRAFT'
}
}
}
}
)

If the retrieved chunks in OpenSearch Serverless do not contain direct evidence for the question, the system falls back to an explicit: "The official gazette does not specify this rule. Please verify with your local Panchayat officer." It does not guess.

Proving it reasons instead of dumping static brochures

Here is the problem with demoing a civic assistant: showing a single query about PM-KISAN proves nothing. Any chatbot can return a summary of PM-KISAN. It looks equally confident whether it actually parsed the user's context or just printed a canned template.
To test whether Niti Mitra evaluates rules or just pattern-matches, consider two users asking the exact same question: "Can I get PM-KISAN?"
Parameter
Case A: 16-Year-Old Student
Case B: 34-Year-Old Landless Laborer
Demographic Signal
Age: 16, Land: 3 Acres (Family)
Age: 34, Land: 0 Acres (Tenant)
Common LLM Failure
Says "Yes, you have 3 acres, here is the form."
Says "No, you are ineligible," and stops there.
What Niti Mitra Does
Flags minor status under the Family Unit clause.
Flags zero registered land revenue records.
Output Action
Clarifies that an adult parent must be the applicant.
Routes directly to MGNREGA and PM-SYM for laborers.
On Case A, the agent explains:
"Under the legal definition of PM-KISAN, an applicant must be an adult head of the family unit (husband, wife, and minor children). As a minor, you cannot apply independently, but your parents can apply if the land title is in their name."
On Case B, it doesn't leave the user at a dead end. It detects zero cultivable title records and redirects immediately to wage-earner protections.
Same scheme. Same prompt. Different personas, evaluated against statutory guidelines.

What went wrong (the useful part)

Almost all the painful debugging was not in the React UI, but in AWS IAM boundaries, vector synchronization, and linguistic edge cases.

1. The OpenSearch Serverless Data Access Trap

Deploying Bedrock Knowledge Bases via AWS SAM looked straightforward on paper. Lambda had full  bedrock:*  execution rights, but every call crashed:
Plaintext
botocore.exceptions.ClientError: An error occurred (AccessDeniedException)
when calling the RetrieveAndGenerate operation:
User: arn:aws:sts::xxx:assumed-role/LambdaRole is not authorized to perform: aoss:APIAccessAll

Bedrock Knowledge Bases orchestrate an OpenSearch Serverless collection in the background. Giving the Lambda execution role permissions to Bedrock was only half the equation. Bedrock's service principal ( bedrock.amazonaws.com ) also required an explicit OpenSearch Serverless Data Access Policy targeting the collection ARN:
JSON
[
{
"Rules": [
{
"Resource": ["collection/bedrock-knowledge-base-vector"],
"Permission": ["aoss:CreateCollectionItems", "aoss:UpdateCollectionItems", "aoss:DescribeIndex"],
"ResourceType": "collection"
}
],
"Principal": ["arn:aws:iam::ACCOUNT_ID:role/BedrockExecutionRole"]
}
]

Until that data access policy was mapped directly to the collection index, Bedrock had no authority to touch its own vector store.

2. The Phonetic Code-Mixing Drift

When a Telugu speaker uses voice search, they rarely speak formal literary Telugu. They code-mix:
"PM-KISAN ki apply cheyyadaniki ration card kavala?"
The browser's Speech Recognition engine frequently transcribed this as  "pm kisaan"  or split it phonetically into  "p m kisan" . When passed straight to Titan Text Embeddings v2, the vector distance drifted away from the official title: "Pradhan Mantri Kisan Samman Nidhi". The retrieval score dropped below the relevance threshold, returning zero chunks.
The fix was adding a pre-retrieval normalizer in Lambda that maps common colloquial abbreviations and phonetic spellings to canonical scheme entities before embedding lookup. Retrieval accuracy went from flaky to deterministic.

3. The 5-Second Voice Latency Trap

Our first implementation attempted to generate audio server-side using Amazon Polly, encoding the output as a Base64 string and returning it in the JSON payload.
Over rural 3G and 4G connections, this was unworkable. The round-trip—voice upload, Bedrock RAG retrieval, Polly synthesis, and base64 transmission—created a 5.8-second delay before the user heard a single sound.
We killed server-side synthesis entirely. Instead, Lambda returns clean, lightweight Markdown JSON, and the frontend hands the text directly to the client browser's native  window.speechSynthesis  API using localized BCP-47 identifiers ( te-IN ,  hi-IN ,  kn-IN ,  ta-IN ). The browser begins vocalizing the first sentence within 200 milliseconds of payload arrival.

What I'd tell you if you're building with Bedrock

  1. Do not use LLMs as databases. Using general knowledge for statutory welfare rules will burn you with subtle hallucinations. Use Bedrock Knowledge Bases to freeze your source of truth to official government PDFs.
  1. Move audio rendering to the edge. Don't stream audio bytes across mobile networks if the client's OS already has high-quality speech engines built in.
  1. Audit IAM from the resource perspective. When using managed RAG, remember that Bedrock acts as an assumed principal to OpenSearch Serverless. Both the IAM role and the collection's data access policy must be configured.
  1. Test the failure cases. Anyone can make an agent look smart on a clean query. The real test is whether it handles minors, landless workers, and missing documents without hallucinating.

Limitations (What it doesn't do)

  • No direct transactional filing: Niti Mitra guides users, generates checklists, and helps them track status. It does not file applications directly on government portals, which require CAPTCHAs and biometric Aadhaar authentication.
  • No PII collection: We deliberately do not store, process, or ask for identity numbers or bank credentials. All status lookups redirect users to official government portals.
  • Browser voice dependencies: Client-side speech synthesis depends on the operating system's installed regional voice packages; older mobile devices may fall back to default system engines.

Try it

  • 
Built with Amazon Bedrock, AWS Lambda, and AWS Amplify for the Bharat Builds Tour.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article