AIdeas: WhisprAI
Learning a new language is about more than memorizing vocabulary and grammar, it’s about confidently navigating real conversations. Yet many language learners struggle when tone, cultural cues, and social expectations. WhisprAI bridges this gap by turning everyday conversations into personalized learning opportunities. In this article, we introduce WhisprAI, explain why conversational confidence is a critical challenge for over 1.5 billion language learners worldwide, and share how we built it.
App Category: Daily Life Enhancement
Team Members: nandinik@, menakaa@
Imagine moving to a foreign country where you understand the language, but you still hesitate every time you speak.
Not because you don’t know the words...
Not because you don’t know the grammar...
Not because you lack the ability...
Our Vision
The hardest part of learning a new language is not memorizing vocabulary or grammar rules. The real challenge begins when you have to speak in a real conversation. Even learners who are “textbook fluent” often hesitate when navigating tone, pronunciation, or unspoken cultural expectations.
WhisprAI is designed to bridge this gap.
WhisprAI turns everyday conversations into a personalized coach for language and culture. By analyzing real-world exchanges, the system identifies patterns in pronunciation, fluency, and tone, and delivers insights through an interactive dashboard after each interaction. Users can also receive optional real-time nudges during conversations to help adjust phrasing, pronunciation, or delivery.
The platform also allows users to practice high-stakes situations,such as interviews, networking conversations, or classroom discussions, through an AI-powered voice interface that simulates realistic dialogue. These practice scenarios allow learners to build confidence before plunging into real conversations.
Why This Matters
More than 1.5 billion people around the world are learning a new language, including over 300 million international migrants and students who must quickly adapt to new social and professional environments.
While many language learning tools focus on grammar and vocabulary, far fewer help learners navigate the real challenges of communication including interpreting tone, understanding social cues, and responding naturally in conversation. This gap often creates what researchers describe as linguistic anxiety, where the fear of making mistakes reduces confidence and cognitive performance during interactions.
When these challenges persist, learners may begin to avoid conversations altogether. This experience, sometimes described as “social ghosting,” can lead to missed opportunities for connection, collaboration, and professional growth.
Blending language learning and daily life, WhisprAI transforms everyday communication into a continuous learning experience where users can practice, reflect, and improve over time.
How We Built This
Backend:
We built WhisprAI's backend entirely on AWS with Kiro as our development environment. We used a DynamoDB database with four tables, users, conversations, feedback, and progress metrics, and used on-demand billing to stay within the free tier.
Then, we built 13 Lambda functions in Node.js covering every core feature: user management, conversation storage, progress tracking, audio transcription via Google Web API, and AI-powered cultural analysis and practice simulations via Amazon Bedrock using Claude Sonnet 4.6.
Each Lambda is exposed through API Gateway with 12 REST endpoints, all secured through Amazon Cognito authentication. Audio recordings are uploaded directly to S3 via presigned URLs, which triggers the transcription pipeline—Transcribe converts speech to text, which then gets analyzed by Bedrock, and the feedback is saved in DynamoDB.
We set up CORS on every endpoint so the Next.js frontend can communicate with the API from the browser, and configured AWS Budgets to monitor spending throughout development. The entire backend was built and deployed incrementally using the AWS CLI, with Kiro helping us debug Lambda functions, fix CORS issues, troubleshoot API Gateway configurations, and iterate on the Bedrock prompts for cultural analysis.
Frontend:
We built the frontend in Next.js 15 with TypeScript and React, using Tailwind CSS for styling. The app has five main pages: a landing page, conversations list with audio upload, individual conversation detail views with AI analysis, a live practice simulator, and user settings. We implemented client-side state management with React Context for authentication (Amazon Cognito integration) and conversations, storing audio files locally in IndexedDB and metadata in localStorage for offline access. The conversation detail page displays cultural analysis from Bedrock including pronunciation, fluency, and tone scores, cultural context with social norms, and coaching cues. We built custom components for audio playback, file uploads with drag-and-drop, and manual transcript input for testing.
Kiro helped us scaffold the Next.js project structure, debug React state management issues, implement the Cognito authentication flow with email verification, fix layout bugs in the conversation analysis cards, and integrate the frontend API client with all 12 backend endpoints.
Demo
Watch a demo of our app here: https://drive.google.com/file/d/1tvG2aLWnchtsEOR8YQhQEQdaz77H5pgs/view?usp=sharing !
What We Learned
Building WhisprAI taught us lessons both about designing a language learning experience and about building a full-stack AI application.
On the product side, we realized that language learning is not just about grammar or vocabulary. Real conversations involve tone, confidence, pacing, and cultural context. Many existing tools focus on structured exercises, but learners often struggle most in unscripted conversations. This reinforced our belief that tools should focus on helping learners reflect on real interactions rather than just practicing isolated exercises.
From a technical perspective, this project gave us hands-on experience building a full-stack AI system. We learned how to design and connect backend services using AWS tools like Lambda, API Gateway, and DynamoDB to manage user data and conversation history. On the frontend, we built a React-based interface that allows users to move smoothly through onboarding, conversation analysis, and feedback.
One of our biggest technical takeaways was how tools like Kiro can significantly accelerate development. Tasks that would normally take many hours—such as scaffolding backend services, connecting APIs, and setting up infrastructure—could be generated and iterated on quickly. This allowed us to focus more on designing the learning experience and refining the AI feedback rather than spending most of our time on boilerplate setup.
We also learned how to integrate multiple AI services into a single pipeline. By combining speech recognition, language analysis, and speech generation, we were able to create a system that analyzes conversations and produces meaningful feedback for users.
Overall, the project helped us understand both the technical challenges of building AI-powered applications and the human-centered design considerations needed to make them useful in everyday life.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article