
ASEANtic Assist: A Multilingual Virtual Assistant Solution using Strands Agents SDK with AWS Bedrock, Transcribe, and SageMaker
Transform language barriers into business opportunities with ASEANtic Assist. This multilingual virtual assistant solution agentically handles queries in ASEAN languages (and can be extended to many more!) using AWS services and Strands Agents SDK.
The ASEAN Challenge
In ASEAN markets, language barriers create significant operational friction. At an airport, customer service agents spend valuable time answering basic directional queries in multiple languages, while more complex issues wait. Similar challenges exist across sectors - retail staff struggle with international customers, and manufacturing teams face delays coordinating with overseas suppliers. Traditional solutions like maintaining multilingual staff or using separate translation services are costly and inefficient.
Solution Overview
ASEANtic Assist demonstrates a horizontal solution using AWS services to enable real-time multilingual communication. It supports both text and voice interactions across Thai, Vietnamese, and Bahasa Indonesia. It has knowledge base retrieval, web search, translation and multi-modal interpretation capabilities. This solution can be extended to 30+ other supported languages, and for different vertical use cases in most industries.
The core architecture leverages:
- Amazon Bedrock for natural language processing
- Amazon Transcribe for speech-to-text
- Custom SageMaker endpoint deploying multilingual text-to-speech solution using open source seamless-m4t-v2-large model
- Bedrock Knowledge Base for multi-modal document understanding
- Bedrock Guardrails for multilingual content filtering guardrails
- Amazon S3 for knowledge base documents, multi-modal content storage
- Strands Agents SDK for agentic multi-tool architecture
The image below is the detailed architecture diagram of the ASEANtic Assist solution.

Key Capabilities
ASEANtic Assist is created as a horizontal virtual assistant that can be adopted in any vertical with minimal enhancements (prompt engineering, knowledge base documents). It therefore has a few key capabilities that is applicable across many industries within ASEAN.
- Multilingual Understanding: Real-time processing of Thai, Vietnamese, and Bahasa Indonesia in both text and voice
- Knowledge Integration: Retrieves and interprets information from uploaded documents
- Web Search Capability: Real-time web search capabilities for up-to-date information
- Multi-modal Interpretation: Analyze and interpret charts, schematic diagrams, and visual content
- Agentic Actions: Performs transcription, translation, and summarization automatically
- Conversational Interface: Streaming voice input processing with natural conversation-like outputs
Watch the demo showcasing these capabilities here!
Demo Scenarios Showcased in the Video
Travel Itinerary
- Query: "What is my Day 5 itinerary?"
- System retrieves relevant itinerary from knowledge base
- Demonstrates: Knowledge base retrieval across multiple documents
Multi-modal Asset Interpretation
- Query: "Based on the shopping street map, what is the road between Zen and Gaysorn Plaza?"
- System interprets schematic diagram from knowledge base
- Demonstrates: Interpretation of multi-modal asset schematic diagram in knowledge base
Agentic Live Transcription
- Voice query: "Can you transcribe something for me"
- System executes agentic action of live transcription tool to transcribe speech, generates both audio and text outputs
- Demonstrates: Real-time transcription + multi-format audio and text output
Cross-language Communication
- Voice query in Thai: “What is my Day 5 itinerary?”
- System detects language, provides Thai response although documents in knowledge base are in English
- For the voice query in English: “Can you tell me what is my day 5 itinerary in Thai?”, the virtual assistant answers in English to address the speaker, while also provides the translated itinerary in spoken Thai
- Voice queries in Vietnamese and Bahasa Indonesia with responses in the detected language
- Demonstrates: Language detection + contextual translation + voice synthesis in user’s language
Agentic Summarization with Web Search
- Voice query: “Can you summarize the book Atomic Habits by James Clear”
- System executes agentic web search to gather information about the book
- Demonstrates: Web search capabilities for summarization purposes
Multilingual Guardrail Content Filtering
- Query: “I hate you” in both English and Thai
- System detects hateful content in both languages and blocks inputs
- Demonstrates: Multilingual guardrails with content filtering checking inputs and outputs
Other Features
- Intermediate tool names and outputs shown clearly to earn user’s trust that answers provided by agent is grounded
- Intermediate voice output of assistant messages such as “Let me search for a concise summary...” for conversationality
- Metrics of each tool call at the end for observability of number of tokens used, cost of agent call, execution time and tools used
Example of a Real-World Application
Taking an airport as a specific vertical implementation:
- Current Challenge: Customer service agents spend most of time answering basic directional queries in multiple languages
- Solution Implementation:
- Deployment of voice-enabled kiosks across terminals
- Integration with airport map database
- Real-time wayfinding in native languages
- Impact:
- Reduced wait times
- 24/7 multilingual support
- Scalable during peak travel seasons
This can be implemented across other industries. Another example is the manufacturing sector - in cases where there are international vendors that require conversing in multiple languages. Documents regarding the product catalogue can added to the knowledge base to support customer queries, including more complicated quotes such as bulk discounts.
Future Scope
ASEANtic Assist is a proof of concept that tackles multilingual barriers for generic capabilities applicable to, and can be customized for various industries. This solution has therefore been designed to be extensible, with the following possible improvements and enhancements that can be implemented when adopting to a specific vertical:
- 30+ additional languages that is supported by both Transcribe and text-to-speech model seamless-m4t-v2-large
- Supported languages: Modern Standard Arabic, Bengali, Catalan, Czech, Mandarin Chinese, Welsh, Danish, German, Estonian, Finnish, French, Hindi, Italian, Japanese, Korean, Maltese, Dutch, Western Persian/Farsi, Polish, Portuguese, Romanian, Russian, Slovak, Spanish, Swedish, Swahili, Telugu, Tagalog, Turkish, Ukrainian, Northern Uzbek
- Agentic multi-tool architecture enables addition of new features through manual creation and addition of tools to the orchestrator, or for generic tools we can leverage Strands Agents’ 40 existing tools out of the box
- Custom vertical adaptations can be executed minimally through prompt engineering, uploading relevant knowledge base documents
Conclusion
ASEANtic Assist is a horizontal solution with foundational multilingual, knowledge base retrieval, web search capabilities that can be leveraged to build targeted vertical solutions across industries. The extensible architecture, built on AWS services and Strands Agents SDK, allows organizations to adapt the solution with minimal effort - whether it's adding new languages from the supported 30+ languages, customizing the knowledge base with industry-specific documents, or introducing new tools beyond the current capabilities.
This means businesses can start addressing their multilingual challenges with a proven architecture, while maintaining the flexibility to evolve the solution as their needs grow. For example, an airport can begin with basic wayfinding assistance in multiple languages, then expand to flight information and retail navigation, all while maintaining the same core architecture. A manufacturer can start with supplier communication tools, then extend to technical documentation translation and quality control processes.
Organizations no longer need to be restricted by language barriers when building their intelligent systems or maintain separate systems for different languages. By leveraging ASEANtic Assist's architecture, businesses can rapidly deploy industry-specific generative AI solutions that transform language barriers into opportunities for enhanced customer experience and operational efficiency.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article