
Building Voca: An AI Voice Proxy with Strands Agents and Cedar
Voca is an AI Voice Proxy that helps users handle important calls when they are unavailable. We built it using Strands Agents SDK, Cedar policies, a local LLM, and a voice interaction layer to create a controlled AI agent for real-world conversations.
Building Voca: An AI Voice Proxy with Strands Agents and Cedar
Introduction
Have you ever missed an important call because you were in class, in a meeting, driving, or simply unavailable?
We built Voca, an AI Voice Proxy designed to help users handle important conversations when they cannot answer a call themselves.
Instead of simply letting the call go unanswered, Voca allows users to create an AI voice agent with specific instructions and permissions. The agent can handle routine conversations, collect relevant information, and provide a summary for the user afterwards.
The Problem
Many phone calls are time-sensitive but do not always require the user's direct involvement.
For example:
- A delivery agent may call to confirm delivery details.
- A recruiter may call regarding a job opportunity.
- A service provider may need basic information from the customer.
Missing these calls can result in missed opportunities, delayed deliveries, or unnecessary callbacks.
Our goal with Voca was to create an AI voice agent that can handle these conversations while keeping the user in control.
Our Solution
The basic Voca workflow is:
Create Agent → Configure Instructions → Activate Agent → Handle Conversation → Generate Summary → Review on Dashboard
The user can define information and rules for their agent. During a conversation, the agent follows those instructions and handles the interaction accordingly.
After the conversation, the user can review information such as the caller, duration, summary, key points, and follow-up actions.
How We Built Voca
Our application consists of several layers:
- React for the frontend interface
- FastAPI for the backend
- Strands Agents SDK for the AI agent orchestration
- Ollama with a local LLM for local model execution
- Cedar for agent permissions and policies
- Browser-based voice interaction
- SQLite for initial local data storage
The overall flow looks like this:
User
↓
React Frontend
↓
FastAPI Backend
↓
Strands Agent
↓
Local LLM
↓
Tools and Policies
↓
Voice Conversation
↓
Call Summary
↓
Dashboard
Using Strands Agents SDK
The AI agent is the core of Voca.
We used the Strands Agents SDK to structure the agent and its interaction flow. The agent receives the user's configured instructions and processes the conversation before generating an appropriate response.
For example, a recruitment-focused agent can be instructed to collect:
- Company name
- Job role
- Recruiter's details
- Callback information
The agent can then summarize the conversation so that the user can quickly understand what happened.
Using Cedar for Agent Permissions
An important part of Voca is controlling what an AI agent is allowed to do.
We used Cedar policies to define boundaries for different types of conversations.
For example, a delivery agent may be allowed to:
- Confirm delivery details
- Provide delivery instructions
But it should not be allowed to:
- Share sensitive information
- Provide an OTP
- Make a payment
Similarly, a recruitment agent can collect job-related information but should not make commitments on behalf of the user.
This policy-based approach helps us keep the AI agent within clearly defined boundaries.
Voice Interaction
Since Voca is a voice proxy, voice interaction is an important part of the experience.
Our frontend provides a voice-focused interface where users can see the current call state and voice activity. The voice layer is designed to support natural interaction between the caller and the AI agent.
The goal is to make the experience feel like a real assistant rather than a traditional text-based chatbot.
What We Learned
Building Voca gave our team practical experience with AI agent architecture, voice interaction, backend APIs, frontend integration, and policy-based permissions.
We also learned that building an AI agent is not only about generating responses. It is equally important to define what the agent is allowed to do and keep its actions within the user's instructions.
Working with Strands Agents SDK and Cedar helped us explore these ideas while building a practical application.
Future Improvements
There are several areas we would like to explore further:
- More advanced voice conversation handling
- Better real-time call processing
- More specialized agent types
- Improved call summaries
- Additional policy templates
- Production cloud deployment
Conclusion
Voca explores how AI voice agents can help people manage important conversations when they are unavailable.
By combining an AI agent with voice interaction and explicit permission policies, we aim to make automated call handling more useful while keeping the user in control.
This project was our exploration of building a practical AI voice application using AWS open-source technologies such as Strands Agents SDK and Cedar.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article