Beyond Vibe Coding: Why Real AI Products Need Real Engineering
Building a real-time AI voice interview system taught us that the hardest part of shipping AI isn't the AI.
There's a popular narrative in tech right now: cool startups ship cool products with vibe coding. Tools like Lovable, Bolt, and v0 let you describe what you want and get a working app in minutes. And honestly? For a lot of products, that works.
But there's a category of product where it doesn't. Where the gap between "demo" and "production" isn't a weekend of polish - it's months of engineering. Real-time AI voice systems live in that category.
We built JuntoAI Kinetic - an AI Chief of Staff that conducts live, speech-to-speech career interviews using Amazon Nova Sonic. A 15-minute conversation that extracts your career DNA and generates a strategic executive profile. No text-to-speech pipeline. No transcribe-then-respond. Direct voice-to-voice, sub-second latency, with barge-in support so you can interrupt the AI mid-sentence like a real conversation.
Here's what it actually took to ship it.
The Iceberg Under the Demo
When people see JuntoAI Kinetic in action, they see a voice conversation. Clean UI. Real-time transcript. A progress bar moving through interview pillars. It looks simple.
Under the surface, there are 28 backend services coordinating in real time:
- A bidirectional streaming engine (
bedrockSonic.js) managing an event-based protocol with Amazon Bedrock — not a simple API call, but a sequenced dance ofsessionStart → promptStart → contentStart → textInput → audioInput → audioOutput → contentEnd → sessionEnd - A finite state machine (
fsmController.js) tracking interview progression across three pillars (Values, Experience, Dream) with automatic transitions - A stream manager (
streamManager.js) that proactively recreates Bedrock connections every 8 minutes before AWS enforces a timeout — replaying full conversation history so the AI doesn't lose context - A metric auditor (
metricAuditor.js) running real-time "pushback loops" — when someone says "I improved sales," the AI fires back: "Give me the percentage and the timeframe" - A pillar detector (
pillarDetector.js) using NLP to determine when a topic is sufficiently covered and it's time to move on - A closing detector (
closingDetector.js) that recognizes when the interview is naturally wrapping up - A persona manager (
personaManager.js) that builds dynamic system prompts following AWS Nova 2 best practices — single one-shot examples instead of phrase lists, because Nova is highly sensitive to suggested phrases and will overuse them
None of this is something you describe in a prompt and get back from a code generation tool.
The Infrastructure Nobody Talks About
Here's what the Terraform directory looks like for Kinetic alone:
1
2
3
4
5
6
7
8
9
10
11
12
infra/modules/core/
├── ecs-careergraph.tf # ECS Fargate service definition
├── cloudfront-careergraph.tf # CDN distribution
├── s3-careergraph-sessions.tf # Session storage with tiered TTLs
├── dynamodb-careergraph.tf # Single-table design for session data
├── dynamodb-careergraph-users.tf # User metadata
├── codepipeline-careergraph-service.tf # Automated deployments
├── careergraph-monitoring.tf # CloudWatch dashboards
├── careergraph-error-alerting.tf # Error alerting
├── ecr-careergraph.tf # Container registry
├── cv-evaluator.tf # CV analysis Lambda
└── ses.tf # Email deliveryThat's 11 Terraform files just for one service. The full infrastructure has 45+ Terraform modules managing ECS clusters, DynamoDB tables, S3 buckets with lifecycle policies, CloudFront distributions, CodePipelines, Lambda functions, CloudWatch alarms, IAM roles, and service discovery via Cloud Map.
You can't vibe code infrastructure. You can't prompt your way to a properly configured Application Load Balancer that routes WebSocket connections to ECS Fargate containers with health checks, auto-scaling from 1 to 5 instances, and blue-green deployments.
The Audio Pipeline Problem
Real-time voice AI has a problem that text-based AI doesn't: audio is continuous, bidirectional, and unforgiving.
Our WebSocket handler (
websocket-handler.js — 3,334 lines) manages:- Binary and JSON messages on the same connection: Node.js receives Buffer objects that could be either JSON events or raw PCM audio. You have to try parsing as JSON first, and if that fails, treat it as audio.
- Different sample rates: Input audio is 16kHz, output is 24kHz. Miss this and your playback sounds like a chipmunk or a demon.
- Heartbeat management: WebSocket connections die silently. We run application-level ping/pong every 30 seconds on top of the protocol-level heartbeat.
- Session reconnection: If a connection drops mid-interview, we rehydrate the session state, recalculate remaining time, and resume without losing context.
- Hard close enforcement: Sessions have a strict 15-minute limit for cost control (~$0.10 per session with Nova Sonic). The system sends a soft close warning at 13:30, a final question at 14:00, and terminates at 15:00.
On the frontend, audio playback uses
AudioWorklet for glitch-free streaming. The AudioStream component handles echo cancellation, noise suppression, and real-time visualization. The VoiceInterface component coordinates microphone state, AI response playback, barge-in detection, and transcript display - all simultaneously.Testing: Not Optional, Not Simple
Our test suite for the CareerGraph service alone includes:
bedrockSonic.test.js- Tests for the bidirectional streaming enginefsmController.test.js- State machine transition testsstreamManager.test.js- Stream recreation and 8-minute safety check testssessionManager.test.js- Session lifecycle and reconnection testspersonaManager.test.js- System prompt generation testsdocumentAI.test.js- CV parsing testsprofileGeneratorV2.test.js- Career graph generation testsfsmStreamManager.integration.test.js- Integration tests for FSM + stream coordinationcareergraph-time-beacon-bug.test.js- Regression test for a specific timing bug
We run Vitest for unit tests, Jest for integration tests, and Playwright for E2E. We have separate CI workflows with path-based triggers - CareerGraph tests only run when CareerGraph code changes. Coverage target is 70% across lines, functions, branches, and statements.
This isn't testing theater. When you're dealing with real-time audio streams, event sequencing, and state machines, a single off-by-one error in your pillar transition logic means the AI asks about your dream job while you're still talking about your first internship.
CI/CD: The Boring Part That Matters Most
Our deployment pipeline:
- Push to
master→ GitHub Actions detects which services changed via path filtering - Changed services trigger their specific AWS CodePipeline
- CodeBuild runs Docker multi-stage builds
- Images push to ECR
- ECS performs blue-green deployment with health checks
- CloudWatch monitors for errors post-deployment
We have 5 GitHub Actions workflows, each scoped to specific paths. The CareerGraph workflow only triggers on changes to
frontend/careergraph-app/** or backend/careergraph-service/**. No wasted CI minutes. No testing services that didn't change.Infrastructure changes go through Terragrunt with mandatory
plan before apply. State is locked in DynamoDB to prevent concurrent modifications. Every resource is tagged with Environment, Project, and ManagedBy.This is the kind of engineering discipline that separates "it works on my machine" from "it works at 3 AM when 200 people are doing simultaneous voice interviews."
The Storage Architecture Nobody Sees
We use a hybrid DynamoDB + S3 storage strategy optimized for cost at scale:
- DynamoDB for small, frequently queried data: session metadata, user status, pillar transcripts, state machine events
- S3 for large, infrequently accessed data: career graph JSON (100KB+), generated PDFs, uploaded CVs, audio chunks
Files move between S3 folders as users progress:
anonymous/ → identified/ → with-profile/, each with different TTL policies (7 days, 7 days, 90 days). This progressive identification pattern handles 100,000+ users/day with linear cost scaling.The cost difference? DynamoDB at $0.25/GB/month vs S3 at $0.023/GB/month. At scale, that's a 92% storage cost reduction.
So What's the Point?
The point isn't that vibe coding is bad. It's great for landing pages, CRUD apps, dashboards, and prototypes. The tools are genuinely impressive and they're making software development more accessible.
But there's a class of product - real-time AI voice, agent-to-agent orchestration, bidirectional streaming, production-grade infrastructure - where the engineering challenge isn't "make it work." It's "make it work reliably, at scale, at 3 AM, for thousands of concurrent users, within a cost budget of $0.15 per session, with graceful degradation when AWS has a hiccup."
That requires:
- Understanding event-based streaming protocols at the byte level
- Building state machines that handle every edge case (user disconnects mid-sentence, stream times out at minute 7:59, user interrupts while the AI is mid-word)
- Infrastructure as code with proper state management, multi-environment deployments, and automated rollbacks
- A test suite that catches timing bugs, race conditions, and audio format mismatches
- Cost optimization that thinks about storage tiers, compute scaling, and per-session budgets
The engineers building these systems? Their job looks different today. They're collaborating with AI tools, using them for code generation, testing, and documentation. But they're making architectural decisions, debugging production issues at the protocol level, and designing systems that handle failure gracefully.
That's not vibe coding.
That's software engineering.
And it's more important than ever.
That's software engineering.
And it's more important than ever.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article