
GhostQueue: Turning Process Abandonment into Actionable Intelligence with AWS
How I built and deployed GhostQueue, a Human Process Abandonment Intelligence platform that uses operational analytics, Ghost Replay, AI investigation, and what-if simulation to understand where people abandon processes.
GhostQueue: Turning Process Abandonment into Actionable Intelligence with AWS
Don't just measure who completed the process. Discover who disappeared before they could.
Most operational dashboards tell us how many people completed a process and how many abandoned it.
But an abandonment number alone doesn't tell us:
- Where did people disappear?
- What happened immediately before they left?
- Which queue or stage is responsible for the concentration?
- What operational change could reduce abandonment?
That was the problem I wanted to explore while building GhostQueue for the First Commit hackathon.
GhostQueue is a Human Process Abandonment Intelligence platform designed to turn abandonment data into an investigation workflow.
The Problem
Consider a contact center.
A dashboard might tell an operations team:
- 49,121 interactions were offered
- 48,211 were completed
- 910 were abandoned
- 1.85% Ghost Rate
But what should the team do next?
Which queue has the problem?
At what time does abandonment concentrate?
Where did an individual interaction disappear?
Would additional staffing change the situation?
GhostQueue was designed around these questions.
Instead of treating abandonment as a final KPI, it investigates the journey behind it.
What GhostQueue Does
1. Abandonment Analytics
GhostQueue analyzes CSV and JSON datasets and automatically maps available fields into a canonical schema.
It calculates metrics such as:
- Total interactions offered
- Completed interactions
- Abandoned interactions
- Ghost Rate
- Average wait time
- Time-based abandonment patterns
- Queue-level performance
The system also detects what capabilities are supported by the uploaded dataset rather than assuming every dataset contains every field.
This makes the analytics engine adaptable to different operational datasets.
2. Ghost Zones
Overall Ghost Rate can hide localized problems.
Ghost Zones identify queues or operational stages where abandonment is concentrated.
Instead of asking:
"Why is our overall abandonment rate high?"
an operations team can ask:
"Which queue is creating the largest concentration of abandoned interactions?"
GhostQueue ranks these areas using abandonment volume and Ghost Rate, helping operations teams focus their investigation.
3. Ghost Replay
When event-level journey data is available, Ghost Replay reconstructs an interaction chronologically.
It shows the journey through stages such as:
Entry → Queue → Waiting → Stage changes → Final event
For abandoned sessions, GhostQueue identifies the final known point before abandonment.
An important design decision was to avoid treating uncertainty as abandonment.
If a session has no terminal event, it can remain unresolved instead of being incorrectly classified as abandoned.
This distinction matters because missing information should not automatically become an assumption.
4. AI Investigator
The AI Investigator turns operational evidence into three separate layers:
Observed Evidence → Hypotheses → Next Actions
This separation is intentional.
The system should not present a hypothesis as if it were a proven causal explanation.
GhostQueue uses a hybrid AI architecture with Amazon Bedrock and a deterministic evidence-based fallback engine.
Amazon Bedrock
In the production deployment, GhostQueue can use Amazon Bedrock for LLM-powered investigation.
The application runs on Amazon EC2 with an IAM instance profile, allowing it to authenticate with AWS services without storing long-lived AWS credentials or API keys in a
.env file.The AI layer is designed around a strict privacy boundary.
Rather than sending raw interaction rows or customer information to the model, GhostQueue passes derived operational summaries such as:
- Ghost Rates
- Queue-level abandonment volumes
- Peak abandonment intervals
- Wait-time statistics
- Aggregated operational signals
This allows the AI Investigator to reason about operational patterns without requiring raw customer records.
Deterministic Fallback
Reliability was also an important design requirement.
If Amazon Bedrock is unavailable, inaccessible, or encounters an AWS/model error, GhostQueue can fall back to its deterministic evidence-based investigation engine.
This means the AI investigation workflow does not have to fail simply because an external model request is unavailable.
The architecture therefore combines:
AWS-native generative AI + deterministic fallback
instead of making the entire product dependent on a single external model call.
5. What-if Simulator
GhostQueue also allows operators to test operational scenarios.
For example:
What happens if two additional agents are available?
The simulator compares the baseline against the scenario and estimates how queue pressure, wait time, and Ghost Rate change under the configured assumptions.
For the contact-center benchmark, one scenario showed a simulated reduction in Ghost Rate from 1.85% to 1.35% with two additional agents.
The result is explicitly presented as a scenario simulation, not a guaranteed prediction.
This distinction is important: the simulator explores the consequences of configured assumptions rather than claiming certainty about future operations.
Building for Real Data
GhostQueue was designed to work beyond a single hardcoded demo.
The application supports:
- Public operational datasets
- Healthcare operational datasets
- Custom CSV uploads
- Custom JSON uploads
- Automatic schema detection
- Capability detection
- Event-level replay when session data exists
For healthcare data, I used a synthetic fixture based on the structure described in published research rather than redistributing restricted institutional data.
This was an important lesson:
Building a realistic demonstration does not require pretending that restricted data is publicly available.
Privacy by Design
Privacy was considered as part of the architecture rather than as a final add-on.
Custom datasets are processed temporarily in memory.
The application does not persist uploaded raw datasets as application data.
For AI investigation, the privacy boundary is even more important.
The Bedrock integration receives derived operational aggregates rather than raw interaction rows, customer records, or PII.
The goal is to keep the AI layer focused on operational patterns while minimizing unnecessary exposure of sensitive data.
Deploying GhostQueue on AWS
Once the application was ready locally, I deployed it to AWS.
The production architecture is:
Internet → Nginx → Next.js + FastAPI → AWS IAM → Amazon Bedrock
The application runs on an Amazon EC2 t3.small instance using Ubuntu 24.04 LTS.
AWS Services Used
Amazon EC2
Amazon EC2 hosts the production Next.js frontend and FastAPI backend.
The application services run under systemd so that the frontend and backend can automatically restart when required.
Amazon VPC and Security Groups
Amazon VPC and Security Groups control network connectivity and inbound access to the EC2 environment.
AWS IAM
AWS IAM provides the EC2 instance role and instance profile.
This allows the application to authenticate with AWS services without embedding long-lived AWS credentials inside the application.
For Amazon Bedrock, the EC2 workload uses its IAM instance profile rather than storing an AWS secret key in the application.
Amazon Bedrock
Amazon Bedrock provides the LLM-powered investigation capability.
GhostQueue uses Bedrock through the backend's AI provider layer, while maintaining a deterministic fallback for reliability.
Nginx
Nginx acts as the production reverse proxy.
It provides a single public entry point for the frontend and API and routes the application traffic to the appropriate services.
HTTPS Production Deployment
The production deployment also uses HTTPS.
The public application is available at:
The deployment uses TLS encryption with automated certificate renewal through Let's Encrypt.
HTTP traffic is redirected to HTTPS.
This was an important final step because the application is intended to be accessed as a real production service rather than only as a local development environment.
Production Verification
After deployment, I verified the production application end-to-end.
The verification included:
- Production health check
- Dataset registry
- Contact-center benchmark analysis
- Ghost Zones
- Ghost Replay
- AI Investigator
- What-if Simulator
- Frontend-to-backend communication
- HTTPS access
- Bedrock integration and fallback behavior
The production system was synchronized with the GitHub repository so that the deployed environment reflects the final project code.
What I Learned
The biggest lesson from this project was that an operational metric is often only the beginning of an investigation.
A Ghost Rate tells you what happened.
Ghost Zones tell you where it happened.
Ghost Replay helps explain what happened immediately before it.
The Investigator structures possible explanations and actions.
And the Simulator lets you explore what might change under a different operational scenario.
Building the project also gave me practical experience moving from a local full-stack application to a publicly accessible AWS deployment.
I worked with:
- Amazon EC2
- Amazon VPC
- Security Groups
- AWS IAM
- Amazon Bedrock
- Nginx
- HTTPS/TLS
- FastAPI
- Next.js
The project also reinforced an important engineering principle:
A production AI system should be useful even when the model is unavailable.
That is why GhostQueue combines an AWS-native LLM path with a deterministic fallback.
What's Next?
The architecture is intentionally extensible.
Future versions could incorporate:
- Real-time operational monitoring
- Richer event streams
- More sophisticated causal analysis
- Streaming queue telemetry
- Additional foundation models
- More advanced simulation models
But the core idea will remain the same:
Don't just measure who completed the process. Discover who disappeared before they could.
That is GhostQueue.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article