AIdeas: Veloquity - The Agentic Evidence Intelligent Platform Turning Raw Feedback into Evidence-Driven Decisions
Most organizations drown in feedback but starve for decisions. VELOQUITY closes that gap - an Agentic Evidence Intelligence System built on AWS that autonomously transforms fragmented feedback signals into validated evidence clusters, scores their confidence, and delivers prioritized, explainable recommendations with full source-to-decision traceability. READ THE ARTICLE BELOW TO SEE HOW.
App Category
"Commercial Solutions" - with strong societal impact crossover (healthcare, education, government use cases are core to the pitch, not afterthoughts).
Every Organization Collects Feedback
Almost None Can Turn It Into Decisions
Mobile apps receive thousands of reviews every week.
Support teams process hundreds of tickets every day.
Hospitals collect patient experience surveys after every visit.
Users constantly submit feedback through forms, chat, and community forums.
Support teams process hundreds of tickets every day.
Hospitals collect patient experience surveys after every visit.
Users constantly submit feedback through forms, chat, and community forums.
Yet despite collecting more feedback than ever before, most organizations still face the same problem:
They cannot reliably turn feedback into decisions.
The signals exist - but they are buried inside noise.
One user reports a crash.
Another describes the same issue differently.
A third leaves a frustrated review on the app store.
Support tickets describe the same problem using completely different language.
Another describes the same issue differently.
A third leaves a frustrated review on the app store.
Support tickets describe the same problem using completely different language.
Soon there are hundreds of signals scattered across multiple platforms.
Organizations worldwide are drowning in feedback - and making worse decisions because of it.
To humans, these reports appear as isolated complaints.
But hidden beneath them are patterns that reveal:
- critical product failures
- usability breakdowns
- emerging feature demands
- systemic user frustration
Most teams today still solve this manually:
Reading reviews.
Scanning support tickets.
Guessing which issues matter most.
Scanning support tickets.
Guessing which issues matter most.
At scale, that approach breaks down.
Organizations may collect thousands of signals, yet still miss the one problem affecting the most users.
This is the gap Veloquity was built to solve.
My Vision
Veloquity is an Agentic Evidence Intelligence System designed to transform raw feedback into structured decision intelligence.
Instead of manually reading thousands of comments, Veloquity automatically:
• collects feedback from multiple sources
• detects hidden semantic patterns
• clusters related user signals
• evaluates signal confidence
• generates actionable product recommendations based on the evidence collected with constraints.
• detects hidden semantic patterns
• clusters related user signals
• evaluates signal confidence
• generates actionable product recommendations based on the evidence collected with constraints.
The result is something most teams currently lack:
Evidence-backed decision making.
Although the current prototype focuses on product feedback, the architecture is intentionally domain-agnostic.
The same intelligence pipeline could help:
Education systems analyze student feedback
Healthcare systems detect recurring patient issues
Governments cluster citizen complaints
Enterprises analyze employee sentiment
Healthcare systems detect recurring patient issues
Governments cluster citizen complaints
Enterprises analyze employee sentiment
Anywhere humans produce feedback, evidence intelligence can emerge.
Why This Matters
Organizations are not suffering from a lack of data. They are suffering from a lack of signal inside the data.
A product team receives 500 support tickets a week. A hospital's patient experience team gets thousands of survey responses a month. A city's digital services portal collects citizen complaints across dozens of services. In every case, the teams responsible for acting on this feedback spend most of their time reading, tagging, and organizing - not deciding.
Traditional analytics tools count things. They tell you "138 users mentioned crashes this week." But they cannot tell you whether those 138 share a root cause, whether the problem is accelerating, or how it compares in urgency to the 94 users reporting a performance regression.
Veloquity treats each feedback item as a signal in a large information system - and finds the evidence hidden inside the noise.
For a hospital quality team, that means surfacing recurring patient safety patterns before they become incidents.
For a university, that means identifying teaching issues quietly hurting student outcomes before end-of-semester grades reflect them.
For a government service team, that means catching the failing public service citizens are complaining about before it becomes a headline.
The underlying challenge is identical everywhere. Veloquity closes that gap.
Example Scenario: Discovering a Hidden
Product Crisis
To understand how the system works, consider a real scenario.
A product team receives feedback from two sources:
• App Store reviews
• Zendesk support tickets
• Zendesk support tickets
Individually, each report seems minor.
But when analyzed collectively, a serious pattern emerges.
Step 1 - Feedback Ingestion
Import Sources
Veloquity begins by ingesting feedback from multiple sources.
In this example the system connects to:
• App Store reviews
• Zendesk support tickets
• Salesforce webhook integrations
• Zendesk support tickets
• Salesforce webhook integrations
The ingestion agent performs several preprocessing tasks automatically:
• data normalization
• PII redaction
• SHA-256 deduplication
• storage in Amazon S3
• PII redaction
• SHA-256 deduplication
• storage in Amazon S3
In this dataset the system ingested:
547 feedback items across multiple sources.
Instead of scattered reports, the system now has a clean signal corpus.
Step 2 - Exploring Raw Signals
Data Studio
Inside Data Studio, each feedback item becomes searchable and analyzable.
Example reports include:
“App crashes every time I switch between projects.”
“Black screen appears after updating.”
“Push notifications arrive 30 minutes late.”
“Export pipeline fails on large datasets.”
“Black screen appears after updating.”
“Push notifications arrive 30 minutes late.”
“Export pipeline fails on large datasets.”
Individually these seem like isolated issues.
But Veloquity is designed to detect hidden patterns across all signals.
Step 3 - Evidence Intelligence
The Evidence Grid is not a summary view.
It is a transparency layer between raw feedback and product decisions.
It is a transparency layer between raw feedback and product decisions.
Each evidence cluster shows exactly where it came from — which sources contributed, which individual feedback items were grouped together, and how confident the system is in that grouping.
For example, a cluster labeled “App crashes when switching projects” does not just display a count.
It reveals the full evidence behind the signal:
It reveals the full evidence behind the signal:
- 138 feedback items that formed the cluster
- 94 unique users reporting the issue
- Two independent sources (App Store and Zendesk) corroborating the signal
- A 91% confidence score derived from how tightly those reports cluster in semantic space
This source-to-evidence mapping is what separates Veloquity from traditional analytics.
A simple count tells you how many people complained.
Evidence mapping tells you that these specific users, across these specific platforms, are describing the same underlying problem - and here is the proof.
Evidence mapping tells you that these specific users, across these specific platforms, are describing the same underlying problem - and here is the proof.
In this example dataset, 547 feedback signals collapse into 6 validated evidence clusters, each retaining full provenance.
Instead of reading hundreds of comments and guessing at patterns, teams see one validated, sourced, confidence-scored signal — and can trace every insight back to the original user voices that generated it.
Step 4 - Confidence Intelligence
Confidence Scores
Not every cluster is equally reliable.
Veloquity therefore calculates confidence scores for each cluster based on:
• similarity variance between embeddings
• number of unique users
• cross-source corroboration
• recency of signals
• number of unique users
• cross-source corroboration
• recency of signals
Example cluster:
App crashes when switching projects
Evidence summary:
138 feedback items
94 affected users
Cross-source validation (App Store + Zendesk)
94 affected users
Cross-source validation (App Store + Zendesk)
Final confidence score:
91%
This transforms anecdotal complaints into quantifiable evidence.
Step 5 - Reasoning Intelligence
Decision Playground
Detecting patterns is only the beginning.
The next challenge is determining what teams should actually do next.
Veloquity uses Claude 3 Haiku through Amazon Bedrock as a reasoning engine.
The reasoning agent evaluates each evidence cluster using a priority model considering:
• cluster confidence
• number of affected users
• recency of signals
• engineering effort estimates
• number of affected users
• recency of signals
• engineering effort estimates
Example reasoning output:
Cluster: App crashes on project switch
Confidence: 91%
Users affected: 94
Confidence: 91%
Users affected: 94
Recommendation:
Prioritize in the next sprint.
Every recommendation includes a traceable explanation, ensuring the system remains transparent and explainable.
Step 6 - Scenario Simulation
Scenario Comparison
Product leaders often ask:
“What if we prioritize differently?”
The Scenario Engine allows teams to simulate multiple strategies.
Examples include:
Conservative Q2
Focus only on high-confidence signals.
Focus only on high-confidence signals.
Aggressive Sprint
Lower thresholds to surface emerging issues earlier.
Lower thresholds to surface emerging issues earlier.
Resource Constrained
Prioritize only low-effort fixes.
Prioritize only low-effort fixes.
Each scenario predicts:
• number of issues resolved
• users impacted
• decision efficiency
• users impacted
• decision efficiency
This transforms planning into evidence-driven strategy.
The Agentic Pipeline
Agents
Under the hood, Veloquity operates as a multi-agent AI system running on AWS.
Four core agents coordinate the pipeline.
Ingestion Agent
Collects feedback and performs preprocessing including deduplication and PII removal.
Evidence Intelligence Agent
Uses Bedrock Titan embeddings and pgvector clustering to detect semantic patterns.
Reasoning Agent
Uses Claude models to generate prioritized product recommendations.
Governance Agent
Runs scheduled checks to detect stale signals, monitor clustering health, and maintain system reliability.
Together these agents convert:
raw feedback → structured evidence → explainable decisions
Conversational Intelligence
Veloquity AI
Veloquity's chat interface is not a general-purpose assistant. It is a direct query layer over the live evidence pipeline.
Every question is answered with full awareness of the current evidence state - active clusters, their confidence scores, the latest reasoning run, and governance health. When a PM asks "which issue should we prioritize this sprint?", the response is grounded in the actual ranked evidence at that moment, not a generic answer.
Teams can query the pipeline in natural language:
- What are the top evidence clusters right now?
- Which issue should we prioritize this sprint?
- Are there emerging trends we should investigate?
The difference from a typical AI assistant: the intelligence is already in the system. The chat interface simply makes it accessible without navigating dashboards.
Observability and Trends
Dashboard
The system dashboard provides real-time intelligence metrics such as:
547 feedback signals analyzed
6 evidence clusters detected
84% average cluster confidence
91% of signals successfully clustered into validated evidence
6 evidence clusters detected
84% average cluster confidence
91% of signals successfully clustered into validated evidence
Trend analytics also highlight rising issues before they escalate into major incidents.
How I Built This
Veloquity is a five-phase multi-agent system built entirely on AWS serverless infrastructure, with 158 automated tests passing end-to-end.
Core services include:
Amazon S3
Feedback ingestion storage
Feedback ingestion storage
AWS Lambda
Event-driven processing agents
Event-driven processing agents
Amazon Bedrock
Embeddings and reasoning models
Embeddings and reasoning models
PostgreSQL with pgvector
Semantic clustering
Semantic clustering
Amazon EventBridge
Agent orchestration and governance
Agent orchestration and governance
This architecture provides:
• horizontal scalability
• low operational overhead
• real-time feedback intelligence
• low operational overhead
• real-time feedback intelligence
Phase 1 - Ingestion Agent
The ingestion agent accepts raw feedback from any source - App Store reviews, Zendesk tickets, Salesforce webhooks, survey exports - and produces a clean, deduplicated, privacy-safe signal corpus. It performs automatic PII redaction, SHA-256 content deduplication (so the same complaint submitted from two platforms produces one record), and date-partitioned storage in Amazon S3. In the test run: 547 feedback items ingested cleanly across two independent sources.
Phase 2 - Evidence Intelligence Agent
This is where raw signals become structured evidence.
The agent generates 1024-dimensional vector embeddings via Amazon Bedrock Titan Embed V2 and clusters them using a greedy cosine similarity algorithm. An embedding cache ensures each unique piece of text is embedded only once - the system achieved 100% cache hit rate on re-runs, making the pipeline cost-efficient at scale.
Confidence scoring uses cosine distance variance across cluster members. Clusters above the acceptance threshold enter active evidence. Ambiguous clusters go through an LLM validation call. Every accepted cluster records full item-level provenance - complete traceability from recommendation back to original user report.
547 feedback signals collapsed into 6 validated evidence clusters.
Phase 3 - Reasoning Agent
The reasoning agent takes validated evidence clusters and produces prioritized, actionable recommendations using a four-component weighted priority formula:
- Cluster confidence score (35%) - the algorithm's own certainty
- Unique users affected (25%) - normalized volume signal
- Cross-source corroboration (20%) - bonus when a theme appears in multiple independent sources
- Recency score (20%) - linear decay over 90 days, so emerging issues surface before they age
The scored clusters are passed as a structured prompt to Claude 3 Haiku via Amazon Bedrock, which returns: recommended action, effort estimate, user impact level, tradeoff explanation, risk flags, and a cross-cluster insight identifying root causes spanning multiple evidence groups. Every output includes full reasoning provenance - no black-box decisions.
Top recommendation from the test run: "App crashes when switching between projects - 91% confidence, 94 affected users, cross-validated across App Store and Zendesk. Prioritize in next sprint."
Phase 4 - Governance Agent
The Governance Agent runs on an EventBridge cron daily and enforces three checks:
- Stale detection - evidence clusters not revalidated in 30 days are flagged and removed from active reasoning.
- Signal promotion - low-confidence staging items that accumulate frequency across multiple time windows are automatically promoted to active evidence as the system learns a pattern is real.
- Health Monitoring - system health monitoring - pipeline integrity and clustering health are tracked continuously to ensure signal quality at scale.
All governance logic is deterministic - the same database state always produces the same actions. Fully auditable, no per-invocation LLM cost.
Phase 5 - Frontend
A React 18 + TypeScript dashboard with seven pages: Dashboard, Agents, Evidence Grid, Recommendations, Scenario Engine, Governance Log, and a conversational AI Chat interface. The Chat page feeds live evidence clusters and recommendations as context to Claude 3 Haiku - teams ask natural language questions and get answers grounded in the actual evidence state, not generic summaries.
Accelerating Development with Kiro Agentic IDE
To architect a five-phase, multi-agent pipeline in a condensed timeframe, I leveraged the Kiro Agentic IDE.
Building an agentic platform using an agentic development environment proved to be a massive force multiplier. Instead of getting bogged down in boilerplate serverless configurations, Kiro acted as an active collaborator - helping to rapidly scaffold the AWS Lambda functions, generate complex pgvector SQL migrations, and seamlessly draft the 158 automated end-to-end tests. It allowed me to stay focused on the high-level system design and LLM orchestration while the IDE handled the granular implementation details.
Beyond Product Teams
Although this prototype demonstrates product feedback analysis, the architecture is intentionally domain-agnostic.
Potential applications include:
Education systems detecting teaching issues from course feedback.
Healthcare systems identifying recurring patient experience problems.
Government portals clustering citizen complaints to identify failing public services.
Enterprise organizations analyzing employee feedback at scale.
In each case the underlying challenge is identical:
finding reliable signals hidden inside massive volumes of human feedback.
What I Learned
Building Veloquity revealed several important lessons about real-world AI systems.
AI systems require governance, not just intelligence. LLM reasoning must be paired with deterministic validation layers. Confidence scoring, threshold evaluation, and daily governance checks exist because a recommendation without verifiable provenance is just a suggestion. The combination of clustering confidence + source corroboration + recency decay + LLM reasoning creates something more trustworthy than any component alone.
Explainability builds trust faster than accuracy does. The single design decision that changed how the output felt was full traceability - every recommendation links back through the reasoning run, through the evidence cluster, to the individual feedback items, to the original user report. A PM can click any recommendation and see exactly why the system surfaced it. Without that chain, even a correct recommendation feels arbitrary.
The domain-agnostic insight is the real value. I built Veloquity thinking about product teams. But halfway through, it became clear: the pipeline doesn't know what kind of feedback it is processing. It doesn't know if the text is a patient survey, a student evaluation, a citizen complaint, or an app review. It finds patterns in semantic space and scores their confidence. The vertical is product feedback because that is where I started - not because it is where the system ends.
Closing
Organizations already possess the data needed to improve their systems.
What they lack is a reliable way to transform that data into decisions.
Veloquity demonstrates how an agentic architecture combining serverless infrastructure, vector intelligence, and LLM reasoning can convert raw feedback into structured evidence and actions
Instead of reacting to isolated complaints, organizations can act on validated signals derived from thousands of users.
Just as observability transformed how engineers monitor software systems, Veloquity introduces a new concept: Evidence Intelligence for human decision-making - and it works wherever humans produce feedback at scale.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article