
TattvaAI: Turning AWS Observability Data into Actionable Incident Intelligence
Production incidents rarely come from a single signal. TattvaAI brings logs, metrics, traces, evidence correlation, reasoning, human review, and reporting into one AWS-powered investigation workflow.
TattvaAI: Turning AWS Observability Data Into an AI-Assisted Incident Investigation Workflow
Observability tells you what happened. TattvaAI is designed to help engineers reason about what the evidence means.
When a production system experiences an incident, the first challenge is usually not a lack of telemetry.
Modern applications already generate logs, metrics, traces, alerts, and service dependency information. The difficult part is bringing those signals together and turning them into an investigation that an engineer can actually use.
During the WeMakeDevs × AWS Bharat Builds Tour – First Commit hackathon, we built TattvaAI, an AI-powered incident investigation platform designed to explore this problem.
TattvaAI sits above existing observability systems. It does not attempt to replace an observability backend. Instead, it provides an investigation layer that collects evidence, normalizes it, correlates signals, generates evidence-grounded hypotheses and recommendations, and gives the engineer an opportunity to review the result before accepting the investigation.
The deployed application also demonstrates this workflow using real AWS observability data from CloudWatch and AWS X-Ray.
The Problem: Telemetry Is Not the Same as Understanding
A typical production incident can involve several different signals.
An engineer might start with an alert, move to application logs, inspect a metric, open a trace, look at service dependencies, and then try to understand whether these pieces are related.
Each individual source can provide useful information.
The challenge is connecting them.
For example, an engineer investigating a slow API might need to answer questions such as:
- Which service is actually involved?
- Did application response time increase?
- Is there evidence of dependency problems?
- What do the traces show?
- Are there related logs?
- Which observations are connected?
- What could be the root cause?
- What should the engineer investigate next?
This creates an important distinction:
Collecting telemetry and investigating an incident are different problems.
TattvaAI was designed around the second problem.
Why We Built TattvaAI
Our central idea was simple:
What if an incident investigation could become a structured workflow instead of a collection of disconnected observability screens?
The goal was not to build another monitoring platform.
Instead, we wanted to create an intelligence and investigation layer that could work with different telemetry providers.
The resulting workflow is:
Incident → Evidence → Normalization → Correlation → Reasoning → Hypotheses → Recommendations → Human Review → Report → Investigation Memory
This structure gives each stage a specific purpose.
Evidence is collected first. It is then transformed into a common investigation format so that different sources can be analyzed together. Correlation connects related signals. The reasoning layer uses that evidence to form hypotheses and recommendations.
Most importantly, the final decision remains with the engineer.
TattvaAI Architecture
The deployed application uses a serverless AWS architecture.
At a high level, the architecture consists of:
React Frontend
↓
AWS Amplify
↓
Amazon API Gateway
↓
AWS Lambda
↓
Investigation Engine
↓
AWS Observability Sources
The application uses several AWS services for different responsibilities:
- AWS Amplify – frontend hosting
- Amazon API Gateway – HTTP API entry point
- AWS Lambda – backend/serverless application
- Amazon DynamoDB – user and investigation persistence
- Amazon S3 – investigation report storage
- Amazon CloudWatch – logs and metrics
- AWS X-Ray – distributed tracing and service dependency information
- AWS Secrets Manager – sensitive configuration and secrets
The frontend is implemented as a React application, while the backend follows Python/FastAPI concepts and is deployed through Lambda and API Gateway.
Inside the Investigation Pipeline
The investigation engine is organized into several stages.
1. Incident Intake
The workflow begins with an incident.
Instead of immediately asking an AI system to generate an answer from a short incident description, TattvaAI treats the incident as the beginning of an evidence investigation.
The objective is to gather relevant information before forming conclusions.
2. Evidence Collection
TattvaAI can retrieve observability evidence through its telemetry provider abstraction.
In the deployed AWS environment, the verified live provider retrieves:
- CloudWatch Logs
- CloudWatch Metrics
- AWS X-Ray traces
- X-Ray service dependency information
This is an important part of the demonstration.
The deployed AWS investigation is not based on synthetic telemetry. It can retrieve real AWS observability data.
The investigation also preserves provenance information, including the provider, source, mode, and region.
For example:
provider = aws source = aws_xray / cloudwatch mode = LIVE region = us-east-1 This makes it possible to distinguish live AWS evidence from mock or offline sources.
3. Evidence Normalization
Logs, metrics, traces, and dependency information naturally have different structures.
For an investigation engine to reason across them, they need to be represented consistently.
TattvaAI therefore uses an evidence normalization stage that converts telemetry into a common investigation-oriented format.
This allows later stages to work with evidence without being tightly coupled to a single telemetry source.
4. Correlation
Collecting evidence is only part of the problem.
The next question is:
Which pieces of evidence are related?
The correlation stage looks for relationships between collected signals and creates findings that can be used by the reasoning layer.
For example, an investigation may contain several observations related to API performance.
The correlation stage can bring those observations together rather than leaving the engineer to manually connect them.
From Correlated Evidence to Root-Cause Hypotheses
After evidence collection and correlation, TattvaAI moves into reasoning.
The output is not intended to be a generic chatbot response.
Instead, the investigation produces structured information such as:
- Root-cause hypotheses
- Confidence
- Supporting evidence
- Recommendations
This distinction is important.
An AI-assisted incident investigation should be able to show why a hypothesis was produced rather than simply returning an answer.
The interface therefore separates evidence, correlation, hypotheses, and recommended actions.
Human-in-the-Loop by Design
Production incident response is a high-responsibility workflow.
For that reason, TattvaAI does not automatically make the final production decision.
The reasoning layer produces an investigation that an engineer can review.
The engineer remains responsible for deciding whether the hypothesis and recommendations make sense in the context of the actual system.
This creates a human-in-the-loop workflow:
AI-assisted investigation → Evidence review → Engineer decision
The objective is not to replace the incident responder.
It is to reduce the amount of manual work required to move from scattered telemetry toward an understandable investigation.
Generating Investigation Reports
An investigation should not disappear after the incident is resolved.
TattvaAI supports investigation report generation in:
- JSON
- Markdown
Reports can be stored using Amazon S3.
This creates an additional layer of investigation memory.
Instead of treating every incident as an isolated event, the investigation can be preserved for later review.
This can be useful when teams want to understand what evidence was collected, what hypotheses were considered, and what recommendations were produced during an incident.
Provider Abstraction: Not Locked to One Observability Platform
Another architectural decision was to avoid tightly coupling the investigation engine to one telemetry provider.
The project includes provider concepts such as:
-
MockTelemetrySource
-
OTLPTelemetrySource
-
SigNozTelemetrySource
-
AWSObservabilitySource
The idea is that the investigation workflow should operate on evidence rather than being fundamentally dependent on one observability vendor.
In the deployed demonstration, AWS is the verified live provider.
The broader provider abstraction also fits the OpenTelemetry-oriented design of the project.
OpenTelemetry-compatible concepts can be used for telemetry collection and export, while the provider layer determines how investigation evidence becomes available to the investigation engine.
Where MCP Fits
TattvaAI also uses the concept of MCP as an evidence and tool access layer.
The important architectural idea is that the reasoning workflow should have structured access to evidence and tools rather than simply receiving an incident description and generating a response.
That means the investigation flow is centered around:
Evidence access → Evidence correlation → Reasoning → Recommendations
rather than:
Incident description → LLM answer
This distinction shaped the architecture of TattvaAI.
AWS Deployment
The deployed application uses AWS services throughout the application lifecycle.
The frontend is hosted using AWS Amplify.
Requests enter through Amazon API Gateway and are handled by AWS Lambda.
Investigation and user persistence use Amazon DynamoDB, while generated reports can be stored in Amazon S3.
The application also integrates with CloudWatch and X-Ray for observability.
Sensitive configuration is handled through AWS Secrets Manager rather than keeping JWT secrets directly in source code.
Authentication uses JWT-based authentication, and passwords are hashed server-side.
The API also uses restricted CORS rather than allowing every origin.
These decisions were important because the project was not only about demonstrating an investigation interface. We also wanted the deployed application to follow a practical serverless AWS architecture.
The Reasoning Layer: An Important Limitation
One of the most important things we learned during the hackathon was the difference between an architecture being configured to support a service and that service actually being available in the deployed environment.
TattvaAI provides a provider abstraction for Amazon Bedrock as the intended cloud reasoning provider.
However, the current deployed environment did not have live Bedrock model authorization during the hackathon.
Therefore, the current deployed demonstration uses an offline fallback/rule-based reasoning engine.
This means the article should not describe the current deployment as being powered by Amazon Bedrock or claim that Claude generated the investigation.
The architecture is prepared for a Bedrock-based reasoning provider, but the deployed demonstration uses the fallback engine.
We consider this distinction important because an engineering project should clearly separate what is implemented, what is configured, and what is actually running.
Challenges During Development
One challenge was bringing together different types of observability evidence without making the investigation engine dependent on one source.
Logs, metrics, traces, and service dependency information have different structures and meanings.
The provider abstraction and normalization stage helped address this by separating evidence retrieval from investigation logic.
Another challenge was designing the reasoning workflow so that the system did not simply produce an unexplained AI response.
The investigation therefore focuses on evidence, correlations, hypotheses, confidence, and recommendations.
A third challenge was maintaining a realistic boundary between development and deployment.
The project contains an Incident Lab for local development and testing. These controlled telemetry workloads are not deployed as production Lambda services.
The verified production AWS investigation instead operates against the deployed TattvaAI Lambda workload and retrieves real AWS telemetry.
What We Learned
The biggest lesson from building TattvaAI was that observability and incident investigation are related, but they are not the same thing.
Observability systems are extremely good at exposing telemetry.
But during an incident, an engineer still has to interpret that telemetry.
That interpretation involves connecting signals, understanding dependencies, comparing evidence, developing hypotheses, and deciding what to investigate next.
This led us to think about incident response as an evidence-to-decision workflow.
We also learned that AI is more useful in this workflow when it is grounded in evidence.
A useful investigation should be able to connect its conclusions back to the information collected during the investigation.
Finally, building the project reinforced the importance of being precise about system capabilities.
A configured provider is not necessarily a live provider.
A local testing workload is not necessarily a production service.
And an architectural integration should not be presented as a live capability when the required authorization or infrastructure is not available.
Limitations and Future Improvements
TattvaAI is a hackathon project and has several known limitations.
The current deployed environment does not have live Amazon Bedrock model authorization, so the production demonstration uses the fallback reasoning engine.
The Incident Lab is intended for local and development testing rather than production deployment.
The project also contains development history from before the hackathon, which has been preserved rather than rewritten.
Future development could build on the provider architecture, including enabling the Bedrock reasoning provider when the required authorization is available and continuing to improve evidence correlation and investigation capabilities.
The broader goal would remain the same: make incident investigations more structured, evidence-grounded, and reviewable.
Conclusion
TattvaAI started with a simple observation:
Having telemetry does not automatically mean having an explanation.
When an incident occurs, engineers need to move from raw evidence toward understanding.
TattvaAI explores that space by putting an investigation layer above existing observability systems.
It collects AWS observability evidence, normalizes it, correlates related signals, generates evidence-grounded hypotheses and recommendations, keeps an engineer involved in the final decision, and preserves the investigation through reports and history.
The architecture also deliberately separates telemetry collection, observability storage, evidence access, investigation reasoning, and human decision-making.
For the deployed demonstration, AWS provides the infrastructure and the live observability data through CloudWatch and X-Ray.
The reasoning architecture is prepared for Amazon Bedrock, while the current deployment uses the fallback reasoning engine because live Bedrock authorization was not available during the hackathon.
That distinction is part of the story too.
We built TattvaAI not to replace the observability systems engineers already use, but to explore the question that comes after telemetry:
“Given all this evidence, what should the engineer understand from it?”
And that is where we believe an investigation layer can add value.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article