
How We Built Pulse: A "Datadog" for ONDC Orders
Moving beyond aggregate metrics to trace exact transaction failures on ONDC. This is the engineering story of how we used EventBridge, Aurora PostgreSQL, and a deterministic state machine to build a highly reliable order tracking system.
Introduction
When you operate on the Open Network for Digital Commerce (ONDC), observability is usually presented at the network or participant level. As a seller network participant, you might get a dashboard showing a 96% "Attempt-to-Confirm" rate or high-level volume metrics.
But when a specific customer complains that their order is stuck, an aggregate percentage doesn't help you. You need to know exactly where the transaction broke down. Did the buyer app fail to send the
/confirm callback? Did our logistics partner miss the /on_status ping?We built ONDC Pulse to solve this. Instead of network-level metrics, we wanted transaction-level observability. We needed to ingest every raw ONDC protocol message, normalize them into a canonical event stream, and reconstruct each order's state machine to detect failures and SLA breaches in real-time.
Here is the story of how we built it, the architecture we settled on, the failure modes we hit, and how we solved them.
The first Attempt : A monolithic Fastify Server
We started with a straightforward stack: a monolithic Node.js server using Fastify and a PostgreSQL database with Prisma as our ORM. We structured our repository using npm workspaces, splitting out apps/api for the core ingestion logic and packages/database for shared schemas.
The initial flow was simple:
ONDC callbacks (like
/search or /on_confirm ) hit our Fastify endpoints.
We parse the payload, extract the tenant_id (essential for our multi-tenant isolation), and write the raw JSON into our OrderEvent table.
We synchronously evaluate the order's state and update the Order table.
This worked perfectly in local testing. It failed the moment we simulated the unpredictable, spiky traffic patterns common in ONDC during peak hours.Because we were doing heavy synchronous writes and state-machine computations inside the HTTP request handler, our API latency spiked. If the server became overloaded, we dropped webhooks. In ONDC, a dropped callback means a lost order state. We realized quickly that we needed to decouple ingestion from processing.
Decoupling the Architecture with AWS Serverless
We pivoted to an event-driven serverless architecture using AWS CDK. The goal was to accept the webhook as fast as possible, durably store it, return a
200 OK , and move all the heavy state-machine logic to an asynchronous background worker.Here is the data flow for the final architecture:

Why we chose these specific AWS services:
- API Gateway + Lambda for Ingestion: We needed to absorb massive spikes in webhooks without provisioning idle servers. The API Gateway securely exposes the endpoint, and the ingestion Lambda does nothing but validate the schema (using Zod), push the raw JSON payload to S3 for auditing, and forward a normalized event to EventBridge.
- Amazon EventBridge + SQS: EventBridge acts as our routing layer. It routes normalized order events into an SQS queue. We chose SQS to buffer the events before they hit our database processor, smoothing out write spikes and allowing us to control the concurrency of our database connections.
- Amazon Aurora PostgreSQL: We needed strict ACID compliance to enforce order state transitions. A NoSQL database would have been faster, but relational integrity, strict
tenant_idforeign keys, and the ability to run analytical queries (like percentile latencies) outweighed the raw write speed benefit.
The Prisma + Lambda Failure Mode
We hit a major roadblock when deploying our database migration Lambda via AWS CDK.
We wrote a Node.js Lambda function (
migrate.ts ) to run Prisma migrations against our Aurora cluster inside the VPC. But every time the Lambda executed in AWS, it crashed, complaining that the Prisma query engine binary could not be found for the host platform.It turned out that our CDK deployment pipeline was running on a Mac, and the Prisma CLI was bundling the macOS binary, not the Linux binary required by AWS Lambda.
To fix this, we had to heavily customize the
NodejsFunction bundling process in our DatabaseStack.ts . We hooked into the afterBundling command to explicitly copy the Prisma schemas into the output directory and manually force the generation of the RHEL binary:commandHooks: { afterBundling(inputDir: string, outputDir: string): string[] { return [ `mkdir -p ${outputDir}/node_modules`, `cp -R ${inputDir}/packages/database/dist ${outputDir}/node_modules/@ondc-pulse/database/`, `cp -R ${inputDir}/packages/database/prisma/migrations ${outputDir}/`, `cp ${inputDir}/packages/database/prisma/schema.prisma ${outputDir}/`, // Explicitly generate the RHEL schema engine inside the output directory! `cd ${outputDir} && PRISMA_CLI_BINARY_TARGETS="rhel-openssl-3.0.x" npx prisma generate --schema=./schema.prisma` ]; }}
This ensured the correct query engine was bundled, regardless of the operating system running the CDK deployment.
Rebuilding the State Machine
With the infrastructure stable, we focused on the core logic: the Order Processor Lambda.
Its job is to pull from SQS, determine the current state of an order, and attempt a transition based on the incoming event type. Because ONDC is a distributed system, network latency means events can arrive out of order (e.g., we might receive an
/on_status update before an /on_confirm ).We built a strict, deterministic state machine in TypeScript to catch these anomalies. The logic explicitly defines what transitions are allowed.
// Inside state-machine.service.tsexport const stateMachineService = { isValidTransition: (currentState, nextState) => { // We strictly reject backward transitions // isValidTransition(ONDC_STATES.CONFIRMED, ONDC_STATES.INITIALIZED) === false // We also reject invalid state skips // isValidTransition(ONDC_STATES.SEARCHED, ONDC_STATES.CONFIRMED) === false }}
If the transition is valid, we update the
orders table. If it's invalid, the transition is rejected, and the anomaly is forwarded to the Incident Engine.The SLA Engine and Incident Grouping
Catching invalid state transitions was only half the battle. The harder problem was detecting missing events. For instance, if an order is confirmed, the seller is supposed to assign logistics within 2 minutes. If that callback never arrives, the order is stuck, but no explicit error event is ever fired.
Our SLA Engine handles this. It is a purely deterministic rules engine that measures time gaps between key milestones. When a new event arrives, it queries the
order_events table, calculates the delta ( assign_time - confirm_time ), and checks it against a configured threshold.If the threshold is breached, the Incident Engine groups the failures. Rather than bombarding operators with 50 individual alerts for 50 stuck orders, the Incident Engine correlates them by
tenant_id and failure type, creating a single incident ticket: "Logistics callbacks failing for 50 orders since 2:00 PM."What's Next
ONDC Pulse gives merchants a way to finally see exactly where their transaction lifecycles are breaking down. By combining scalable serverless ingestion with a rigid relational state machine, we built a system that survives webhook traffic spikes without corrupting order states.
The next step for the architecture is integrating Amazon Timestream to calculate real-time P95 and P99 latencies for API stages. This will allow us to move from just detecting incidents to proactively predicting systemic network degradation before it impacts buyers.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article