The AI That Can't Touch the Database: Building a Guardrailed Decision Explainer for Smart Cancel
Smart Cancel decides what happens to a parcel after a late cancellation. Here is how we built its AI agent to explain every decision but never change one, using read-only access, contract validation and a deterministic fallback, alongside a serverless AWS backend.
Software Engineer
The one question that matters
A parcel is already on the road when the customer taps cancel. Our engine weighs six options and picks one, say HOLD at a nearby hub. Then the operations manager asks the only question that matters: why?
"Because the model said so" is not an answer, and an LLM with write access to your database is not a safe one either. This post is about the middle path we built for Smart Cancel: an AI agent that explains every decision in plain language and is technically unable to change any of them.
Smart Cancel in 30 seconds
Today, a late cancellation sends the parcel all the way back to the warehouse, wasting the fuel, packaging and handling already spent. Smart Cancel treats it as a decision instead.
A rules engine evaluates six options (stop, hold, continue, re-match, transfer, return) and recommends the cheapest safe one. An operator approves it, and every action lands in an audit trail and an impact ledger with estimated cost and carbon. Customers still cancel in one tap. We don't stop cancellations, we make them cheaper.
It is a team project. My part was the AI agent and its integration with the backend, which is what this post is about.
The rule: AI explains, never decides
The recommendation always comes from deterministic rules, because money and safety are involved. The AI comes after the decision: it reads a saved decision and explains it.
Decision engine → saved decision (PostgreSQL) → AI agent → plain-language explanation → operator Three guardrails make this a property of the system, not a promise in a prompt:
- Read-only. The agent cannot generate candidates, change a recommendation or score, write to the database, or execute an action.
- Contract validation. It checks the decision it fetched against a strict schema before anything reaches the model. Bad data stops there.
- Deterministic fallback. If the API key is missing, the model times out or fails, or the response is invalid, the agent returns a rules-based explanation instead. Cancellation recovery never waits on an LLM.
How the agent works
The agent is a standalone TypeScript service on Node.js, deliberately separate from the frontend and the decision engine. One request looks like this:
- The backend calls
POST /v1/decisions/{id}/analyzewith only a decision ID.
- The agent fetches the real decision from the backend API itself, so it never trusts a payload someone could have edited. An inline-snapshot mode exists only for isolated tests and is off by default.
- It validates the decision against the contract.
- It asks Gemini for a grounded explanation, or uses the deterministic one if anything above goes wrong.
-
GET /healthshows whether the service is running on Gemini or on the fallback.
Step 2 is the choice I like most. An explainer that accepts its input from the caller can be talked into explaining things that never happened. One that reads the system of record can only explain what is actually there.
The AWS side
The platform around the agent is serverless and defined as code with AWS CDK, targeting ap-southeast-2 (Sydney):
- Amazon API Gateway (HTTP API) with a Cognito JWT authorizer on every route except the liveness check
- AWS Lambda running the Express API
- Amazon S3 holding the single-page frontend
- Amazon RDS for PostgreSQL in private subnets, with credentials generated into Secrets Manager
- Amazon CloudWatch logs with one-week retention
One lesson shaped the design. Our first plan put CloudFront in front of the site, but an account-verification block stopped it, and Cognito's Lite tier had configuration limits of its own. So we redesigned: API Gateway and a small Lambda serve the private S3 site, which puts the browser and the API on one origin with no CORS rules and no NAT Gateway to pay for. The constraint made the architecture simpler.
What broke, and what it taught us
During integration, the live model call failed with a 404 because the default model ID was wrong. We switched the default, ran the full flow again, and it came back healthy: provider Gemini, status READY. Thanks to the fallback design, a failure like this degrades the explanation, not the cancellation flow.
Three takeaways from that moment:
- Treat the model as a flaky dependency and design for the day it fails.
- Test the live path end to end, not only against mocks.
- Keep the model ID in configuration, so a fix is a settings change, not a redeploy of logic.
Honest limits
Smart Cancel is a prototype. The data is synthetic, execution is simulated (no riders are dispatched and no refunds are issued), and our cost and carbon figures are modelled estimates, not measured savings. We say so in the product too, because a number you cannot defend is worse than no number.
Takeaways and what's next
- Put the AI after the decision, not inside it.
- Make the safe path the default path: validation and fallback beat careful prompt wording.
- Let constraints shape the architecture, as the Cognito and CloudFront limits did for us.
Next, because the model call lives inside one service, swapping the provider (for example to Amazon Bedrock) should be a contained change, and it is the first thing we want to test. We also want to backtest the savings against real cancellation data.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article