How I Built a Real-Time Fraud Detection System on AWS (and What I Learned About False Alarms)
The problem that got me started Picture an online bank receiving thousands of transactions every minute. Most of them are completely normal: someone buying groceries, paying a bill, topping up a transport card. A few are not.
The idea
I wanted a pipeline that does four things:
- Listens to every transaction as it arrives
- Scores it with a machine learning model
- Decides what to do: approve, ask for extra verification, or block
- Tells the fraud team when a human should take a look
The key word is decides. A score on its own helps no one. The value comes from connecting the prediction to an action.
The toolbox
This project let me wear several hats at once: data engineer, ML engineer, analyst, and a bit of security architect. Here is what I used and why:
| Job | AWS service | Why I picked it |
|---|---|---|
| Receive transactions continuously | Amazon Kinesis Data Streams | Handles a constant flow of events without me managing servers |
| Enrich events on the fly | AWS Lambda | Small, cheap functions to compute things like “how many payments in the last 10 minutes?” |
| Remember customer behavior | DynamoDB and SageMaker Feature Store | Fast lookups of each customer’s normal habits |
| Train the models | Amazon SageMaker | XGBoost for known fraud patterns, Random Cut Forest for unusual behavior |
| Score in real time | SageMaker real-time endpoint | A risk score in milliseconds |
| Make the decision | AWS Step Functions | A clear, visual workflow: approve, challenge, or block |
| Raise the alarm | Amazon SNS and EventBridge | Notify analysts the moment something looks wrong |
| Keep the history | Kinesis Data Firehose and Amazon S3 | Every transaction archived for future retraining |
| Understand what’s happening | Athena and QuickSight | Dashboards for investigators and managers |
| Stay safe | IAM, KMS, CloudWatch, CloudTrail | Encryption, least privilege, and an audit trail |
How it fits together
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Transactions → Kinesis Data Streams
│
Lambda (enrichment)
│
Feature Store + DynamoDB (customer profile)
│
SageMaker endpoint (risk score)
│
Step Functions (decision)
┌──────────┼───────────┐
Approve Challenge Block
│ │
SNS / EventBridge → fraud team
│
Firehose → S3 → Athena → QuickSightWhat I learned along the way
1. Fraud is rare, and that makes it tricky.
Fraud is often less than 1% of all transactions. That means a lazy model that always says “not fraud” would be right 99% of the time and completely useless. Accuracy is the wrong yardstick here. I focused on recall, precision on the fraud class, and AUC-PR instead.
Fraud is often less than 1% of all transactions. That means a lazy model that always says “not fraud” would be right 99% of the time and completely useless. Accuracy is the wrong yardstick here. I focused on recall, precision on the fraud class, and AUC-PR instead.
2. Behavior matters more than amount.
A €900 payment is normal for one customer and alarming for another. The most useful signals weren’t the amount itself but how far a transaction strayed from that person’s habits: unusual hour, new country, new device, sudden burst of payments.
A €900 payment is normal for one customer and alarming for another. The most useful signals weren’t the amount itself but how far a transaction strayed from that person’s habits: unusual hour, new country, new device, sudden burst of payments.
3. Two models are better than one.
A supervised model is good at spotting fraud patterns it has seen before. An anomaly detector catches the strange stuff nobody has labeled yet. Together they cover each other’s blind spots.
A supervised model is good at spotting fraud patterns it has seen before. An anomaly detector catches the strange stuff nobody has labeled yet. Together they cover each other’s blind spots.
4. Three outcomes beat two.
“Approve or block” is too blunt. Adding a middle option, like asking for a one-time code, meant fewer legitimate customers were turned away without taking extra risk.
“Approve or block” is too blunt. Adding a middle option, like asking for a one-time code, meant fewer legitimate customers were turned away without taking extra risk.
5. Humans still matter.
Uncertain cases go to an analyst, and every decision they make becomes a new label for retraining. The system gets smarter because people stay in the loop.
Uncertain cases go to an analyst, and every decision they make becomes a new label for retraining. The system gets smarter because people stay in the loop.
6. The threshold is a business decision.
How many missed frauds are you willing to accept to avoid annoying one honest customer? There is no purely technical answer. It depends on cost, brand, and risk appetite.
How many missed frauds are you willing to accept to avoid annoying one honest customer? There is no purely technical answer. It depends on cost, brand, and risk appetite.
Things to watch out for
- Latency: every step has to fit into a tight time budget.
- Drift: fraudsters change tactics, so keep an eye on model performance with SageMaker Model Monitor.
- Explainability: if you block someone, you should be able to say why. SageMaker Clarify helps here.
- Privacy: pseudonymize and encrypt personal data, and respect regulations like GDPR.
- Cost: a permanent endpoint and a streaming pipeline run around the clock, so size them carefully.
Where this could go next
- Use Amazon Neptune to detect networks of linked accounts
- Automate retraining with SageMaker Pipelines
- Add Amazon Bedrock to write a short plain-language summary of each alert for the analyst
- Run in shadow mode first, where the model scores transactions but doesn’t block anything, to build trust before going live
The same pattern works well beyond banking: hijacked accounts, promo-code abuse in e-commerce, or unusual readings from industrial sensors.
Final thoughts
The most important lesson wasn’t about any single AWS service. It was that fraud detection is as much about people as about algorithms: the customer who just wants to pay, the analyst who needs clear alerts, and the business that has to balance safety and experience.
Kinesis, SageMaker, Step Functions, and QuickSight make it possible to build all of this without managing a single server. The harder and more interesting part is deciding what “good” looks like.
If you’ve worked on something similar, I’d love to hear how you handled the false-positive problem.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article