AWS Builder Center

Vouch: What Happens After a Pull Request Gets Approved?

Vouch is a code review intelligence platform that uses machine learning to identify risky pull requests and detect when reviews may be rushed or incomplete. This article explores how we built Vouch using GitHub, DistilBERT, and AWS services including Elastic Beanstalk, DynamoDB, IAM, Secrets Manager, and CloudWatch.

Vouch: What Happens After a Pull Request Gets Approved?

Picture this: it's late, a developer opens a pull request touching the authentication flow, and a tired reviewer skims it, sees nothing obviously wrong, and clicks Approve. GitHub logs it as reviewed. Branch protection is satisfied. The change ships.
But was it actually reviewed — or just glanced at?
That gap between "approved" and "actually scrutinized" is where Vouch lives. It's a Code Review Intelligence and Risk-Aware Governance platform for GitHub that we built for the WeMakeDevs Bharat Builds Tour Hackathon 2026, and it treats approval not as a yes/no checkbox, but as a signal worth interrogating.

The Problem: "Approved" Isn't the Same as "Safe"

Branch protection can tell you whether someone clicked approve. It can't tell you whether the reviewer actually caught what mattered.
A pull request that touches authentication or another sensitive part of the codebase can be large, complex, and genuinely risky — and still sail through with a fast, low-engagement review. The approval is real. The scrutiny might not be.
As teams grow, this gap widens. Reviewers get stretched thin, reviews get shorter and more repetitive, and the risk that slips through doesn't disappear — it just becomes invisible. Left unaddressed, that shows up later as production bugs, missed security issues, emergency debugging, and delayed releases.
We wanted to make that invisible risk measurable. That's the whole premise of Vouch.

Three Ideas at the Core of Vouch

1. Change Risk — how risky is the code itself? Vouch weighs things like the size and complexity of a diff and whether it touches sensitive areas of the codebase. A one-line docs fix and a rewrite of the auth flow are not the same kind of change, and they shouldn't be treated the same way.
2. Review Quality — how meaningful was the review that happened? A 500-line change that gets a detailed back-and-forth in the comments is a very different situation from the same change getting a rubber-stamp approval and nothing else. Vouch looks at the review activity itself to gauge how much real engagement occurred.
3. Residual Risk — this is where the first two ideas combine:
Residual Risk = Change Risk × (1 − Review Confidence)
In plain terms: it's the risk that's still sitting in the codebase after the review happened. A high-risk change paired with a genuinely thorough review can end up with low residual risk. That same high-risk change paired with a rushed, low-engagement review is a different story entirely — and when residual risk crosses a defined threshold, Vouch flags the pull request for a second look.

Teaching a Model to Tell "LGTM" From an Actual Review

Not every comment carries the same weight. "LGTM" and a reviewer flagging a subtle edge case or a behavioral change are worlds apart — but a naive system might count them identically just because a comment exists.
To fix that, Vouch uses DistilBERT, a transformer-based language model, to analyze the substance of review comments rather than just their presence. This lets the system move past a binary "was there a comment?" check and toward a real answer to "was there meaningful discussion?"

The AWS Stack Behind Vouch

Vouch needed hosting, a data layer, security boundaries, observability, and AI — and AWS covered all of it. Here's how each piece earned its place.
Elastic Beanstalk — deployment, without babysitting servers Our Flask backend and dashboard run on Elastic Beanstalk, which let us deploy without hand-managing infrastructure. During a hackathon, that trade-off matters: every hour not spent wrangling servers is an hour spent on the actual product. It also gave us hands-on exposure to environment configuration, versioning, environment variables, and production debugging — the unglamorous parts of shipping that you only really learn by doing them under time pressure.
DynamoDB — a database that forces you to think differently DynamoDB became our serverless data layer, and it demanded a mental shift. Relational thinking starts with tables and joins; DynamoDB starts with a single question: how will this data actually be accessed? Designing around access patterns instead of schema first was one of the more valuable lessons of the whole build.
IAM — permissions as architecture, not afterthought Rather than giving the application broad, unrestricted access, IAM let us scope permissions down to exactly what each component needed. The bigger lesson here: cloud security isn't just about who can log in — it's about what each piece of the system is actually allowed to do. Fittingly, permission errors were also some of the most stubborn bugs we chased.
Secrets Manager — credentials that live outside the code Vouch talks to external services, which means credentials. Secrets Manager kept those out of source code entirely, flipping the pattern from "code contains the credential" to "code retrieves the credential when it needs it." That distinction matters a lot more once you leave your laptop and go to production.
CloudWatch — because deploying successfully isn't the finish line Once Vouch was live, the real question became: what's actually happening at runtime? CloudWatch gave us logging and monitoring so we could see failures, errors, and behavior in production — instead of guessing based on what worked locally.

When the Plan Meets Reality: Our Bedrock Detour

We'd originally planned to use Amazon Bedrock to generate plain-language explanations of Vouch's risk scores — turning "residual risk: high" into something a developer could actually act on.
Then we hit a wall: our model access quota started at zero, and despite requesting access, we couldn't get it approved in time for the hackathon.
It was a useful, slightly humbling lesson: having an architecture that supports a service and having actual access to that service in your account are two different things. So we designed Vouch's explanation layer to be provider-agnostic — able to plug into an alternative AI provider instead of hard-depending on Bedrock. It's a small design decision that came directly out of a real constraint, and it's a good reminder that quota and access timelines deserve a place in your planning, not just your architecture diagram.

From Approval to Intelligence

Put together, the pipeline looks like this:
GitHub Pull Request
↓
Analyze the Change
↓
Calculate Change Risk
↓
Analyze Review Quality
↓
Calculate Residual Risk
↓
Flag High-Risk Reviews
↓
Generate an Explanation
↓
Help the Team Decide What Needs Another Look

That last step is the real point. Vouch doesn't try to replace human judgment — it tries to point it at the right places.

It's Not Just About the Code — It's About the People Reviewing It

A team can be full of strong engineers and still have a broken review process, if too much of the reviewing load falls on too few people. So Vouch also looks at patterns around reviews: reviewer workload, review duration and pacing, repeated reviewer pairings, and signs of fatigue building up over time.
At a team level, that opens up questions worth asking:
  • Is review workload concentrated on a handful of people?
  • Are the codebase's most sensitive areas actually getting enough attention?
  • Are reviews trending faster and shallower over time?
None of this is about scoring individual developers. It's about giving teams visibility into whether their process — not any one person — needs adjustment.

What We Learned

  • Architecture matters early. The moment auth, a database, AI, monitoring, and deployment are all in play, a clear architecture stops being optional.
  • Managed services buy back time. Elastic Beanstalk and DynamoDB meant less time on infrastructure, more time on the actual product — which is exactly what you want in a hackathon.
  • Permissions are a design decision, not a cleanup task. IAM needs to be considered while you're architecting, not bolted on at the end.
  • Cloud services have real constraints. Quotas and access approval timelines are part of your project timeline whether you plan for them or not.
  • Monitoring isn't optional once you're live. Logs and observability matter as much as the code itself the moment something's in production.

What's Next for Vouch

There's a clear list of what we want to push further: deeper GitHub integration, more automated review recommendations, smarter reviewer workload balancing, richer risk explanations, team-level analytics, and stronger automation around pull-request governance overall.
The long-term goal hasn't changed since day one: make code review aware of both the code being changed and the humans reviewing it.

Closing Thought

Code review was never really about who clicked Approve. It's about whether a change got the attention it needed — and under deadline pressure, those two things quietly drift apart. Vouch is our attempt to catch that drift before it becomes a production incident.
Built with: GitHub, DistilBERT, and AWS (Elastic Beanstalk, DynamoDB, IAM, Secrets Manager, CloudWatch, and an in-progress Bedrock integration)
Built by Team Quantified Minds — WeMakeDevs Bharat Builds Tour Hackathon, 2026
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article