
Churnalist: The One Rule That Made Everything Else Possible
How we learned that the number everyone argues about is the one thing you can't use to find the story.
On February 20th, 2020, a KSRTC bus collided with a container lorry near Avinashi, Tamil Nadu. In the days that followed, the reported death toll moved 17, then 19, then 20 depending on which outlet you read and when you read it.
The easy reaction is to ask: which number was right? That's the wrong question, and it's the question we deliberately built Churnalist not to answer.
Nobody necessarily did anything wrong here. Casualty figures update as hospitals report in, as wire services correct earlier counts, as one newsroom copies another's early estimate before a revision lands. The problem isn't that the number changed. The problem is that a reader scrolling their feed that week had no way to see it changing they saw one version, assumed it was complete, and moved on. Five people reading five different outlets walked away with five different facts about the same tragedy, each fully convinced they'd read the news.
That's the actual thing we set out to build: not a fact-checker, not a fake-news detector, not a system that adjudicates who was right. A system that shows the escalation itself this outlet said this, at this point, this one said that, and here's the shape of how it moved and leaves the judgment to the person looking at it.
The feature we refused to rush
Everything else in Churnalist claim extraction, source discovery, report generation exists to support one output: an honest, ordered, sourced picture of how a number (or a claim, or an attribution) changed across coverage. We treated that output as the one thing that had to be right before anything else mattered, because it's the only part of the system a reader actually looks at and trusts or doesn't.
Getting it right meant being disciplined about two things we could easily have gotten wrong.
First: never present the drift as a verdict.
It would have been easy to build a system that says "Tamil coverage inflated the death toll" or "the correction was suppressed" That's a claim about intent, and we have no basis for it a number moving from 17 to 20 is equally consistent with careless copying, an editor rounding up, or new information arriving. So the output never says why something changed. It says what changed, where, and when, with the original sentence next to it, and stops there. The reader supplies the interpretation, because they're allowed to and we're not.
Second: never let the thing that's escalating decide which articles belong together.
This one is subtler and it's where most of our early build time actually went. If you try to group articles by matching the reported number, you've built something that can only notice drift when the number happens to survive unchanged the moment it changes, your matcher fails to connect the articles at all, and you report nothing. So the system groups articles by incident type, location, named entities, and timing deliberately excluding the contested figure from that decision precisely so it can still show you the escalation instead of losing it.
def event_similarity(article_a: Article, article_b: Article) -> float:
score = 0.0
score += 0.30 * incident_type_match(article_a.incident, article_b.incident)
score += 0.25 * location_overlap(article_a.entities, article_b.entities)
score += 0.20 * temporal_proximity(article_a.published_at, article_b.published_at)
score += 0.15 * named_entity_overlap(article_a.entities, article_b.entities)
score += 0.10 * semantic_similarity(article_a.embedding, article_b.embedding)
return scoreNeither of these is a modeling problem. Both are restraint problems deciding, ahead of time, what the tool is not allowed to claim, and building the pipeline so it's structurally incapable of claiming it, rather than trusting a prompt to behave.
What the output actually looks like
The output isn't a score, a label, or a red flag. It's a timeline:
17 injured → Source A, [time] → "17 people were injured, according to officials"
19 injured → Source B, [time] → "19 people were injured"
20 injured → Source C, [time] → "Around 20 people were injured"No claim of which is correct. No red X next to the outlet that said 20. Just the sequence, the sourcing, and the exact wording each time so the reader can see, in about five seconds, that this number moved, and decide for themselves what to make of that.
That restraint was the actual design goal from day one, and it's also the hardest part to build, because every instinct in building a "detection" tool pulls toward producing a verdict a score, a flag, a confidence percentage. Those all feel more impressive in a demo. They're also all a claim we can't actually back, and the first time the tool is wrong about one, it stops being trustworthy about all of them. Showing evidence and staying quiet doesn't fail that way. It can be incomplete, but it can't be wrong in the way a verdict can be wrong.
Why this mattered more than adding features
We had a longer feature list early on sentiment scoring on the drift, a "reliability index" per outlet, an auto-generated verdict on which version was most accurate. We cut all three, not because they were hard to build, but because each one turns the tool from "here's what happened" into "here's what to think" and that's a line we didn't want to cross with a system built on data we can't fully verify in real time. A reliability score implies confidence we don't have. A verdict implies authority we shouldn't claim.
What we kept instead was the one feature that does exactly what a reader actually needs: the escalation, ordered, sourced, and undecorated. Getting that to work correctly matching the right articles without letting the disputed number do the matching, and presenting the result without editorializing on top of it took most of the build. Everything else in the pipeline exists to make that one output trustworthy.
The Architecture, and Why It's Shaped the Way It Is
Churnalist's pipeline runs across eight layers, and the shape of it follows directly from the one rule the whole system is built around: never let the thing under investigation decide how the system reasons about it. Every layer below exists either to gather evidence honestly, to keep that evidence traceable, or to present it without editorializing.

Frontend : investigation, not verdict, as the unit of the UI.
The frontend isn't built around a headline result. Its main surfaces are a live news feed, an investigation view, a provenance/claim-lineage view, and story clustering all framed around tracing, not scoring. There's no page in this app whose job is to tell you a story is fake. There's a page whose job is to show you its lineage.
FastAPI ingestion layer.
Requests come in through a standard API router, but the ingestion service itself does the unglamorous, load-bearing work: pulling candidate articles via SerpAPI and RSS, scraping and parsing them, and streaming progress back to the frontend over Server-Sent Events so an investigation doesn't feel like a black box while it runs you watch sources get pulled in, not just wait for an answer to appear.
Multi-agent orchestration, on Strands.
This is the layer that matters most for the Build It track specifically. The orchestrator is built on the AWS Strands Agents SDK, an open-source, model-agnostic agent framework meaning the same orchestration code that runs today against a local model would point at Amazon Bedrock without a rewrite if this ever moved to Ship It. Strands is what turns "an LLM figures it out" into a pipeline with defined stages and defined handoffs, which matters because each stage here has a different, narrower job than "understand the news":
- Source Hunter : finds candidate articles
- Event Weaver : decides which of those candidates are actually the same event, using incident type, location, entities, and timing, deliberately excluding whatever value is being investigated
- Claim Miner : extracts the atomic factual claims from each article once the event boundary is settled
- DRIFT Investigator : compares those claims across the set and orders the escalation
- Truth Trail : links every claim back to its exact source sentence, so nothing in the output is unsourced
No single agent is trying to do all of this at once, which is the same discipline as the event-matching rule applied one level up: narrow, sequential responsibility instead of one model asked to reason about everything simultaneously.
Vector and lexical retrieval, on OpenSearch.
Semantic matching the part that lets Event Weaver recognize that a Tamil-language rewrite and an English wire story are describing the same accident despite completely different phrasing runs on Amazon OpenSearch, using multilingual sentence embeddings for the semantic side and lexical search alongside it for exact-term matches like place names and organization names that embeddings alone can blur. OpenSearch is the second piece, alongside Strands, that puts this squarely inside the Build It track's own open-source list rather than reaching for it as an afterthought.
Evidence storage.
Every report and every piece of source evidence gets persisted rather than held only in memory, so a finding can be re-opened and re-verified later instead of trusted once and forgotten the same instinct as citing a sentence instead of asserting a conclusion, just applied to storage instead of language.
Telemetry.
Metrics and logs get captured across the pipeline, mainly so a failure in one agent stage is visible and diagnosable rather than silently swallowed into a wrong-looking final output.
LLM inference.
Model calls run through Groq's hosted inference for speed during development, swapped in cleanly because Strands treats the model as a provider, not a hard dependency the same swappability the SchemeProof writeup describes with Ollama and Bedrock.
In Simple Words
Churnalist doesn't tell you what's true it shows you what changed, who said it, and when, and lets you decide the rest. That restraint, applied to the news and to our own architecture alike, is the whole project.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article