AIdeas: WorldMind Evidence
WorldMind Evidence is designed for regulated industries such as pharmaceutical research, clinical trials, and regulatory submissions, where every claim must be verifiable and traceable to authoritative documentation.
App category
Commercial Solutions
WorldMind Evidence is designed for regulated industries such as pharmaceutical
research, clinical trials, and regulatory submissions, where every claim must be
verifiable and traceable to authoritative documentation.
We achieve 90% accuracy against competitors 76.5%.
Our vision
Large language models can generate fluent answers to almost any question, but they cannot guarantee that those answers are grounded in verifiable evidence. In high-stakes environments such as medicine, law, and finance, this limitation prevents AI systems from being trusted in real workflows.
Our vision was to build an AI system that behaves less like a conversational chatbot and more like a responsible research assistant. Instead of producing plausible responses to every question, the system should answer only when the available evidence actually supports the claim.
To explore this idea, we built WorldMind Evidence, a clinical evidence intelligence system designed to query FDA clinical review documents. Instead of retrieving arbitrary text fragments like typical Retrieval-Augmented Generation (RAG) systems, WorldMind Evidence retrieves structured evidence units that we call holons.
A holon represents an atomic unit of clinical evidence extracted from a source document. Each holon contains the measurement, the patient population, and the precise location in the original document.
Before the system generates an answer, it passes through a Licensing Oracle, which is a verification layer that determines whether the AI system is allowed to answer the query. The oracle evaluates whether the available evidence matches the entities contained in the question and whether the evidence is sufficiently similar to support a reliable answer.
If the evidence passes these checks, the system is licensed to answer. If not, the system abstains.
This simple architectural change fundamentally alters the behavior of the AI system. Instead of attempting to answer every question, the system first determines whether it is allowed to answer at all.
We implemented a working prototype using AWS serverless infrastructure and evaluated it against a traditional RAG pipeline using a real FDA clinical review document. In testing on 200 clinical queries, the system achieved 90% overall accuracy and produced four times fewer false positives than a standard RAG system, while maintaining full traceability to the original clinical evidence.
The result is a working prototype that demonstrates a new design pattern for trustworthy AI. It demonstrates evidence-bounded reasoning, where a system verifies that evidence supports a claim before it is allowed to generate an answer.
Why this matters
AI hallucination is often treated as a nuisance problem. In many applications, a slightly incorrect answer may not cause serious harm. In regulated domains, however, the consequences are very different.
Pharmaceutical researchers, regulatory reviewers, and clinicians frequently rely on precise numerical values from clinical trial reports. If an AI system fabricates or misinterprets those values, the consequences could include incorrect regulatory conclusions, flawed analyses, or even risks to patient safety.
The fundamental problem is architectural. Most language models generate answers by predicting statistically likely token sequences. Even when they retrieve documents, they still attempt to synthesize answers from fragments that appear relevant. This means a system can appear confident even when the underlying evidence does not actually support the claim.
WorldMind Evidence demonstrates a different approach. Instead of asking the AI to answer every question, the system enforces a verification step before generation. If the evidence does not match the query, the system simply abstains.
In other words, the AI system is designed to know its limits.
This approach has implications far beyond clinical research. Any domain that depends on authoritative documents and verifiable claims could benefit from evidence-bounded AI systems. Examples include regulatory filings, financial reporting, legal contracts, and scientific literature.
By combining structured evidence units, entity-level validation, and licensing decisions before generation, systems like WorldMind Evidence could enable a new generation of AI assistants that are not only powerful but also trustworthy.
The Team
Richard Ackermann - @holonic ( WorldMind founder, co-author of Licensing Oracle framework) led system design. With 30+ years in regulated domains including large-scale assessment systems serving millions of users, he focuses on AI architectures that are structurally reliable in high-stakes environments.
Simeon Emanuilov (PhD, AWS Certified, UnfoldAI founder) led technical implementation and AWS architecture, using Kiro IDE and Spec-Driven Development to accelerate development.
How we built this
To test whether evidence-bounded AI systems could reduce hallucinations, we built a working prototype that answers questions about FDA clinical review documents while ensuring that every answer is grounded in verifiable evidence.
The system combines three core components:
• structured evidence extraction
• a licensing oracle that validates queries before answering
• a serverless AWS architecture that orchestrates the workflow
The pipeline retrieves evidence, verifies whether the system is allowed to answer, and only then generates a response.
Step 1: Creating structured evidence
Traditional RAG systems retrieve chunks of text from documents. While effective for general knowledge tasks, text fragments often contain incomplete or ambiguous information in technical domains.
Instead of storing text fragments, we extracted structured evidence units, called holons, from the tables in the FDA clinical review for the Pfizer-BioNTech COVID-19 vaccine (Comirnaty Monovalent XBB.1.5, August 27, 2025).
Example holon:
Using Gemini 2.5 Flash, we extracted structured measurements from the document tables.
Dataset summary:
• 86 pages analyzed
• 32 clinical tables processed
• 590 structured holons extracted
Each holon contains the measurement, context, and exact source location in the document.
Step 2: Entity-based query understanding
When a user submits a question, the system first extracts key entities from the query.
Using GPT-4o-mini, the system identifies three entity types:
• clinical endpoint
• patient population
• timepoint
Example query:
“What were the SARS-CoV-2 neutralization titers in children 5-11 years?”
Extracted entities:
- Endpoint: NT50 neutralization titer
- Population: Children 5-11 years
- Timepoint: unspecified
This structured representation allows the system to compare the query directly with the entities stored in the holons.
Step 3: The Licensing Oracle
The Licensing Oracle determines whether the AI system is allowed to answer the question.
The oracle combines two forms of validation.
Entity validation
Entities extracted from the query are compared with entities stored in the holons.
Endpoint matches are required. Population and timepoint must match if they appear in the query.
This deterministic filtering prevents common hallucination scenarios where similar text refers to a different study context.
Similarity validation
Remaining candidate holons are evaluated using embedding similarity.
If similarity is above a defined threshold, the system is licensed to answer. Otherwise, the system abstains.
This two-stage validation prevents what we call subtle misbinding, where queries resemble evidence semantically but refer to a different experimental context.
Step 4: Answer generation
Only after the query is licensed does the system generate an answer.
Matching holons are passed to Gemini 2.5 Flash, which generates a concise response and includes the exact source location.
Example response:
“The NT50 titer was 271.0 (Table 13, Page 34).”
Users can then jump directly to the source document to verify the claim.
If the system cannot license the query, it returns an abstention message indicating that the available evidence is insufficient.
A note on entity extraction alternatives
GLiNER2 is a lightweight open-source Named Entity Recognition model well-suited for structured clinical text. The model can be fine-tuned on domain-specific entity types, making it effective for consistently identifying clinical endpoints, patient populations, and timepoints. Prior experiments confirmed strong performance on this extraction task. In the current implementation, GPT-4o-mini was selected over GLiNER2 primarily due to deployment constraints — GLiNER2's model size makes it less practical for AWS Lambda cold-start environments. For deployments running dedicated inference infrastructure or containerized workloads, GLiNER2 remains a compelling and cost-effective alternative for entity extraction.
AWS architecture
The system runs on a serverless AWS architecture.
Key services include:
AWS Lambda
Orchestrates the query pipeline, including entity extraction, holon retrieval, licensing decisions, and answer generation.
Amazon DynamoDB
Stores holons, embeddings, and metadata.
Amazon API Gateway
Provides the REST interface for query requests.
Amazon S3 and Amazon CloudFront
Serve the web dashboard and provide secure access to the FDA documents referenced by the holons.
AWS Secrets Manager and AWS Certificate Manager
Manage credentials and secure communication.
Kiro IDE
We used SDD (Spec-Driven-Development) approach and Kiro IDE to build this prototype.
This architecture allowed us to deploy a fully functional prototype while keeping infrastructure costs extremely low, approximately $1.15 per month during development.
Evaluating the system
To measure performance, we compared WorldMind Evidence with a traditional RAG system using the same:
• embeddings
• language model
• source document
• query dataset
We evaluated both systems on 200 clinical queries, including in-scope and out-of-scope questions.
Results:
WorldMind Evidence
• 90% overall accuracy
• 11 false positives
Traditional RAG
• 76.5% overall accuracy
• 45 false positives
The most important result is the reduction in false positives. In clinical research, a system that confidently produces the wrong answer is far more dangerous than one that declines to answer.
By introducing a licensing stage before generation, the system reduced incorrect answers by a factor of four.
Demo
You can explore the system yourself.
Video overview:
Example query with verified answer
Query:
“What were the SARS-CoV-2 neutralization titers in children 5-11 years?”
Response:
“The NT50 titer was 271.0 (Table 13, Page 34).”
Users can jump directly to the original FDA document to verify the result.
Example query where the system abstains
Query:
“How many investigators participated in the study?”
This information is not present in the dataset.
The system responds:
“I cannot provide a confident answer based on the available evidence.”
This behavior prevents the system from generating speculative answers.
What we learned
Building WorldMind Evidence revealed several important lessons about designing AI systems for high-stakes environments.
Structured evidence improves retrieval
Transforming documents into atomic evidence units dramatically improved alignment between queries and retrieved information. Structured evidence reduces ambiguity that often arises when models rely on narrative text fragments.
AI systems must be able to abstain
Many hallucinations occur because AI systems are designed to answer every question. Introducing a licensing stage allows the system to abstain when evidence is insufficient, making the system far more trustworthy.
Entity validation prevents subtle hallucinations
Semantic similarity alone is not sufficient for reliable retrieval. Explicitly matching endpoints, populations, and timepoints prevents queries from being answered using evidence from the wrong context.
Serverless architecture accelerates experimentation
Using AWS serverless services allowed us to focus on the AI architecture rather than infrastructure management. The system scaled easily while remaining inexpensive during development.
Evidence-bounded AI may be a useful design pattern
The most important insight from this project is architectural. AI systems used in professional workflows must prioritize verification and traceability.
WorldMind Evidence demonstrates a new design pattern for AI systems: evidence-bounded reasoning, where the system verifies that evidence supports a claim before it generates an answer.
As AI systems become integrated into critical decision-making processes, architectures that combine powerful language models with strong verification mechanisms may become essential.
Closing note
This project demonstrates how cloud-native architectures on AWS can be used to prototype new AI systems that prioritize verification, traceability, and trust in real-world decision-making environments.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article