Three Failure Modes of GenAI Agents and how to fix it

I have been thinking, for several weeks, about a question that started in healthcare and ended somewhere broader. So…I decided to deep dive into this matter.

Healthcare is the domain where a particular distinction stops being a philosophical preference and becomes a design constraint with real consequences. The distinction is the one between an AI assistant and an AI decision-maker. Most workflows tolerate ambiguity on this line because the cost of getting it wrong is low. Clinical workflows do not. In February 2026, Brazil’s Conselho Federal de Medicina codified this distinction in Resolution 2.454/2026, requiring AI in medicine to operate exclusively as a support tool, with the physician as the final decision-maker, with effect from August 2026.

That is the regulatory context. It is not the subject of this article.

The subject is what I noticed when I started pulling on the thread. The patterns that make a Generative AI (GenAI) agent a good assistant in a clinical workflow are the same patterns that make it a good assistant anywhere the stakes are high. They are GenAI architecture patterns. Healthcare is where you stop being able to skip them. Financial advisory, legal review, security operations, infrastructure automation: every domain where a wrong-but-confident answer carries real cost has the same three failure modes, and benefits from the same three patterns.

This is a practitioner’s note about those failure modes and what to do about them.

If you have built agentic systems with Strands, AgentCore, the Model Context Protocol (MCP), or any orchestrator that delegates between language model calls, you have seen at least one of these in the wild. Most teams have seen all three and called them by different names.

The first is hallucination in a context where the model lacks coverage. Foundation models are pretrained on a corpus where common topics dominate the head and rare topics occupy the long tail. When an agent operates in a domain whose terminology, taxonomy, or evidence base sits in that long tail, the base rate of confident but wrong output rises. Rare conditions in medicine, niche regulations in finance, edge cases in infrastructure: same mechanism. The model produces fluent text that has the shape of an answer but does not correspond to verified knowledge.

The second is prompt injection through a tool boundary. Agentic systems pull external content through MCP tools, Retrieval-Augmented Generation (RAG) pipelines, or direct application programming interface calls. When that content includes free-text fields written by humans or extracted from unstructured documents, the payload can contain imperative sentences that the language model reads as instructions. The MCP specification treats tool responses as opaque content at the protocol layer, which is correct for the protocol and unsafe at the application layer.

The third is chain break with stale or missing context in multi-agent flows. A supervisor delegates to a specialist. The specialist expects a context payload conforming to an implicit contract. The payload arrives with a missing field, an outdated timestamp, or a confidence indicator below the operational threshold. The specialist, trained to produce coherent output, fills the gap with a plausible inference rather than refusing. The chain completes. The forensic trail later shows the inference was the failure point, not the model.

These three failures share a common root. The agent was allowed to act when it should have refused, or to fill a gap that it should have surfaced. The architectural fix is to remove the option to do either.

The first failure mode disappears when the agent is structurally prevented from invoking a language model without a complete, verified context. The pattern has three parts.

The supervisor agent owns the context capture as a deterministic step before any model call. Deterministic here is precise. The supervisor uses non-probabilistic mechanisms to assemble the minimum viable context for the operation. Database queries against authoritative sources. Structured lookups against verified registries. Hash comparisons against known-good references. None of these involve a language model.

Each piece of captured context carries three pieces of metadata: where it came from, when it was captured, and the confidence level if the source provides one. This metadata is not optional. It is the audit substrate.

When the supervisor cannot assemble the minimum viable context for the requested operation, the correct behavior is not to invoke the specialist with partial context. The correct behavior is to return a structured refusal to the upstream caller, explicitly listing what is missing.

In Strands Agents, the supervisor’s system prompt is written to make this contract explicit. It does not invite the model to reason about missing information. It instructs the supervisor to evaluate the completeness of the capture and to surface gaps when the check fails. The system prompt is part of the architecture, not part of the conversation.

For a clinical assistant: a supervisor preparing context for a differential diagnosis suggestion captures patient demographics, current vitals, active medications, and recent laboratory results from the Electronic Health Record. If the most recent laboratory result is older than the operational freshness threshold, or if the medication list returns with a “reconciliation pending” flag, the supervisor returns insufficient context to the physician. The model never runs on stale data.

The same shape works for a financial advisory assistant that needs current account balances and recent transactions, or an infrastructure operations assistant that needs current deployment state. The domain changes. The pattern does not.

What you buy with this pattern is an agent whose refusals are explicit and auditable. The team operating the system can review the refusal logs and see exactly which queries the system declined and why. The system that always answers is the system that has always been guessing when it should have refused.

The second failure mode disappears when the agent treats every tool response as classified data, not as opaque text.

The MCP specification gives you a clean tool abstraction. It does not require tools to classify what they return. That classification is a convention you adopt in your tool catalog, and the convention has to be mandatory for every tool that touches the agent, including third-party tools you wrap in adapters.

Every tool response carries four structured fields beyond the payload itself.

A classification tag indicating the sensitivity tier of the content. The tiers reflect your domain. For health data under Brazil’s Lei Geral de Proteção de Dados, the relevant tiers are public, personal, sensitive_health, and restricted_phi. For financial data, the tiers map to the regulatory regime you operate under. The classification is decided by the tool implementation based on the source, not by the language model after the fact.

A source authority indicating how the content was produced. A clinical guideline retrieved from a verified terminology service has a different authority than a free-text note typed by a patient through a portal. The supervisor uses authority to decide what enters the language model prompt and what stays out.

A content hash that lets the audit trail prove which exact payload was used in which exact inference. When an operator or a regulator asks what data the agent saw when it produced a given recommendation, the hash is the answer.

A freshness timestamp that the supervisor uses to enforce operational time constraints. Stale data triggers re-capture, not silent use.

The supervisor’s policy then becomes explicit and reviewable. Content classified above a sensitivity threshold enters the model context only when the operation requires it and only under defined guardrails. Content from low-authority sources passes through a non-language-model sanitizer first. Imperative sentences embedded in free-text fields are stripped or escaped before any model sees them. The model is not asked to recognize a prompt injection. The pipeline removes the injection vector before the model is involved.

Amazon Bedrock Guardrails belong in this picture, but they operate on content, not on context. Guardrails filter both input prompts and model responses against denied topics, word lists, and sensitive information patterns. They do not classify a piece of input by where it came from or which authority produced it. You need both layers. Source-aware classification owned by your tool catalog. Guardrails on the content itself.

For a clinical assistant: the supervisor pulling free-text content from a prior consultation note classifies the payload as restricted_phi from a physician_authored source. A patient portal submission for the same patient classifies as restricted_phi from a patient_authored source. Both are restricted, but the second goes through an injection sanitizer first, because patient-submitted free text is a different threat profile than physician-authored notes in a controlled Electronic Health Record system.

The same pattern protects financial inputs from advisor notes versus client-uploaded documents, or infrastructure inputs from operator-authored runbooks versus auto-generated incident reports. Source authority is the discriminator. Classification is the contract.

The third failure mode disappears when every specialist agent in the chain validates its incoming context against a contract before producing any output, and refuses cleanly when the contract is not met.

The pattern is structurally simple and behaviorally hard. Language models are trained to produce coherent output. A specialist agent that receives a context payload with a missing field will, by default, produce something. The “something” will be syntactically plausible and semantically wrong. The architectural decision is to remove that default.

Every specialist boundary declares an explicit context contract. The contract specifies which fields must be present, which types they must conform to, and what minimum confidence threshold applies for the operation. The validation runs before any inference. When the contract is not met, the specialist returns a structured insufficient_context response, not a guess.

The supervisor has a defined response to insufficient_context. It does not retry the same specialist with the same context. It attempts deterministic re-capture of the missing field exactly once. If re-capture fails, the chain terminates and the human in the loop sees a transparent reason. Retry-with-the-same-input is the anti-pattern that turns intermittent context failures into systematic hallucination.

Strands Agents gives you the hooks for this. The policy decision is yours. Decide explicitly, at architecture time, how every agent in your topology behaves when it receives invalid context. Encode the decision in the agent definition, not in the orchestrator, because orchestrator logic drifts and per-agent contracts persist.

For a clinical assistant: a specialist agent producing a medication interaction check expects patient identifier, full active medication list, recent laboratory results relevant to renal and hepatic function, and known allergies. If the allergy list arrives with a “patient unable to confirm” flag, the specialist returns insufficient_context with the specific reason. The physician sees a transparent gap and decides whether to proceed with the partial information or to capture the allergy data first. The system did not silently assume "no known allergies." Silent assumption is the failure mode.

The same shape protects a financial portfolio rebalancing specialist that needs current positions, a security incident triage specialist that needs current alert state, or any specialist agent whose output depends on context the supervisor cannot always guarantee.

What you buy with this pattern is a system that surfaces its uncertainty as a feature, not as a defect. The user trusts the system more, not less, when it refuses cleanly. A system that always answers eventually answers wrong, and the trust collapse from one bad confident answer costs more than a hundred clean refusals.

Four notes that are not always obvious from reading the official documentation for these services.

The first is that Bedrock Guardrails are content filters, applied to input or output, not source-aware data classifiers. Guardrails decide whether a piece of text crosses a content policy boundary. They do not decide whether that piece of text came from an authoritative source or from a free-text field a user typed last Tuesday. You need a classification layer in front of the model, owned by your tool catalog, that the model cannot influence.

The second is that AgentCore Memory is operational memory for conversation continuity, not an audit trail for regulated workflows. Memory in AgentCore stores conversation context across sessions. An audit trail has different obligations: tamper-evident storage, defined retention windows per data category, segregation from operational logs, signed records for forensic review. Build the audit trail as a separate concern, with separate storage, separate access policies, and separate retention.

The third is that Strands Agents does not export a built-in “fail-closed” retry policy. The framework gives you the hooks to implement it. The policy is yours to define. Make the definition explicit at architecture time, encode it per agent rather than per orchestrator, and document the refusal behavior alongside the success behavior. The agent that refuses correctly is half the agent.

The fourth is that the MCP specification does not mandate envelope-level classification. Classification is a convention you adopt in your tool catalog. Every tool you write, and every third-party tool you connect, must return a response that conforms to your classification contract. Third-party tools that do not classify their payload need an adapter that classifies on the way out. The adapter is non-negotiable for any tool whose response will reach a model.

These three patterns share a single conviction.

A GenAI agent is most useful when it is structurally prevented from doing the work that belongs to a human. Capture before invoke prevents the agent from inventing context it does not have. Classify at the boundary prevents the agent from treating user-submitted text as a trusted instruction. Fail-closed by default prevents the agent from filling a gap that should have been surfaced.

The agent that says “I do not have enough information to answer this safely” is not a worse agent. In any workflow where the stakes are real, it is the only kind of agent worth deploying. The agent that always has an answer is the one that has been guessing all along.

Build for the refusal. The answers take care of themselves.

LinkedIn  · Medium  · AWS Builder Center  · Golden Jackets Brazil