AWS Builder Center

Designing Error Messages as a Product Surface: An Evidence-Based Method

For a developer tool, the error response is read at the exact moment a user is blocked, and increasingly it is read by an autonomous agent. This article argues it should be designed and reviewed like a product interface, not treated as an implementation detail. It draws on platform usability guidance, an IETF standard, and current agent-recovery research, and distinguishes what is standardized from what is still a single-study finding.

Series: agentic-ai (4 articles)

  1. 1
    Designing Error Messages as a Product Surface: An Evidence-Based Method This article
  1. The argument
A user who completes a happy path may never read a sentence of your prose. A user who hits an error reads the response word for word. The error path therefore carries a disproportionate share of the product's actual UX, and Microsoft's design guidance states the consequence directly: poorly written error messages are a leading cause of avoidable technical support costs [1]. The word "avoidable" matters — it implies the information needed to recover exists and is being withheld or obscured, rather than the failure being inherently unsolvable.
The design goal is not friendlier wording. It is an error response that lets the caller recover without a human intermediary.
  1. Designing for the human reader
The established human-facing contract is three-part: state the problem, explain the cause, and provide a solution the user can actually perform [1]. Microsoft lists the characteristics of a good error as relevant, actionable, user-centered, brief, clear, specific, courteous, and rare [1]. Two of these do most of the work.
Specificity is the antidote to the generic error. Guidance is to avoid vague wording such as "syntax error" or "illegal operation" and instead supply the specific names, locations, and values of the objects involved [1]. Echoing the rejected value lets a developer recognize their own input rather than reproduce the request mentally. The same guidance requires a unique error code per problem and that the relevant parameters be preserved through the handling layer, because a specific message cannot be produced at the boundary if the underlying code discards the context [1].
Actionability constrains length and tone. A message should change what the user does next; if there is no action the user can take, the guidance says to suppress the message rather than display a dead-end notification [1][2]. Conversely, dumping a full stack trace or a multi-step troubleshooter into the message is over-communication — detail beyond the most likely fix should sit behind a documentation link [1]. This is also where a security boundary belongs: diagnostic detail should not leak internal implementation or secrets, a normative constraint rather than a stylistic one.
  1. Designing for the machine: a standard format
Prose is for humans; programmatic handling needs stable structure. RFC 9457 defines Problem Details for HTTP APIs, a machine-readable format serialized as application/problem+json, explicitly to avoid every API inventing its own error schema [3]. Its five core members are:
Member
Purpose
type
A URI identifying the problem type; the primary identifier consumers should branch on
title
A short, human-readable summary of the problem type
status
The HTTP status code, advisory and kept consistent with the response
detail
An occurrence-specific human-readable explanation
instance
A URI identifying the specific occurrence
Two design positions in the RFC are worth noting. The detail field "ought to focus on helping the client correct the problem, rather than giving debugging information," and consumers are told not to parse it; machine-readable data belongs in typed extensions instead [3]. The format is explicitly extensible — the RFC's own validation example adds an errors array where each entry carries a detail and a JSON Pointer locating the offending field [3]. This separation is the structural version of the human-facing principle: stable codes for branching, prose for comprehension, extensions for data.
  1. Why this matters more when the caller is an agent
When an LLM drives an API, the error response is an input to the model's next reasoning step, not just text on a screen. Research on environment-feedback self-correction finds that agents recover more reliably when intermediate feedback is rich and specific, and that self-correction should be triggered selectively rather than on every step [6]. Work on structured reflection for tool use similarly frames the tool error message as the basis for diagnosing a failure and emitting a corrected call [5].
The most direct empirical evidence comes from a recent study introducing self-reflective APIs, whose validation failures include a machine-readable recovery_feedback.suggestions[] payload naming the parameter changes needed to retry [4]. On a leak-audited pilot across adversarial, domain-specific tasks, structured suggestions raised task-completion over plain-English diagnoses by 36.7 to 40.0 percentage points on two Anthropic models (Fisher's exact, p ≤ 0.0022) while improving per-success token efficiency by 1.8 to 2.2 times [4].
The same study supplies the caveats that keep this claim honest. The improvement was not statistically significant on gpt-4o-mini (p = 0.435) at the tested sample size, and the largest gains appeared specifically on proprietary constraints that a general model cannot reconstruct from training data; on tasks the model already understood, verbose prose sometimes saturated and structure added little [4]. The defensible conclusion is therefore narrower than "structured always wins": any per-rule diagnostic substantially beats a generic code, while a typed recovery payload adds its strongest value on context-specific, non-public rules.
  1. A consolidated design contract
Combining the standard and the guidance, an error response should satisfy all of the following:
  1. A stable, unique machine-readable type the client can branch on, mapped to the correct HTTP status.
  1. A human-readable problem, cause, and an action the caller can perform — no dead-end messages.
  1. Specific names, locations, and the echoed rejected value; parameters preserved end to end.
  1. Machine-readable data in typed extensions rather than encoded into parseable prose.
  1. A documentation link for anything beyond the most likely fix.
  1. A review that checks for information leakage and for whether the suggested action is genuinely likely to work.
  1. For domain-specific validation, an optional structured recovery payload an agent can consume directly.
  1. Limits of the evidence
The human-facing principles are long-standing vendor guidance rather than controlled experiments, and the machine format is a standard whose adoption is voluntary. The quantified agent-recovery results rest largely on one recent, carefully audited study with a modest per-cell sample and a null result on one model; they should be treated as directional and re-validated against your own workload rather than as settled effect sizes. None of these sources removes the need for a reviewer who understands the underlying business rule — a coherent error can still encode a wrong assumption.



References

  1. IETF, RFC 9457: Problem Details for HTTP APIs —  https://www.rfc-editor.org/rfc/rfc9457 
  1. Self-Reflective APIs: Structure Beats Verbosity for AI Agent Recovery (arXiv:2606.05037) —  https://arxiv.org/html/2606.05037 
  1. Failure Makes the Agent Stronger: Structured Reflection for Reliable Tool Interactions (arXiv:2509.18847) —  https://arxiv.org/html/2509.18847 
  1. QueryAgent: Reliable Reasoning with Environmental Feedback based Self-Correction, ACL 2024 —  https://aclanthology.org/2024.acl-long.274/ 
#developer-tools #api-design #error-handling #ai-agents
The views expressed here are my own and do not represent those of my employer.

Series: agentic-ai (4 articles)

  1. 1
    Designing Error Messages as a Product Surface: An Evidence-Based Method This article
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article