
Reversible Sensitive Data (PII, PHI) Masking in Agent Logs
AI agents produce rich traces for auditing — but those traces often contain PII/PHI. This article proposes a technique for keeping sensitive data out of agent logs on AWS.
AI agents that process real customer data generate rich reasoning traces. Those traces are essential for debugging, auditing, and understanding what your agent did and why. They're also a compliance problem — the same detail that makes a trace useful tends to contain sensitive data.
Sensitive data is any information that needs to be protected from unauthorised access: personally identifiable information (PII) like names and addresses, protected health information (PHI), financial data like account numbers and transaction details, credentials, and anything else your organisation or regulators require you to handle carefully.
If you are building your AI agents on AWS with AgentCore, Amazon Bedrock AgentCore Observability is where those traces live. It gives you real-time visibility into agent performance — execution paths, intermediate outputs, token usage, latency — by emitting spans and logs into Amazon CloudWatch. The richer the trace, the more useful it is for debugging and auditability. It also means more sensitive data flowing into your logs.
The Approach
CloudWatch Data Protection (CWDP) is the natural first line of defence, and you should enable it. But relying on CWDP alone means relying on an ML-based classifier to find sensitive data in free text after the fact. There's a better layer to add on top: instead of asking a classifier to find sensitive data, have the agent mark it as it writes — then use a deterministic regex to catch it at egress.
The agent already knows which fields contain sensitive data — it extracted them. That's the key insight. Rather than asking an unrelated classifier to re-derive sensitive values from raw text, you instruct the agent to bracket them with a fixed delimiter as it writes them into free-text fields. A regex-based CloudWatch custom data identifier then matches that delimiter reliably.
In three steps: 1/ Instruct the agent to wrap sensitive values in a marker delimiter as it writes them 2/ Register a custom data identifier in CWDP with a regex that matches your chosen delimiter 3/ Keep the managed identifiers as a backstop for anything not explicitly marked
I use
[[ ]] as the delimiter throughout this article — it's readable, unlikely to appear naturally in agent output, and straightforward to match with a regex. Any delimiter that is distinctive, consistent, and regex-friendly works. Pick one that doesn't conflict with formats already present in your agent's output.The result is deterministic coverage where it matters most: sensitive values the agent already extracted and re-mentions in free-text reasoning fields like
reasoning or summary .Implementation
Marking Sensitive Data at the Source
The most reliable place to add the marking instruction is the Pydantic field description. That description becomes part of the tool schema passed to the model via the Bedrock Converse API — so it travels with the field it protects, rather than living only in the system prompt where it's easier to miss or for future prompt edits to drop. See the following examples.
class PayslipData(BaseModel): employee_name: str | None = Field( default=None, description=( "Full name of the employee. Use [[ Full Name ]] format for " "printing Full Name of a person." ), )
class LegitimacyAssessment(BaseModel): reasoning: str = Field( description=( "Full step-by-step reasoning behind the verdict, in prose. " "Use [[ Full Name ]] format for printing Full Name of a person." ) )The
reasoning field is the important one. It's a free-text field where the model re-mentions names it already extracted into structured fields like employee_name . Structured fields are easy — you know exactly where the sensitive data is. Free-text fields are where it leaks.The Custom Data Identifier Regex
Register this regex as a custom data identifier in your CWDP policy, alongside the managed identifiers:
\[\[([\p{L}\p{M}\s.'\-]+)\]\]-
\p{L}/\p{M}— Unicode letter and combining-mark categories, so text using non-Latin scripts or accented characters (José,Müller,Nguyễn) are matched
-
.,',-— covers common name punctuation (St. Clair,O'Brien,Jean-Luc)
- Configure it to mask the full match including the brackets, so the markers themselves don't appear in the masked log as a signal that something was redacted
The result:
[[John Doe]] in the trace becomes *********** in the log.Why Reversibility Matters
A flat replace — swapping
John Doe with a static placeholder — destroys the data permanently. A bracketed marker preserves it behind a layer that privileged principals can lift using logs:Unmask . That distinction matters when you need to:- Audit an agent decision that had material consequences for a customer
- Satisfy a retrospective investigation or Responsible AI obligation
- Retain data in a compliant form without exposing sensitive data to the log aggregation pipeline
The custom and managed identifiers are complementary. The custom identifier catches sensitive values the agent explicitly marks — deterministically. The managed identifiers remain a backstop for anything not marked.
Callouts
Use a direct regex identifier for structured sensitive fields. This technique is designed for free-text fields where sensitive values appear in unpredictable positions — like a
reasoning field that re-mentions a name mid-sentence. If a field has a known, consistent format — an Employee ID like EMP-12345 — you should define a direct regex custom data identifier in CWDP that matches that format, rather than relying on the agent to bracket it.Coverage must be audited, not assumed. Go through every free-text field in your schema and decide explicitly whether it needs the
[[ ]] instruction. Treat it as a required question for any new field added going forward — the only symptom of a missed field is an unmasked sensitive data reaching logs.Generation-time compliance is probabilistic. The model can fail to apply the instruction — especially on repeated mentions of the same name within a long free-text field (bracketed on first mention, dropped on the second). That's why the managed CWDP identifiers backstop matters.
The regex needs validation against your data. The pattern handles common Latin-adjacent name punctuation and Unicode letters, but validate it against your actual customer name distribution before treating it as a finished control. Build an adversarial test set — hyphenated names, apostrophes, multi-part names, non-Latin scripts — and run it as a regression test whenever the model version, system prompt, or schema changes.
Conclusion
Enabling CWDP is the right starting point for sensitive data in agent logs. Adding a
[[ ]] marker and a matching custom data identifier gives you a deterministic layer on top of it — one that works precisely because the agent already knows which values are sensitive. The masking is reversible, so privileged access can recover data where retention or governance obligations require it. Coverage and testing require ongoing attention, but the mechanism itself is simple to add and low-friction to maintain.Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article