My AWS Account Started Keeping a Diary

I have dashboards. Good ones. Forty-some widgets across three tabs, arranged with care, color-coded with love.

I stopped reading them months ago.

Not out of laziness — out of arithmetic. A dashboard is pull observability: it sits there, complete and silent, waiting for a human to show up and interrogate it. Every morning, the deal is the same: bring your own attention, bring your own questions, bring your own memory of what looked odd on Tuesday. Most mornings I didn't hold up my end. I suspect you don't either.

So last week I tried something different. I stopped asking myself to read the account's day — and made the account write it down instead. That's what you're seeing in the image below.

Nobody wrote a rule for that. Nobody scripted "watch for recurring IAM attachments." My account noticed someone attaching an IAM policy three days running, promised to keep an eye on it, held the thread across a quiet day, and picked it up again when a new event landed at 23:10. Every number in those entries — 08:13, 12:36, six, 23:10 — is verbatim from measured telemetry, and there's a validator that makes sure of it.

This post is how it works, what it costs, what went wrong, and what I still don't trust.

Let me be honest about the prior art first, because "point an LLM at your logs" is not new. AWS's own Cloud Operations blog has a walkthrough of summarizing CloudWatch Logs with Bedrock, and the Marketplace has tools that turn security findings into plain English. Those are summaries, and summaries are fine.

A diary is a different animal, in three structural ways.

A diary has memory. A summary starts from zero every time you run it. A diary's entries reference each other — "the deploy I promised to watch, now on its second day." That August 11 entry above is my favorite thing the system produces, precisely because it's the least impressive-looking: closing a loop on a promise made days earlier, across a day where nothing happened, is something no stateless summarizer can do at all.

A diary selects. My collector gathered every deploy, alarm, metric window, and cost line for a week. The diary mentioned almost none of them. A summary is judged by coverage; a diary is judged by taste — the day's two things that mattered, or the honest admission that nothing did.

A diary has a narrator. "IAM AttachRolePolicy event count: 6" is a row you skim past. "The IAM policy I've been watching added another event today, making it six in total" is a sentence you actually read with your coffee. First person isn't decoration. It's the difference between a report and a habit.

Dashboards are pull. A diary is a habit. That's the whole thesis.

The design law underneath everything — the same one I used in my document-accessibility and prompt-caching work — is deterministic detect, generative explain. Code decides what happened and what mattered. The model only tells the story. And a validator checks the story against the facts, because "only tells the story" is a rule models follow right up until they don't.

Everything it's allowed to say is on that sheet. Everything else is a contract violation, and the contract is enforced by code, not by asking nicely.

Salience is editorial judgment, done with arithmetic. Each collected fact gets scored on deviation (z-score against a three-week, same-hour-of-week baseline — an account has a weekly rhythm, and Tuesday 2 p.m. should be compared to Tuesday 2 p.m.), novelty (a first-seen actor or event type outranks a familiar one), cost movement, and blast radius. Clear the bar, get selected; nothing clears the bar, and the diary is legally obligated to say so briefly.

The two failure modes pull in opposite directions, and you need both defenses at once. Salience without baselines flags everything — every deploy is an event, every blip is a story, and your diary becomes the boy who cried wolf with better prose. Baselines without novelty miss the new thing entirely, because the new thing has no baseline to deviate from.

I found out how this actually plays out on the second day of backfilling, staring at a week of fact sheets and slowly realizing every single one was CloudTrail: deploys and IAM writes, nothing else. My carefully-built metrics path had produced not one fact. On my account, it turned out, the metrics contributed nothing all week. My Lambdas are watched but low-traffic — flat-or-empty baselines — so the z-score engine had nothing defensible to say, and it correctly said nothing. There was even a genuinely odd 381-second execution in the window, and the engine stayed silent — no three-week baseline, no verdict. I'll come back to that in the residuals, because a tool's silence should be as honest as its speech.

Threads are the crown jewel — the mechanism that turns entries into a story.

A thread opens the first time a topic clears the salience bar. It advances on recurrence, carrying streak math ("now on its third day"). It resolves when a watched storyline goes quiet, and expires if untouched long enough. Every fact carries a stable topic key, which is what lets Friday's event find Tuesday's storyline instead of starting a new one.

And then there's my favorite rule in the whole codebase: the promise rule. If yesterday's entry contained a watch phrase — matched by an actual regex, nothing fancy: "worth watching", "keep an eye", "see if it holds", "revisit tomorrow" — then that thread is flagged as promised, it's force-fed into tomorrow's prompt, and the validator fails any draft that doesn't close the loop. The diary is not allowed to ghost you. When it wrote "I'll keep an eye on it" about that IAM policy, the follow-up the next day wasn't good manners. It was a contract.

That's what you're seeing in the August 9 → 12 storyline above: open, advance, promise, quiet-day loop-closure, advance again. On real CloudTrail data, unscripted.

This section is really three small war stories, because every honesty gate in the code exists for one reason: something got through without it.

Beat one: numbers verbatim. The validator extracts every numeric token from the draft — including times like 23:10 and money like $7.83 — and requires each one to exist on the fact sheet. Arithmetic in prose is structurally impossible: the narrator can't write a number it wasn't handed, so it can't helpfully "compute" a rate, a total, or a percentage. Fail the check, regenerate; fail repeatedly, and a deterministic template writes the entry instead.

That fallback isn't theoretical. Two of the seven entries in my real week are the template — the quiet days, where Nova kept flunking the brutal combination of two lines maximum, must close the promise, zero stray numbers. Here's one: "Quiet day. The lambda code I said I'd watch held fine — nothing moved." You can't tell from reading it. That's the point. The safety net engaged in production and the diary got more boring, which for a truth-telling system is the correct direction to fail in.

Beat two: prose can lie without numbers. My persona prompt includes example entries, because models imitate voice far better than they obey adjectives. One night, Nova handed back a quiet-day entry that read beautifully: "Nothing much today. The batch ran, the bill held steady, and nobody touched anything they shouldn't have."

My account doesn't have a batch.

It had copied my example entry word for word — and sailed through validation, because the sentence contains no numbers, and my only gate was numeric. A claim-shaped sentence with zero digits was invisible to the contract. The fix was twofold: the exemplars now describe a deliberately fictional account (an image-resize queue, a staging stack) whose resources can never match a real sheet, and the validator grew a shingle gate — any six-word run shared with an exemplar kills the draft. Deterministic, five lines, loophole closed.

Beat three: "six" is a number too. The August 12 entry says "making it six in total." Spelled out. My numeric gate only tokenized digits, so word-numbers evaded it entirely — a narrator could write "seven" with nothing on the sheet to back it. The tokenizer now normalizes spelled cardinals one through ten to digits ("six" is checked exactly like "6" — and yes, the six is real; the fact carried events: 6). Ordinals stay unmapped on purpose, so "its third day" — streak math the thread legitimately computed — doesn't false-flag.

The residual edge, named honestly: spelled numbers above ten still evade the gate. Nothing in the current prompt produces them, but the hole exists, it's logged, and now you know it too.

That's what "honest by construction" means in practice. Not one clever gate — a sequence of embarrassments, each one converted into a check.

Seven days, my real account. The deploys are real, the IAM storyline is real, the timestamps are real. So are the flaws — read August 9 and 10 back to back and you'll notice they nearly rhyme; the narrator's range is still narrow, and I'd rather you see that than a cherry-picked week. And look specifically at the quiet days — the two italic one-liners.

A diary that never shuts up is just a dashboard with pronouns. The quiet-day rule — at most two short lines when nothing cleared the bar — is the test I care most about, because it's the one that separates a narrator from a generator. Anyone can make a model write four paragraphs about nothing; the entire trick is making it not. I learned this lesson on a previous project and it's become a design reflex: a demo that never fails proves nothing, and a diary that's never quiet proves less.

CloudTrail was the bottleneck. LookupEvents is rate-limited and chatty; my first backfill crawled. Filtering server-side by event category and paging carefully got a seven-day backfill down to something you'd actually wait for. If you build on CloudTrail, budget your patience for this API and not the others.

Novelty kept re-firing. The backfill didn't initially seed the "have I seen this actor before?" set from prior days, so the same IAM actor read as first-seen every single day. The threads machinery masked it in prose — the entries correctly said "again" and "third day" — but a diary that thinks every morning is the first morning is one careful reader away from losing all credibility. Novelty is now seeded from persisted history.

Quiet-plus-promise is the hardest prompt in the system. Two lines maximum, must reference the promised thread, zero unsourced numbers — Nova failed it three drafts running on two separate days, and the template took over. I could tune the prompt until the model passes. I probably will. But I'm oddly fond of the current behavior: the strictest target in the contract is guarded by the dumbest, most reliable writer on the team.

Frozen days need their receipts carried forward. When I re-rendered approved entries through a replay path, their token costs vanished — the replay narrator had no usage to report. Bodies were byte-identical; the accounting wasn't. If you freeze generated content, freeze its metadata with it.

The repo is here: https://github.com/Felina14/infrastructure-diary 

I backfilled the week you've been reading from my real account's data, and the nightly schedule went live this week — from here on, the diary writes itself while I sleep.

Deploy it tonight and yours starts writing too.

I still have the dashboards. I just don't have to remember to read them anymore. My account tells me about its day every morning, in a couple of paragraphs — and when it says "worth watching," I know it will remember tomorrow whether I do or not.