AWS Builder Center

A write-up about Lambda durable functions got the docs fixed

I tested Lambda durable functions by building a human-in-the-loop AI agent pipeline and wrote up two sharp edges: an undocumented IAM pattern for callbacks and stale SDK examples. Eight days later the examples were removed and the developer guide gained a callback permissions page. Here's the timeline.

Cloud engineer.
Originally published on Medium .
A few weeks ago I built a three-agent content pipeline (researcher → writer → editor) on Lambda durable functions, the checkpoint/replay feature AWS shipped at re:Invent 2025, and wrote up the results . The review was mostly positive. My pipeline suspended overnight waiting for human approval and billed nothing, which is the whole promise of the feature. But it ended with a list of sharp edges, and two of them cost me real time.
Then a developer on the Lambda product team commented on the post with links to a merged PR and a new docs section fixing both. I'm writing down what happened because the path from "builder writes up friction" to "docs get fixed" is worth seeing end to end.

The two problems

The first was IAM. My approval endpoint calls send_durable_execution_callback_success to wake the suspended function. I granted that action on the function ARN and got AccessDeniedException anyway. It took about an hour to figure out that callbacks are authorized against the durable execution's ARN, which is a sub-resource of a function version, so a policy on the unqualified function ARN never matches. The grant has to be:
1
"Resource": "arn:aws:lambda:us-east-1:123456789012:function:my-function:*"
That trailing :* wasn't documented anywhere I could find at the time. I worked it out from the shape of the error and put it in my Terraform module's design notes.
The second was a mismatch between the docs and the SDK. The examples I was working from showed a one-argument submitter function for wait_for_callback and a plain timeout=86400. The SDK I actually installed invokes the submitter with two positional arguments and wants a WaitForCallbackConfig. If you copy the example, you get a runtime error.

What happened next

I published the original on Medium on July 6. On July 11 I shared it in an AWS community Slack channel and asked for honest feedback. On July 14 a PR merged in the SDK repo, and the same day someone from AWS replied in that Slack thread to say the team had fixed the issue, with the PR linked. The next day, a developer on the Lambda product team left a longer comment on the Medium post:
"…your detailed write-up helped us to find and close the docs gaps to save the next person the hour you lost :-)"
Two concrete things came out of it.
First, the Lambda developer guide now has a security and permissions page for durable functions  that covers exactly the thing I lost an hour to. It explains that a durable execution ARN extends the function ARN with a qualifier, that an unqualified ARN in a policy will never match one, and it shows the :* pattern in a worked example for the callback actions.
Second, the stale examples were tracked down and removed in aws-durable-execution-sdk-python#538 . The PR confirms the submitter bug (it really is invoked with two positional arguments, callback_id and a WaitForCallbackContext) and turned up a third problem I hadn't caught: the old examples imported WaitForCallbackConfig from the wrong module path.

The part I didn't expect

The team traced the stale examples not to the developer guide but to AGENTS.md in the SDK repo, the file that tells AI coding agents how to use the SDK. It had grown to 929 lines of duplicated documentation, drifted out of sync with the actual code, and the fix was to cut it down to 69 lines that defer to the developer guide.
I built an AI agent pipeline, with an AI coding agent, and the wrong examples we kept hitting were in the file written specifically to teach AI agents the SDK. The agent was doing exactly what it was told. The instructions were just wrong.
I suspect this will become a common failure mode. AGENTS.md-style files are turning into a second documentation surface, and they rot the same way READMEs do, except that when a README rots a human squints at it, and when an agent-instructions file rots the bad example gets typed into your codebase at 60 tokens a second. What AWS did here is the right fix: keep the agent file thin and point it at a single source of truth.

The docs, before and after

Before, the callback permissions story was an AccessDeniedException, an hour of head-scratching, and an educated guess about ARN sub-resources. Now the docs have a worked policy:
1
2
3
4
5
6
7
8
9
10
{
"Sid": "SendCallbacks",
"Effect": "Allow",
"Action": [
"lambda:SendDurableExecutionCallbackSuccess",
"lambda:SendDurableExecutionCallbackFailure",
"lambda:SendDurableExecutionCallbackHeartbeat"
],
"Resource": "arn:aws:lambda:us-east-1:123456789012:function:myDurableFunction:*"
}
The next person building a human-in-the-loop approval flow reads one paragraph instead of debugging IAM by trial and error.

Two things they taught me back

The comment also had implementation tips worth passing on.
You can reject with an exception instead of a payload. I modeled rejection as send_durable_execution_callback_success with {"approved": false}. There's also send_durable_execution_callback_failure, which surfaces the rejection to the orchestrator as an exception. Which one fits depends on whether "human said no" is a normal outcome or an error in your domain. For my pipeline the payload approach still makes sense; for a compliance gate that should halt everything, the failure callback is cleaner.
And if you don't like inline lambdas for steps, there's a @durable_step decorator  for top-level functions.

If you hit a docs gap

I never filed a GitHub issue, and I'm not recommending that. An issue would have been the proper channel, and PR #538 shows the team acts on them. But a few things made the post work anyway.
It had a reproduction. The exact error, the exact fix, the exact wrong example, all tested against a live deployment. Underneath the prose it was a bug report with better distribution.
It was fair. The first post was mostly positive because the feature is mostly good, and criticism inside an honest review travels further than a takedown. People who build the thing will actually read it to the end.
It was early. Durable functions were a few months old, so a detailed real-world account was rare enough to get passed around by people who could act on it.
And I didn't just publish it and wait. Sharing it in the Slack channel is visibly what worked here, since the reply about the fix came back through that same thread three days later. If the post had only lived on Medium, I doubt any of this happens.
So if you're building on something new and the docs let you down, write it up with the reproduction. Worst case, you save the next builder an hour. Best case, the docs change and someone from the product team shows up in your comments with the merge links.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article