Let AWS DevOps Agent query Amazon S3 logs with Amazon Athena managed query results

Queries fail with "Access denied when writing output to url", and adding s3:PutObject to the Agent Space role changes nothing, because the guardrail is a session policy with no S3 write in it. Here's the whole configuration, plus four traps the documentation skips.

Cheng

Published Oct 20, 2026

Last modified Oct 20, 2026

Your logs are in Amazon Simple Storage Service (Amazon S3), the Amazon Athena table is already built, and you want AWS DevOps Agent to query them for you during an investigation. The query fails, and the reason is a missing AWS Identity and Access Management (IAM) permission. So you add the Athena actions to the Agent Space role, and add s3:PutObject on the results bucket while you're there. Now the query submits and scans data, then fails every time at the point where it writes the results back to S3:

1
2
Access denied when writing output to url: s3://.../xxx.csv .
Please ensure you are allowed to access the S3 bucket...

A different results bucket, a different prefix, same error.

The cause is the DevOps Agent guardrail. The agent passes a session policy when it assumes your Agent Space role, the effective permissions are the intersection of the role's identity policy and that session policy, and the session policy has no S3 write action in it. Nothing you add to the role will change that.

The fix is to stop asking Athena to write to S3. Create a workgroup with managed query results and Athena keeps the results for you, so no part of the query needs to write anywhere.

The documentation does say this. It's one line in the guardrail table on Limiting Agent Access in an AWS Account :

The workgroup must use managed query results because the agent lacks s3:PutObject permissions.

That line is the whole answer, and it's also all you get. It doesn't cover how to create the workgroup, what to put on the role, who creates the table, or how to make the agent use the workgroup. It also never connects to the error you're actually holding: search for Access denied when writing output to url and you won't land on that page. This post fills in the rest.

Where this applies

AWS WAF logs, AWS CloudTrail, Application Load Balancer and Amazon CloudFront access logs, VPC Flow Logs, anything that sits in S3 and gets queried through an Athena table runs into the same two layers of IAM limits. How the logs got into the bucket makes no difference. The examples below use WAF logs.

Layer one: the role has no Athena query permissions, so athena:StartQueryExecution comes back as AccessDenied.
Layer two: s3:PutObject isn't in the guardrail, so Athena fails when it writes the results.

Layer one: the role has no Athena query permissions

When you create an Agent Space from the console, it attaches three policies to the role: the AWS managed AIDevOpsAgentAccessPolicy, plus two the console generates on the spot (AIDevOpsCreateServiceLinkedRolePolicy-* and AIDevOpsAllowSecretsManagerActionsPolicy-*). The last two cover the service-linked role and AWS Secrets Manager, and have nothing to do with Athena.

Inside AIDevOpsAgentAccessPolicy, Athena gets read actions only (athena:GetWorkGroup, athena:List*, and so on). There's no athena:StartQueryExecution and no athena:GetQueryResults. S3 gets bucket-level Get and List, but not the s3:GetObject you need to read the objects themselves.

This layer is the easy one. Add the permissions to the role and you're past it.

Layer two: s3:PutObject isn't in the guardrail

DevOps Agent (aidevops.amazonaws.com) carries a session policy every time it assumes the role. That session policy doesn't live on the role. It arrives as two request parameters on the AssumeRole call: policyArns points at a few AWS managed policies, and policy carries an inline document. You can't edit it, and you won't find it in the IAM console.

Here is what CloudTrail recorded in requestParameters for the session that ran the Athena query:

1
2
3
4
5
6
7
8
9
10
11
12
13
"requestParameters": {
  "roleArn": "arn:aws:iam::<account-id>:role/service-role/DevOpsAgentRole-AgentSpace-<suffix>",
  "roleSessionName": "monitorAssociationRoleSession",
  "policyArns": [
    {
      "arn": "arn:aws:iam::aws:policy/AIDevOpsAgentAccessPolicy"
    },
    {
      "arn": "arn:aws:iam::aws:policy/ReadOnlyAccess"
    }
  ],
  "policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"Statement1\",\"Effect\":\"Allow\",\"Action\":[\"aidevops:CreateKnowledgeItem\",\"aidevops:UpdateKnowledgeItem\",\"athena:S*QueryExecution\",\"bedrock:Retrieve\",\"kms:Decrypt\"],\"Resource\":[\"*\"]}]}"
}

The policy field is that inline document. CloudTrail stores it as a JSON string, which is why every quote inside it is escaped.

AIDevOpsAgentAccessPolicy and ReadOnlyAccess both give S3 read access only. The inline document adds a handful of actions on top: starting and stopping queries (athena:S*QueryExecution matches StartQueryExecution and StopQueryExecution, nothing else), kms:Decrypt, and a few more. No S3 write action appears anywhere in the session policy. Which means:

s3:PutObject on the role does nothing, because it isn't in the intersection.
Switching the Athena results bucket does nothing either.
Athena writes query results to S3 by default, either to the workgroup's OutputLocation or to the ResultConfiguration the caller passes. The agent can't write there, so the query fails.

The same role gets assumed for other sessions too. Those carry an even narrower session policy, holding only AIDevOpsAgentAccessPolicy, with no ReadOnlyAccess and no inline document, so there's no S3 write there either.

Why it's hard to pin down

The role page in the IAM console doesn't show session policies. The policy simulator doesn't account for them either: aws iam simulate-principal-policy, which is what the IAM Policy Simulator in the console calls, only evaluates the policies attached to the role. Put s3:PutObject on the role and it answers allowed, while the agent's real call gets denied.

When the result write fails, Athena's message says nothing about a session policy, so there's no way to tell from the error what denied it. The session policy only shows up in the AssumeRole event in CloudTrail, and you have to look in the Region where the Agent Space lives:

1
2
3
aws cloudtrail lookup-events --region <agent-space-region> \
  --lookup-attributes AttributeKey=EventName,AttributeValue=AssumeRole \
  --query 'Events[].CloudTrailEvent' --output text | tr '\t' '\n' | grep <agent-space-role-name>

Don't lean on the agent's own diagnosis. Ours worked through "there's an explicit Deny in the managed policy", then "the policy cache hasn't caught up", then "try a fresh bucket", before it guessed session policy and recommended an elevated role. An elevated role doesn't fix this. With one configured and elevated actions turned on, asking the agent in chat to write an object to S3 does raise an approval. But when it runs an Athena query, nothing gets approved and no role gets swapped. It submits under the ordinary role and fails at the same step. Investigation turns down write operations outright.

How to set it up

Step 1: Create a workgroup with managed query results

Create it in the Region where the Athena table lives, which is the Region of the AWS Glue Data Catalog that holds the table. The query runs there, and the log bucket is usually there as well. The Agent Space can sit in a different Region. We tested that combination and it works.

1
2
aws athena create-work-group --region <table-region> --name devops-agent \
  --configuration '{"ManagedQueryResultsConfiguration":{"Enabled":true},"PublishCloudWatchMetricsEnabled":true}'

Note: devops-agent is only an example name, so pick whatever you like. The rest of this post uses it, and if you choose another name, use the same one in the steps below.

Don't modify primary. Create a new workgroup. A workgroup with managed query results can no longer set OutputLocation, and it doesn't support result reuse. If other clients pass OutputLocation explicitly on primary or rely on result reuse, changing primary breaks them. We tested this: a query that passes ResultConfiguration gets rejected on a managed workgroup right at StartQueryExecution, with ManagedQueryResultsConfiguration and ResultConfiguration cannot be set together., and turning on Enforce for the workgroup makes no difference.

Step 2: Add a custom policy to the Agent Space role

The role to edit is the one created along with the Agent Space, named DevOpsAgentRole-AgentSpace-<suffix> by default (you can rename it at creation time). To see which role an Agent Space uses, open it in the DevOps Agent console and look at the Cloud section of the Capabilities tab.

Give the role the Athena actions needed to query this one workgroup, plus read access to the log bucket. Don't add s3:PutObject, it won't take effect.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AthenaManagedWorkgroup",
      "Effect": "Allow",
      "Action": [
        "athena:StartQueryExecution",
        "athena:StopQueryExecution",
        "athena:GetQueryExecution",
        "athena:BatchGetQueryExecution",
        "athena:GetQueryResults",
        "athena:GetQueryResultsStream"
      ],
      "Resource": "arn:aws:athena:<table-region>:<account-id>:workgroup/devops-agent"
    },
    {
      "Sid": "LogBucketRead",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": ["arn:aws:s3:::<log-bucket>", "arn:aws:s3:::<log-bucket>/*"]
    }
  ]
}

A few notes on that policy:

The Glue read actions (glue:GetDatabase, GetTable, GetTables, GetPartitions) are already in the default policy, so leave them out. Tables that use partition projection and tables with partitions registered in Glue both worked in our tests without any extra Glue permission, glue:BatchGetPartition included.
The s3:GetObject here is for reading the log data, which belongs to layer one. It isn't in the default policy, and adding it to the role does work, because the ReadOnlyAccess in the session policy that runs Athena queries covers it. s3:ListBucket and s3:GetBucketLocation are already there, and listing them again does no harm.
The s3:PutObject needed to write query results belongs to layer two. Adding it to the role has no effect, so don't bother. Managed query results from step 1 is what gets you around it.
If the log bucket is encrypted with an AWS Key Management Service (AWS KMS) customer managed key, add kms:Decrypt for that key to the role and make sure the key policy allows the role. kms:Decrypt is in the session policy, so this one takes effect. Without it you get no identity-based policy allows the kms:Decrypt action.

Step 3: Create the Athena table up front

glue:CreateTable isn't in the session policy either, so the agent can't create the table. Create it first yourself, with AWS CloudFormation, a Glue crawler, or a one-off CREATE EXTERNAL TABLE under your own credentials.

Step 4: Tell the agent which workgroup to use, and not to pass ResultConfiguration

When the agent calls StartQueryExecution it has to name the workgroup from step 1 and leave out ResultConfiguration and OutputLocation. It won't go looking for the workgroup on its own, so you have to tell it.

Put that in the Agent Space's agent instructions (AGENTS.md). Here's the sample we used:

1
2
3
4
# Environment notes

- AWS WAF logs for the CloudFront WebACL `<webacl-name>` are queryable in Amazon Athena in us-east-1 as table `<database>.<table>` (partition column `log_time`, string `yyyy/MM/dd/HH/mm`).
- Run every Athena query in workgroup `devops-agent` in us-east-1. It uses Athena managed query results. Never pass `ResultConfiguration` or an `OutputLocation`; the agent cannot write to S3.

Swap in your own table name, Region and workgroup name. What matters is spelling out which workgroup DevOps Agent should use, and that it must not pass a result location.

There are two ways to get this in. One is the Operator Web App, the DevOps Agent web interface, following the path in the Agent instructions user guide: open the Agents page, open any managed agent, then edit Global instructions (AGENTS.md) on the Agent definition tab and save. Global instructions apply to every agent in that Agent Space.

The other is the API:

1
2
3
4
5
aws devops-agent create-asset --region <agent-space-region> \
  --agent-space-id <agent-space-id> \
  --asset-type agents_md \
  --metadata '{"agent_type": "GENERIC"}' \
  --content '{"file": {"path": "AGENTS.md", "body": {"text": "<the markdown above>"}}}'

metadata has to carry agent_type. Leave it out and you get agent_type is required for AgentsMd knowledge items. Per the Managing assets documentation, agent_type can be GENERIC, CHAT, INCIDENT_TRIAGE, INCIDENT_RCA (the one investigation uses) and others, you get one asset per type, the content is markdown, and the limit is 25 KB. In our tests a single agent_type: GENERIC asset was enough for both chat and investigation. We never had to mention the workgroup in a question after that: the agent named devops-agent on its own and passed no ResultConfiguration.

It isn't completely unusable without the instructions, but it isn't dependable. Sometimes the agent gave up after being denied, sometimes it found the workgroup by itself and answered correctly. So either write the instructions, or name the workgroup in every question you ask.

Step 5: Check that it works

Ask DevOps Agent something it can only answer by reading the logs, say the request count for a URI over some time window. If it runs the query and comes back with the number, your configuration is live. If it can't answer, open the recent queries in the Athena console and the error there will tell you which piece is missing.

What you give up

Managed query results are kept for 24 hours. After that GetQueryResults returns The result for this query has expired. Please run the query again to view the results, and seeing them again means rerunning the query and paying for the scan again. Conclusions the agent already wrote down aren't affected.
A managed workgroup doesn't support result reuse, so running the same query twice inside those 24 hours scans the data twice.
Athena managed query results do support customer managed keys, but DevOps Agent can't use them, which leaves you on the default AWS owned key. With a customer managed key the query is rejected at StartQueryExecution with User is denied access to the KMS key for encrypting query results, because the kms:GenerateDataKey it needs isn't in the session policy.
AIDevOpsAgentAccessPolicy and the session policy are both managed by AWS, so what's in them can change.

If your logs are in Amazon CloudWatch Logs

None of the above applies. logs:StartQuery, logs:GetQueryResults and logs:StopQuery are already in AIDevOpsAgentAccessPolicy, and a CloudWatch Logs Insights query doesn't write to S3. With nothing but the default policies, under either session policy, Logs Insights queries ran to completion in our tests.

Conclusion

The agent's effective permissions are the intersection of your role and the guardrail, the guardrail has no S3 write permission, so don't make Athena write to S3. Every step above follows from that one point: a workgroup with managed query results, read-only additions to the role, a table built ahead of time, and the workgroup named in the agent instructions.

Next time Access denied when writing output to url shows up, you can skip the bucket policy, KMS and caching, and you don't need an elevated role.

With this in place, DevOps Agent can read the WAF, CloudTrail and CloudFront logs in your S3 buckets and take the investigation work off your hands.

# devops # agents # ai-agents # amazon-athena # iam

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Let AWS DevOps Agent query Amazon S3 logs with Amazon Athena managed query results

Cheng

Where this applies

Layer one: the role has no Athena query permissions

Layer two: s3:PutObject isn't in the guardrail

Why it's hard to pin down

How to set it up

Step 1: Create a workgroup with managed query results

Step 2: Add a custom policy to the Agent Space role

Step 3: Create the Athena table up front

Step 4: Tell the agent which workgroup to use, and not to pass ResultConfiguration

Step 5: Check that it works

What you give up

If your logs are in Amazon CloudWatch Logs

Conclusion