Automating AWS DevOps Agent Generic Webhook Creation with Terraform
How do I automate the webhook creation? There's no create-webhook API in the documentation.
Introduction
AWS DevOps Agent is a frontier AI agent that resolves and proactively prevents incidents, continuously improving reliability and performance. One of its most powerful integration points is the generic webhook — an HMAC-authenticated HTTP endpoint that allows any external system (CloudWatch Alarms, EventBridge rules, custom monitoring tools) to trigger automated investigations.
A common question from teams adopting Infrastructure as Code is: "How do I automate the webhook creation? There's no
create-webhook API in the documentation."This article shows you how to fully automate the end-to-end setup — from Agent Space creation through webhook provisioning and secret management — using Terraform and the AWS CLI. No console clicks required.
The Problem
If you search the DevOps Agent API reference , you won't find a
CreateWebhook operation. The register-service API registers an event channel service type but does not generate webhook credentials. This leads many teams to believe webhook creation is console-only.The Solution
The webhook is created via the
associate-service API — the same call the console makes internally. When you associate an event channel service, the response includes the webhook URL and HMAC secret:aws devops-agent associate-service \ --agent-space-id "<your-agent-space-id>" \ --service-id "event_channel" \ --configuration '{"eventChannel":{}}' \ --region us-east-1 \ --output jsonResponse:
{ "association": { "agentSpaceId": "2c3ffc02-b9fb-4f08-a795-a69a68a41fe4", "associationId": "ab11c7ee-8fce-43e0-83be-6e3398bf0ce0", "serviceId": "event_channel", "configuration": { "eventChannel": {} } }, "webhook": { "webhookUrl": "https://event-ai.us-east-1.api.aws/webhook/generic/<id>", "webhookId": "<id>", "webhookType": "hmac", "webhookSecret": "<base64-encoded-secret>" }}The secret is returned only once — treat it like an IAM access key.
Architecture Overview
┌─────────────────────────────────────────────────────────────────┐│ Terraform │├─────────────────────────────────────────────────────────────────┤│ ││ 1. awscc_devopsagent_agent_space (Agent Space) ││ 2. awscc_devopsagent_association (AWS Account Link) ││ 3. null_resource + local-exec (associate-service → ││ webhook URL + secret) ││ 4. aws_secretsmanager_secret (Store credentials) ││ 5. aws_lambda_function (Alarm forwarder) ││ │└─────────────────────────────────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────────────┐│ Runtime Integration │├─────────────────────────────────────────────────────────────────┤│ ││ CloudWatch Alarm → Lambda → HMAC-signed POST → Webhook URL ││ EventBridge Rule → Lambda → HMAC-signed POST → Webhook URL ││ │└─────────────────────────────────────────────────────────────────┘Prerequisites
- AWS CLI v2.34+ (must include the
devops-agentsubcommand)
- Terraform >= 1.0 with the
awsccprovider (~> 1.0)
-
jqinstalled
- An AWS account in a supported region (us-east-1, us-west-2, ap-southeast-2, ap-northeast-1, eu-central-1, eu-west-1)
Step 1: Deploy the Agent Space and Association
terraform { required_providers { awscc = { source = "hashicorp/awscc", version = "~> 1.0" } aws = { source = "hashicorp/aws", version = "~> 5.0" } random = { source = "hashicorp/random", version = "~> 3.0" } time = { source = "hashicorp/time", version = "~> 0.9" } }}
provider "aws" { region = var.aws_region }provider "awscc" { region = var.aws_region }
variable "aws_region" { type = string; default = "us-east-1" }variable "agent_space_name" { type = string }
data "aws_caller_identity" "current" {}resource "random_id" "suffix" { byte_length = 4 }
# --- IAM Role for DevOps Agent ---resource "aws_iam_role" "devops_agent_role" { name = "DevOpsAgentRole-${random_id.suffix.hex}" assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [{ Effect = "Allow" Principal = { Service = "aidevops.amazonaws.com" } Action = "sts:AssumeRole" Condition = { StringEquals = { "aws:SourceAccount" = data.aws_caller_identity.current.account_id } } }] })}
resource "aws_iam_role_policy_attachment" "devops_agent_policy" { role = aws_iam_role.devops_agent_role.name policy_arn = "arn:aws:iam::aws:policy/AIDevOpsAgentAccessPolicy"}
# Wait for IAM propagationresource "time_sleep" "iam_propagation" { depends_on = [aws_iam_role.devops_agent_role] create_duration = "30s"}
# --- Agent Space ---resource "awscc_devopsagent_agent_space" "this" { depends_on = [time_sleep.iam_propagation] name = var.agent_space_name}
# --- AWS Monitor Association ---resource "awscc_devopsagent_association" "monitor" { agent_space_id = awscc_devopsagent_agent_space.this.agent_space_id service_id = "aws" configuration = { aws = { account_id = data.aws_caller_identity.current.account_id account_type = "monitor" assumable_role_arn = aws_iam_role.devops_agent_role.arn } }}Step 2: Create the Webhook via null_resource
This is the key step. We call
associate-service to create the event channel, which returns the webhook credentials, then store them in Secrets Manager:# --- Secrets Manager for webhook credentials ---resource "aws_secretsmanager_secret" "webhook_creds" { name = "devops-agent/${var.agent_space_name}/webhook-${random_id.suffix.hex}" description = "DevOps Agent generic webhook HMAC credentials" recovery_window_in_days = 0}
# --- Create webhook and store credentials ---resource "null_resource" "create_webhook" { depends_on = [ awscc_devopsagent_association.monitor, aws_secretsmanager_secret.webhook_creds ]
triggers = { agent_space_id = awscc_devopsagent_agent_space.this.agent_space_id }
provisioner "local-exec" { interpreter = ["/bin/bash", "-c"] command = <<-EOT set -euo pipefail
# Create the event channel association — this generates the webhook RESULT=$(aws devops-agent associate-service \ --agent-space-id "${awscc_devopsagent_agent_space.this.agent_space_id}" \ --service-id "event_channel" \ --configuration '{"eventChannel":{}}' \ --region ${var.aws_region} \ --output json \ --no-cli-pager)
echo "associate-service response:" echo "$RESULT" | jq .
# Extract webhook credentials WEBHOOK_URL=$(echo "$RESULT" | jq -r '.webhook.webhookUrl') WEBHOOK_SECRET=$(echo "$RESULT" | jq -r '.webhook.webhookSecret') WEBHOOK_ID=$(echo "$RESULT" | jq -r '.webhook.webhookId')
if [ -z "$WEBHOOK_URL" ] || [ "$WEBHOOK_URL" = "null" ]; then echo "ERROR: No webhook URL in response" exit 1 fi
# Store in Secrets Manager aws secretsmanager put-secret-value \ --secret-id "${aws_secretsmanager_secret.webhook_creds.id}" \ --secret-string "$(jq -n \ --arg url "$WEBHOOK_URL" \ --arg secret "$WEBHOOK_SECRET" \ --arg id "$WEBHOOK_ID" \ '{webhookUrl: $url, webhookSecret: $secret, webhookId: $id}')" \ --region ${var.aws_region} \ --no-cli-pager
echo "Webhook URL: $WEBHOOK_URL" echo "Credentials stored in Secrets Manager" EOT }}Step 3: Deploy the Lambda Forwarder
resource "aws_iam_role" "forwarder_role" { name = "devops-agent-forwarder-${random_id.suffix.hex}" assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [{ Effect = "Allow", Principal = { Service = "lambda.amazonaws.com" }, Action = "sts:AssumeRole" }] })}
resource "aws_iam_role_policy" "forwarder_policy" { name = "forwarder-policy" role = aws_iam_role.forwarder_role.id policy = jsonencode({ Version = "2012-10-17" Statement = [ { Effect = "Allow", Action = ["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"], Resource = "arn:aws:logs:*:*:*" }, { Effect = "Allow", Action = ["secretsmanager:GetSecretValue"], Resource = aws_secretsmanager_secret.webhook_creds.arn } ] })}
data "archive_file" "forwarder" { type = "zip" source_file = "${path.module}/lambda/index.py" output_path = "${path.module}/.build/forwarder.zip"}
resource "aws_lambda_function" "forwarder" { filename = data.archive_file.forwarder.output_path function_name = "devops-agent-webhook-forwarder-${random_id.suffix.hex}" role = aws_iam_role.forwarder_role.arn handler = "index.lambda_handler" runtime = "python3.12" timeout = 30 source_code_hash = data.archive_file.forwarder.output_base64sha256 environment { variables = { WEBHOOK_SECRET_ARN = aws_secretsmanager_secret.webhook_creds.arn } }}
resource "aws_lambda_permission" "allow_cloudwatch" { statement_id = "AllowCloudWatchAlarms" action = "lambda:InvokeFunction" function_name = aws_lambda_function.forwarder.function_name principal = "lambda.alarms.cloudwatch.amazonaws.com"}Lambda Function Code ( lambda/index.py )
import json, os, hmac, hashlib, base64, urllib3, boto3from datetime import datetime, timezone
http = urllib3.PoolManager()secrets_client = boto3.client("secretsmanager")_cached = None
def get_creds(): global _cached if not _cached: r = secrets_client.get_secret_value(SecretId=os.environ["WEBHOOK_SECRET_ARN"]) _cached = json.loads(r["SecretString"]) return _cached
def sign(secret, ts, payload): return base64.b64encode( hmac.new(secret.encode(), f"{ts}:{payload}".encode(), hashlib.sha256).digest() ).decode()
def lambda_handler(event, context): print(json.dumps(event)) creds = get_creds()
alarm = event.get("alarmData", {}) name = alarm.get("alarmName", "Unknown") state = alarm.get("state", {}) if state.get("value") != "ALARM": return {"statusCode": 200, "body": "skipped"}
payload = json.dumps({ "eventType": "incident", "incidentId": f"{name}-{state.get('timestamp', '')}", "action": "created", "priority": "HIGH", "title": f"CloudWatch Alarm: {name}", "description": f"Alarm: {name}\nReason: {state.get('reason', '')}", "timestamp": state.get("timestamp", datetime.now(timezone.utc).isoformat()), "service": name, "data": {"metadata": {"alarmName": name, "region": event.get("region", "")}} })
ts = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.000Z") sig = sign(creds["webhookSecret"], ts, payload)
r = http.request("POST", creds["webhookUrl"], body=payload, headers={ "Content-Type": "application/json", "x-amzn-event-timestamp": ts, "x-amzn-event-signature": sig })
print(f"Webhook: {r.status}") if r.status not in (200, 202): raise Exception(f"Failed: {r.status} {r.data.decode()}") return {"statusCode": 200, "body": "triggered"}Step 4: Connect a CloudWatch Alarm
resource "aws_cloudwatch_metric_alarm" "high_cpu" { alarm_name = "high-cpu-production" comparison_operator = "GreaterThanThreshold" evaluation_periods = 2 metric_name = "CPUUtilization" namespace = "AWS/EC2" period = 300 statistic = "Average" threshold = 80 alarm_description = "CPU > 80% for 10 minutes"
dimensions = { InstanceId = var.monitored_instance_id }
alarm_actions = [aws_lambda_function.forwarder.arn]}How It Works
terraform apply │ ├─► awscc_devopsagent_agent_space → Agent Space created ├─► awscc_devopsagent_association → AWS account linked ├─► null_resource (associate-service) → Webhook URL + Secret returned ├─► aws_secretsmanager_secret → Credentials persisted └─► aws_lambda_function → Forwarder deployed │ ▼CloudWatch Alarm (ALARM state) → Lambda reads secret from Secrets Manager → Signs payload with HMAC-SHA256 → POST to webhook URL → DevOps Agent investigation triggeredKey Findings
The API is associate-service , not register-service
API | What it does | Returns webhook? |
|---|---|---|
register-service --service eventChannel | Registers the service type | ❌ Only serviceId |
associate-service --service-id event_channel | Creates the webhook | ✅ URL + Secret |
The correct CLI command
aws devops-agent associate-service \ --agent-space-id "<agent-space-id>" \ --service-id "event_channel" \ --configuration '{"eventChannel":{}}' \ --region us-east-1 \ --output jsonWhy the confusion exists
- The API reference lists
AssociateServicebut doesn't clearly document that it generates webhook credentials for event channels
-
register-servicewith--service eventChannelsounds like the right call but only registers the service type
- The webhook documentation describes console-only generation steps
- The AWSCC Terraform provider's
awscc_devopsagent_associationresource withEventChannelconfiguration fails due to CloudFormation schema validation issues
Idempotency
associate-service is not idempotent — calling it again returns "already registered". To regenerate credentials:# 1. Find the associationaws devops-agent list-associations --agent-space-id <id> --region us-east-1
# 2. Remove itaws devops-agent disassociate-service --agent-space-id <id> --association-id <assoc-id> --region us-east-1
# 3. Re-create (generates new credentials)aws devops-agent associate-service --agent-space-id <id> --service-id event_channel --configuration '{"eventChannel":{}}' --region us-east-1Security Best Practices
- Store secrets in Secrets Manager — never in Lambda environment variables or Terraform state
- Use
recovery_window_in_days = 0only for dev — use 7+ days in production
- Restrict Lambda permissions — only
secretsmanager:GetSecretValueon the specific secret ARN
- Rotate credentials — disassociate + re-associate to generate new keys periodically
- Enable CloudTrail — monitor
AssociateServiceandDisassociateServicecalls
Cleanup
# Disassociate the event channel first (not managed by Terraform)aws devops-agent disassociate-service \ --agent-space-id <id> --association-id <id> --region us-east-1
# Then destroy Terraform resourcesterraform destroyConclusion
Despite the documentation gap, AWS DevOps Agent generic webhooks can be fully automated using the
associate-service CLI command. The key insight is that associate-service --service-id "event_channel" (not register-service ) is the API that generates webhook credentials — returning both the URL and HMAC secret in a single response.Combined with Terraform's
null_resource pattern and AWS Secrets Manager, you achieve a zero-touch, IaC-managed webhook setup that integrates DevOps Agent with any monitoring system.References
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article