Kiro for AWS Infrastructure: Spec-Driven Terraform and Lambda Development

<!-- .kiro/steering/tech.md -->
# Technology Stack

- IaC tool: Terraform >= 1.7, AWS provider ~> 5.0
- Lambda runtime: Python 3.12
- State backend: S3 with DynamoDB locking (see infra/backend.tf)
- Naming convention: {env}-{service}-{resource}, all lowercase, hyphenated
- All Lambda roles must use least-privilege inline policies, never AdministratorAccess
- API Gateway: REST APIs only (no HTTP APIs) for this project, for WAF compatibility
- Every Lambda must have a corresponding CloudWatch alarm on Errors and Throttles

<!-- .kiro/specs/file-processor/requirements.md -->
# Requirements: S3 File Processor

## User Story
As a data engineer, I want uploaded CSV files to be validated and
transformed automatically, so that downstream jobs only see clean data.

## Acceptance Criteria (EARS)

1. WHEN a file is uploaded to the `raw/` prefix of the input S3 bucket
THE SYSTEM SHALL invoke the processing Lambda within 30 seconds.

2. WHEN the uploaded file is not valid CSV
THE SYSTEM SHALL move it to a `rejected/` prefix and publish a
CloudWatch metric `FileValidationFailure`.

3. WHEN the uploaded file is valid CSV
THE SYSTEM SHALL write the transformed output to the `processed/`
prefix of the output bucket.

4. WHEN the Lambda invocation fails after 3 retries
THE SYSTEM SHALL send the event to a dead-letter queue for manual review.

5. WHERE the Lambda requires S3 read/write access
THE SYSTEM SHALL scope IAM permissions to only the two buckets
involved, not to `s3:*`.

<!-- .kiro/specs/file-processor/design.md -->
# Design: S3 File Processor

## Architecture
- S3 bucket (input) with EventBridge notifications on `raw/*` prefix
- Lambda (Python 3.12) triggered via EventBridge rule
- Lambda IAM role scoped to: s3:GetObject on input bucket/raw/*,
s3:PutObject on input bucket/rejected/* and output bucket/processed/*
- SQS dead-letter queue attached to the Lambda's async invocation config
- CloudWatch alarm on Lambda Errors > 0 over 5 minutes -> SNS topic

## Sequence

1. Object PUT to raw/ triggers EventBridge rule
2. EventBridge invokes Lambda asynchronously
3. Lambda validates CSV structure (headers, row count > 0)
4. On failure: copy to rejected/, emit custom metric, delete from raw/
5. On success: transform, write to processed/, delete from raw/
6. On repeated invocation failure: EventBridge DLQ captures the event

## Traceability
- Req 1, 3 -> EventBridge rule + Lambda handler (Step 2-3, 5)
- Req 2 -> validation branch + custom metric (Step 4)
- Req 4 -> DLQ configuration
- Req 5 -> IAM policy scoping (see security.md steering)

<!-- .kiro/specs/file-processor/tasks.md -->
# Tasks: S3 File Processor

- [ ] 1. Create S3 buckets (input, output) with versioning + public access block
_Req: 1, 3_
- [ ] 2. Create IAM role and least-privilege policy for the Lambda
_Req: 5_
_Depends on: 1_
- [ ] 3. Write Lambda handler: CSV validation + transform logic
_Req: 2, 3_
- [ ] 4. Wire EventBridge rule (raw/* PUT -> Lambda invoke)
_Req: 1_
_Depends on: 1, 2, 3_
- [ ] 5. Configure DLQ + async invocation config on the Lambda
_Req: 4_
_Depends on: 3_
- [ ] 6. Add CloudWatch alarm on Errors metric -> SNS
_Req: 4_
_Depends on: 3_

# infra/s3.tf
resource "aws_s3_bucket" "input" {
bucket = "prod-fileproc-input"
}

resource "aws_s3_bucket_versioning" "input" {
bucket = aws_s3_bucket.input.id
versioning_configuration { status = "Enabled" }
}

resource "aws_s3_bucket_public_access_block" "input" {
bucket = aws_s3_bucket.input.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}

resource "aws_s3_bucket" "output" {
bucket = "prod-fileproc-output"
}
# ... versioning + public access block mirrored for output bucket

# infra/iam.tf
data "aws_iam_policy_document" "lambda_assume" {
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = ["lambda.amazonaws.com"]
}
}
}

resource "aws_iam_role" "file_processor" {
name = "prod-fileproc-lambda-role"
assume_role_policy = data.aws_iam_policy_document.lambda_assume.json
}

data "aws_iam_policy_document" "file_processor_scoped" {
statement {
actions = ["s3:GetObject"]
resources = ["${aws_s3_bucket.input.arn}/raw/*"]
}
statement {
actions = ["s3:PutObject", "s3:DeleteObject"]
resources = [
"${aws_s3_bucket.input.arn}/rejected/*",
"${aws_s3_bucket.output.arn}/processed/*",
]
}
}

resource "aws_iam_role_policy" "file_processor" {
name = "scoped-s3-access"
role = aws_iam_role.file_processor.id
policy = data.aws_iam_policy_document.file_processor_scoped.json
}

# src/handler.py
import boto3
import csv
import io
import os

s3 = boto3.client("s3")
cloudwatch = boto3.client("cloudwatch")

INPUT_BUCKET = os.environ["INPUT_BUCKET"]
OUTPUT_BUCKET = os.environ["OUTPUT_BUCKET"]

def handler(event, context):
record = event["Records"][0]
key = record["s3"]["object"]["key"]

obj = s3.get_object(Bucket=INPUT_BUCKET, Key=key)
body = obj["Body"].read().decode("utf-8")

if not _is_valid_csv(body):
rejected_key = key.replace("raw/", "rejected/", 1)
s3.put_object(Bucket=INPUT_BUCKET, Key=rejected_key, Body=body)
s3.delete_object(Bucket=INPUT_BUCKET, Key=key)
cloudwatch.put_metric_data(
Namespace="FileProcessor",
MetricData=[{"MetricName": "FileValidationFailure", "Value": 1}],
)
return {"status": "rejected", "key": key}

transformed = _transform(body)
processed_key = key.replace("raw/", "processed/", 1)
s3.put_object(Bucket=OUTPUT_BUCKET, Key=processed_key, Body=transformed)
s3.delete_object(Bucket=INPUT_BUCKET, Key=key)
return {"status": "processed", "key": processed_key}

def _is_valid_csv(body: str) -> bool:
try:
reader = csv.reader(io.StringIO(body))
rows = list(reader)
return len(rows) > 1 and len(rows[0]) > 0
except csv.Error:
return False

def _transform(body: str) -> str:
reader = csv.DictReader(io.StringIO(body))
output = io.StringIO()
writer = csv.DictWriter(output, fieldnames=reader.fieldnames)
writer.writeheader()
for row in reader:
writer.writerow({k: v.strip() for k, v in row.items()})
return output.getvalue()

# .kiro/hooks/terraform-plan-on-save.yml
name: Terraform Plan on Save
event: file_saved
file_pattern: "infra/**/*.tf"
instructions: |
Run `terraform validate` and `terraform plan` for the infra/ directory.
Summarize any resource changes, especially IAM policy or public-access
changes, in a comment back to the developer. Flag anything that widens
permissions beyond what design.md specifies.