AWS Builder Center

Builder Center

AWS Well-Architected Benchmark with AgriNova AI

AWS Well-Architected Benchmark with AgriNova AI

Benchmarking Cursor, Kiro, and Claude using AWS Well-Architected Skills on AgriNova AI to evaluate architecture-aware AI coding agents.

Introduction

AI coding assistants are rapidly becoming the default engineering companion. Developers increasingly ask AI systems to design architectures, review workloads, generate code, and recommend production improvements instead of manually consulting documentation.
But an important question remains:
Can AI coding agents produce architecture-aware engineering guidance, or do they remain code-focused assistants?
To explore this, I benchmarked AWS Well-Architected Skills & Steering using a real AI workload — AgriNova AI, an agriculture-focused assistant — across multiple AI coding agents:
  • Cursor
  • Kiro
  • Claude
Instead of evaluating toy applications, the experiment used a production-style AI system involving tools, APIs, datasets, memory layers, and external integrations.
The objective was simple:
Evaluate how different AI coding agents interpret the same AWS Well-Architected review workflow and measure their architectural awareness.

Project Overview – AgriNova AI

AgriNova AI is an agriculture intelligence assistant designed to support farmers with contextual recommendations and domain-specific guidance.
Capabilities include:
  • Weather insights
  • Rainfall prediction
  • Soil health recommendations
  • Irrigation guidance
  • Market price tracking
  • Government schemes
  • Water management
  • Equipment subsidies
  • Crop planning and calendar recommendations

Technology Stack

LayerTechnology
Foundation ModelAmazon Nova Pro
Agent FrameworkStrands Agents SDK
Memory LayerMem0 + FAISS
Knowledge LayerAWS Docs MCP
APIswttr.in + data.gov.in
Data LayerCSV / Excel
Evaluation FrameworkAWS Well-Architected

Why AWS Well-Architected Skills?

AWS recently introduced Well-Architected Skills & Steering for AI coding environments.
Instead of architecture reviews happening after development, these skills bring AWS guidance directly into AI-assisted workflows.
The evaluation covers six pillars:
PillarObjective
SecurityIdentity, secrets, access boundaries
ReliabilityRecovery and fault tolerance
Performance EfficiencyOptimization and scaling
Cost OptimizationResource efficiency
Operational ExcellenceMonitoring and observability
SustainabilityEnergy and resource optimization

Benchmark Setup

Skill Used

1
wa-review

Benchmark Workload

1
AgriNova AI

AI Coding Agents Evaluated

  • Cursor
  • Kiro
  • Claude

Evaluation Prompt

1
2
3
4
5
6
7
8
9
10
Review AgriNova AI architecture using AWS Well-Architected principles.

Evaluate:

1. Security
2. Reliability
3. Performance Efficiency
4. Cost Optimization
5. Operational Excellence
6. Sustainability
The same workload and evaluation prompt were used across all tools to ensure consistency.

Benchmark Results

1. Cursor – Architecture Reviewer

Cursor approached AgriNova primarily from an architectural perspective.
Recommendations focused on AWS design maturity and operational improvements.

Security Recommendations

  • Amazon Cognito
  • API Gateway + WAF
  • IAM Least Privilege
  • Tool allowlists

Reliability Recommendations

  • EventBridge
  • Amazon SQS
  • Retries
  • Circuit breakers
  • Fallback caches

Performance Improvements

  • Response caching
  • Step Functions orchestration
  • Lightweight model routing

Operational Excellence

  • CloudWatch
  • AWS X-Ray
  • SLO tracking
  • CI/CD quality gates

Observation

Cursor behaved similarly to an architecture consultant.
Its outputs emphasized:
System design → AWS integration → Operational maturity

2. Kiro – Production Auditor

Kiro produced the deepest implementation-aware review.
Instead of stopping at architecture diagrams, Kiro inspected runtime behavior and source implementation.
Analysis scope included:
  • agriNovaAI.py
  • Tool registrations
  • Mem0
  • FAISS
  • AWS Docs MCP
  • External APIs
  • Dataset loading patterns

Security Findings

Critical issue detected:
1
os.environ["BYPASS_TOOL_CONSENT"] = "true"
Impact:
  • Unrestricted tool execution
  • Government API access without approval
  • Memory writes without confirmation
Recommendation:
Remove bypass and introduce approval boundaries.

Reliability Findings

Detected:
1
requests.get(...)
without:
  • timeout
  • retry
  • fallback
Risk:
External API delays could block the complete workflow.
Recommended fix:
1
requests.get(url, timeout=(3,10))
Add:
  • retries
  • exponential backoff
  • cached fallback responses

Cost Findings

Current execution:
1
2
3
17 tools
+ MCP manifest
→ every inference call
Recommended optimization:
1
2
3
4
5
Intent Classifier
↓
Relevant Tool Selection
↓
Nova Pro
Expected improvement:
40–60% reduction in token consumption

Observation

Kiro behaved less like a coding assistant and more like a production engineering auditor.
It identified:
  • Runtime issues
  • API risks
  • Reliability gaps
  • Operational concerns

3. Claude – Well-Architected Assessor

Claude generated the most structured Well-Architected review.
Analysis scope:
  • 1178 lines analyzed
  • 17 registered tools
  • 6 datasets
  • Weather API integration
  • data.gov.in integration
  • AWS Docs MCP

AWS Well-Architected Scorecard

PillarScoreStatus
Security3 / 10🔴 Critical
Reliability5 / 10🟠 Medium
Performance5 / 10🟠 Medium
Cost6 / 10🟡 Moderate
Operational Excellence4 / 10🔴 Critical
Sustainability6 / 10🟢 Foundation

Security Findings

Highest-priority observations:

Tool approval bypass

1
os.environ["BYPASS_TOOL_CONSENT"] = "true"
Risk:
  • Uncontrolled execution
  • Reduced production safety

Secret management

Current API key handling requires improvement.
Recommended:
  • AWS Secrets Manager
  • Systems Manager Parameter Store

Reliability Findings

Current implementation depends heavily on external APIs.
Detected:
1
requests.get(...)
without resilience mechanisms.
Risk:
A stalled weather endpoint could stop the entire execution flow.

Performance Findings

Claude identified duplicate execution paths.
Examples:
  • Duplicate tool registration
  • Repeated weather calls
  • Redundant inference overhead
Recommendations:
  • Cache responses
  • Reuse HTTP sessions
  • Remove duplicate registrations

Operational Excellence Findings

Current observability relies mainly on:
1
print()
Missing:
❌ Structured logs
❌ Request tracing
❌ Metrics
❌ Dashboards
Recommended stack:
  • CloudWatch Logs
  • AWS X-Ray
  • Request IDs
  • Operational dashboards

Sustainability Findings

AgriNova naturally aligns with sustainability goals.
Strengths:
  • Irrigation optimization
  • Water conservation
  • Soil-based recommendations
Suggested improvement:
Move from always-on execution to:
  • AWS Lambda
  • ECS Fargate
  • Scale-to-zero workloads
Result:
Reduced idle compute and energy consumption.

Evaluation Criteria and Scoring Framework

To make the comparison more structured, each AI coding agent was evaluated against a common set of criteria derived from AWS Well-Architected review objectives.
The goal was not to measure model intelligence, but to assess how effectively each agent identified architectural risks, implementation issues, and actionable improvements within the AgriNova AI workload.

Evaluation Metrics

MetricDescriptionWeight
Architecture AwarenessAbility to identify architectural patterns, AWS service recommendations, and system-level improvements25%
Code & Implementation AwarenessAbility to inspect source code, runtime behavior, APIs, and integrations25%
Security AnalysisAbility to identify security risks, secrets management issues, and access control concerns15%
Operational ExcellenceQuality of observability, monitoring, logging, and operational recommendations15%
ActionabilityPracticality and implementation readiness of recommendations10%
Output Structure & ClarityOrganization, readability, and ease of consumption of findings10%

Scoring Scale

Each category was scored on a scale of 1–10:
  • 1–3 → Limited Coverage
  • 4–6 → Moderate Coverage
  • 7–8 → Strong Coverage
  • 9–10 → Excellent Coverage
The scores are based on qualitative analysis of the generated outputs and should be viewed as comparative observations rather than absolute performance measurements.

Benchmark Results

MetricCursorKiroClaude
Architecture Awareness989
Code & Implementation Awareness5109
Security Analysis799
Operational Excellence889
Actionability899
Output Structure & Clarity7810

Overall Assessment

ToolOverall Score (/10)Primary Strength
Cursor7.3Architecture Guidance
Kiro8.7Deep Implementation Analysis
Claude9.2Structured Well-Architected Assessment

Benchmark Limitations

This benchmark focuses on qualitative architectural evaluation rather than statistical model performance. The results may vary depending on workload complexity, prompt design, available project context, and tool integrations.
Future iterations may include:
  • Finding detection rate
  • Severity coverage scoring
  • Response consistency across multiple runs
  • Time-to-insight measurements
  • Recommendation implementation success rate
These additions would further strengthen the benchmarking methodology and provide more quantitative comparisons.

Benchmark Summary

ToolRoleStrengthReview Style
CursorArchitecture ReviewerAWS architecture guidanceArchitecture-first
KiroProduction AuditorRuntime + implementation inspectionCode-first
ClaudeWA AssessorStructured scoring and analysisFramework-first

Key Engineering Findings

Highest priority improvements identified across all agents:
PriorityImprovement
P1Remove tool bypass
P1Protect secrets
P1Add API timeout and retries
P2Introduce caching
P2Tool routing
P2Structured observability
P3Lambda + scale-to-zero
P3Memory persistence

Recommended Target State

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Farmer Query
↓

API Gateway

↓

AgriNova Agent

↓

Tool Routing Layer

↓

Weather / Market / Soil Services

↓

Memory + Training Data

↓

CloudWatch + X-Ray

↓

Lambda + Scale-to-zero

Final Thoughts

This experiment demonstrated that AWS Well-Architected Skills significantly influence AI coding outputs.
The same workload produced three different engineering perspectives:
Cursor → Architecture Reviewer
Focused on AWS service design and operational maturity.
Kiro → Production Auditor
Focused on implementation, runtime behavior, and operational risks.
Claude → Well-Architected Assessor
Focused on structured scoring and production readiness.
This highlights an important shift:
AI coding agents are evolving beyond code generation into architecture-aware engineering assistants.
Using a real workload such as AgriNova AI made the evaluation significantly more meaningful than benchmarking against toy examples.

Thank You for Reading 🙌

Thank you for taking the time to read this article and explore this benchmark journey with AgriNova AI and AWS Well-Architected Skills.
This experiment was an attempt to understand how AI coding agents evolve from code assistants into architecture-aware engineering companions, and how the same workload can produce completely different engineering perspectives.
If you found this article useful:
⭐ Feel free to like and share it with the community
💬 Drop your thoughts, observations, or experiences in the comments — I would genuinely love to hear different viewpoints and benchmark approaches from other builders.
I am also continuously improving AgriNova AI. If you have suggestions around:
  • Architecture improvements
  • AWS Well-Architected practices
  • Bedrock / Agent workflows
  • Cost optimization ideas
  • Security enhancements
  • Production readiness recommendations
please share them — feedback from the community is always valuable.
Future improvements planned for AgriNova AI include:
✅ Better guardrails and security controls
✅ Observability with CloudWatch and X-Ray
✅ Tool routing and token optimization
✅ Serverless scale-to-zero architecture
✅ Memory persistence improvements
✅ Production-grade reliability patterns
Thank you again for reading and being part of this learning journey 🌾🤖
# agentic-ai# agents# ai-agents# amazon-bedrock-agents# amazon-nova
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article