AWS Builder Center

Builder Center

From Polygon to Insight: Building an Open-Source Geospatial AI Agent on AWS

From Polygon to Insight: Building an Open-Source Geospatial AI Agent on AWS

An open-source AI geospatial agent that lets you draw a polygon on a map, ask a question in plain English, and get back satellite analysis — NDVI, burn severity, water detection, thermal maps — as interactive overlays. Built as a coding agent (not tool-calling) so satellite data stays in memory and never floods the LLM context. Deployed on Amazon Bedrock AgentCore with a serverless React UI. Fully extensible: add a Python function with a docstring, and the agent discovers it automatically.

Series: Strands Agents CodeAct (3 articles)

  1. 1
    Introducing strands-code-agent: CodeAct as the Action Paradigm for Strands Agents
  2. 2
    Building a Data Analyst Agent in Amazon Bedrock AgentCore
  3. 3
    From Polygon to Insight: Building an Open-Source Geospatial AI Agent on AWS This article

Emilio

Senior Applied Scientist @ AWS Prototyping
You draw a polygon over a wildfire-affected region in California. You type: "What is the burn severity here?"
Within seconds, the agent fetches the most recent cloud-free Landsat scene, computes the Normalized Burn Ratio before and after the fire, classifies the result into severity categories, and overlays a color-coded map directly on your browser's interactive map — with statistics showing that 34% of the area experienced high-severity burns.
No configuration. No code. No GIS software.
This post explains how we built this — the key architectural decisions, the code that makes it work, and how to deploy and extend it yourself. The project is open-source on GitHub .

The Core Decision: Coding Agent, Not Tool-Calling Agent

Most AI agents are tool-calling agents: the LLM picks a function, calls it with arguments, gets a result back in its context, and decides what to do next. This works well when tools return small, structured values. It gets awkward when tools return large data structures.
For geospatial analysis, the practical workaround is to pass data through files — write intermediate results to GeoTIFF, NetCDF, or NumPy .npy files and pass file paths between tool calls. Efficient binary formats exist, so raw serialization cost is not prohibitive. But the pattern adds friction: every handoff requires an explicit write-to-file and read-from-file step, temporary files need to be tracked and cleaned up, and for a multi-step pipeline — fetch bands, compute a vegetation index, run a thermal analysis, correlate the two — each step is a separate LLM round trip, each waiting for the previous tool to return before the next call can be made.
Coding agents take a different approach. Instead of a fixed set of tools, the LLM gets a Python interpreter. It writes a script; the interpreter runs it; intermediate results stay in Python variables that persist across code blocks within the same turn. There are no files to write, no paths to track, no round trips between steps. The LLM only sees what the code explicitly prints — statistics, summaries, confirmations. A multi-step analysis that would require several sequential tool calls executes as a single script, with data flowing directly from one function to the next in memory.
We built this with the Strands Agents SDK  and the strands-code-agent  library:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
self.agent = CodeAgent(
system_prompt="You are an expert geospatial analyst proficient in Python...",
tools=[visualize_image, visualize_map_raster_layer, share_file_with_client],
toolkits=[
VISUALIZATION_TOOLKIT,
DATA_ANALYSIS_TOOLKIT,
Toolkit(
libraries=['PIL.*', 'pycocotools.*'],
initialization_code=INITIALIZATION_CODE.render(AOI_COORDINATES=coordinates),
domain_specific_code=[
get_satellite_data, compute_NDVI, compute_NDWI,
compute_NBR, compute_dNBR, generate_overlay,
convert_lwir11_to_celsius,
],
),
],
model=BedrockModel(model_id=model_id, temperature=0.1),
)
The four tools attached to the agent are notable: python_repl (the primary tool — executes Python), visualize_image (sends a PNG to the UI), visualize_map_raster_layer (adds a geo-referenced raster to the map), and share_file_with_client (uploads a file to S3 and returns a pre-signed URL). The visualization tools are "fire-and-forget" — they stream content to the client but return nothing to the LLM, keeping context clean.

The Geospatial Library: Where the Real Intelligence Lives

The LLM doesn't understand satellite imagery. It writes Python code that calls functions from a purpose-built geospatial library. That library is where the domain expertise lives, and extending the agent means extending the library.

Satellite Data Retrieval

Fetching satellite imagery follows a four-step pipeline built on open standards:
1. STAC search. The SpatioTemporal Asset Catalog  standard is a search engine for geospatial data. We query the Element84 Earth Search API , filtering by polygon intersection, date range, and cloud cover threshold. The result is a list of scenes grouped by satellite grid cell.
2. Scene selection. We pick the best scene in two steps: first, find the grid cell with the largest overlap with the user's area of interest; then, among scenes in that cell, pick the one with the lowest cloud cover percentage.
3. Band fetching. We use stackstac  to load spectral bands from Cloud-Optimized GeoTIFFs  on S3. COGs are the key technology here — they allow partial reads, so we download only the bounding box region we care about, not the full 100×100 km scene. The result is an xarray.DataArray with dimensions (band, y, x).
4. Polygon clip. We clip the loaded data precisely to the user's drawn polygon using rioxarray, masking everything outside as NaN.
The whole pipeline is a single function call for the agent:
1
2
3
4
5
6
7
scene = get_satellite_data(
satellite_name="sentinel",
bands=["red", "green", "blue", "nir"],
polygon_coordinates=AOI_COORDINATES,
max_cloud=20
)
# scene['data'] is an xr.DataArray ready for analysis
We support three satellites: Sentinel-2 (10–20m, visible + NIR/SWIR bands), Landsat (30m, includes the thermal lwir11 band), and NAIP (0.3–0.6m aerial imagery, CONUS only, used for object detection).

Spectral Indices

Spectral indices are mathematical combinations of bands that highlight surface properties. NDVI uses NIR and red bands to measure vegetation density; NDWI uses green and NIR for water bodies; NBR uses NIR and SWIR2 for fire damage.
We built a small framework to make adding new indices trivial. An Index dataclass carries all the metadata:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
NDVI = Index(
name="NDVI (Normalized Difference Vegetation Index)",
bands=["red", "nir"],
valid_range=(-1.0, 1.0),
index_cmap='RdYlGn',
classes=[
("Water/No Data", (-1.00, 0.00), "#0000FF"),
("Bare Soil/Rock", ( 0.00, 0.15), "#8B4513"),
("Sparse Vegetation", ( 0.15, 0.33), "#DEB887"),
("Moderate Vegetation", ( 0.33, 0.50), "#90EE90"),
("Dense Vegetation", ( 0.50, 0.66), "#228B22"),
("Very Dense Vegetation", ( 0.66, 1.00), "#006400")
]
)
A compute_ function wraps the math:
1
2
3
4
5
6
7
8
9
10
def compute_NDVI(data: xr.DataArray) -> ComputedIndex:
"""
Compute Normalized Difference Vegetation Index (NDVI).
NDVI = (NIR - Red) / (NIR + Red)
...
"""
bands = get_bands(data, NDVI.bands)
return ComputedIndex(
NDVI, safe_divide(bands['nir'] - bands['red'], bands['nir'] + bands['red'])
)
The ComputedIndex object that comes back provides get_statistics() (min, max, mean, median, percentiles), get_class_percentages() (pixel distribution per class), and class_to_rgba() (an RGBA array ready for visualization). The pattern is the same for every index — you define the metadata, write the formula, and everything else just works.
To add a completely new index, it's about fifteen lines of code. You define an Index instance, write a compute_ function with a docstring, export it from __init__.py, and register it in the agent's toolkit list. That's the whole extension process.

Auto-Documentation via inspect

Here's a subtle but important design detail. The strands-code-agent library uses Python's inspect module to automatically extract docstrings and type hints from every function in the toolkit's domain_specific_code list. It then injects this documentation into the agent's system prompt.
This means the LLM always has accurate, up-to-date documentation of every available function — including the exact argument names, types, and what the function returns. You never need to maintain a separate schema. Write a good docstring, and the agent knows how to use your function.

Infrastructure: Serverless and Managed End-to-End

The infrastructure is defined entirely in AWS CDK (Python), across three stacks that deploy in dependency order.
1
VPCStack → AgentCoreStack → WebAppStack

Amazon Bedrock AgentCore

AgentCore is the managed service that hosts the agent. You provide a Docker image; AgentCore handles running it, exposing an HTTPS endpoint, managing sessions, and scaling. No server configuration, no Kubernetes, no load balancer to manage.
The container is built automatically. When you run cdk deploy, CDK uploads the agent/ source directory to S3, computes a content hash, and triggers a CodeBuild project if the hash has changed. CodeBuild builds the ARM64 Docker image and pushes it to ECR. A Lambda-backed Custom Resource waits for the build to complete before the AgentCore runtime is created. The result: code changes automatically trigger a fresh container build with no manual steps.

Security Model

The security model is deliberately minimal. Cognito authenticates users. On successful login, an Identity Pool issues temporary IAM credentials. Those credentials have exactly one permission: bedrock-agentcore:InvokeAgentRuntime scoped to the specific ARN of this agent runtime. Users cannot invoke any other AWS resource.
The agent container runs in private VPC subnets. Its security group allows only HTTPS outbound (TCP 443) via a NAT Gateway — necessary to reach the Element84 STAC API and download COG files. Nothing else is reachable.

UI Hosting

The React frontend is a serverless static site. S3 stores the build artifacts. CloudFront serves them with TLS 1.2, HTTPS redirect, and SPA routing (404s and 403s redirect to index.html so React Router handles them). Zero backend servers.
Deploying the entire stack, from infrastructure to uploaded UI, is a single command:
1
./scripts/deploy.sh --cdk
The script deploys CDK stacks, reads CloudFormation outputs, generates the React .env file, runs npm run build, syncs to S3, and invalidates the CloudFront cache.

The UI: Map and Chat, Streaming in Real Time

The user interface has two panels: an interactive map and a chat sidebar.
The map is built with MapLibre GL JS . Users switch between base maps (CARTO Dark, Google Satellite, Esri Satellite), draw polygons defining their area of interest, and see analysis results overlaid as geo-referenced raster layers. When the agent calls visualize_map_raster_layer, the PNG and its geographic bounds arrive in the event stream and are placed at exactly the right position on the map.
The chat panel renders Markdown with syntax-highlighted code blocks — the agent's generated Python code is fully visible to the user, making the analysis transparent and auditable. Images arrive inline. Download links appear as clickable buttons. After each response, the interface displays the on-demand cost of that invocation in USD.
The agent streams results as Server-Sent Events. The frontend's api.ts is an async generator that yields typed events:
EventWhat it carries
textLLM reasoning narrative
python_codeGenerated Python script
execution_outputPython stdout
imageBase64-encoded PNG
file_linkS3 pre-signed download URL
resultFinal answer + token counts + cost
This event model means the UI can react immediately to each piece of output — displaying text as it streams, adding map layers when analysis completes, showing code blocks as they're generated — without waiting for the full response.

Optional: Object Detection with Scale-to-Zero GPUs

For tasks like counting aircraft at an airport or ships in a harbor, we integrated Falcon-Perception , a 0.6B vision-language model that supports open-vocabulary object detection — you describe what you want to find in natural language, and the model finds it.
The model runs on a SageMaker real-time endpoint (ml.g6e.xlarge, L40S GPU, 48 GB VRAM). The interesting engineering challenge: GPU instances are expensive (~$2.61/hour), but the model doesn't need to run all the time. We implemented full scale-to-zero using two CloudWatch alarms and Application Auto Scaling with SageMaker Inference Components.
Scale-in: After 60 consecutive minutes with zero invocations, a CloudWatch alarm fires and a step scaling policy sets DesiredCopyCount = 0. The GPU instance is released. Billing stops.
Scale-out: When an invocation arrives but there are no instances, SageMaker emits a NoCapacityInvocationFailures metric. A CloudWatch alarm fires immediately, triggering a step scaling policy that sets DesiredCopyCount = 1. A new instance provisions (~5–8 minutes cold start).
The cold start is handled gracefully in the agent: if it catches a ModelNotReadyException, it surfaces a friendly message to the user: "The object detection model is starting up. Please retry in 5–8 minutes." The endpoint is entirely self-managing — no cron jobs, no manual start/stop scripts.
1
2
3
4
def detect_objects(image: np.ndarray, query: str) -> list[dict]:
"""Detect objects in a high-resolution image using open-vocabulary detection.
Uses Falcon-Perception via a SageMaker endpoint...
"""
A typical workflow: get_high_resolution_image() fetches NAIP imagery at 0.6m resolution for the area; detect_objects(result['rgb'], "airplane") finds instances; the agent draws bounding boxes, counts detections, and reports back to the user.

Evaluation: Measuring What Matters

A geospatial agent is only as good as the accuracy of its analysis. We built an evaluation framework that benchmarks the agent against predefined test cases.
Each test case specifies a question, an expected answer, and the polygon coordinates of the area of interest:
1
2
3
4
5
6
7
{
"id": "1",
"use-case": "NDVI",
"question": "What is the NDVI in this area?",
"answer": "0.45",
"aoi": [[lon1, lat1], [lon2, lat2], ...]
}
Scoring uses LLM-as-Judge: we pass the question, expected answer, and generated answer to Claude Haiku 4.5 (temperature 0.2), which returns a score between 0 and 1 with a rationale. For numerical answers, the score degrades proportionally to the percentage error — a 25% error gives a 0.75 score. This handles the fact that satellite scenes change over time, so the "right answer" for NDVI on a given date may differ slightly from the expected answer recorded in the test suite.
The AgentClient class in evaluation/agent_client.py mirrors the web UI exactly: it reads CloudFormation stack outputs, authenticates via Cognito with real credentials, and invokes AgentCore with the same SSE streaming interface. If the web UI works, the evaluation client works.
All evaluations are cached with diskcache, keyed by model ID, temperature, system prompt, question, and both answers. Re-running the benchmark after fixing an unrelated bug costs nothing for already-evaluated pairs.
1
2
3
cd evaluation
python benchmark_agent.py # Full benchmark
python benchmark_agent.py --use_case "NDVI" # Single use case
Cost tracking is built in at the agent level. The result event from the Strands SDK carries accumulated input and output token counts. We multiply these by the per-token costs defined in bedrock_models.py to compute the exact on-demand USD cost of each invocation — displayed in the UI and logged during evaluation runs.

Key Takeaways

Coding agents outperform tool-calling agents when data is large and multi-step. If your domain involves data that can't be serialized without information loss, or analyses that require holding state across multiple operations, a coding agent is the right architecture. The Python interpreter is the tool.
The library, not the prompt, is the intelligence surface. We invested engineering effort in a well-designed geospatial library — clean data structures, consistent conventions, thorough docstrings. The system prompt is short. The LLM's job is to write Python; our library's job is to make that Python correct and efficient. This makes the system more reliable, more testable, and more extensible than trying to put domain knowledge into the prompt.
Auto-documentation removes a whole class of maintenance burden. By using inspect to extract docstrings into the agent's context automatically, we never have a schema that drifts from the implementation. The contract between the LLM and the library is always in sync.
AgentCore + CDK makes managed agent hosting genuinely simple. The combination of a managed container runtime (no server to configure) and infrastructure-as-code (reproducible, reviewable, version-controlled deployment) significantly reduces operational overhead. The automatic container rebuild on code changes — triggered by a content hash in a CloudFormation Custom Resource — is a small pattern with large payoff.
Scale-to-zero GPUs are practical. The two-alarm Application Auto Scaling pattern for SageMaker Inference Components is straightforward to implement and eliminates GPU idle billing entirely. For optional capabilities that aren't used constantly, this is the right default.

Getting Started

The project is open-source under the MIT-0 license. To deploy your own instance:
1
2
3
git clone https://github.com/aws-samples/sample-geospatial-agent
cd sample-geospatial-agent
./scripts/deploy.sh --cdk
You'll need AWS credentials, CDK bootstrapped in your account, and Node.js for the React build. The deploy script handles everything else. After deployment, create a Cognito user:
1
2
3
4
aws cognito-idp admin-create-user \
--user-pool-id <user-pool-id> \
--username <username> \
--temporary-password <password>
Open the CloudFront URL, log in, draw a polygon, and ask a question.
To run the agent locally against your own AWS account (useful for development):
1
2
python -m geospatial_agent.agent_service # starts on localhost:8080
./scripts/start-ui-local.sh --local # UI connects to local agent
To extend the agent with new analysis capabilities, add a function with a docstring and type hints to agent/geospatial/, export it in __init__.py, register it in the toolkit in agent.py, and redeploy. CDK detects the source change and rebuilds the container automatically.
We'd love to see what you build with it. Issues, PRs, and questions are welcome on GitHub.
# amazon-bedrock-agentcore# agentic-ai# strands-agents

Series: Strands Agents CodeAct (3 articles)

  1. 1
    Introducing strands-code-agent: CodeAct as the Action Paradigm for Strands Agents
  2. 2
    Building a Data Analyst Agent in Amazon Bedrock AgentCore
  3. 3
    From Polygon to Insight: Building an Open-Source Geospatial AI Agent on AWS This article
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article