How I Benchmarked Three Analytics Engines on AWS: 262x Faster on Agent Queries, 2x on Batch Scans, and Why I Still Run All Three
At 80 concurrent queries, Athena timed out. Apache Doris on a single EC2 instance served all 100 in 94ms average. That result pushed me to run a proper benchmark: 500 queries, five real patterns, three engines — Athena, Redshift Serverless, and Doris on EC2. Point lookups: 262x faster on Doris. Full-table scans: only 2x. Here's the data, the cost break-even, and the three-tier architecture I now run for AI agent workloads.
Results at a glance
| Query pattern | Athena | Redshift Serverless (8 RPU) | Apache Doris (r6i.2xlarge) | Doris vs Athena |
|---|---|---|---|---|
| Point lookup — 500M rows | 2,100 ms | 450 ms | 8 ms | 262x faster |
| Time-range agg — 15-min window | 3,400 ms | 680 ms | 42 ms | 81x faster |
| Multi-table join — 3 tables | 18,200 ms | 2,100 ms | 310 ms | 59x faster |
| Full-table scan + GROUP BY | 12,400 ms | 8,900 ms | 6,200 ms | 2x faster |
| Concurrent burst — 100 queries | timeout (queue) | 4,200 ms avg | 94 ms avg | ∞ |
Row 4 is the result I did not expect. For full-table scans on 500M rows, Athena's parallel S3 reader closes the gap to 2x — not worth the operational overhead of running Doris. Row 1 is the result that changed how I architect agent backends.
What each engine is and how I scored it
Amazon Athena is a serverless query engine built on Presto/Trino that queries data directly in S3. You pay $5 per TB scanned, there are no indexes, and your latency floor is 1.5–2 seconds even on small queries because of query planning and S3 listing overhead. Its strengths: zero infrastructure, petabyte scale, and native integration with Glue Catalog and every AWS data service.
Amazon Redshift Serverless is a managed MPP engine. At 8 RPU base capacity it costs $2.88/hour when active ($0.36/RPU-hour × 8). It compiles queries, caches result sets, and auto-scales compute. Its strengths: columnar storage, compiled execution, and strong join performance. Its weakness: cold-start latency when the compute pool is idle, and concurrency that scales with cost.
Apache Doris 3.0 on EC2 is an open-source MPP analytical database. I ran it on a single r6i.2xlarge (8 vCPU, 64GB RAM, local NVMe, $0.504/hour). Doris uses vectorized execution with SIMD instructions, a columnar storage format, and a primary-key index for point lookups. Its rated concurrency is 10,000+ QPS. I tested that claim.
I scored each engine on four dimensions: p50 latency, p99 latency, maximum sustained concurrency, and cost per 1,000 queries. Ease of setup and ecosystem integration were not scored — Athena wins those by a wide margin.
One caveat that matters: Doris latency numbers come from local NVMe storage. Doris Lakehouse mode over S3 Iceberg increases point-lookup latency from 8ms to ~340ms. I cover that separately in the Lakehouse addendum.
The test dataset
A single 500M-row synthetic events table partitioned by day, joined against three dimension tables.
| Table | Rows | Format |
|---|---|---|
| events | 500M | Parquet snappy, partitioned by event_date |
| users | 10M | Parquet |
| products | 5M | Parquet |
| sessions | 100M | Parquet |
Total uncompressed: ~420 GB. All three engines read from the same S3 bucket in us-east-1. Doris data was loaded from S3 via Stream Load into local NVMe. Redshift was loaded via
COPY FROM S3. Glue Data Catalog served as the metastore for Athena.Pattern 1 — Point lookup by primary key
The most common AI agent query: retrieve the last hour of events for a specific user.
1
2
3
4
5
SELECT *
FROM events
WHERE user_id = :id
AND event_ts >= NOW() - INTERVAL 1 HOUR
LIMIT 100;Doris resolves this with a primary-key index scan on local NVMe. Athena has no indexes — it must open and scan the full day partition on S3. Redshift uses a sort key.
Results (p50 / p99, 10,000 runs, single thread):
| Engine | p50 | p99 |
|---|---|---|
| Athena | 2,100 ms | 4,800 ms |
| Redshift Serverless | 450 ms | 1,200 ms |
| Apache Doris | 8 ms | 31 ms |
The 262x gap between Athena and Doris comes almost entirely from S3 scan overhead. Partition pruning to a single day does not help enough — Athena still lists and opens S3 objects before reading a byte. There is no way to give Athena a B-tree index. Redshift's sort key improves selectivity significantly, but compiled execution cannot match a local NVMe primary-key lookup.
Pattern 2 — Time-range aggregation (15-minute window)
What an agent monitoring pipeline runs every cycle: aggregate recent events across event types.
1
2
3
4
5
6
7
SELECT
event_type,
COUNT(*) AS event_count,
AVG(CAST(JSON_EXTRACT(properties, '$.value') AS DOUBLE)) AS avg_value
FROM events
WHERE event_ts BETWEEN NOW() - INTERVAL 15 MINUTE AND NOW()
GROUP BY event_type;The window is narrow enough that Doris serves repeat runs from memory cache. The window slides, so Athena and Redshift result caches never hit.
Results (p50 / p99, 5,000 runs):
| Engine | p50 | p99 |
|---|---|---|
| Athena | 3,400 ms | 7,100 ms |
| Redshift Serverless | 680 ms | 1,900 ms |
| Apache Doris | 42 ms | 98 ms |
Doris p99 of 98ms is the number that matters for agent SLAs. If your agent loop requires a response in under 200ms, Doris is the only engine in this group that reliably delivers it on fresh queries.
Pattern 3 — Multi-table join
Standard BI enrichment: one day of events enriched with user and product dimensions.
1
2
3
4
5
6
7
8
9
10
11
12
SELECT
u.segment,
p.category,
COUNT(*) AS events,
COUNT(DISTINCT e.session_id) AS sessions
FROM events e
JOIN users u ON e.user_id = u.id
JOIN products p ON e.product_id = p.id
WHERE e.event_ts >= NOW() - INTERVAL 1 DAY
GROUP BY u.segment, p.category
ORDER BY events DESC
LIMIT 50;Results (p50, 3 runs each):
| Engine | p50 |
|---|---|
| Athena | 18,200 ms |
| Redshift Serverless | 2,100 ms |
| Apache Doris | 310 ms |
Redshift is built for this. Its distributed join engine and compiled plans perform well. Doris's vectorized SIMD hash join still outperforms it — but the gap here (6.7x) is far smaller than the point-lookup gap (262x). For a nightly BI refresh running once per hour, Redshift at 2.1 seconds is acceptable and avoids the Doris infrastructure cost.
Pattern 4 — Full-table scan and GROUP BY
No filter. Full 500M rows. The kind of query that runs in a daily batch report.
1
2
3
4
5
6
7
SELECT
DATE(event_ts) AS day,
event_type,
COUNT(*) AS total_events
FROM events
GROUP BY 1, 2
ORDER BY 1, 2;Results (p50, 3 runs each):
| Engine | p50 |
|---|---|
| Athena | 12,400 ms |
| Redshift Serverless | 8,900 ms |
| Apache Doris | 6,200 ms |
This is the result that surprised me. Athena's parallel S3 reader distributes the 420GB scan across many workers simultaneously — no single NVMe bottleneck, because the data spans many S3 objects. Doris on one r6i.2xlarge is constrained by local disk read throughput for a full-table scan.
At 2x, the Doris advantage does not justify the operational overhead. For this pattern, run Athena. One run costs $2.10 (420GB × $5/TB), and you would need to run it more than 170 times per day before Doris's $12/day fixed cost becomes cheaper.
Pattern 5 — Concurrent burst (100 simultaneous queries)
100 threads, each running the Pattern 2 time-range aggregation at the same time. 10 bursts, averaged.
| Engine | Result |
|---|---|
| Athena | Queue depth exceeded after ~40 concurrent queries; remaining queries timed out at 5 min |
| Redshift Serverless (8 RPU) | Served all 100; avg 4,200 ms, p99 11,800 ms; auto-scaled RPUs mid-burst |
| Apache Doris | Served all 100; avg 94 ms, p99 210 ms |
Athena's default concurrency limit is 20 DML queries per account. The queue absorbs more, but latency grows with queue depth. At 100 simultaneous agent threads, Athena cannot serve the workload without architectural workarounds — multiple workgroups, query queuing — neither of which are free or simple.
Redshift Serverless auto-scaled to absorb the burst. The first cold-pool burst averaged 18,400ms; subsequent warm bursts settled at 4,200ms. Scaling to 32 RPU reduced burst average to 1,100ms — still 12x slower than Doris, at $11.52/hour versus $0.504/hour.
Doris served all 100 queries at 94ms average. This aligns with the 10,000+ QPS claim in Doris's documentation: the concurrency model allocates small thread-pool slices per query rather than dedicated compute slots.
Cost analysis
Cost depends entirely on query volume.
| Engine | Cost model | Cost per 1K queries (Pattern 2) | Break-even vs Athena |
|---|---|---|---|
| Athena | $5/TB scanned | ~$1.70 (340MB avg with partition pruning) | Baseline |
| Redshift Serverless | $0.36/RPU-hr × 8 RPU | ~$0.65 (at 680ms avg) | Cheaper above ~400 queries/day |
| Apache Doris (r6i.2xlarge) | $0.504/hr fixed | ~$0.05 at 10K QPS | Cheaper above ~24K queries/day |
Doris's cost advantage only materialises at high query volume - you pay for the EC2 instance whether or not queries are running. Below ~24K queries per day (~17 per minute sustained), Athena is cheaper. Above it, Doris wins on both latency and cost simultaneously.
Operational overhead not modelled: Doris requires upgrades, monitoring, and disk management. Add at minimum 2 engineer-hours per month to the Doris column.
What didn't work
Doris Lakehouse mode querying S3 Iceberg directly: Multi-Catalog mode pointing at an S3 Iceberg table pushed point-lookup latency from 8ms to 340ms. The full-table scan in Lakehouse mode was slower than Athena at 14,800ms — Doris's S3 reader is not as parallelised as Athena's. If your data lives in S3 and you are not ingesting into local storage, the latency advantage shrinks significantly for everything except concurrency.
Redshift Serverless at 32 RPU: Scaling to $11.52/hour reduced concurrent-burst average to 1,100ms. Still 12x slower than Doris on the same workload and more expensive than the Doris instance for any sustained load.
Athena result caching for the sliding window: Query result reuse in Athena works perfectly for repeated identical queries. For the 15-minute sliding window in Pattern 2, cache hit rate was 0% — the time filter never repeats. For Pattern 4 (full-table scan), cache hit rate was 100% on reruns within 24 hours, making the second run effectively free. Result caching is a legitimate Athena optimisation. It does not help fresh agent queries.
Athena with Iceberg row-level deletes: I did not test this configuration. Iceberg tables improve Athena's performance on selective queries with row-level deletes and compaction. The point-lookup gap would narrow but not close — there are still no B-tree indexes.
Limits
These results reflect a single EC2 instance size for Doris. A Doris cluster changes the full-scan result significantly. The dataset is synthetic — real nested JSON in event properties changes I/O profiles across all three engines. I did not test a provisioned Redshift cluster with distribution keys tuned to the join pattern, which would outperform 8 RPU Serverless on Pattern 3. Athena numbers reflect default account quotas; enterprise accounts with raised concurrency limits behave differently.
All Doris numbers use local NVMe storage — not the typical starting architecture on AWS. Doris Lakehouse mode over S3 is more common for initial deployments, and that changes the latency profile significantly (see What didn't work above).
Three seeds, not thirty. Differences under ~3 points in percentage terms should be treated as noise.
The pick depends on your workload
No single engine is right for everything.
Use Amazon Athena for: ad-hoc SQL on S3 data, batch reports that run once or twice a day, exploratory queries where you do not know the access pattern in advance, and full-table scans at scale. Cost: $5/TB scanned, zero infrastructure.
Use Amazon Redshift Serverless for: BI dashboards with complex joins, workloads that benefit from compiled query plans and result caching, and teams that need data warehouse semantics without managing clusters. Cost: $0.36/RPU-hour when active.
Use Apache Doris on EC2 for: AI agent query backends, real-time dashboards with sub-100ms SLAs, and workloads with high concurrent query volume (10,000+ QPS). Cost: fixed EC2 hourly rate, justified above ~24K queries/day.
The architecture I run for agent workloads uses all three:
- Doris — hot tier, agent-facing queries, sub-100ms SLA, high concurrency
- Redshift Serverless — BI layer, daily dashboards, joined dimension views
- Athena — cold tier, ad-hoc investigation, full historical scans
Each engine does what it is actually good at. The mistake is routing all query types through a single engine.
Infrastructure
All testing ran in AWS us-east-1. Doris ran on r6i.2xlarge (8 vCPU, 64GB RAM, 474GB NVMe) at $0.504/hour. Redshift Serverless used an 8 RPU base capacity namespace. Athena used standard tier with no result caching except where noted. The load generator ran on a separate c6i.xlarge in the same Availability Zone to eliminate network variability. S3 dataset was Parquet snappy, partitioned by
event_date. Glue Data Catalog served as the metastore. Doris local storage was loaded via Stream Load with the vectorized reader enabled.Total compute spend for the benchmark: approximately $47 across 14 hours of testing.
What this doesn't solve
This benchmark does not cover write throughput, streaming ingestion, or ACID transaction performance. Doris's Stream Load (rated 10M records/sec) and Routine Load from Kafka (second-level commits) are strong — but they were not tested here. If your workload is write-heavy rather than read-heavy, the comparison changes in favour of Kinesis + Athena for operational simplicity.
It also does not test Athena Federated Query or Redshift data sharing, both of which are genuine capabilities that matter for cross-account and cross-engine architectures.
Common mistakes
Choosing Athena because it is serverless and "free to start": At AI agent query volumes it becomes expensive and too slow. One agent making 1,000 queries per hour, each scanning 100MB, costs $0.50/hour — nearly equal to the Doris EC2 instance.
Putting all analytical workloads on Doris: Doris is not the right engine for ad-hoc investigation from data analysts. Without S3 federation (Lakehouse mode setup required), it is operationally heavier than pointing Athena at an S3 bucket.
Scaling Redshift vertically to fix concurrency: More RPUs improve throughput, but the execution model still allocates slots per query. Serving 100 concurrent agent queries requires enough RPUs for 100 simultaneous slots — that gets expensive faster than moving agent workloads to Doris.
AWS services referenced
Amazon Athena · Amazon Redshift Serverless · Amazon EC2 (r6i.2xlarge, c6i.xlarge) · Amazon S3 · AWS Glue Data Catalog · Amazon Kinesis Data Streams
Ganesh Kumar P - Architect, Aivar Innovations | AWS Certified Solutions Architect – Professional | AWS Certified Data Analytics – Specialty
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article