AWS Builder Center

Builder Center

Benchmarking Amazon S3 Vectors vs OpenSearch Serverless at Scale

A hands-on benchmark comparing latency, throughput, cost, and filtered search behavior across both services — with practical guidance on when to use each.

Katia

Vector search is showing up everywhere — RAG pipelines, product recommendations, image similarity, semantic document search. And when vector storage comes up in architecture reviews, the same question keeps appearing: where should I store and query my vectors?
On AWS, two serverless options stand out for vector workloads: Amazon S3 Vectors and Amazon OpenSearch Serverless (AOSS). Both let you avoid infrastructure management. Both support metadata filtering. But they have fundamentally different architectures and pricing models. AOSS offers a rich feature set — aggregations, hybrid search, complex DSL queries — that Amazon S3 Vectors doesn't provide. However, for workloads that only need vector similarity search with simple metadata filtering, both services work well. The cost difference is clear from the pricing pages — S3 Vectors has zero idle cost while AOSS requires a minimum of 2 OCUs (OpenSearch Compute Units — each a combination of 6 GiB RAM and vCPU; pricing varies by region — ~$366/month in eu-west-1, $350/month in us-east-1) for production collections with redundancy enabled.  Disabling redundancy  reduces this to 1 OCU total ($183/month in eu-west-1), and additional collections sharing the same KMS key can share OCUs.
The questions I keep hearing are practical ones. What happens to query latency when I add a filter that only matches 10% of my vectors? How much does it actually cost at 100K vectors versus 10 million? And how do different filtering strategies affect result completeness?
I couldn't point builders to a clear answer, so I decided to run the benchmark myself. I indexed 75,000 vectors with 10 metadata fields, tested 7 different filter types at varying selectivity levels, and measured everything from ingestion throughput to cold-start behavior. The results also highlight an important difference between OpenSearch filtering strategies — post-filtering vs efficient filtering — that directly impacts result completeness and is worth understanding before going to production.
Here's what I found.

What I Tested and Why

The benchmark simulates a common pattern: vector embeddings with metadata that needs to be filtered at query time. Think product catalogs filtered by category and availability, document stores scoped by access level or department, or recommendation engines narrowed by user segment. The metadata distribution covers a range of selectivity — from  premium  (only 10% of vectors match) to  active  (67% match) — to stress-test how each service handles restrictive filters.
I needed to answer four questions:
  1. How fast can each service ingest vectors at scale?
  1. What's the query latency when I add metadata filters — especially restrictive ones?
  1. How does performance scale with different k-values and index sizes?
  1. What does this actually cost at different scales?

The Setup

Parameter
Value
Vectors
Up to 100,000 (1024 dimensions, cosine similarity)
Metadata fields
10 (category, priority, region, department, premium, active, score, year, created_at, vector_id)
Filter selectivity
10% to 67%
AOSS vector mode
 on_disk  (32× binary quantization, Faiss HNSW engine)
Region
eu-west-1
Test period
December 2025 – March 2026
For each service, I ran 4 test scenarios: ingestion optimization, filter performance, k-value scaling, and end-to-end workflow simulation. Both services got identical vector data, metadata distributions, and query patterns.

The Two Services

Amazon S3 Vectors is a serverless vector storage service that extends Amazon S3 with native vector search. You store embeddings directly on S3, query them with a simple  query_vectors()  API call, and pay only when you actually use it. No clusters, no capacity planning, no idle cost.
Amazon OpenSearch Serverless (AOSS) is the serverless flavor of Amazon OpenSearch Service. For vector workloads, it offers VECTORSEARCH collections using the HNSW algorithm with Faiss engine. It auto-scales compute, but requires a minimum of 2 OCU for collections with redundancy enabled. Disabling redundancy reduces this to 1 OCU (0.5 indexing + 0.5 search), but removes standby nodes and replicas — meaning a single node failure can cause downtime. This works for development, testing, or workloads where availability is not critical.

Why I Chose on_disk Mode for AOSS

AOSS supports two vector workload modes:  in_memory  and  on_disk . This is an important architectural choice that directly affects cost and latency.
In-memory mode keeps full-precision vectors in RAM. It delivers the lowest latency (single-digit milliseconds) but requires more OCUs because vector data consumes RAM directly. This is the default mode and the most expensive option.
On-disk mode ( on_disk ) uses  binary quantization  to compress each 32-bit vector dimension down to a single bit — a 32× compression ratio that  reduces memory requirements by 97% . The compressed index stays in RAM for fast candidate retrieval, while full-precision vectors are stored on disk. Queries run in two phases: first, the compressed index is searched in memory for candidates; then, full-precision vectors are loaded from disk to rescore and refine the results. This rescoring step is what adds latency compared to in-memory mode.
To configure on_disk mode, set the  mode  parameter in the index mapping:
PUT my-vector-index
{
  "settings": {
    "index": { "knn": true }
  },
  "mappings": {
    "properties": {
      "vector": {
        "type": "knn_vector",
        "dimension": 1024,
        "space_type": "cosinesimil",
        "mode": "on_disk",
        "method": {
          "name": "hnsw",
          "engine": "faiss"
        }
      }
    }
  }
}
By default,  on_disk  mode uses Faiss HNSW with 32× compression. No additional configuration is needed.
According to  AWS benchmarks , on_disk mode can run vector search at a third of the cost of in-memory mode while maintaining high recall (0.93–0.99) and P90 latency in the 100–200ms range. It's an excellent option when sub-millisecond response times aren't critical — which covers most semantic search, recommendation, and RAG use cases.
I chose on_disk mode for this benchmark because it offers an interesting cost-latency tradeoff worth evaluating against S3 Vectors. Both services target builders who want serverless vector search without managing infrastructure. On_disk mode brings AOSS closer to S3 Vectors in terms of cost profile, making the comparison more meaningful. If your use case requires single-digit millisecond latency, in-memory mode would be the better AOSS configuration, but at a higher cost point.
Mode
Memory Usage
Latency
Cost
Best For
 in_memory 
Full vectors in RAM
Single-digit ms
Higher (more OCUs)
Ultra-low latency requirements
 on_disk 
32× compressed in RAM
100–200ms P90
~1/3 of in-memory
Cost-sensitive workloads tolerant of higher latency
The fundamental difference is the cost model: S3 Vectors charges per query and per GB stored, while AOSS bundles compute into always-on OCUs.

Ingestion: Batch Size Matters, and AOSS Is Faster

I started by testing how batch size affects ingestion throughput, using 10,000 vectors with batch sizes of 100, 250, and 500.
Batch Size
S3 Vectors
AOSS
Gap
100
168 vec/s
201 vec/s
AOSS 1.2× faster
250
226 vec/s
414 vec/s
AOSS 1.8× faster
500
253 vec/s
477 vec/s
AOSS 1.9× faster
Larger batches benefit both systems, but AOSS scales better with batch size thanks to the OpenSearch bulk API.

Production-Scale Ingestion (Batch Size 500)

Dataset Size
S3 Vectors
AOSS
Gap
50K vectors
250 vec/s (200s)
497 vec/s (101s)
AOSS 2.0× faster
100K vectors
253 vec/s (396s)
488 vec/s (205s)
AOSS 1.9× faster
Both services scale linearly and maintain 100% success rates. S3 Vectors held steady at ~250 vec/s regardless of dataset size (avg batch latency: ~1,950ms). AOSS maintained ~490 vec/s at scale.
If you're doing heavy initial data loads or frequent bulk updates, AOSS gives you nearly double the throughput.

OpenSearch Filtering Strategies: Post-Filtering vs Efficient Filtering

Before looking at the latency results, it's important to understand how OpenSearch handles filtered vector search — because the filtering strategy directly impacts both performance and result completeness.

Post-Filtering: The Default Trap

When I first wrote my AOSS queries, I used what felt like the natural pattern — a  bool  query with a  knn  search and a  filter  clause:
{
  "query": {
    "bool": {
      "must": [{ "knn": { "vector": { "vector": [...], "k": 20 } } }],
      "filter": [{ "term": { "premium": true } }]
    }
  }
}
This is post-filtering. OpenSearch first finds the 20 nearest neighbors from ALL vectors, then applies the filter to those 20 results. With a filter that matches only 10% of vectors, you get roughly 20 × 10% = 2 results.
In my tests, the premium filter (10% selectivity) returned an average of 4.2 results instead of 20. That's 21% of what I asked for.

Efficient Filtering: The Fix

Moving the filter inside the  knn  clause changes the execution strategy entirely:
{
  "query": {
    "knn": {
      "vector": {
        "vector": [...],
        "k": 20,
        "filter": { "term": { "premium": true } }
      }
    }
  }
}
This is efficient filtering. OpenSearch traverses the HNSW graph with filter constraints, only evaluating vectors that match the filter. It always returns exactly k results from the filtered set.
The difference between the two approaches comes down to where the  filter  clause is placed in the query DSL.

S3 Vectors: Filtering Just Works

With S3 Vectors, filtering is inline by design — the filter is always applied during the search. According to the  S3 Vectors metadata filtering documentation : "S3 Vectors performs vector search and filter evaluation in tandem. S3 Vectors searches through candidate vectors in the index to find the top K similar vectors while simultaneously validating if each candidate vector matches your metadata filter conditions." This tandem approach is more likely to return full k results compared to post-filtering:
result = s3vectors_client.query_vectors(
    vectorBucketName="my-bucket",
    indexName="products",
    queryVector={"float32": query_vector},
    topK=20,
    filter={"premium": True}
)
It always returns k results from the filtered set. No special syntax to remember, no subtle execution strategy differences.

The Impact Across All Filters

Filter
Selectivity
S3 Vectors Results
AOSS Post-Filter Results
AOSS Efficient Filter Results
Premium
10%
20/20 ✅
4.2/20 ❌
20/20 ✅
Department
20%
20/20 ✅
8.2/20 ❌
20/20 ✅
Category
20%
20/20 ✅
8.2/20 ❌
20/20 ✅
Priority
25%
20/20 ✅
10.1/20 ❌
20/20 ✅
Region
25%
20/20 ✅
9.7/20 ❌
20/20 ✅
Active
67%
20/20 ✅
20/20 ✅
20/20 ✅
As  documented by OpenSearch , post-filtering "might receive fewer than k results" when filter selectivity is low. This is by design — post-filtering is useful when combining kNN scoring with boolean query logic. For use cases requiring exactly k filtered results, efficient filtering or S3 Vectors' tandem approach are more appropriate.

Query Latency: The Full Picture

Unfiltered Queries (75K Index)

Service
Avg Latency
Min
Max (Cold-Start)
S3 Vectors
200ms
78ms
447ms
AOSS
1,007ms
79ms
25,127ms (25s)
For AOSS, the average is skewed by cold-start outliers — see the  Cold-Start vs Warm Latency  section for details.

Filtered Queries — Latency Comparison (75K Index, k=20, 50 Queries Each)

Filter
Selectivity
S3 Vectors (avg)
AOSS Post-Filter (avg)
AOSS Efficient Filter (avg)
Premium
10%
62ms
46ms
141ms
Department
20%
61ms
54ms
236ms
Category
20%
128ms
70ms
243ms
Priority
25%
74ms
66ms
279ms
Region
25%
65ms
60ms
301ms
Active
67%
64ms
66ms
131ms

Filtered Queries — Result Completeness (75K Index, k=20)

Filter
Selectivity
S3 Vectors
AOSS Post-Filter
AOSS Efficient Filter
Premium
10%
20/20 ✅
4.2 ❌
20/20 ✅
Department
20%
20/20 ✅
8.2 ❌
20/20 ✅
Category
20%
20/20 ✅
8.2 ❌
20/20 ✅
Priority
25%
20/20 ✅
10.1 ❌
20/20 ✅
Region
25%
20/20 ✅
9.7 ❌
20/20 ✅
Active
67%
20/20 ✅
20.0 ✅
20/20 ✅
S3 Vectors always returns complete results across all filter selectivities. AOSS post-filtering shows the lowest latency for filtered queries (46–70ms) but returns fewer than k results at low selectivity. For use cases where latency is the priority and partial results are acceptable — such as "show me a few similar items" recommendations or exploratory search — post-filtering can be a valid choice. For use cases requiring exactly k results, efficient filtering or S3 Vectors are more appropriate.
When comparing S3 Vectors and AOSS efficient filtering — both of which return the full k results — S3 Vectors is consistently 2–4.6× faster across all filter selectivities.

K-Value Scaling: How Latency Changes with Result Count

I tested k-values from 1 to 30 on a 50K vector index (100 queries per k-value, no filter) to understand how the number of requested results affects latency. This test ran on a smaller index (50K vs 75K in the filter tests), without any metadata filtering. The queries ran sequentially — so k=1 absorbed the initial warm-up penalty while later k-values benefited from a warm index.
K Value
S3 Vectors (avg)
AOSS (avg)
Fastest
k=1
151ms
65ms
AOSS
k=5
91ms
49ms
AOSS
k=10
68ms
51ms
AOSS
k=15
64ms
54ms
AOSS
k=20
64ms
64ms
Tie
k=25
64ms
61ms
AOSS
k=30
63ms
66ms
S3 Vectors
Interesting patterns here:
  • S3 Vectors latency decreases from k=1 to k=10, then plateaus around 63ms. The k=1 average (151ms) is inflated by the warm-up penalty on the first queries.
  • AOSS latency increases mildly from k=5 (49ms) to k=30 (66ms), which is expected — more results means more graph traversal.
  • At k=20, they're essentially tied at 64ms on a warm 50K index.
Note that these unfiltered results show AOSS performing well because there's no efficient filtering overhead. When efficient filtering is applied (as in the filtered query tables on the 75K index), AOSS adds latency from both the filter-aware HNSW traversal and the on_disk two-phase rescore.

K=20 With Category Filter (50K Index)

Variant
S3 Vectors
AOSS Post-Filter
AOSS Efficient Filter
Results
k=20, no filter
64ms
64ms
64ms
All 20/20
k=20, category=tech
72ms
45ms
155ms
S3V: 20/20, Post: 8.2/20, Eff: 20/20
Without a filter, all three approaches perform identically at 64ms — filtering strategy only matters when a filter is applied. Adding a category filter makes S3 Vectors slightly slower (+13%) but AOSS post-filter faster (−30%). However, AOSS post-filter only returns 8.2 results. AOSS efficient filtering returns the full 20 results but at higher latency.

End-to-End Workflow: Simulating Production

This test simulates a complete production workflow: ingest data, query it, add more data, query again, then add filters. It measures how both services handle a realistic lifecycle.

Phase-by-Phase Results (25K → 50K Vectors)

Phase
S3 Vectors
AOSS
Phase 1: Initial ingestion (25K)


Throughput
249 vec/s (100s)
389 vec/s (64s)
Phase 2: Baseline queries (25K index)


Avg latency
129ms
86ms
Max latency (cold-start)
431ms
439ms
QPS
7.8
11.6
Phase 3: Incremental ingestion (+25K)


Throughput
249 vec/s (101s)
534 vec/s (47s)
Phase 4: Scaled queries (50K index)


Avg latency
167ms
62ms
Max latency
371ms
75ms
QPS
6.0
16.1
Phase 5: Filtered queries (priority=high)


Avg latency
100ms
63ms
Max latency
216ms
120ms
QPS
10.0
15.9
Results per query
20.0
10.5
A few things stand out:
  • AOSS incremental ingestion got faster (534 vec/s vs 389 vec/s initial). The collection was warmed up.
  • S3 Vectors degraded with scale: QPS dropped from 7.8 (25K) to 6.0 (50K) — a 23% decrease. This is genuine scaling impact.
  • AOSS appeared to improve from Phase 2 to Phase 4 (11.6 → 16.1 QPS), but this reflects warm-up, not that more data made queries faster.
  • Phase 5 uses post-filtering, which is why AOSS returns only 10.5 results instead of 20. With efficient filtering, AOSS would return 20 results but with higher latency.
  • S3 Vectors filtering improved QPS from 6.0 (unfiltered) to 10.0 (filtered) — a 1.67× improvement. Filters narrow the search space.

Scaling Impact Summary

Metric
S3 Vectors
AOSS
Scaling impact (50K QPS / 25K QPS)
0.77 (23% slower)
1.39 (warm-up effect)
Filter improvement (filtered QPS / unfiltered QPS)
1.67× faster
0.99× (no change)

Cold-Start vs Warm Latency

S3 Vectors: Infrequent Query Penalty

The  S3 Vectors documentation  states: "S3 Vectors delivers sub-second response times for cold queries, leveraging Amazon S3 elastic throughput to efficiently search across millions of vectors. For warm queries, S3 Vectors can deliver response times as low as 100ms, benefiting workloads with repeated or frequent query patterns."
In the  AWS re:Invent 2025 session STG318 — "Transforming AI storage economics with Amazon S3 Vectors" , the speakers describe how S3 Vectors uses an internal query-pattern-aware caching mechanism that detects repeated or similar queries and promotes the relevant index segments to a faster storage tier. The cache persists for several minutes after the last access — beyond that window, the next query becomes cold again. This is fully managed with no user configuration required. The benchmark data is consistent with this description.
The benchmark data is consistent with this behavior:
Scenario
First Queries (Max)
Warm Queries (Min)
Warm Avg
Unfiltered (75K index)
447ms
78ms
200ms
Unfiltered k=1 (first test, 50K)
346ms
58ms
151ms
Unfiltered k=30 (later test, 50K)
69ms
58ms
63ms
The k=1 and k=30 tests ran on the same 50K index with the same unfiltered query type — the only difference is that k=1 ran first. The latency drop from 151ms avg to 63ms avg shows a warm-up effect, not a k-value effect. Once the index is warm, unfiltered queries settle around 60–65ms regardless of k.
Filtered queries show a similar pattern but with slightly lower warm latency (56–62ms), because filters narrow the search space.

AOSS: Cold-Start Penalty

AOSS shows a much larger gap between cold and warm performance:
Service
Cold Query (Max)
Warm Query (Min)
S3 Vectors
447ms
78ms
AOSS Efficient Filter
25,127ms (25s)
79ms
AOSS Post-Filter
34,316ms (34s)
92ms
For applications with variable traffic patterns — user-facing APIs with idle periods, development environments, or batch processing with gaps — the difference between sub-second and 25–34 seconds on the first query is significant. For applications with constant traffic, AOSS warm performance (P50 102ms) is competitive with S3 Vectors for unfiltered queries.

Decision Framework

After running these benchmarks, here's how I think about the choice:

Choose S3 Vectors when:

  • Your queries use metadata filters
  • You need predictable cold-start performance (sub-second)
  • Cost matters and you can't justify $366/month minimum with redundancy ($183/month without) in eu-west-1
  • You want the simplest possible API (native boto3, key-value filters)
  • Your traffic is variable or intermittent
  • Your workload fits within S3 Vectors  limits : up to 2 billion vectors per index, up to 100 top-K results, up to 4,096 dimensions, up to 2 KB filterable metadata per vector, and hundreds of queries per second per index

Choose AOSS with efficient filtering when:

  • You exceed S3 Vectors limits — for example: need more than 100 top-K results, more than 4,096 dimensions, or higher QPS than S3 Vectors supports per index
  • You need OpenSearch features: aggregations, hybrid search (keyword + vector), complex DSL queries, or dimensions up to 16,000
  • You already have OpenSearch expertise on your team
  • Your traffic is constant enough to keep the service warm

Avoid AOSS post-filtering for:

  • Any filter with selectivity below 50%
  • Production APIs requiring exact k results
  • Anywhere silent partial results would cause problems

Implementation Tips

If you use AOSS, here are three practical tips from this benchmark:
1. Always verify your filter placement. The difference between post-filtering and efficient filtering is a single level of JSON nesting. Add a completeness check to your integration tests:
actual_results = len(response['hits']['hits'])
if actual_results \< k:
    raise ValueError(
        f"Incomplete results: {actual_results}/{k}. "
        f"Check filter placement in query DSL."
    )
2. Test with your actual filter selectivity. Generic unfiltered benchmarks miss the most common production failure mode. If your filters match 10–20% of data, that's exactly where post-filtering breaks down.
3. Plan for cold-start. If you choose AOSS, implement connection pooling and consider periodic keep-alive queries for user-facing endpoints. A 25-second first-query latency will not go unnoticed.

Reproducing These Results

The benchmark ran in eu-west-1 with identical vectors (up to 100K, 1024 dimensions, cosine similarity), 10 metadata fields, and the same query patterns across 7 filter types for both services.
I'd encourage you to set up a similar benchmark with your own data and query patterns. Vector search performance is workload-dependent, and your results may differ with different vector dimensions, dataset sizes, or filter distributions.

Cost: It Depends on Scale

The cost models are fundamentally different. The estimates below use eu-west-1 (Ireland) pricing, matching the benchmark region. For other regions, use the  AWS Pricing Calculator .
S3 Vectors unit pricing (eu-west-1,  source ): Storage $0.06/GB/month, PUT $0.20/GB, Query API $0.0025/1,000 requests ($2.50/million), Query data processing $0.004/TB (first 100K vectors) / $0.002/TB (above 100K).
AOSS unit pricing (eu-west-1,  source ): Indexing OCU $0.251/hour, Search OCU $0.251/hour. Production collections require a minimum of 2 OCU (0.5×2 indexing + 0.5×2 search); disabling redundancy reduces this to 1 OCU (0.5 indexing + 0.5 search). Additional collections sharing the same KMS key can share OCUs. Managed storage $0.024/GB/month. The OCU rate is the same for both in_memory and on_disk modes. The cost benefit of on_disk mode is that the 32× compression lets the same OCU count handle larger indexes — at the benchmark scale (75K–100K vectors), 2 OCU is sufficient for both modes.
Vector size in this benchmark: ~5.17 KB per vector for query processing (vector data + filterable metadata + key), ~6.17 KB for storage (including non-filterable metadata).

Small-Medium Scale (100K vectors, 1M queries/month)

Component
S3 Vectors
AOSS
Storage
$0.04 (0.59 GB × $0.06/GB)
$0.01 (0.49 GB × $0.024/GB)
Compute (OCU)
—
$366.46 (2 OCU × $0.251/hr × 730 hrs)
Query API
$2.50 (1M × $2.50/M)
Included
Data Processing
$1.93 (481.5 TB × $0.004/TB)
Included
Monthly Total
~$4.46
~$366.47
S3 Vectors is ~82× cheaper at this scale.

Medium Scale (1M vectors, 1M queries/month)

Component
S3 Vectors
AOSS
Storage
$0.35
$0.12
Compute (OCU)
—
$366.46
Query API
$2.50
Included
Data Processing
$10.59 (Tier 1: 481.5 TB × $0.004 + Tier 2: 4,333 TB × $0.002)
Included
Monthly Total
~$13.45
~$366.58
Still ~27× cheaper with S3 Vectors. Data processing becomes the dominant S3 Vectors cost at this scale.

Large Scale (10M vectors, 10M queries/month)

Component
S3 Vectors
AOSS
Storage
$3.53
$1.18
Compute (OCU)
—
$732.92 (4+ OCU)
Query API
$25.00
Included
Data Processing
$972.62 (Tier 1: 4,815 TB × $0.004 + Tier 2: 476,679 TB × $0.002)
Included
Monthly Total
~$1,001.15
~$734.10+
At this scale, AOSS becomes ~27% cheaper than S3 Vectors. The data processing cost ($972.62) dominates the S3 Vectors bill because it scales with index size × query volume. The 4 OCU estimate assumes on_disk mode — in_memory mode would likely require more OCUs for the same dataset, increasing the AOSS cost further.

The Crossover

Scale
S3 Vectors
AOSS (redundancy enabled, 2 OCU)
AOSS (redundancy disabled, 1 OCU)
100K vectors, 1M queries
~$4.46
~$366.47 (82× more)
~$183.24 (41× more)
1M vectors, 1M queries
~$13.45
~$366.58 (27× more)
~$183.35 (14× more)
10M vectors, 10M queries
~$1,001.15
~$734.10 (AOSS 27% cheaper)
~$367.64 (AOSS 63% cheaper)
S3 Vectors is dramatically cheaper at small-to-medium scale. At large scale (10M+ vectors with high query volume), AOSS becomes more cost-effective because S3 Vectors' data processing cost scales with index size × query count. AOSS cost is also justified at any scale when you need OpenSearch-specific features (hybrid search, aggregations, complex DSL) that S3 Vectors doesn't provide.

Overall Comparison

Use Case
Recommended
Why
Unfiltered vector search (warm)
AOSS
102ms P50 warm, lower than S3 Vectors' 200ms avg
Unfiltered vector search (variable traffic)
S3 Vectors
Sub-second cold queries, no 25s cold-start penalty
Filtered queries (latency + completeness)
S3 Vectors
61–128ms avg, always returns k results
Filtered queries (lowest latency, partial results OK)
AOSS Post-Filter
46–70ms avg, but returns fewer than k at low selectivity
High-throughput ingestion
AOSS
~490 vec/s at scale, ~1.9× faster than S3 Vectors
Predictable cold/warm performance
S3 Vectors
Sub-second cold queries, ~60ms warm
Serverless / no cluster management
Both
No infrastructure to manage
Complex query patterns (hybrid search, aggregations)
AOSS
Full OpenSearch DSL, hybrid search, aggregations
Cost at small-medium scale (≤1M vectors)
S3 Vectors
$0 idle cost, 27–82× cheaper than AOSS (eu-west-1)
Cost at large scale (10M+ vectors, high QPS)
AOSS
Data processing cost makes AOSS more cost-effective at scale

Head-to-Head at a Glance

Metric
S3 Vectors
AOSS (on_disk)
Unfiltered query latency (75K)
200ms avg
1,007–1,188ms avg (102–130ms P50 warm)
Filtered query latency (75K, complete results)
61–128ms
131–301ms (efficient filter)
Filtered query latency (75K, partial results)
—
46–70ms (post-filter)
k=20 latency (50K, warm, unfiltered)
64ms
64ms
Ingestion rate (batch 500, 100K)
253 vec/s
488 vec/s
Cold query worst case
447ms
25,127–34,316ms
Filter result completeness
Always k
Always k (efficient filter) / Partial (post-filter)
Minimum monthly cost
$0 (idle)
~$366 (2 OCU, redundancy enabled) / ~$183 (1 OCU, redundancy disabled), eu-west-1
API simplicity
Simple (boto3, key-value filters)
Complex (OpenSearch DSL, SigV4 auth)

Benchmark Limitations

  • Single-threaded queries. All tests issued queries sequentially from one thread. Concurrent query behavior may differ, especially for AOSS which has OCU-based scaling.
  • Random query vectors. Queries are random and not related to the indexed data. Real workloads with clustered queries may produce different results.
  • Small scale. The largest index tested was 100K vectors. Behavior at 1M+ vectors may differ substantially, especially for on_disk mode where I/O patterns change.
  • AOSS on_disk mode only. In-memory mode would show lower AOSS latencies at a higher cost point.

Key Takeaways

  1. Filtering strategy fundamentally changes results. AOSS post-filtering returns 4.2/20 results at 10% selectivity. Efficient filtering returns 20/20. Same index, same data — different query syntax.
  1. S3 Vectors excels at filtered queries. 2–4.6× faster than AOSS efficient filtering with identical result completeness, and higher QPS (up to 16 QPS vs 7.7 QPS).
  1. K-value scaling behaves differently. S3 Vectors plateaus at k=10; AOSS increases mildly. At k=20 on a warm index, they're tied at ~64ms.
  1. AOSS wins on ingestion and warm unfiltered queries. 2× faster ingestion, and 102ms P50 for unfiltered queries when warm. If that's your primary workload, AOSS is worth considering.
  1. End-to-end workflow reveals scaling patterns. S3 Vectors QPS degrades 23% from 25K to 50K vectors. AOSS maintains throughput but post-filtering returns incomplete results.
  1. Cost scales differently. With eu-west-1 pricing, S3 Vectors is 82× cheaper at 100K vectors and 27× cheaper at 1M vectors. At 10M vectors with 10M queries/month, AOSS becomes ~27% cheaper because S3 Vectors' data processing cost scales with index size × query count.
  1. Cold-start varies dramatically. <500ms for S3 Vectors vs 25–34s for AOSS. Worth factoring in if your traffic pattern includes idle periods.
  1. Monitor result completeness. Silent partial results are worse than obvious failures. Add assertions to your vector search pipeline.

Conclusion

This benchmark set out to answer a simple question: for workloads that need vector similarity search with metadata filtering, how do S3 Vectors and AOSS in  on_disk  mode compare? The data tells a clear story. S3 Vectors delivers faster filtered queries (2–4.6×), sub-second cold starts, complete results by default, and dramatically lower cost at small-to-medium scale. AOSS brings faster ingestion (2×), competitive warm unfiltered latency, the full OpenSearch feature set, and becomes more cost-effective at large scale with high query volumes. Neither service is universally better; the right choice depends on your query patterns, scale, and feature requirements. The benchmark data and methodology are available for you to validate against your own workloads.

I ran all benchmarks in eu-west-1 with up to 100,000 vectors (1024 dimensions, cosine similarity) across 350+ queries and 7 filter types. Test period: March 2026.
# amazon-s3-vectors# opensearch# vector-search# generative-ai# retreival-augmented-generation
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article