AWS Builder Center
Amazon OpenSearch Serverless Now Supports 10,000 Collections per Collection Group

Amazon OpenSearch Serverless Now Supports 10,000 Collections per Collection Group

Learn how the new 10,000-collection limit in Amazon OpenSearch Serverless helps multi-tenant applications scale isolated search workloads, share compute across different KMS keys, and reduce operational complexity and per-collection cost.

Amazon OpenSearch Serverless Now Supports 10,000 Collections per Collection Group

Multi-tenant SaaS platforms often need to balance two competing goals:
  • Isolate each tenant’s data and access
  • Share infrastructure efficiently to control cost
Creating a separate Amazon OpenSearch Serverless collection for every tenant can provide clear security and data boundaries. However, operating thousands of isolated collections can create fragmented capacity, higher per-tenant costs, and additional management overhead.
Amazon OpenSearch Serverless collection groups were introduced to address this challenge by allowing multiple collections even collections encrypted with different AWS KMS keys to share OpenSearch Compute Units.
AWS has now expanded the capability significantly.
Next-generation Amazon OpenSearch Serverless collection groups can support up to 10,000 collections, increased from the previous limit of 1,500.
The higher limit applies automatically to new and existing next-generation collection groups in all supported AWS Regions.

Understanding the basic components

Before examining the release, it helps to understand three OpenSearch Serverless concepts.

Collection

A collection is a logical group of OpenSearch indexes supporting a specific workload.
OpenSearch Serverless provides collection types for:
  • Search
  • Time-series analytics
  • Vector search
In a multi-tenant architecture, a SaaS provider might create one collection for each customer.
This can provide:
  • Clear data separation
  • Tenant-specific access policies
  • Independent encryption keys
  • Easier deletion and lifecycle management
  • Reduced risk of querying another tenant’s indexes accidentally

OpenSearch Compute Unit

An OpenSearch Compute Unit, or OCU, represents the compute resources used for indexing and searching data.
OpenSearch Serverless automatically manages and scales these resources according to workload demand and configured capacity limits.

Collection group

A collection group organizes multiple collections and allows them to share OCUs.
Collections in the same group can retain:
  • Separate data
  • Separate indexes
  • Collection-level access controls
  • Different AWS KMS encryption keys
At the same time, they can use a common compute pool for indexing and search.
This separation between logical security boundaries and shared compute is the central value of collection groups.

What has changed?

The maximum supported collections in one next-generation collection group has increased:
Capability
Previous limit
New limit
Collections per collection group
1,500
10,000
This is more than a simple quota increase.
Previously, a SaaS platform with 10,000 tenant collections could require at least seven collection groups to remain within the 1,500-collection limit.
With the new limit, those collections can potentially be consolidated into one group, subject to workload, capacity, performance, and isolation requirements.
This can reduce:
  • Collection-group sprawl
  • Repeated capacity configurations
  • Operational overhead
  • Fragmented compute pools
  • Underutilized OCUs
  • Per-collection infrastructure cost
The increase is particularly relevant for multi-tenant applications that provision a dedicated collection for every tenant.

Why separate collections can become expensive

Suppose a SaaS application gives every tenant a collection encrypted with its own customer-managed KMS key.
Without collection groups, different encryption keys can require separate compute resources. A large number of small or irregular tenant workloads may therefore result in inefficient capacity utilization.
For example:
  • Tenant A is active during business hours.
  • Tenant B runs searches mainly at night.
  • Tenant C has infrequent activity.
  • Tenant D generates a temporary indexing spike.
  • Thousands of other tenants remain mostly idle.
If every security boundary requires independent compute, much of that capacity may remain unused.
A collection group allows these collections to share OCUs. The serverless platform can allocate resources according to combined demand instead of maintaining a separate compute pool for each encryption boundary.
This does not combine the tenants’ data. Each collection remains logically separate and continues using its assigned encryption and access policies.

A multi-tenant SaaS example

Consider a document-management SaaS platform serving 8,000 organizations.
Each customer requires:
  • A dedicated OpenSearch collection
  • A customer-specific KMS key
  • Collection-level access controls
  • Independent indexes
  • Separate retention and deletion processes
The architecture could look like this:
Tenant A ── Collection A ── KMS Key A ┐
Tenant B ── Collection B ── KMS Key B │
Tenant C ── Collection C ── KMS Key C ├── Shared Collection Group
... │ ├── Shared Indexing OCUs
Tenant 8,000 ─ Collection 8,000 ─ Key ┘ └── Shared Search OCUs

The tenants retain separate collections and encryption keys, but their workloads share the collection group’s compute capacity.
Before the new limit, this design would have required multiple collection groups. It can now potentially fit within one next-generation group.
Whether it should use one group is a separate architectural decision.

Security isolation remains important

Sharing compute does not mean removing collection-level security.
Collections within a group can continue to use:
  • Different AWS KMS keys
  • Individual data-access policies
  • Collection-specific indexes
  • Separate tenant authorization
  • Independent lifecycle management
OpenSearch Serverless accesses the necessary KMS keys when serving each collection while maintaining encryption at rest.
When customer-managed KMS keys are supplied during collection creation, the relevant identity requires permissions including:
  •  kms:DescribeKey 
  •  kms:CreateGrant 
Using a shared compute pool should never be treated as a substitute for properly designed tenant authorization.
The application must still ensure that:
  • Each user is mapped to the correct tenant.
  • Access policies restrict users to approved collections.
  • KMS permissions follow least privilege.
  • Administrative operations cannot cross tenant boundaries unintentionally.
  • Tenant context is validated rather than trusted from user input alone.

Capacity management

Collection-group capacity is managed using minimum and maximum OCU limits for indexing and search.

Minimum capacity

Minimum capacity establishes baseline compute for the group.
A non-zero minimum can improve predictability and avoid cold-start latency for sensitive workloads. However, the minimum is maintained even when demand is below that level, so billing reflects the reserved baseline.

Maximum capacity

Maximum capacity limits how far the group can scale.
This provides an important cost-control mechanism, but the limit must be selected carefully.
If utilization reaches the maximum:
  • Requests may slow down.
  • Indexing can fall behind.
  • Query latency may increase.
  • Tenant workloads may affect one another.
  • The platform may experience throttling or degraded performance.
CloudWatch metrics such as  OCUUtilization  should be monitored to determine whether the maximum remains appropriate.

Scale to zero

Next-generation collection groups can scale indexing and search capacity to zero when minimum capacity is configured as zero.
After approximately ten minutes without incoming requests across the group, compute resources can scale down to zero.
Indexing and search scale independently.
This is useful for:
  • Development environments
  • Infrequently used tenant groups
  • Batch workloads
  • Trial accounts
  • Applications with predictable inactive periods
The trade-off is that the first request after an idle period may experience roughly 10–30 seconds of additional latency while capacity is restored.
Scale to zero may therefore be inappropriate for premium or latency-sensitive tenants.

Should all 10,000 collections use one group?

The new limit makes consolidation possible, but it does not mean every application should place 10,000 collections in one group.
A single group creates a shared capacity boundary.
A noisy tenant or unexpected indexing spike could consume a large portion of the available OCUs and affect other collections in the same group.
A stronger design may divide tenants according to workload characteristics.
For example:
Collection group
Tenant type
Capacity approach
Premium
Latency-sensitive customers
Non-zero minimum capacity
Standard
Normal production customers
Shared elastic capacity
Trial
Free or evaluation accounts
Scale to zero
Analytics
High-ingestion workloads
Higher indexing limit
Vector
AI and semantic-search workloads
Memory-focused capacity planning
This structure preserves the benefits of shared compute while reducing the blast radius of unusually demanding tenants.

Isolation models

A multi-tenant OpenSearch architecture generally has three broad choices.

Shared collection and shared indexes

Multiple tenants use the same collection and potentially the same indexes, with tenant identifiers applied to documents and queries.
Advantages:
  • High resource efficiency
  • Fewer resources
  • Simpler infrastructure provisioning
Challenges:
  • Strong application-level isolation is essential
  • Accidental cross-tenant queries become a greater risk
  • Per-tenant deletion and encryption may be harder

Shared collection with separate indexes

Tenants use different indexes inside one collection.
Advantages:
  • Better logical separation
  • Lower resource count than collection-per-tenant
  • Shared collection capacity
Challenges:
  • Collection-level encryption remains shared
  • Index quotas must be considered
  • Access-policy design can become complex

Separate collection per tenant

Every tenant receives a dedicated collection.
Advantages:
  • Stronger logical boundary
  • Separate KMS keys
  • Easier tenant lifecycle management
  • Clear collection-level access controls
Challenges:
  • More resources to provision
  • More policies and configurations
  • Potential compute fragmentation
The increased collection-group limit makes the third option more financially and operationally practical at a much larger scale.

Important limitations and considerations

Only next-generation collection groups

The 10,000-collection capacity applies to next-generation Amazon OpenSearch Serverless collection groups.
Confirm the generation of existing groups before depending on the new limit.

Same collection type

Collections within a group must use the same collection type.
Search, time-series, and vector-search collections cannot be mixed within one collection group.
Separate collection groups are required for different workload types.

Existing collections

Collection-group architecture should be planned during collection creation. Existing independent collections may not simply be moved into a group.
Review current AWS documentation before planning consolidation or migration.

Capacity can become the actual constraint

Supporting 10,000 collection objects does not guarantee that one group can handle the combined workload of all 10,000 collections.
Actual scalability depends on:
  • Data volume
  • Index count
  • Query rate
  • Indexing throughput
  • Vector memory requirements
  • Query complexity
  • Maximum configured OCUs
  • Tenant concurrency
The resource quota and workload capacity are different concerns.

Shared performance boundary

Different KMS keys and access policies maintain security boundaries, but collections still share compute.
Tenants with strict performance-isolation requirements may need separate collection groups.

Cost considerations

The release can lower per-collection cost by improving OCU utilization, but it does not guarantee savings automatically.
Costs still depend on:
  • Search OCU consumption
  • Indexing OCU consumption
  • Managed storage
  • Data transfer
  • KMS key usage
  • CloudWatch monitoring
  • Configured minimum capacity
  • Workload peaks
To control cost:
  1. Group tenants with compatible workload patterns.
  1. Avoid unnecessary minimum capacity.
  1. Set realistic maximum OCU limits.
  1. Monitor  OCUUtilization .
  1. Identify noisy tenants.
  1. Use scale to zero for eligible workloads.
  1. Separate premium workloads requiring predictable performance.
  1. Review collection and index growth regularly.
The architectural goal is to maximize safe compute sharing without creating unacceptable performance coupling.

Operational recommendations

Automate tenant provisioning

For collection-per-tenant architectures, provisioning should be automated using:
  • AWS CloudFormation
  • AWS CDK
  • AWS CLI
  • AWS SDKs
  • Internal platform APIs
Automation should create and verify:
  • The tenant collection
  • KMS configuration
  • Data-access policy
  • Network policy
  • Collection-group assignment
  • Tags
  • Monitoring
  • Capacity-related controls

Maintain a tenant-to-collection registry

The application should maintain an authoritative mapping between:
  • Tenant ID
  • Collection name
  • Collection endpoint
  • KMS key
  • Collection group
  • Region
  • Data-access policy
  • Service tier
This prevents tenant routing from depending on user-controlled values.

Monitor at tenant and group levels

Group-level monitoring alone may hide an individual tenant’s behaviour.
Track:
  • Query volume
  • Indexing throughput
  • Storage growth
  • Error rates
  • Query latency
  • OCU utilization
  • Throttling
  • Tenant-specific consumption

Plan tenant movement

Large or unpredictable tenants may eventually need a separate collection group.
The platform should have a documented process for identifying and isolating these tenants before they affect shared performance.

Solution architect’s perspective

This announcement improves the economics of collection-per-tenant designs, but architecture should still begin with business requirements.
I would ask:
  1. Does every tenant require a dedicated collection?
  1. Does each tenant require its own KMS key?
  1. What performance isolation is promised?
  1. Which tenants can safely share compute?
  1. Can idle workloads scale to zero?
  1. What happens when one tenant creates an indexing spike?
  1. How will maximum OCU limits be selected?
  1. How will cross-tenant access be prevented?
  1. How will tenant lifecycle operations be automated?
  1. When should a tenant be moved to another group?
The new quota offers more design flexibility. It does not eliminate the need to model tenant workload and isolation requirements.

Final thoughts

Increasing the collection-group limit from 1,500 to 10,000 makes Amazon OpenSearch Serverless significantly more practical for large multi-tenant platforms.
SaaS providers can maintain separate collections, encryption keys, and access controls for thousands of customers while allowing those collections to share compute.
The architectural principle is simple:
Keep the tenant boundaries. Share the capacity where it makes sense.
Used carefully, this model can provide:
  • Stronger tenant isolation
  • Better OCU utilization
  • Lower per-collection cost
  • Fewer collection groups
  • Simpler large-scale operations
The best architecture will not necessarily place every tenant into one group. Instead, it will group tenants according to their security, performance, availability, and cost requirements.

References

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article