
How do I apply the correct permissions in Amazon OpenSearch Service?
Is your search or RAG application showing users documents they shouldn't have access to? You can copy each ACL from the source system into the OpenSearch index, filter every query in your application, and sync changes without re-embedding.
Does your search index hold documents from systems with different permissions? Then every query needs a rule that consumer search doesn't: users shouldn't see a result, snippet or generated answer from a document they can't open in the source system. This article shows you how to copy each source document's permissions into the index and filter every query by the user's groups on Amazon OpenSearch Serverless . The last section covers Amazon OpenSearch Service domains. It's part 3 of a six-part series. Part 2 added
allowed_groups to the common schema, and this part puts it to work.Three rules for permission-aware search
- Copy permissions onto every document at ingest. A document whose permissions you can't read gets no permissions applied, so nobody sees it.
- Filter every query in one place, with groups from the user's verified sign-in. Your application is the only thing enforcing permissions, so one missed code path exposes every document.
- Sync permission changes separately. Send them as partial updates, remove access before granting it, and expect a few minutes of delay.
Terms used in this article
- Access control list (ACL). The users and groups allowed to open a document in its source system, like the guest list at a door.
- Data access policy. The OpenSearch Serverless setting that says which AWS Identity and Access Management (IAM) roles can read or write each collection and index. It can't hide one document from one user.
- App role. The IAM role your search application signs its requests with. Every user's query arrives as this role.
- Permission filter. The filter on
allowed_groupsandallowed_usersthat your application adds to every query.
When you copy permissions into the index, queries stay fast, but permissions are only as current as your last sync. If yours change constantly, you can check each result with the source system at query time instead, as Authorizing access to data with RAG implementations describes.
How permissions flow through the pipeline
In Figure 1, permissions arrive with documents and user groups arrive with queries.

Figure 1: Document permissions arrive at ingest, and the search application adds each user's groups to every query.
When a user signs in, your identity provider gives the search application their groups. The application adds them as a filter and signs the query as the app role, so OpenSearch Serverless never sees the user. Permission changes arrive separately, as partial updates.
What you need
- An OpenSearch Serverless collection. We tested on a current-generation vector search collection in us-east-1.
- Two IAM roles: an app role for searching and an ingestion role for writing.
- An identity provider, such as Okta, Microsoft Entra ID or Amazon Cognito, that gives your application each user's groups.
- An index built with the common schema from Part 2. The examples use one named
enterprise-docs.
- If you sign requests yourself instead of with an AWS SDK, add the
x-amz-content-sha256header from the Serverless signing requirements . Without it, our writes got a403 Forbiddenwith no detail.
Step 1: Add permission fields to the mapping
Create the index with an explicit mapping that adds two permission fields. Each holds a list of group or user names.
PUT enterprise-docs{ "settings": { "index.knn": true }, "mappings": { "properties": { "title": { "type": "text" }, "body": { "type": "text" }, "allowed_groups": { "type": "keyword" }, "allowed_users": { "type": "keyword" }, "body_vector": { "type": "knn_vector", "dimension": 1024, "method": { "name": "hnsw", "space_type": "l2" } } } }}- Map both fields as
keyword. Astext,eng-teamis split intoengandteam, and a filter oneng-teamstops matching.
- Leave out
engine. Current-generation collections choose the engine themselves and reject the setting withField parameter 'engine' is not supported.
- Use one name per group. If one source says
eng-teamand another saysEngineering Team, change both to the name your identity provider returns. Otherwise, users with access get empty results.
- Hide documents with unknown permissions. If the pipeline can't read a document's ACL, write an empty
allowed_groups. In our test, those documents didn't match any filter. If you index chunks as separate documents, copy the fields onto each chunk.
Step 2: Lock down the collection
[ { "Rules": [ { "ResourceType": "index", "Resource": ["index/<collection-name>/enterprise-docs"], "Permission": ["aoss:ReadDocument", "aoss:DescribeIndex"] }, { "ResourceType": "model", "Resource": ["model/<collection-name>/*"], "Permission": ["aoss:ExecuteMLResource"] } ], "Principal": ["arn:aws:iam::<account-id>:role/app-role"] }]Give the ingestion role its own policy with write permissions, such as
aoss:WriteDocument .- Name only the app role and the ingestion role. In our test, every other role got
403"Bad Authorization", even withaoss:APIAccessAllin its IAM policy.
- Keep the app role read-only. Anyone who can write
allowed_groupscan grant themselves access.
- Add the
modelrule only forneuralqueries. Without it, the app gets403"Forbidden". If your app sends its own vectors, leave the rule out. The machine learning setup page lists the other model permissions.
- Expect policy changes to take a minute. In our test, requests switched between 403 and 200 for 15–60 seconds.
- Keep the app role's credentials out of users' hands. Anyone who can sign as the app role can skip the filter.
Step 3: Filter every query in one place
Take the user's groups from their verified sign-in, not from a request parameter, which a user could edit. Then have one function in your code add this filter to every query:
"filter": { "bool": { "should": [ { "terms": { "allowed_groups": ["eng-team", "enterprise-search-users"] } }, { "term": { "allowed_users": "user-a" } } ], "minimum_should_match": 1 }}For keyword queries, add it to a
bool query. For vector queries, put it inside knn or neural , as efficient k-NN filtering describes:GET enterprise-docs/_search{ "size": 10, "query": { "knn": { "body_vector": { "vector": [ ... ], "k": 10, "filter": { "bool": { "should": [ ... ], "minimum_should_match": 1 } } } } }}GET enterprise-docs/_search?search_pipeline=hybrid-pipeline{ "size": 10, "query": { "hybrid": { "queries": [ { "match": { "body": "torque spec" } }, { "knn": { "body_vector": { "vector": [ ... ], "k": 10 } } } ], "filter": { "bool": { "should": [ ... ], "minimum_should_match": 1 } } } }}- Filter inside the query, not after it.
post_filterremoves results after the nearest neighbors are picked. In our test atk: 1, filtering insideknnreturned 8 permitted documents andpost_filterreturned 4. Hybrid queries work the same way:hybrid.filterreturned 11 andpost_filterreturned 6.
- Use
sizeto set how many results you get.kcounts per shard, so in our testk: 1still returned 8.
- Send every hybrid query with
search_pipeline. Without a normalization processor , the query returns 200 but doesn't combine the scores.
Step 4: Keep permissions in sync
When a permission changes at the source, update only the permission fields. OpenSearch Serverless doesn't support
_update_by_query , so find the affected documents first: open a point in time (PIT) and page through the matches with search_after .POST enterprise-docs/_search/point_in_time?keep_alive=5m
GET _search{ "size": 100, "query": { "term": { "allowed_groups": "contractors-2025" } }, "pit": { "id": "<pit-id>", "keep_alive": "5m" }, "sort": [ { "_shard_doc": "asc" } ], "_source": false}Then send each document's new ACL with
_bulk update actions, and close the PIT with DELETE _search/point_in_time when you finish:POST _bulk{ "update": { "_index": "enterprise-docs", "_id": "wiki:2" } }{ "doc": { "allowed_groups": ["eng-team"] } }- Send the full new list.
docreplaces the wholeallowed_groupsarray. To remove one group without knowing the rest, run a scripted update withremoveIfon each document.
- Partial updates don't re-embed. In our test,
_updateand_bulkupdatemade no embedding calls, even with atext_embeddingdefault pipeline, and kept the stored vector. Use them only for fields that aren't embedded, such as permissions. When we changedbodywith_update, vector search kept matching the old text. To change embedded text, re-send the whole document through the pipeline.
- Don't use
skip_existingon Serverless. In our test, every write through a pipeline with it set failed with404 index_not_found_exception. To re-send a whole document with unchanged text, include its stored vector and add?pipeline=_none. You need both: without_none, the model runs anyway, and without the vector, the vector is deleted.
- Plan for a few minutes of delay. In our test, a removed group stopped matching after 125–248 seconds, and an added group appeared after about 300 seconds.
- Compare source and index on a schedule. Event-driven updates miss changes.
Check that filtering works
Run the same query as two test users from different groups. A document restricted to one group should appear only for that user. Then run it once without the filter: in our test, that returned all 30 documents, including the five with empty ACLs. Repeat with your vector and hybrid queries.
Give some documents a hard boundary
For documents that parts of your app must not return, such as export-controlled files, use a separate index with its own policy and app role. OpenSearch Serverless then blocks access itself, with or without the filter. In our test, a wildcard search such as
docs-* failed entirely with 403 when it included an index the role couldn't read. Search only the indexes each role can read, or use _msearch , which returns results per index.On an OpenSearch Service domain
The same pattern works on a domain with fine-grained access control . Create a read-only role, map it to your application's identity, and send the same filtered queries:
PUT _plugins/_security/api/roles/enterprise_search_app{ "cluster_permissions": ["cluster_composite_ops_ro"], "index_permissions": [ { "index_patterns": ["enterprise-docs*"], "allowed_actions": ["read"] } ]}In our test on OpenSearch 3.7, each user got the right documents, including from hybrid queries. Domains also support
_update_by_query ; add pipeline=_none so it doesn't re-run embedding. If other clients query the index directly, or compliance requires the search engine itself to enforce permissions, give those clients document-level security (DLS). Keep the search application on the role without DLS: in our tests on OpenSearch 2.19 and 3.7, hybrid queries failed for DLS users.Quick reference: what happens if…
If you… | What happens | Fix |
|---|---|---|
Map allowed_groups as text | eng-team is split into eng and team , and the filter stops matching | Map it as keyword |
Set engine on a knn_vector field | 400 Field parameter 'engine' is not supported | Remove engine |
Miss the filter in one code path | That path returns every document | Build every query in one function |
Take groups from a request parameter | A user can add any group | Take groups from the verified sign-in |
Name extra roles in the data access policy | They can read every document | Name only the app and ingestion roles |
Run neural queries without a model rule | 403 internal_server_exception "Forbidden" | Add aoss:ExecuteMLResource on model/<collection-name>/* |
Use post_filter on vector or hybrid results | Fewer results (4 instead of 8 in our test) | Filter inside knn or in hybrid.filter |
Send a hybrid query without search_pipeline | 200 , but the scores aren't combined | Add search_pipeline |
Call _update_by_query on Serverless | 404 with an empty body | Use a PIT search and _bulk updates |
Set skip_existing on text_embedding | Writes fail with 404 index_not_found_exception | Remove it, and send permission changes as partial updates |
Change embedded text with a partial update | Vector search still matches the old text | Re-send the whole document through the pipeline |
Sign requests without x-amz-content-sha256 | 403 Forbidden with no detail | Add the header |
Next steps
- List each source system's permission model and group names.
- Decide how long a permission change may take to show up, including the few minutes OpenSearch Serverless needs.
- Read Part 4, which moves from who can see a document to how its vectors are stored at scale.
Permissions that travel with the document
Store each document's permissions with its text, and let your application match them against the user's groups on every query. Lock the collection to two roles, build every query in one place, and sync changes as partial updates.
Learn more
- Hybrid query in the OpenSearch docs. OpenSearch Service is built on the OpenSearch project, and some features may work differently or be unavailable in the managed service.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article