AWS Builder Center
How do I apply the correct permissions in Amazon OpenSearch Service?

How do I apply the correct permissions in Amazon OpenSearch Service?

Is your search or RAG application showing users documents they shouldn't have access to? You can copy each ACL from the source system into the OpenSearch index, filter every query in your application, and sync changes without re-embedding.

Does your search index hold documents from systems with different permissions? Then every query needs a rule that consumer search doesn't: users shouldn't see a result, snippet or generated answer from a document they can't open in the source system. This article shows you how to copy each source document's permissions into the index and filter every query by the user's groups on  Amazon OpenSearch Serverless . The last section covers  Amazon OpenSearch Service  domains. It's part 3 of a six-part series.  Part 2  added  allowed_groups  to the common schema, and this part puts it to work.

Three rules for permission-aware search

  1. Copy permissions onto every document at ingest. A document whose permissions you can't read gets no permissions applied, so nobody sees it.
  1. Filter every query in one place, with groups from the user's verified sign-in. Your application is the only thing enforcing permissions, so one missed code path exposes every document.
  1. Sync permission changes separately. Send them as partial updates, remove access before granting it, and expect a few minutes of delay.

Terms used in this article

  • Access control list (ACL). The users and groups allowed to open a document in its source system, like the guest list at a door.
  • Data access policy. The OpenSearch Serverless setting that says which AWS Identity and Access Management (IAM) roles can read or write each collection and index. It can't hide one document from one user.
  • App role. The IAM role your search application signs its requests with. Every user's query arrives as this role.
  • Permission filter. The filter on  allowed_groups  and  allowed_users  that your application adds to every query.
When you copy permissions into the index, queries stay fast, but permissions are only as current as your last sync. If yours change constantly, you can check each result with the source system at query time instead, as  Authorizing access to data with RAG implementations  describes.

How permissions flow through the pipeline

In Figure 1, permissions arrive with documents and user groups arrive with queries.
Permissions flow: source ACLs indexed as allowed_groups; the search app adds each user's groups as a filter and signs as the app role

Figure 1: Document permissions arrive at ingest, and the search application adds each user's groups to every query.
When a user signs in, your identity provider gives the search application their groups. The application adds them as a filter and signs the query as the app role, so OpenSearch Serverless never sees the user. Permission changes arrive separately, as partial updates.

What you need

  • An OpenSearch Serverless collection. We tested on a current-generation vector search collection in us-east-1.
  • Two IAM roles: an app role for searching and an ingestion role for writing.
  • An identity provider, such as Okta, Microsoft Entra ID or Amazon Cognito, that gives your application each user's groups.
  • An index built with the common schema from Part 2. The examples use one named  enterprise-docs .
  • If you sign requests yourself instead of with an AWS SDK, add the  x-amz-content-sha256  header from the  Serverless signing requirements . Without it, our writes got a  403 Forbidden  with no detail.

Step 1: Add permission fields to the mapping

Create the index with an  explicit mapping  that adds two permission fields. Each holds a list of group or user names.
PUT enterprise-docs
{
"settings": { "index.knn": true },
"mappings": {
"properties": {
"title": { "type": "text" },
"body": { "type": "text" },
"allowed_groups": { "type": "keyword" },
"allowed_users": { "type": "keyword" },
"body_vector": { "type": "knn_vector", "dimension": 1024, "method": { "name": "hnsw", "space_type": "l2" } }
}
}
}
  • Map both fields as  keyword . As  text ,  eng-team  is split into  eng  and  team , and a filter on  eng-team  stops matching.
  • Leave out  engine . Current-generation collections choose the engine themselves and reject the setting with  Field parameter 'engine' is not supported .
  • Use one name per group. If one source says  eng-team  and another says  Engineering Team , change both to the name your identity provider returns. Otherwise, users with access get empty results.
  • Hide documents with unknown permissions. If the pipeline can't read a document's ACL, write an empty  allowed_groups . In our test, those documents didn't match any filter. If you index chunks as separate documents, copy the fields onto each chunk.

Step 2: Lock down the collection

Only your application should read the index. Create a  data access policy  for the app role:
[
{
"Rules": [
{ "ResourceType": "index", "Resource": ["index/<collection-name>/enterprise-docs"], "Permission": ["aoss:ReadDocument", "aoss:DescribeIndex"] },
{ "ResourceType": "model", "Resource": ["model/<collection-name>/*"], "Permission": ["aoss:ExecuteMLResource"] }
],
"Principal": ["arn:aws:iam::<account-id>:role/app-role"]
}
]
Give the ingestion role its own policy with write permissions, such as  aoss:WriteDocument .
  • Name only the app role and the ingestion role. In our test, every other role got  403   "Bad Authorization" , even with  aoss:APIAccessAll  in its IAM policy.
  • Keep the app role read-only. Anyone who can write  allowed_groups  can grant themselves access.
  • Add the  model  rule only for  neural  queries. Without it, the app gets  403   "Forbidden" . If your app sends its own vectors, leave the rule out. The  machine learning setup page  lists the other model permissions.
  • Expect policy changes to take a minute. In our test, requests switched between 403 and 200 for 15–60 seconds.
  • Keep the app role's credentials out of users' hands. Anyone who can sign as the app role can skip the filter.

Step 3: Filter every query in one place

Take the user's groups from their verified sign-in, not from a request parameter, which a user could edit. Then have one function in your code add this filter to every query:
"filter": {
"bool": {
"should": [
{ "terms": { "allowed_groups": ["eng-team", "enterprise-search-users"] } },
{ "term": { "allowed_users": "user-a" } }
],
"minimum_should_match": 1
}
}
For keyword queries, add it to a  bool  query. For vector queries, put it inside  knn  or  neural , as  efficient k-NN filtering  describes:
GET enterprise-docs/_search
{
"size": 10,
"query": {
"knn": {
"body_vector": {
"vector": [ ... ],
"k": 10,
"filter": { "bool": { "should": [ ... ], "minimum_should_match": 1 } }
}
}
}
}
For  hybrid queries , use the top-level  filter , which applies to every subquery:
GET enterprise-docs/_search?search_pipeline=hybrid-pipeline
{
"size": 10,
"query": {
"hybrid": {
"queries": [
{ "match": { "body": "torque spec" } },
{ "knn": { "body_vector": { "vector": [ ... ], "k": 10 } } }
],
"filter": { "bool": { "should": [ ... ], "minimum_should_match": 1 } }
}
}
}
  • Filter inside the query, not after it.  post_filter  removes results after the nearest neighbors are picked. In our test at  k: 1 , filtering inside  knn  returned 8 permitted documents and  post_filter  returned 4. Hybrid queries work the same way:  hybrid.filter  returned 11 and  post_filter  returned 6.
  • Use  size  to set how many results you get.  k  counts per shard, so in our test  k: 1  still returned 8.
  • Send every hybrid query with  search_pipeline . Without a  normalization processor , the query returns 200 but doesn't combine the scores.

Step 4: Keep permissions in sync

When a permission changes at the source, update only the permission fields. OpenSearch Serverless doesn't support  _update_by_query , so find the affected documents first: open a  point in time (PIT)  and page through the matches with  search_after .
POST enterprise-docs/_search/point_in_time?keep_alive=5m

GET _search
{
"size": 100,
"query": { "term": { "allowed_groups": "contractors-2025" } },
"pit": { "id": "<pit-id>", "keep_alive": "5m" },
"sort": [ { "_shard_doc": "asc" } ],
"_source": false
}
Then send each document's new ACL with   _bulk    update  actions, and close the PIT with  DELETE _search/point_in_time  when you finish:
POST _bulk
{ "update": { "_index": "enterprise-docs", "_id": "wiki:2" } }
{ "doc": { "allowed_groups": ["eng-team"] } }
  • Send the full new list.  doc  replaces the whole  allowed_groups  array. To remove one group without knowing the rest, run a scripted  update  with  removeIf  on each document.
  • Partial updates don't re-embed. In our test,  _update  and  _bulk   update  made no embedding calls, even with a  text_embedding  default pipeline, and kept the stored vector. Use them only for fields that aren't embedded, such as permissions. When we changed  body  with  _update , vector search kept matching the old text. To change embedded text, re-send the whole document through the pipeline.
  • Don't use  skip_existing  on Serverless. In our test, every write through a pipeline with it set failed with  404 index_not_found_exception . To re-send a whole document with unchanged text, include its stored vector and add  ?pipeline=_none . You need both: without  _none , the model runs anyway, and without the vector, the vector is deleted.
  • Plan for a few minutes of delay. In our test, a removed group stopped matching after 125–248 seconds, and an added group appeared after about 300 seconds.
  • Compare source and index on a schedule. Event-driven updates miss changes.

Check that filtering works

Run the same query as two test users from different groups. A document restricted to one group should appear only for that user. Then run it once without the filter: in our test, that returned all 30 documents, including the five with empty ACLs. Repeat with your vector and hybrid queries.

Give some documents a hard boundary

For documents that parts of your app must not return, such as export-controlled files, use a separate index with its own policy and app role. OpenSearch Serverless then blocks access itself, with or without the filter. In our test, a wildcard search such as  docs-*  failed entirely with 403 when it included an index the role couldn't read. Search only the indexes each role can read, or use   _msearch  , which returns results per index.

On an OpenSearch Service domain

The same pattern works on a domain with  fine-grained access control . Create a read-only role,  map it  to your application's identity, and send the same filtered queries:
PUT _plugins/_security/api/roles/enterprise_search_app
{
"cluster_permissions": ["cluster_composite_ops_ro"],
"index_permissions": [
{ "index_patterns": ["enterprise-docs*"], "allowed_actions": ["read"] }
]
}
In our test on OpenSearch 3.7, each user got the right documents, including from hybrid queries. Domains also support   _update_by_query  ; add  pipeline=_none  so it doesn't re-run embedding. If other clients query the index directly, or compliance requires the search engine itself to enforce permissions, give those clients  document-level security  (DLS). Keep the search application on the role without DLS: in our tests on OpenSearch 2.19 and 3.7, hybrid queries failed for DLS users.

Quick reference: what happens if…

If you…
What happens
Fix
Map  allowed_groups  as  text 
 eng-team  is split into  eng  and  team , and the filter stops matching
Map it as  keyword 
Set  engine  on a  knn_vector  field
 400   Field parameter 'engine' is not supported 
Remove  engine 
Miss the filter in one code path
That path returns every document
Build every query in one function
Take groups from a request parameter
A user can add any group
Take groups from the verified sign-in
Name extra roles in the data access policy
They can read every document
Name only the app and ingestion roles
Run  neural  queries without a  model  rule
 403   internal_server_exception   "Forbidden" 
Add  aoss:ExecuteMLResource  on  model/<collection-name>/* 
Use  post_filter  on vector or hybrid results
Fewer results (4 instead of 8 in our test)
Filter inside  knn  or in  hybrid.filter 
Send a hybrid query without  search_pipeline 
 200 , but the scores aren't combined
Add  search_pipeline 
Call  _update_by_query  on Serverless
 404  with an empty body
Use a PIT search and  _bulk  updates
Set  skip_existing  on  text_embedding 
Writes fail with  404 index_not_found_exception 
Remove it, and send permission changes as partial updates
Change embedded text with a partial update
Vector search still matches the old text
Re-send the whole document through the pipeline
Sign requests without  x-amz-content-sha256 
 403 Forbidden  with no detail
Add the header

Next steps

  • List each source system's permission model and group names.
  • Decide how long a permission change may take to show up, including the few minutes OpenSearch Serverless needs.
  • Read Part 4, which moves from who can see a document to how its vectors are stored at scale.

Permissions that travel with the document

Store each document's permissions with its text, and let your application match them against the user's groups on every query. Lock the collection to two roles, build every query in one place, and sync changes as partial updates.

Learn more

  •  Hybrid query  in the OpenSearch docs. OpenSearch Service is built on the OpenSearch project, and some features may work differently or be unavailable in the managed service.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article