AWS Builder Center

Troubleshooting low disk storage in Amazon OpenSearch Service

This article provides an overview of how to troubleshoot low disk space in Amazon OpenSearch Service. It covers common causes, warning signs, and best practices for managing storage in your OpenSearch cluster.

Cloud Enginner
Low Disk alerts are common in Amazon OpenSearch Service environments that receive large volumes of logs, metrics, and events. At first, the obvious response may be to increase EBS storage. But before scaling the cluster, we need to understand why storage reached its limit and whether another factor is putting pressure on the cluster.
This article presents a simple and practical process for investigating this scenario.
Let's go!

What is Amazon OpenSearch Service?

Amazon OpenSearch Service is an AWS managed service based on OpenSearch that simplifies the deployment, operation, and scaling of an open-source search and analytics engine. It allows you to store, search, and analyze large volumes of data in near real time, with capabilities such as log monitoring, metrics analysis, and data visualization.
It is commonly used for scenarios such as:
  • Centralizing and analyzing application and infrastructure logs
  • Observability and system troubleshooting
  • Metrics and event analysis
  • SIEM and security analysis
  • Text search and search applications
  • Monitoring distributed environments
  • Application data analysis
A common use case is centralized logging:
Applications / Containers / Infrastructure
↓
Ingestion Pipeline
↓
Amazon OpenSearch
↓
OpenSearch Dashboards
↓
Search / Metrics / Analysis
Because these environments can receive millions or billions of documents, storage capacity, shard count, retention, and node sizing need to be continuously monitored.
This is where Low Disk alerts can become a major operational issue.

Troubleshooting Low Disk

1. Understand the domain

Before changing anything, collect information about:
  • OpenSearch version
  • Instance type (Serverless or provisioned)
  • Number of data nodes
  • EBS size and type per node
  • Availability Zones
  • UltraWarm/Cold, if used
This defines the current capacity and helps determine whether the cluster is properly sized for the workload.

2. Look at the trend, not just the current value

In AWS CloudWatch, start by looking at metrics such as:
  •  FreeStorageSpace 
  •  ClusterUsedSpace 
  •  ClusterIndexWritesBlocked 
  •  JVMMemoryPressure 
  •  CPUUtilization 
 FreeStorageSpace  is especially important, but the current value alone does not tell the whole story.
Ask:
How much storage are we consuming per day?
If the cluster has 200 GiB free but is consuming 50 GiB per day, the problem is not solved. There is a growth trend that needs to be addressed.
Also look at the minimum per node, because a single node may be much more full than the others.
In OpenSearch Service,  ClusterIndexWritesBlocked = 1  indicates that the cluster is blocking write operations. Common causes include insufficient storage and high JVM pressure. AWS recommends monitoring these metrics through CloudWatch.

3. Check cluster health

If OpenSearch Dashboards is available, several endpoints are useful for checking cluster health. Go to Dev Tools and run:
GET /_cluster/health
GET /_cat/allocation?v
GET /_cat/indices?v
GET /_cat/shards?v
GET /_cluster/stats
 _cat/allocation  helps identify shard and storage distribution across nodes.  _cat/indices  helps identify the largest indices.
If Dashboards is unavailable and you do not have API access, do not stop troubleshooting. Start the analysis through the AWS Console and CloudWatch.

4. Look for the root cause

Several questions can help narrow down the most likely causes.

4.1. Retention

Are there old indices that should have been removed?

4.2. Ingestion growth

Has the amount of incoming data increased recently?

4.3. Shards

Are there very large shards or an unbalanced distribution?

4.4. JVM

Is there a history of  High JVM Usage  or  Insufficient GC ?

4.5. Capacity

Has the data volume grown beyond the cluster's current capacity?
In real-world incidents, a Low Disk alert can appear alongside high JVM pressure events. Therefore, looking only at disk usage can lead to an incomplete conclusion.

5. Do not ignore shards

Check shard allocation:
GET /_cat/allocation?v
and:
GET /_cat/shards?v
Look for:
  • Nodes with significantly different utilization
  • Very large shards
  • A high number of shards per node
  •  UNASSIGNED  shards
  • Shards in  INITIALIZING  or  RELOCATING 
It is important to validate cluster health. A  Yellow  cluster means that primary shards are allocated but some replicas are unallocated. Insufficient storage can also prevent replica allocation.
A  Red  cluster is more critical because one or more primary shards are unallocated.
Therefore, Low Disk + cluster health issues + many unassigned shards should be investigated before making any changes.

6. Check ISM

For time-based indices, such as logs, Index State Management (ISM) should be one of the first things to check.
Ask:
  • Are there ISM policies?
  • Are the indices associated with those policies?
  • Are any actions failing?
  • Is rollover working?
  • Is the transition to  delete  working?
  • Are there indices significantly older than the expected retention period?
A common workflow may look like:
Hot
↓
Warm
↓
Delete
If the transition to  delete  does not happen, the indices continue consuming storage.
ISM can automate operations such as rollover, tier migration, and index deletion based on age, size, and other conditions.

7. Validate backups before deleting data

Before removing any index:
  • Is there a valid snapshot?
  • When was the last successful snapshot?
  • What is the retention period?
  • Can the data actually be discarded?
Low Disk is not a reason to blindly delete indices.
Deleting an index is a destructive operation. First validate retention, business requirements, and recovery options.

8. Choose remediation only after the diagnosis

The remediation depends on the root cause.
Incorrect retention
Fix the ISM policy or retention settings.
Abnormal growth
Investigate the source of the ingestion before simply increasing capacity.
Insufficient storage
Increase EBS storage when the growth is legitimate.
Insufficient processing capacity
Consider adding nodes or using larger instances.
Shard-related issue
Review shard count, size, and distribution.
JVM pressure
Investigate shards, queries, mappings, workload, and instance sizing.
AWS recommends considering larger instances, additional instances, or additional EBS storage for capacity issues, depending on the root cause.

9. The key takeaway

A Low Disk alert does not automatically mean:
"Increase the disk."
The troubleshooting flow should be:
Alert
↓
Understand the domain
↓
Analyze FreeStorageSpace
↓
Analyze growth trends
↓
Check write blocks and JVM
↓
Identify indices and shards
↓
Check retention and ISM
↓
Validate snapshots
↓
Identify the root cause
↓
Choose the remediation
In OpenSearch environments, different signals can appear at the same time: Low Disk, JVM pressure, unassigned shards, write blocks, and configuration issues.
The key takeaway is simple:
Use Low Disk as the starting point for your investigation, not as the final diagnosis.

Quick checklist

[ ] Version / instance type
[ ] Data nodes / EBS / AZ / UltraWarm

[ ] FreeStorageSpace
[ ] FreeStorageSpace per node
[ ] 24h / 7d trend
[ ] ClusterUsedSpace
[ ] ClusterIndexWritesBlocked
[ ] JVMMemoryPressure
[ ] CPUUtilization

[ ] Cluster health
[ ] Shard allocation
[ ] Largest indices
[ ] UNASSIGNED shards
[ ] Very large shards

[ ] Retention
[ ] ISM
[ ] Snapshots

[ ] Identify root cause
[ ] Remediate safely
[ ] Escalate when necessary

Final thoughts

Troubleshooting Low Disk in Amazon OpenSearch Service goes beyond monitoring free storage. The goal is to understand the relationship between storage, ingestion, shards, retention, ISM, JVM, and cluster capacity before taking action. A structured approach reduces the risk of treating only the symptom and helps build OpenSearch environments that are more predictable, resilient, and easier to operate.

References

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article