
Amazon OpenSearch Service performance can be optimized by monitoring storage and shard skew using CloudWatch metrics deployed via a Lambda function to ensure even data distribution across cluster nodes.


<div><p>This article explains how to optimize Amazon OpenSearch Service performance by monitoring storage and shard skew using CloudWatch metrics and a Lambda-based solution.</p><ul><li>Proper shard strategy distributes workload evenly across data nodes for optimal performance</li><li>Target shard sizes: 10-30 GB for search, 30-50 GB for log analytics workloads</li><li>Storage skew above 10% from average significantly impacts CPU, network, and disk usage</li><li>CloudFormation template deploys Lambda function to extract shard metadata and push metrics to CloudWatch</li><li>Solution monitors NodeStorageSkew, NodeShardSkew, and ShardDiskUsed metrics daily by default</li><li>Use Index State Management (ISM) plugin to rotate indexes based on shard size, not just time</li><li>Minimal monthly cost; solution can be deleted via CloudFormation console anytime</li></ul><p>This solution helps identify and remediate uneven shard distribution to improve OpenSearch domain performance and reduce operational costs.</p></div>


Related articles