Home icon
Improved scalability and resiliency for Amazon EMR on EC2 clusters

Blog



This article announces over 30 new features for Amazon EMR on EC2 clusters, focusing on improved scalability, resiliency, and availability for big data processing workloads.

  • Enhanced Spot Instance interruption handling prevents cluster scaling operations from getting stuck
  • Recommissioned nodes now accept Spark tasks within seconds instead of 60 minutes
  • Improved YARN exclude file management reduces scale-down failures from disk over-utilization
  • Local storage file system more resilient to instance reconfigurations and EBS volume remounting
  • Fixed timing sequence issues reducing Kerberos cluster startup time by up to 200%
  • Log management daemon automatically restarts if interrupted, preventing disk over-utilization
  • Log rotation enabled for long-running clusters to prevent disk space exhaustion
  • Expanded log monitoring covers additional log folders for better cluster stability

These enhancements make Amazon EMR clusters more stable and efficient for long-running and large-scale data processing workloads.



Go to article

The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.

Related articles

The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.