Home icon

Introducing the Apache Spark Optimization Agent for Amazon EMR Serverless

News



Amazon EMR Serverless now offers the Apache Spark optimization agent, which analyzes completed Spark job runs and delivers cost and performance optimization recommendations in seconds.

  • Analyzes job configuration, logs, and telemetry to identify performance bottlenecks like parallelism issues, skewed joins, and memory pressure
  • Returns specific configuration changes with evidence and reasoning for each recommendation
  • Compares single runs against baselines to pinpoint changes affecting performance
  • Accessible via EMR Serverless console or SageMaker Unified Studio MCP Server
  • Evaluates best practices including Spark version upgrades and Graviton processor usage
  • Available at no additional charge in all regions except GovCloud and China for EMR 7.6+

The agent helps data engineers optimize Spark jobs during development and production by balancing cost and performance trade-offs while maintaining security through IAM authentication and CloudTrail logging.



Go to article

The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.

Related articles

Dec 16
2025
Introducing Apache Spark upgrade agent for Amazon EMR
Dec 2
2025
Announcing the Apache Spark upgrade agent for Amazon EMR
Nov 21
2025
Amazon EMR Serverless now supports Apache Spark 4.0.1 (preview)
Sep 4
2024
Use Apache Spark on Amazon EMR Serverless directly from Amazon Sagemaker Studio

The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.