Announcing the Apache Spark upgrade agent for Amazon EMR
News
This article announces the Apache Spark upgrade agent for Amazon EMR, which automates and accelerates Apache Spark version upgrades on EC2 and EMR Serverless.
- Reduces upgrade timelines from months to weeks through automated code analysis
- Automatically identifies API changes and behavioral modifications in PySpark and Scala
- Provides conversational natural language interface for expressing upgrade requirements
- Engineers review and approve all suggested changes before implementation
- Validates data quality and functional correctness throughout upgrade process
- Supports upgrades from Spark 2.4 to 3.5
- Accessible via SageMaker Unified Studio, Kiro CLI, or IDE with MCP compatibility
- Available in all AWS Regions where SageMaker Unified Studio is available
The Apache Spark upgrade agent significantly reduces engineering effort and time required for Spark version upgrades while maintaining code control and data accuracy.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
Dec 16
2025
2025
Introducing Apache Spark upgrade agent for Amazon EMR
Dec 16
2025
2025
Introducing the Apache Spark troubleshooting agent for Amazon EMR and AWS Glue
May 27
2026
2026
Amazon EMR now supports Apache Spark 4.0.2 in general availability
Jun 9
2026
2026
Announcing general availability of Apache Spark 4.0 on Amazon EMR
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.