Run interactive workloads on Amazon EMR on EC2 with Spark Connect
News
This article announces support for interactive Apache Spark sessions with Spark Connect on Amazon EMR on EC2, enabling developers to work interactively from managed notebooks and IDEs.
- Develop and debug Spark applications interactively from SageMaker Unified Studio and IDEs like Jupyter and VS Code
- Spark Connect's client-server architecture decouples application clients from Spark drivers
- Persistent Spark context spans cells and scripts, blending local Python with remote Spark operations
- Monitor active and completed sessions via Spark UI, History Server, and EMR console
- Supports ad hoc exploration, iterative debugging, and incremental PySpark development before production deployment
- Available on EMR on EC2 with AWS runtime for Apache Spark 8.0 and later in all AWS Regions except GovCloud and China
Interactive Sessions with Spark Connect enables data engineers to maintain preferred development environments while running Spark infrastructure on managed EMR clusters.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
Jun 9
2026
2026
Run Interactive Workloads on Amazon EMR Serverless with Spark Connect
Dec 18
2025
2025
Modernize Apache Spark workflows using Spark Connect on Amazon EMR on Amazon EC2
Aug 3
2026
2026
Introducing Apache Spark troubleshooting agent for Amazon EMR on EKS
Jun 17
2026
2026
AWS Glue Interactive Sessions now support Spark Connect for interactive workloads
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.