Amazon EMR on EKS now supports Apache Spark troubleshooting agent
News
Amazon EMR on EKS now supports the Apache Spark troubleshooting agent, enabling data engineers to diagnose job failures through natural language queries.
- Analyzes Spark History Server data, executor logs, and cluster configurations to identify issues
- Identifies problems like memory errors, data skew, resource contention, and connectivity failures
- Provides automated root cause analysis and PySpark code recommendations
- Accessible via "Troubleshoot with AI" option in EMR on EKS console
- Available through MCP with compatible AI coding agents (Kiro, Claude Code, Cursor)
- All operations are read-only, authenticated with IAM roles, and logged in CloudTrail
- Covers all EMR deployment options: EC2, Serverless, and EKS
The Spark troubleshooting agent simplifies EMR job diagnostics by eliminating manual log navigation and providing intelligent insights through natural language.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.