Amazon EMR now supports Apache Spark 4.0.2 in general availability
News
This article announces the general availability of Apache Spark 4.0.2 on Amazon EMR across all deployment models.
- ANSI SQL support makes data engineering accessible without Spark-specific syntax knowledge
- VARIANT data types natively handle JSON and semi-structured data formats
- Fine-grained access control (FGAC) enforces row and column-level security on Lake Formation tables
- Apache Iceberg v3 provides transaction guarantees and data lineage tracking for compliance
- Enhanced streaming capabilities enable faster real-time application deployment
- Available in all regions where EMR is available
- Apache Spark upgrade agent available to accelerate existing application upgrades
Spark 4.0.2 on EMR simplifies data pipeline development, improves security and compliance, and accelerates real-time application deployment.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.