Introducing Apache Iceberg materialized views in AWS Glue Data Catalog

This article introduces Apache Iceberg materialized views in AWS Glue Data Catalog, a new capability that simplifies data transformation pipelines and accelerates query performance.

• Materialized views store pre-computed query results as Iceberg tables, automatically updated when base data changes
• Eliminates need for complex custom data pipelines, change detection, and manual orchestration
• Supports full and incremental refresh mechanisms to keep views synchronized with source tables
• Automatic query rewriting across Athena, EMR, and Glue intelligently uses materialized views for performance gains
• Create views using standard SQL syntax with optional scheduled refresh intervals (SECONDS, MINUTES, HOURS, DAYS)
• Manual refresh available via SQL commands or AWS Glue APIs for updates outside scheduled intervals
• Requires AWS optimized Spark runtime 3.5.6+ (not open source Spark)
• Views eventually consistent; queries may see outdated data during refresh windows

Iceberg materialized views streamline data lake operations by automating transformations and optimizing queries without requiring complex pipeline maintenance.