Accelerating Spark queries with Iceberg materialized views

This article explains how to accelerate Apache Spark queries using Apache Iceberg materialized views with automatic query rewrite, without changing any SQL code.

• Automatic query rewrite analyzes query plans and transparently substitutes materialized views when they match, supporting structural and exact-match patterns
• Available in Amazon EMR 7.12.0+ and AWS Glue 5.1+, with support for Iceberg and Parquet source tables
• Three MV design tiers demonstrate trade-offs: Tier 1 (5x speedup, broad reuse), Tier 2 (10x speedup, medium reuse), Tier 3 (20x+ speedup, narrow reuse)
• Materialized views store precomputed results as Iceberg tables with automatic refresh on configurable schedules
• Enable with spark.sql.optimizer.answerQueriesWithMVs.enabled=true; validate rewrite via query plan inspection or EMR logs
• Common issues include stale MVs, metadata cache delays, and Spark version mismatches; optimization adds overhead to all queries

Automatic query rewrite enables significant performance gains for expensive analytical queries with joins, aggregations, and window functions without application-level changes.