Home icon

Unlocking real-time analytics: Streaming Aurora DSQL changes into Apache Iceberg

Database Blog

This article demonstrates how to stream Amazon Aurora DSQL change data capture (CDC) events into Apache Iceberg tables using Amazon Data Firehose and Kinesis Data Streams.

  • Maintains two Iceberg tables: cdc_events (append-only audit trail) and current_state (merged by primary key)
  • Handles DELETE events using tombstone flags to preserve audit trail integrity
  • Processes out-of-order CDC events with transaction commit timestamps for ordering
  • Includes Streamlit dashboard for exploring pipeline settings, current state, audit trail, and time-travel queries
  • Provides CloudFormation deployment script for automated setup of Aurora DSQL, Kinesis, Firehose, Lambda, and Glue
  • Enables real-time analytics, event-driven applications, and data synchronization without impacting operational workloads

The solution provides a complete CDC pipeline from Aurora DSQL to Iceberg with audit compliance and real-time analytics capabilities.

Go to article

The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.

Related articles

Jul 28
2026
Building scalable applications on Amazon Aurora DSQL
Aug 31
2026
Deliver real-time data to streaming tables for Apache Iceberg with Amazon Kinesis Data Streams
Aug 21
2026
Amazon Aurora DSQL now supports Amazon CloudWatch Database Insights
Aug 31
2026
Amazon Redshift now supports Apache Iceberg v3 tables

The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.