Real-time CDC from Aurora PostgreSQL to Amazon S3 Tables using Debezium and Firehose
Big Data Blog
This article demonstrates building a real-time CDC pipeline from Aurora PostgreSQL to Apache Iceberg tables in Amazon S3 Tables using Debezium, MSK, Firehose, and Lambda.
- Debezium captures row-level changes from Aurora PostgreSQL logical replication WAL
- ByLogicalTableRouter SMT consolidates multiple tables into single MSK topic
- Lambda transforms Debezium envelope to flattened JSON with routing metadata
- Firehose delivers records to S3 Tables with automatic Iceberg operations
- S3 Tables handles compaction and snapshot management automatically
- Supports inserts, updates, deletes with row-level Iceberg operations
- Lake Formation provides fine-grained access control for analytics teams
- Infrastructure deployed via AWS CDK with six modular stacks
- Single Firehose stream serves multiple destination tables, reducing costs
- Iceberg time travel enables querying historical table states
The solution provides a fully managed, governed lakehouse architecture enabling near real-time analytics on transactional data without impacting OLTP performance.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
Jun 3
2026
2026
Implementing real-time change data capture with Debezium for Amazon Aurora PostgreSQL and Amazon RDS for PostgreSQL
Aug 12
2026
2026
Migrate Amazon Aurora PostgreSQL across major versions with active Debezium CDC connectors using native logical replication
Jul 8
2026
2026
Amazon Aurora DSQL change data capture (CDC) Is now generally available
Jun 11
2026
2026
Amazon Aurora now supports PostgreSQL major version 18
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.