
Automate archiving and purging of historical PostgreSQL data to Amazon S3 using AWS Glue workflows with pg_partman, then restore it on demand.


<div><p>This article demonstrates automating data archival and purge processes for Amazon RDS PostgreSQL using pg_partman, Amazon S3, and AWS Glue.</p><ul><li>Uses pg_partman for native PostgreSQL range partitioning of hot and cold data</li><li>AWS Glue automates archival of historical partitions to Amazon S3</li><li>Reduces database compute load by running business logic outside the database</li><li>Solution includes two workflows: "Maintain and Archive" and "Restore from S3"</li><li>Architecture uses Multi-AZ RDS, Secrets Manager, S3 endpoints, and private subnets</li><li>AWS CDK project provisions all required infrastructure automatically</li><li>Supports on-demand or scheduled triggers via EventBridge</li><li>Solution works for both RDS PostgreSQL and Aurora PostgreSQL</li></ul><p>This approach modernizes database maintenance by automating partition management and archival while maintaining data availability and reducing operational overhead.</p></div>


Related articles