Migrate Delta tables from Azure Data Lake Storage to Amazon S3 using AWS Glue
Big Data Blog
This article explains how to migrate Delta tables from Azure Data Lake Storage Gen2 (ADLS Gen2) to Amazon S3 using AWS Glue. It provides a step-by-step guide on how to configure the Azure Data Lake Storage Connector for AWS Glue, author an AWS Glue ETL script, and load the extracted data into Amazon S3.
Specifically, the article covers:
- Overview of the Azure Data Lake Storage Connector for AWS Glue
- Prerequisites for the migration
- Configuring ADLS Gen2 account credentials in AWS Secrets Manager
- Subscribing to the Azure Data Lake Storage Connector for AWS Glue
- Creating a custom connection in AWS Glue
- Reading Delta tables from ADLS Gen2 using the connector in an AWS Glue ETL job
- Reading Delta tables in an AWS Glue notebook (optional)
- Cleaning up the resources
- Conclusion
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
Aug 29
2024
2024
Migrate Amazon RDS for Oracle BLOB column data to Amazon S3
Sep 12
2024
2024
AWS Glue Data Catalog now supports storage optimization of Apache Iceberg tables
Sep 12
2024
2024
The AWS Glue Data Catalog now supports storage optimization of Apache Iceberg tables
Oct 30
2024
2024
Modernize your legacy databases with AWS data lakes, Part 1: Migrate SQL Server using AWS DMS
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.