Migrate Delta tables from Azure Data Lake Storage to Amazon S3 using AWS Glue
Big Data Blog
This article explains how to migrate Delta tables from Azure Data Lake Storage Gen2 (ADLS Gen2) to Amazon S3 using AWS Glue. It provides a step-by-step guide on how to configure the Azure Data Lake Storage Connector for AWS Glue, author an AWS Glue ETL script, and load the extracted data into Amazon S3.
Specifically, the article covers:
- Overview of the Azure Data Lake Storage Connector for AWS Glue
- Prerequisites for the migration
- Configuring ADLS Gen2 account credentials in AWS Secrets Manager
- Subscribing to the Azure Data Lake Storage Connector for AWS Glue
- Creating a custom connection in AWS Glue
- Reading Delta tables from ADLS Gen2 using the connector in an AWS Glue ETL job
- Reading Delta tables in an AWS Glue notebook (optional)
- Cleaning up the resources
- Conclusion
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
Aug 29
2024
2024
Migrate Amazon RDS for Oracle BLOB column data to Amazon S3
Dec 10
2024
2024
Build a managed transactional data lake with Amazon S3 Tables
May 20
2026
2026
How to Migrate from Azure Blob Storage to Amazon S3 Using Agentless AWS DataSync
May 29
2025
2025
Optimizing data lakes with Amazon S3 Tables and Apache Spark on Amazon EKS
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.