Best practices for migrating from Apache Airflow 2.x to Apache Airflow 3.x on Amazon MWAA
Big Data Blog
This article provides comprehensive guidance for migrating from Apache Airflow 2.x to Apache Airflow 3.x on Amazon Managed Workflows for Apache Airflow (MWAA). The migration requires careful planning due to several critical breaking changes.
- Key breaking changes include:
- Removal of direct metadata database access
- Deprecation of SubDAGs
- Changes to default scheduling behavior
- Library and dependency updates
- Recommended migration strategy includes:
- Phased approach with pilot migration
- Blue/green deployment
- Wave-based production migration
- Post-migration review
- Critical steps involve:
- Compatibility checks using ruff linter
- Updating DAG code and requirements
- Creating a new MWAA 3.x environment
- Migrating metadata and configurations
The migration requires thorough preparation, systematic testing, and clear documentation to minimize risks and disruptions to workflow operations.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
2025
2025
2026
2024
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.