How LaunchDarkly migrated to Amazon MWAA to achieve efficiency and scale
Big Data Blog
LaunchDarkly successfully migrated its internal analytics platform to Amazon Managed Workflows for Apache Airflow (MWAA) and Amazon ECS, achieving significant scalability and efficiency improvements.
- Migrated from a vendor-managed Airflow solution to AWS
- Scaled from 100 to 14,000 Airflow tasks per day across three teams
- Implemented Amazon ECS on Fargate for task isolation and improved performance
- Created a reusable Terraform module for quick MWAA environment deployment
- Integrated monitoring using CloudWatch and Datadog
The migration solved challenges around service integration, data locality, and centralized orchestration, enabling LaunchDarkly to more efficiently manage and scale their data pipelines across multiple AWS services.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.