Troubleshooting AWS Glue ETL Jobs using Amazon CloudWatch Logs Insights enhanced queries
AWS Cloud Operations Blog
This article explains how to use AWS CloudWatch Logs Insights to troubleshoot and optimize AWS Glue ETL jobs by analyzing their logs.
Specifically, the article covers:
- Setting up log collection from AWS Glue jobs to CloudWatch Logs
- Querying CloudWatch Logs Insights to identify inefficient long-running jobs
- Analyzing job stages and their durations to pinpoint performance bottlenecks
- Using the Pattern feature in Logs Insights to find job configuration issues impacting performance
- Optimizing AWS Glue jobs based on insights from log analysis
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
May 9
2024
2024
Use AWS Glue Data Catalog views to analyze data
May 23
2024
2024
Get started with AWS Glue Data Quality dynamic rules for ETL pipelines
Mar 12
2024
2024
Measure performance of AWS Glue Data Quality for ETL pipelines
Mar 29
2024
2024
Enhance monitoring and debugging for AWS Glue jobs using new job observability metrics, Part 3: Visualization and trend analysis using Amazon QuickSight
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.