AWS Glue now provides job queuing, enabling you to automatically queue and retry jobs if account-level quotas or resource limits are insufficient, freeing you from needing to manage quotas and limits.

<div>
<p>AWS Glue now provides job queuing, a new capability that enables users to submit AWS Glue job runs without needing to manage account level quotas and limits.</p>
<p>Specifically, the article covers:</p>
<ul>
<li>AWS Glue job queuing monitors account level quotas and limits, and automatically queues jobs if limits are insufficient to start a job run</li>
<li>Once limits become available, AWS Glue retries the queued job run</li>
<li>Job queuing can handle limits like max concurrent job runs per account, max concurrent Data Processing Units (DPU), and resource unavailability due to IP address exhaustion in Amazon VPC</li>
<li>Job queuing can be enabled via the AWS Management console or API/CLI</li>
<li>Job queuing is available in all AWS commercial regions where AWS Glue is generally available</li>
</ul>
</div>


AWS Glue now provides job queuing

Related articles