Mastering Amazon Bedrock throttling and service availability: A comprehensive guide

This article provides a comprehensive guide to handling 429 ThrottlingException and 503 ServiceUnavailableException errors in Amazon Bedrock production applications.

• 429 errors indicate exceeded quotas (RPM/TPM); 503 errors signal temporary service capacity issues
• Rate-based throttling (RPM) occurs when requests per minute exceed account limits across all applications
• Token-based throttling (TPM) happens when input/output tokens per minute exceed quotas
• Model-specific throttling indicates particular model endpoints are overloaded
• Implement exponential backoff with jitter for intelligent retry strategies
• Use token-aware rate limiting to track sliding 60-second token consumption windows
• Increase boto3 connection pool size to prevent 503 errors from pool exhaustion
• Deploy circuit breaker pattern to prevent hammering failing services
• Use cross-Region inference (CRIS) for automatic failover and improved resilience
• Monitor CloudWatch metrics: InvocationThrottles, InvocationServerErrors, token counts by model/region
• Configure CloudWatch alarms for quota utilization thresholds and service success rates

The guide emphasizes understanding error root causes, implementing appropriate retries, designing for scale with connection pooling and circuit breakers, and establishing comprehensive monitoring for production-grade generative AI workloads.