Home icon

Keep inference running through regional capacity shortfalls with cross-Region inference routing in Amazon SageMaker AI

Machine Learning Blog



Amazon SageMaker AI introduces cross-Region inference routing, a new feature that automatically routes inference requests across multiple AWS Regions and accounts to maintain availability when local capacity is exhausted.

  • Inference router resource provides single invocation path across endpoints and inference components in multiple Regions and accounts
  • Router prefers local Region destinations while healthy, automatically shifts traffic to remote destinations during capacity shortfalls
  • Supports real-time and asynchronous inference with automatic retry logic for non-model errors
  • Endpoint owners can set cross-Region traffic caps to protect local traffic from overflow
  • Per-request routing records and CloudWatch metrics show which destination served each request and why
  • Cross-account deployments supported via resource-based policies with source router ARN conditions
  • No additional charge; pricing follows standard SageMaker endpoint rates

Cross-Region inference routing eliminates the need for custom failover logic, enabling highly available model serving across Regions without application changes.



Go to article

The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.

Related articles

Nov 18
2026
Amazon SageMaker AI now supports cross-Region inference routing
Nov 18
2026
How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI
Nov 17
2026
Amazon SageMaker AI now supports KV cache-aware routing for real-time inference endpoints
Oct 30
2026
Amazon SageMaker AI inference now supports GPU partitioning for real-time endpoints

The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.