Announcing sticky session routing for Amazon SageMaker Inference

The article announces the availability of sticky session routing for Amazon SageMaker Inference, which helps improve performance and user experience for generative AI applications by leveraging previously processed information.

Specifically, the article covers:

• Sticky session routing allows all requests for the same session to be routed to the same instance, enabling reuse of previously processed information to reduce latency and improve user experience.
• This feature is particularly valuable for applications with large data payloads or seamless interactive experiences.
• Customers can create a session ID with their first request and use it for subsequent requests to route them to the same instance.
• Sessions can be deleted when no longer needed to free up resources.
• The feature is available in all regions where SageMaker is available.