Amazon SageMaker HyperPod enhances support for Ray

Amazon SageMaker HyperPod enhances support for Ray, a popular open-source framework for scaling AI workloads, with built-in observability, resilient training, accelerated inference, and managed development environments.

• Create, edit, monitor, and delete Ray clusters from SageMaker Studio web interface
• Attach JupyterLab, Code Editor, or local IDE to running clusters for interactive development
• Grafana dashboards with Amazon Managed Service for Prometheus provide built-in observability
• Node auto-recovery and hung job detection handle GPU faults and job hangs
• Tiered checkpointing and task governance maximize compute utilization and GPU-hour goodput
• Tiered KV cache for Ray Serve reduces inference latency and time to first token
• Available for HyperPod clusters orchestrated by Amazon EKS

HyperPod simplifies Ray deployment at production scale by reducing operational burden and enabling interactive development without kubectl familiarity.