Amazon SageMaker AI now supports optimized generative AI inference recommendations

This article announces optimized generative AI inference recommendations in Amazon SageMaker AI, which automates the process of finding optimal deployment configurations for generative AI models.

• Reduces model deployment time from weeks to hours with automated configuration optimization
• Three-stage process: narrows configuration space, applies goal-aligned optimizations, benchmarks on real GPU infrastructure
• Users specify single performance goal: optimize for cost, minimize latency, or maximize throughput
• Automatically applies techniques like speculative decoding, tensor parallelism, and kernel tuning
• Uses NVIDIA AIPerf for rigorous benchmarking with statistical confidence intervals
• Returns ranked, deployment-ready recommendations with validated performance metrics
• No additional costs; uses standard compute or existing ML Reservations
• Available in seven AWS regions including US East, US West, Asia Pacific, and Europe

SageMaker AI eliminates weeks of manual infrastructure tuning, enabling teams to deploy generative AI models faster with validated configurations and right-sized costs.