Optimizing Salesforce’s model endpoints with Amazon SageMaker AI inference components

This article details how Salesforce optimized its AI model deployment using Amazon SageMaker AI inference components, achieving significant cost and performance improvements.

• Salesforce was struggling with inefficient GPU utilization across different model sizes
• SageMaker inference components allowed multiple models to share GPU resources on a single endpoint
• The solution enabled model-specific resource allocation and independent scaling
• Key benefits included optimized resource allocation and up to 8x reduction in deployment costs
• The approach allows efficient hosting of both large and small models with dynamic scaling

By implementing SageMaker AI inference components, Salesforce transformed its AI infrastructure management, maximizing GPU utilization while maintaining high performance standards across diverse AI workloads.