Supercharge your LLM performance with Amazon SageMaker Large Model Inference container v15

AWS has announced Amazon SageMaker Large Model Inference (LMI) container v15, which offers significant improvements for deploying large language models (LLMs) with enhanced performance and expanded model support.

• Introduces async mode with vLLM's AsyncLLMEngine for improved request handling
• Supports vLLM V1 engine, delivering up to 111% higher throughput
• Expanded API schema support with OpenAI Chat Completions, Completions, and TGI formats
• Added multimodal support and function calling capabilities
• Supports latest models like Llama 4, Gemma 3, Qwen, Mistral AI, and DeepSeek-R

The new container provides improved performance, flexibility, and ease of deployment for generative AI models across various use cases and model types.