Accelerate Generative AI Inference on Amazon SageMaker AI with G7e Instances

This article announces the availability of G7e instances powered by NVIDIA RTX PRO 6000 Blackwell GPUs on Amazon SageMaker AI for generative AI inference workloads.

• G7e instances offer 96 GB GPU memory per GPU, double that of G6e instances
• Delivers up to 2.3x inference performance improvement over previous-generation G6e
• Single G7e.2xlarge GPU can host 35B parameter models; 8-GPU G7e.48xlarge supports 300B models
• 1,600 Gbps networking throughput enables low-latency multi-node inference scenarios
• Benchmarks show 2.6x cost reduction ($0.79 vs $2.06 per million tokens) at production concurrency
• Combined with EAGLE speculative decoding, achieves 2.4x throughput and 75% cost reduction
• Well-suited for chatbots, RAG pipelines, long-context inference, and multimodal AI workloads

G7e instances provide significant cost and performance improvements for LLM inference, enabling previously multi-GPU workloads to run efficiently on single GPUs while reducing operational complexity.