Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS
Machine Learning Blog
This article demonstrates how to deploy NVIDIA Nemotron 3.5 Super VL, a 120B-parameter multimodal model, on Amazon EKS and SageMaker HyperPod for accelerating agentic AI workloads.
- Nemotron 3.5 Super VL combines reasoning and tool use with text, image, document, and video understanding capabilities
- Hybrid Mamba-2 MoE architecture activates only 12B of 120B parameters per token, reducing compute requirements
- Supports agent workflows for coding, web research, document intelligence, cybersecurity, and video analysis
- Deployment validated on p6-b200.48xlarge and p5en.48xlarge instances with performance benchmarking tools
- Reference implementation available in aws-do-eks project with cluster creation, model deployment, and testing automation
- Concurrency sweep analysis helps optimize deployment topology and capacity planning decisions
The article provides deployment patterns and performance evaluation approaches for running NVIDIA Nemotron models on AWS infrastructure with NVIDIA Dynamo and vLLM serving frameworks.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
2026
2026
2026
2026
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.