AWS Neuron introduces support for Trainium2 and NxD Inference
News
AWS has released Neuron SDK 2.21, introducing significant updates for machine learning infrastructure:
- Added support for Trainium2 chips and EC2 Trn2 instances
- Introduced NxD Inference, a PyTorch-based library for simplified large language model deployment
- Launched Neuron Profiler 2.0 (beta) with enhanced profiling capabilities
- Enabled Llama 3.1 405B model inference on a single trn2.48xlarge instance
- Added support for Llama 3.2, Llama 3.3, and Mixture-of-Experts models
- Introduced new inference features like FP8 weight quantization and flash decoding
The SDK supports training and deploying models on Trn1, Trn2, and Inf2 instances across various AWS purchasing options.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
May 27
2025
2025
AWS Neuron introduces NxD Inference GA, new features, and improved tools
Jul 2
2025
2025
New features for AWS Neuron 2.24 include PyTorch 2.7 and inference enhancements
Sep 17
2024
2024
AWS Neuron introduces Neuron Kernel Interface (NKI), NxD Training, and JAX support for training
Jun 11
2024
2024
Get started quickly with AWS Trainium and AWS Inferentia using AWS Neuron DLAMI and AWS Neuron DLC
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.