Under the hood: Amazon EKS ultra scale clusters

Amazon EKS has announced support for ultra-scale Kubernetes clusters with up to 100,000 nodes, enabling massive AI/ML workloads with unprecedented computational capabilities. Key highlights of this breakthrough include:

• Support for up to 1.6 million AWS Trainium chips or 800,000 NVIDIA GPUs in a single cluster
• Architectural innovations in etcd data store, including consensus offloading and in-memory database
• Significant improvements in API server throughput and controller performance
• Enhanced Karpenter node management with static capacity and auto-repair features
• Network scaling optimizations to support massive cluster sizes

The improvements enable customers like Anthropic to run cutting-edge AI model training and inference workloads at an unprecedented scale, with robust performance, reliability, and Kubernetes conformance.