Distributed training with Ray Train DLCs and EFA on EKS

This article demonstrates distributed PyTorch training with Ray Train Deep Learning Containers (DLC) and Elastic Fabric Adapter (EFA) on Amazon EKS, eliminating the need to build custom container images.

• Ray Train DLC includes pre-assembled CUDA, PyTorch, NCCL, EFA libraries, and AWS OFI NCCL plugin tested as a single unit
• KubeRay operator manages Ray head and worker pods from a single RayCluster manifest on EKS
• Sample fine-tunes Qwen2.5-1.5B using PyTorch FSDP across eight L4 GPUs on two g6.12xlarge instances
• Configuration via environment variables and manifest templates; no image rebuilds needed per job
• EFA networking automatically engaged through resource requests and environment variables for high-speed inter-node communication
• Training code is model-agnostic; Ray Train handles distributed setup and process group formation
• Scripts automate capacity discovery, cluster deployment, and job submission with idempotent operations

The approach eliminates container image management complexity while enabling reproducible, scalable distributed training on Kubernetes with optimized networking.