AWS adds support for NIXL with EFA to accelerate LLM inference at scale

This article announces AWS support for NVIDIA Inference Xfer Library (NIXL) with Elastic Fabric Adapter (EFA) to accelerate disaggregated LLM inference on Amazon EC2.

• NIXL with EFA increases KV-cache throughput and reduces inter-token latency
• Enables efficient KV-cache movement between storage layers
• Compatible with all EFA-enabled EC2 instances across AWS regions
• Integrates natively with NVIDIA Dynamo, SGLang, and vLLM frameworks
• Available at no additional cost with NIXL 1.0.0+ and EFA installer 1.47.0+

NIXL with EFA provides flexible, performant disaggregated LLM inference at scale on EC2.