Home icon

Using NVIDIA Run:ai GPU Fractioning on Amazon EKS

Machine Learning Blog



This article demonstrates how to configure NVIDIA Run:ai GPU Fractioning on Amazon EKS to enable multiple Pods to share a single GPU with enforced memory limits.

  • GPU Fractioning adds Kubernetes scheduling integration and enforced per-Pod GPU-memory limits to software-based sharing
  • Validated example uses a g5.xlarge instance with NVIDIA A10G GPU, allowing two Pods to share one GPU with 8 GiB memory contracts each
  • Custom Amazon Linux 2023 image built with NVIDIA R615 driver, NVIDIA Container Toolkit, and containerd with NRI integration
  • KAI Scheduler and GPU Fractioning Operator enable placement of multiple Pods on same GPU while enforcing memory boundaries
  • Walkthrough includes baseline testing, positive sharing test with two Pods, and verification of rejection for allocations exceeding limits
  • Production considerations include least-privilege IAM, namespace controls, performance testing under concurrent load, and pinned component versions

GPU fractioning improves GPU utilization for interactive notebooks, evaluation jobs, and small inference services when workload memory fits within enforced limits.



Go to article

The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.

Related articles

Jun 9
2025
Maximizing GPU Utilization using NVIDIA Run:ai in Amazon EKS
Oct 28
2025
Extending GPU Fractionalization and Orchestration to the edge with NVIDIA Run:ai and Amazon EKS
Oct 28
2026
Amazon ECS now supports GPU sharing with NVIDIA Multi-Process Service (MPS)
Sep 4
2025
How to run AI model inference with GPUs on Amazon EKS Auto Mode

The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.