Optimizing AI implementation costs with Automat-it

This article details how Automat-it helped a customer optimize AI implementation costs for video intelligence solutions using YOLOv8 models on AWS Elastic Kubernetes Service (EKS).

• Initial approach used dedicated GPU instances per customer, costing $353 per camera monthly
• Implemented GPU time-slicing to share GPU resources across multiple AI models
• Used NVIDIA Kubernetes device plugin to enable GPU resource fractionalization
• Reduced inference costs to $27.81 per camera monthly - a twelvefold cost reduction
• Successfully maintained model performance while processing up to 54 pods on a single GPU

The solution demonstrates how strategic GPU resource sharing can dramatically lower AI infrastructure costs without compromising performance, using Kubernetes and time-slicing techniques.