GPU Cost Attribution in Amazon EKS Using Amazon Managed Service for Prometheus, Amazon Managed Grafana, and OpenTelemetry

This article explains how to implement GPU cost attribution in Amazon EKS using observability tools to track and allocate GPU expenses across teams.

• DCGM Exporter provides detailed GPU metrics including utilization and memory usage
• Kube-state-metrics adds Kubernetes context for mapping GPU usage to applications
• OpenTelemetry Collector enriches and pipelines metrics to Amazon Managed Prometheus
• Three cost calculation types: allocated (requested), effective (actual), waste (difference)
• Grafana dashboards visualize cost allocation by business unit and identify optimization opportunities
• NVIDIA MIG technology partitions GPUs for better multi-tenancy and cost tracking
• Organizations can implement request-based, utilization-based, or hybrid allocation methods

This observability-first approach enables accurate GPU cost chargeback, waste identification, and informed capacity planning for AI/ML workloads on EKS.