Home icon

Three questions to ask about AI inference in Amazon EKS, and how Amazon CloudWatch answers them

AWS Cloud Operations Blog



This article demonstrates how to use Amazon CloudWatch to monitor GPU utilization, workload efficiency, and hardware health for AI inference workloads running on Amazon EKS.

  • Use CloudWatch accelerated compute insights to identify which workloads hold GPUs and detect idle, expensive GPUs with no traffic
  • Compare model performance using GPU utilization, DRAM activity, model compute efficiency, and KV cache headroom metrics
  • Monitor GPU hardware health through DCGM metrics including XID errors, ECC counters, and power/thermal warnings
  • Create PromQL alarms in CloudWatch Query Studio that track GPU, host, and pod labels for precise incident alerting
  • Install the CloudWatch Observability add-on to automatically attach Kubernetes labels to DCGM GPU metrics

CloudWatch provides unified visibility across GPU hardware, Kubernetes workloads, and inference application metrics to optimize expensive GPU resources on shared EKS clusters.



Go to article

The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.

Related articles

Sep 1
2026
Fast model loading for AI inference on Amazon EKS
Oct 21
2026
Observability patterns for AI agents on Amazon EKS
Nov 13
2026
Announcing Accelerated Compute Insights for Amazon CloudWatch
May 29
2025
Introducing AI on EKS: powering scalable AI workloads with Amazon EKS

The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.