Fix pod distribution drift in Amazon EKS with the Kubernetes descheduler

This article explains how pod distribution drift occurs in Amazon EKS under soft topology spread constraints and demonstrates using the Kubernetes descheduler to restore balanced pod placement across availability zones.

• Soft topology spread constraints (ScheduleAnyway) permit drift when zones become unavailable; running pods are never redistributed by the scheduler
• Drift persists silently for stable workloads even after capacity returns, reducing resilience and increasing cross-AZ data transfer costs
• Hard constraints (DoNotSchedule) prevent drift but queue pods as Pending; soft constraints with descheduler offer a middle ground
• The Kubernetes descheduler evaluates running pods against topology policies and evicts violators, relying on PodDisruptionBudgets to throttle disruption
• Measured test: 1,000-pod fleet drifted to 500/500/0 distribution across three zones and held for 16+ hours until descheduler restored balance in 183 evictions
• Enable descheduler only for stateless, restartable workloads; start with narrow scope, enforce PDBs first, and use absolute budget values for small deployments

The descheduler restores topology spread intent for running pods with controlled disruption, but requires careful PDB configuration and operational discipline to balance resilience gains against restart costs.