
H2O.ai achieved a 100-fold improvement in ML workload provisioning time by implementing Karpenter for intelligent node autoscaling and Bottlerocket OS with pre-baked container images, reducing deployment from 15 minutes to seconds.


<div><p>This article describes how H2O.ai optimized their AI/ML infrastructure on AWS using Karpenter and Bottlerocket, achieving 100x faster workload provisioning.</p><ul><li>Replaced Kubernetes Cluster Autoscaler with Karpenter for faster, more flexible node provisioning</li><li>Adopted Bottlerocket OS for improved security, reduced attack surface, and automatic patching</li><li>Created multiple weighted Karpenter provisioners to optimize GPU and instance type selection</li><li>Pre-baked container images into EBS snapshots to eliminate image pull delays</li><li>Reduced workload provisioning time from 15 minutes to seconds using combined approach</li><li>Implemented CIS Benchmark for Bottlerocket using bootstrap containers</li></ul><p>H2O.ai successfully improved performance, reliability, and security of their SaaS platform through Karpenter's intelligent scaling and Bottlerocket's lightweight, secure OS design.</p></div>


Related articles