Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot
Machine Learning Blog
This article describes how AssemblyAI optimizes speech-to-text inference costs on AWS using Amazon EC2 Spot Instances while maintaining service level agreements.
- Combines Amazon ECS capacity providers with EC2 Spot and On-Demand GPU instances for cost-effective inference at scale
- Diversifies across multiple GPU instance families (g4dn, g5, g6) to increase Spot capacity availability and reduce interruptions
- Uses Amazon SQS queue-based retry mechanism to gracefully handle Spot interruptions and recover in-flight jobs
- Deploys across multiple AWS Regions (us-west-2, eu-west-2) to meet data residency requirements for regulated industries
- Achieved 70% cost savings on inference compute compared to On-Demand pricing while supporting 4x year-over-year growth
The architecture demonstrates how batch and near real-time AI workloads can leverage Spot Instances with proper fault tolerance design to achieve significant cost optimization without sacrificing availability.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
2025
2026
2026
2026
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.