How Socure achieved 50% cost reduction by migrating from self-managed Spark to Amazon EMR Serverless

This article details how Socure migrated its Transaction ETL (TETL) streaming pipeline from self-managed Apache Spark on Amazon EKS to Amazon EMR Serverless, achieving significant improvements in cost, latency, and operational efficiency.

• Achieved 50% cost reduction and 47-51% average latency improvement with EMR Serverless
• Resolved performance issues from inefficient autoscaling causing 5x latency increases
• Eliminated Spark executor out-of-memory failures through larger executor sizing (27GB/4 cores vs 14GB/2 cores)
• EMR Serverless scaled down 12% more efficiently on low-traffic days versus EKS
• Transitioned to AWS Graviton for additional cost efficiencies
• Reduced operational overhead by eliminating self-managed EKS and OSS Spark maintenance
• Pipeline processes identity verification data from 1M-5M records daily across raw and processed layers

EMR Serverless provided Socure with a fully managed, scalable solution that improved reliability, reduced costs by over half, and accelerated customer POC delivery while eliminating infrastructure management burden.