Attribute Amazon EMR on EC2 costs to your end-users
Big Data Blog
This article explains how to attribute the costs of running Spark applications on Amazon EMR on EC2 clusters to different users, teams, or lines of business. It describes a solution that captures usage and cost metrics from the EMR cluster and AWS Cost Explorer, stores the data in an Amazon RDS database, and uses SQL queries to derive chargeback figures and reporting trends through Amazon QuickSight.
Specifically, the article covers:
- Solution overview and architecture
- Prerequisites for implementing the solution
- Steps to create the required RDS tables
- Steps to deploy the AWS CDK stack with a Lambda function, RDS database, and QuickSight dashboard
- Testing the solution by running sample Spark jobs and querying the RDS tables
- Sample SQL queries and charts for calculating chargeback costs
- Estimating the cost of running this solution
- Instructions for cleaning up the deployed resources
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
2024
2024
2024
2026
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.