Build an internal SaaS service with cost and usage tracking for foundation models on Amazon Bedrock
Machine Learning Blog
This article provides a technical guide on building an internal SaaS platform for accessing foundation models on Amazon Bedrock, with a focus on cost and usage tracking for different teams or tenants.
Specifically, the article covers:
- Solution architecture and key components for capturing model usage and tracking costs per team
- Controlling usage limits and throttling requests for each team using API Gateway usage plans
- Configuring access control to specific models for each team
- Deploying the solution using an AWS CDK template
- Invoking the service with code examples for text generation and embeddings
- Cost estimation example based on input/output token usage
- End-to-end view of a multi-tenant serverless SaaS environment
- Support for different Amazon Bedrock consumption plans (on-demand and provisioned throughput)
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.