Building a serverless pipeline to deliver reliable messaging
Compute Blog
This post describes an architecture for building a reliable, high-performance serverless streaming pipeline to manage large audit records between 256k and 6MB in size. Specifically, the article covers:
- Ingesting large messages with low latency using API Gateway and AWS Lambda
- Orchestrating message processing with AWS Step Functions
- Storing message data in Amazon S3 and metadata in Amazon DynamoDB
- Providing traceability through Amazon SNS notifications at different stages
- Ensuring high data durability by leveraging multiple services
- Conclusion and deployment instructions
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
Mar 4
2024
2024
Comparing design approaches for building serverless microservices
Mar 21
2024
2024
Build an end-to-end serverless streaming pipeline with Apache Kafka on Amazon MSK using Python
Mar 12
2024
2024
Empower Your WebXR App Using AWS Serverless Architecture
Mar 28
2024
2024
Introducing the AWS Message Processing Framework for .NET (Preview)
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.