Leveraging Open Source at Barclays to Enable Lambda Event Filtering with AWS Glue Schema Registry
Blog
This article describes how Barclays implemented AWS Glue Schema Registry integration with AWS Lambda event filtering by customizing an open source library.
- Barclays needed schema validation for event-driven systems processing millions of transactions daily
- AWS Lambda event filtering didn't support Glue Schema Registry serialized payloads (non-JSON format)
- Default implementation prepends schema version ID as bytes, breaking JSON structure and filtering
- Solution: Add schema version ID as JSON metadata node instead of byte prefix
- Custom serializer extends default to inject schemaVersionId into metadata during encoding
- Custom deserializer retrieves schemaVersionId from JSON to load producer schema for decoding
- Maintains JSON compliance, enables Lambda filtering, and reduces unnecessary invocations and costs
Barclays leveraged the open source Glue Schema Registry library to create custom serialization/deserialization wrappers, enabling schema validation while preserving Lambda event filtering capabilities.
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.