Publish and enrich real-time financial data feeds using Amazon MSK and Amazon Managed Service for Apache Flink

This article discusses a solution for publishing and enriching real-time financial data feeds using Amazon MSK (Managed Streaming for Apache Kafka) and Amazon Managed Service for Apache Flink.

Specifically, the article covers:

• An overview of the solution architecture, which includes an Apache Flink application for enriching a raw stock quotes feed, an MSK cluster for storing the raw and enriched feeds, an Amazon OpenSearch Service cluster for persistent data storage, and a consumer application for consuming the enriched feed.
• Steps for deploying the solution, including setting up the MSK cluster, multi-VPC connectivity, SASL/SCRAM authentication, and the consumer application.
• Running the applications and viewing the enriched data feed in OpenSearch Dashboards.
• Additional use cases for this architecture in the capital markets domain, such as real-time data visualization, stock implied volatility, technical indicator engine, market alert engine, market surveillance, and markets risk management.
• Conclusion highlighting the benefits of the solution and encouraging readers to try it out.