Build Spark Structured Streaming applications with the open source connector for Amazon Kinesis Data Streams
Big Data Blog
This article discusses the open source Amazon Kinesis Data Streams Connector for Spark Structured Streaming, which enables building high-throughput streaming applications with Spark Structured Streaming.
Specifically, the article covers:
- Introduction to the Kinesis Data Streams connector for Spark Structured Streaming
- How the connector works internally
- Solution overview and architecture
- Prerequisites for getting started
- Steps to create a Spark Structured Streaming application using the connector
- Deploying the Kinesis Data Generator to stream sample data
- Cleaning up resources after the deployment
The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.
Related articles
May 24
2024
2024
Introducing the Amazon Kinesis Data Streams Apache Spark Structured Streaming Connector for Amazon EMR
Sep 10
2025
2025
Build a streaming data mesh using Amazon Kinesis Data Streams
Nov 22
2024
2024
Amazon Managed Service for Apache Flink releases a new Amazon Kinesis Data Streams connector
Jan 17
2023
2023
Build highly available streams with Amazon Kinesis Data Streams
The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.