Home icon

Introducing the new Amazon Kinesis source connector for Apache Flink

Big Data Blog

This article introduces the new Amazon Kinesis source connector for Apache Flink, released in version 5.0.0. Key highlights include:

  • Introduces a new source connector for reading from Kinesis Data Streams
  • Compatible with Flink 2.x and uses the new Source interface
  • Supports stream ARN instead of stream name for cross-Region and cross-account access
  • Provides improved ordering guarantees during stream resharding
  • Uses a new default shard assigner for more uniform data distribution
  • Significantly reduced JAR file size (from 60 MB to 200 KB)
  • Based on AWS SDK for Java 2.x

Migration requires careful planning, especially for existing applications, with different approaches needed for DataStream API and Table/SQL applications. The new connector prepares Flink applications for future versions and improves overall streaming performance.

Go to article

The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.

Related articles

Nov 22
2024
Amazon Managed Service for Apache Flink releases a new Amazon Kinesis Data Streams connector
Nov 20
2020
Amazon Kinesis Data Analytics now supports the Apache Flink Dashboard
Nov 12
2024
Amazon Managed Service for Apache Flink now supports Amazon DynamoDB Streams as a source
Apr 30
2021
Amazon Kinesis Data Analytics for Apache Flink introduces custom maintenance windows in preview

The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.