Home icon

Real-time serverless data ingestion from your Kafka clusters into Amazon Timestream using Kafka Connect

Database Blog

This article explains how to stream events in real-time from Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters to Amazon Timestream tables using the Timestream Sink Connector for Apache Kafka.

Specifically, the article covers:

  • An overview of the solution architecture for ingesting data from MSK clusters to Timestream
  • Provisioning the required AWS resources (VPC, subnets, MSK cluster, EC2 instance, Timestream database and table, etc.) using a CloudFormation template
  • Configuring the EC2 instance as a Kafka producer client and publishing messages to an MSK topic using Apache JMeter
  • Deploying the Timestream Sink Connector in Amazon MSK Connect and configuring it to stream data from the MSK topic to the Timestream table
  • Verifying that the published messages are written as time-series data in the Timestream table
  • Troubleshooting steps and clean-up instructions
Go to article

The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.

Related articles

Mar 21
2024
Build an end-to-end serverless streaming pipeline with Apache Kafka on Amazon MSK using Python
Nov 21
2025
Improving throughput of serverless streaming workloads for Kafka
Feb 3
2026
Use Amazon MSK Connect and Iceberg Kafka Connect to build a real-time data lake
Mar 30
2026
Securely connect Kafka client applications to your Amazon MSK Serverless cluster from different VPCs and AWS accounts

The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.