
Amazon OpenSearch Ingestion now supports ingesting streaming data directly from Amazon MSK, enabling real-time data indexing and processing for observability use cases without third-party connectors.


<div><p>This article announces that Amazon OpenSearch Ingestion now supports ingesting streaming data from Amazon MSK, enabling seamless data indexing without third-party connectors.</p><ul><li>Ingest streaming data from Amazon MSK topics directly to OpenSearch</li><li>Perform near real-time aggregations, sampling, and anomaly detection</li><li>Transform data before writing to OpenSearch or Amazon S3</li><li>Configure consumer counts and fetch parameters per topic</li><li>Optionally use AWS Glue Schema Registry for dynamic schema specification</li><li>Data Prepper 2.4.0 adds S3 batch processing and Avro/Parquet codecs</li><li>Available in all AWS commercial regions supporting OpenSearch Ingestion</li></ul><p>This integration simplifies building efficient data pipelines for observability use cases by connecting Amazon MSK directly to OpenSearch Ingestion.</p></div>


Related articles