Cribl Stream
Cribl Stream allows you to implement an observability pipeline which helps you parse, restructure, and enrich data in flight - before you pay to analyze it. Get the right data, where you want, in the formats you need. Route data to the best tool for the job - or all the tools for the job - by translating and formatting data into any tooling schema you require. Let different departments choose different analytics environments without having to deploy new agents or forwarders. As much as 50% of log and metric data goes unused – null fields, duplicate data, and fields that offer zero analytical value. With Cribl Stream, you can trim wasted data streams and analyze only what you need. Cribl Stream is the best way to get multiple data formats into the tools you trust for your Security and IT efforts. Use the Cribl Stream universal receiver to collect from any machine data source - and even to schedule batch collection from REST APIs, Kinesis Firehose, Raw HTTP, and Microsoft Office 365 APIs
Learn more
Amazon Managed Service for Apache Flink
Thousands of customers use Amazon Managed Service for Apache Flink to run stream processing applications. With Amazon Managed Service for Apache Flink, you can transform and analyze streaming data in real-time using Apache Flink and integrate applications with other AWS services. There are no servers and clusters to manage, and there is no computing and storage infrastructure to set up. You pay only for the resources you use. Build and run Apache Flink applications, without setting up infrastructure and managing resources and clusters. Process gigabytes of data per second with subsecond latencies and respond to events in real-time. Deploy highly available and durable applications with Multi-AZ deployments and APIs for application lifecycle management. Develop applications that transform and deliver data to Amazon Simple Storage Service (Amazon S3), Amazon OpenSearch Service, and more.
Learn more
Amazon Kinesis
Easily collect, process, and analyze video and data streams in real time. Amazon Kinesis makes it easy to collect, process, and analyze real-time, streaming data so you can get timely insights and react quickly to new information. Amazon Kinesis offers key capabilities to cost-effectively process streaming data at any scale, along with the flexibility to choose the tools that best suit the requirements of your application. With Amazon Kinesis, you can ingest real-time data such as video, audio, application logs, website clickstreams, and IoT telemetry data for machine learning, analytics, and other applications. Amazon Kinesis enables you to process and analyze data as it arrives and respond instantly instead of having to wait until all your data is collected before the processing can begin. Amazon Kinesis enables you to ingest, buffer, and process streaming data in real-time, so you can derive insights in seconds or minutes instead of hours or days.
Learn more
Apache Kafka
Apache Kafka® is an open-source, distributed streaming platform. Scale production clusters up to a thousand brokers, trillions of messages per day, petabytes of data, hundreds of thousands of partitions. Elastically expand and contract storage and processing. Stretch clusters efficiently over availability zones or connect separate clusters across geographic regions. Process streams of events with joins, aggregations, filters, transformations, and more, using event-time and exactly-once processing. Kafka’s out-of-the-box Connect interface integrates with hundreds of event sources and event sinks including Postgres, JMS, Elasticsearch, AWS S3, and more. Read, write, and process streams of events in a vast array of programming languages.
Learn more