Search Results for "kafka"
Sort By:
Showing 38 open source projects for "kafka"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Alpakka Kafka
Alpakka is a Reactive Enterprise Integration library for Java
...Akka Streams is a Reactive Stream and JDK 9+ java.util.concurrent.Flow-compliant implementation and therefore fully interoperable with other implementations. As Kafka’s client protocol negotiates the version to use with the Kafka broker, you may use a Kafka client version that is different than the Kafka broker’s version. -
2
kafka-go
Kafka library in Go
We rely on both Go and Kafka a lot at Segment. Unfortunately, the state of the Go client libraries for Kafka at the time of this writing was not ideal. sarama, which is by far the most popular but is quite difficult to work with. It is poorly documented, the API exposes low level concepts of the Kafka protocol, and it doesn't support recent Go features like contexts. -
3
Confluent's .NET Client for Apache Kafka
Confluent's Apache Kafka .NET client
confluent-kafka-dotnet is Confluent's .NET client for Apache Kafka and the Confluent Platform. Confluent-kafka-dotnet is a lightweight wrapper around librdkafka, a finely tuned C client. There are a lot of details to get right when writing an Apache Kafka client. We get them right in one place (librdkafka) and leverage this work across all of our clients (also confluent-kafka-python and confluent-kafka-go). -
4
Strimzi
Apache Kafka® running on Kubernetes
...You can expose Kafka outside Kubernetes using NodePort, Load balancer, Ingress and OpenShift Routes, depending on your needs, and these are easily secured using TLS. The Kube-native management of Kafka is not limited to the broker. You can also manage Kafka topics, users, Kafka MirrorMaker and Kafka Connect using Custom Resources. This means you can use your familiar Kubernetes processes and tooling to manage complete Kafka applications. -
AI-powered service management for IT and enterprise teamsGive your IT, operations, and business teams the ability to deliver exceptional services—without the complexity. Maximize operational efficiency with refreshingly simple, AI-powered Freshservice.
-
5
AKHQ
Kafka GUI for Apache Kafka to manage topics, topics data, etc.
Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more. Enabling your teams to search and explore data in a unified console, while supporting its administration and integration within your ecosystem. Multi-Cluster vision into a central console, available in Multi-Cloud environments. -
6
Node RDKafka
Node.js bindings for librdkafka
A high-performance Node.js client for Apache Kafka, built on top of librdkafka, providing bindings for efficient Kafka message processing. -
7
PHP RdKafka
Production-ready, stable Kafka client for PHP
php-rdkafka is a PHP extension that provides a binding to librdkafka, the C library for Apache Kafka. It enables PHP applications to produce and consume messages efficiently from Kafka topics, supporting both low-level and high-level APIs. The extension is ideal for integrating Kafka into high-performance, real-time systems built with PHP. -
8
Kafdrop
Kafka Web UI
Kafdrop is a web UI for viewing Kafka topics and browsing consumer groups. The tool displays information such as brokers, topics, partitions, and consumers, and lets you view messages. This project is a reboot of Kafdrop 2.x, dragged kicking and screaming into the world of Java 17+, Kafka 2.x, Helm and Kubernetes. It's a lightweight application that runs on Spring Boot and is dead-easy to configure, supporting SASL and TLS-secured brokers. -
9
AmazonMSK AWS Identity Access Management
Use AWS Identity and (IAM) to connect to Amazon Managed Streaming
...After you've downloaded the code from GitHub, you can build it using Gradle. An uber jar containing the library and all its relocated dependencies except the kafka client and slf4j-api can also be built. At runtime, the uber jar expects to find the kafka client library and the sl4j-api library on the classpath. -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
10
sarama
Sarama is a Go library for Apache Kafka 0.8, and up
Sarama is an MIT-licensed Go client library for Apache Kafka version 0.8 (and later). API documentation and examples are available via godoc. Sarama provides a “2 releases + 2 months” compatibility guarantee: we support the two latest stable releases of Kafka and Go, and we provide a two month grace period for older releases. This means we currently officially support Go 1.12 through 1.14, and Kafka 2.1 through 2.4, although older releases are still likely to work. ... -
11
librdkafka
The Apache Kafka C/C++ library
librdkafka is a C library implementation of the Apache Kafka protocol, providing Producer, Consumer and Admin clients. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer. librdkafka is licensed under the 2-clause BSD license. KAFKA is a registered trademark of The Apache Software Foundation and has been licensed for use by librdkafka. librdkafka has no affiliation with and is not endorsed by The Apache Software Foundation. ... -
12
Broadway
Concurrent and multi-stage data ingestion and data processing
Broadway is a data processing library for Elixir designed to handle high-throughput, concurrent workloads with ease. It provides an abstraction for defining pipelines that consume data from sources like RabbitMQ, Kafka, Amazon SQS, or custom producers. Each pipeline is fault-tolerant and backpressure-aware, ensuring stable throughput even under load. The library integrates seamlessly with GenStage and OTP supervision trees, making it highly resilient in production. Developers can enrich pipelines with batching, concurrency control, and metrics reporting, simplifying the management of complex data ingestion and processing systems. ... -
13
Redpanda
Redpanda is a streaming data platform for developers
Faster data. Fewer nodes. More time for coffee. Redpanda is a Kafka®-compatible streaming data platform that is proven to be 10x faster and 6x more cost-efficient. It is also JVM-free, ZooKeeper®-free, Jepsen-tested and source available. We are building a real-time streaming engine for modern applications - from the enterprise to the solo dev prototyping a react application on her laptop. We go beyond the Kafka protocol, into the future of streaming with inline WASM transforms and geo-replicated hierarchical storage. ... -
14
Zerocode
API automation and load testing framework
A community-developed, free, open source, microservices API automation and load testing framework built using JUnit core runners for Http REST, SOAP, Security, Database, Kafka and much more. Zerocode Open Source enables you to create, change, orchestrate and maintain your automated test cases declaratively with absolute ease. Zerocode makes it easy to create and maintain automated tests with absolute minimum overhead for REST,SOAP, Kafka Real Time Data Streams and much more. It has the best of best ideas and practices from the community to keep it super simple, and the adoption is rapidly growing among the developer/tester community. ... -
15
Nestjs boilerplate Microservice API
Nestjs boilerplate microservice api
A boilerplate for building microservices with NestJS, this provides a template with essential components for microservice architecture. -
16
Watermill
Building event-driven applications the easy way in Go
...It is intended for building event driven applications, enabling event sourcing, RPC over messages, sagas and basically whatever else comes to your mind. You can use conventional pub/sub implementations like Kafka or RabbitMQ, but also HTTP or MySQL binlog if that fits your use case. -
17
EaseProbe
A simple, standalone, and lightweight tool
...Connect to a Redis server and run the PING command. Memcache. Connect to a Memcache server and run the version command or validate a given key/value pair. MongoDB. Connect to a MongoDB server and perform a ping. Kafka. Connect to a Kafka server and perform a list of all topics. -
18
Apache Flink
Stream processing framework with powerful stream
...Developers program against high-level APIs—DataStream and Table/SQL—to express transformations, joins, and stateful patterns, while specialized libraries support CEP, machine learning workflows, and connectors. A rich connector ecosystem integrates with systems like Kafka, Kinesis, filesystems, JDBC sources/sinks, and object stores. Deployments span Kubernetes, YARN, Mesos, and standalone clusters, and operational features such as savepoints, state backends, and metrics make long-running jobs manageable in production. -
19
Bifrost
Middleware for production-oriented MySQL
Heterogeneous middleware for production-oriented MySQL, MariaDB, kafka synchronization to Redis, MongoDB, ClickHouse and other services. And this Bifrost can synchronize the full amount of data to multiple targets in real time. Support MySQL, MariaDB all storage types. Interface-based dynamic and flexible configuration of different data tables and target libraries. Multiple data sources, multiple target library support. -
20
Argo Events
Event-driven Automation Framework for Kubernetes
...It allows you to trigger 10 different actions (such as the creation of Kubernetes objects, invoking workflows or serverless workloads) on over 20 different events (such as webhook, S3 drop, cron schedule, messaging queues - e.g. Kafka, GCP PubSub, SNS, SQS). -
21
Zeebe
Distributed Workflow Engine for Microservices Orchestration
Automate processes at scale with unprecedented performance and resilience. Zeebe is the workflow and decision engine that powers Camunda Platform 8. Zeebe’s cloud-native design provides the performance, resilience, and security enterprises need to future-proof their process orchestration efforts. Zeebe distributes data across all brokers in a cluster with storage directly on the server filesystem. If one broker goes down, another can replace it with no data loss. This pre-configured... -
22
Apache Spark
A unified analytics engine for large-scale data processing
...The built-in MLlib library provides scalable machine learning algorithms, while GraphX enables graph computations integrated with data pipelines. Spark supports multiple languages—Scala, Java, Python, R—and connects with many storage systems like HDFS, S3, Cassandra, and streaming platforms like Kafka, making it a versatile choice for big data workloads in analytics, ETL, and data science. -
23
Pinot
Apache Pinot - A realtime distributed OLAP datastore
...Pinot is designed to answer OLAP queries with low latency on immutable data and mutable data(Upsert Support) Pluggable indexing technologies - Sorted Index, Bitmap Index, Inverted Index, StarTree Index, Bloom Filter, Range Index, Text Search Index(Lucence/FST), Json Index, Geospatial Index. Near Realtime ingestion with Apache Kafka, Apache Pulsar, Kinesissupports JSON, Avro, ProtoBuf, Thrift formats. Apache Pinot supports dimension lookup joins currently. Full SQL join support in Apache Pinot is coming soon. In the interim, full SQL joins can be performed by querying Pinot via the Trino or PrestoDB connector. Apache Pinot provides SQL interface for querying interactively, as well as programmatically via rest-api. -
24
PaaSTA
An open, distributed platform as a service
PaaSTA is a highly-available, distributed system for building, deploying, and running services using containers and Kubernetes. PaaSTA has been running production services at Yelp since 2016. It was originally designed to run on top of Apache Mesos but has subsequently been updated to use Kubernetes. Over time the features and functionality that PaaSTA provides have increased but the principal design remains the same. PaaSTA aims to take a declarative description of the services that teams... -
25
gruntwork.io website
The gruntwork.io website
...Use the Gruntwork Infrastructure as Code Library (IaC Library), a collection of reusable Terraform code that has been proven in production at hundreds of companies, to deploy the rest of your infrastructure, including EKS, ECS, EC2, MySQL, Postgres, Elasticsearch, Kafka, etc. Gruntwork Compliance modules are certified compliant by the Center for Internet Security (CIS), so all your infrastructure is compliant out-of-the-box and includes controls and guard rails to prevent or alert about non-compliant behavior. Gruntwork provides commercial maintenance for all the code in the IaC Library and Reference Architecture, and you can use Patcher (BETA) to automatically update your code to our latest releases.
