Search Results for "kafka"
Sort By:
Showing 85 open source projects for "kafka"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
1
Alpakka Kafka
Alpakka is a Reactive Enterprise Integration library for Java
...Akka Streams is a Reactive Stream and JDK 9+ java.util.concurrent.Flow-compliant implementation and therefore fully interoperable with other implementations. As Kafka’s client protocol negotiates the version to use with the Kafka broker, you may use a Kafka client version that is different than the Kafka broker’s version. -
2
kafka-go
Kafka library in Go
We rely on both Go and Kafka a lot at Segment. Unfortunately, the state of the Go client libraries for Kafka at the time of this writing was not ideal. sarama, which is by far the most popular but is quite difficult to work with. It is poorly documented, the API exposes low level concepts of the Kafka protocol, and it doesn't support recent Go features like contexts. -
3
AKHQ
Kafka GUI for Apache Kafka to manage topics, topics data, etc.
Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more. Enabling your teams to search and explore data in a unified console, while supporting its administration and integration within your ecosystem. Multi-Cluster vision into a central console, available in Multi-Cloud environments. -
4
Strimzi
Apache Kafka® running on Kubernetes
...You can expose Kafka outside Kubernetes using NodePort, Load balancer, Ingress and OpenShift Routes, depending on your needs, and these are easily secured using TLS. The Kube-native management of Kafka is not limited to the broker. You can also manage Kafka topics, users, Kafka MirrorMaker and Kafka Connect using Custom Resources. This means you can use your familiar Kubernetes processes and tooling to manage complete Kafka applications. -
Go From AI Idea to AI App FastAccess Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
-
5
Confluent's .NET Client for Apache Kafka
Confluent's Apache Kafka .NET client
confluent-kafka-dotnet is Confluent's .NET client for Apache Kafka and the Confluent Platform. Confluent-kafka-dotnet is a lightweight wrapper around librdkafka, a finely tuned C client. There are a lot of details to get right when writing an Apache Kafka client. We get them right in one place (librdkafka) and leverage this work across all of our clients (also confluent-kafka-python and confluent-kafka-go). -
6
Node RDKafka
Node.js bindings for librdkafka
A high-performance Node.js client for Apache Kafka, built on top of librdkafka, providing bindings for efficient Kafka message processing. -
7
Kafdrop
Kafka Web UI
Kafdrop is a web UI for viewing Kafka topics and browsing consumer groups. The tool displays information such as brokers, topics, partitions, and consumers, and lets you view messages. This project is a reboot of Kafdrop 2.x, dragged kicking and screaming into the world of Java 17+, Kafka 2.x, Helm and Kubernetes. It's a lightweight application that runs on Spring Boot and is dead-easy to configure, supporting SASL and TLS-secured brokers. -
8
PHP RdKafka
Production-ready, stable Kafka client for PHP
php-rdkafka is a PHP extension that provides a binding to librdkafka, the C library for Apache Kafka. It enables PHP applications to produce and consume messages efficiently from Kafka topics, supporting both low-level and high-level APIs. The extension is ideal for integrating Kafka into high-performance, real-time systems built with PHP. -
9
AmazonMSK AWS Identity Access Management
Use AWS Identity and (IAM) to connect to Amazon Managed Streaming
...After you've downloaded the code from GitHub, you can build it using Gradle. An uber jar containing the library and all its relocated dependencies except the kafka client and slf4j-api can also be built. At runtime, the uber jar expects to find the kafka client library and the sl4j-api library on the classpath. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
EFAK-AI
A AI-Driven, Distributed and high-performance monitoring system
EFAK (Eagle For Apache Kafka) is an open-source monitoring and management platform designed to provide comprehensive visibility and operational control over Apache Kafka clusters through a unified web interface. The project focuses on simplifying Kafka administration by offering real-time insights into cluster health, performance metrics, and consumer activity, allowing engineers to quickly diagnose issues and optimize system behavior. -
11
ThingsBoard Message Queue (TBMQ)
Open-source, scalable, and fault-tolerant MQTT broker
TBMQ is a lightweight message broker built to support ThingsBoard's IoT platform, focusing on telemetry data streaming and device communication. It uses Kafka-compatible APIs and is optimized for high-throughput messaging, device scalability, and low-latency delivery. TBMQ is ideal for IoT backends needing MQTT or Kafka-style pub/sub infrastructure. -
12
sarama
Sarama is a Go library for Apache Kafka 0.8, and up
Sarama is an MIT-licensed Go client library for Apache Kafka version 0.8 (and later). API documentation and examples are available via godoc. Sarama provides a “2 releases + 2 months” compatibility guarantee: we support the two latest stable releases of Kafka and Go, and we provide a two month grace period for older releases. This means we currently officially support Go 1.12 through 1.14, and Kafka 2.1 through 2.4, although older releases are still likely to work. ... -
13
Redpanda Console
Redpanda Console is a developer-friendly UI for managing your workload
...Redpanda is a Kafka®-compatible streaming data platform that is proven to be 10x faster and 6x lower in total costs. It is also JVM-free, ZooKeeper®-free, Jepsen-tested, and source available. A single binary with built-in everything. No ZooKeeper®. No JVM. Deploys in minutes. Spins up in milliseconds. Plays nice with Kubernetes and microservices. Works natively with your Kafka tools. -
14
librdkafka
The Apache Kafka C/C++ library
librdkafka is a C library implementation of the Apache Kafka protocol, providing Producer, Consumer and Admin clients. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer. librdkafka is licensed under the 2-clause BSD license. KAFKA is a registered trademark of The Apache Software Foundation and has been licensed for use by librdkafka. librdkafka has no affiliation with and is not endorsed by The Apache Software Foundation. ... -
15
Canal
MySQL binlog
Canal is an open-source project developed by Alibaba that simulates MySQL slave functionality to parse MySQL binlog files. It enables real-time data synchronization and change data capture (CDC) between MySQL and other systems such as Elasticsearch, Kafka, or HBase. Canal is widely used for data integration, replication, and monitoring across distributed systems, offering high performance and low-latency log parsing. -
16
Bytewax
Python Stream Processing
...You can use Bytewax for a variety of workloads from moving data à la Kafka Connect style all the way to advanced online machine learning workloads. Bytewax is not limited to streaming applications but excels anywhere that data can be distributed at the input and output. -
17
Redpanda
Redpanda is a streaming data platform for developers
Faster data. Fewer nodes. More time for coffee. Redpanda is a Kafka®-compatible streaming data platform that is proven to be 10x faster and 6x more cost-efficient. It is also JVM-free, ZooKeeper®-free, Jepsen-tested and source available. We are building a real-time streaming engine for modern applications - from the enterprise to the solo dev prototyping a react application on her laptop. We go beyond the Kafka protocol, into the future of streaming with inline WASM transforms and geo-replicated hierarchical storage. ... -
18
Broadway
Concurrent and multi-stage data ingestion and data processing
Broadway is a data processing library for Elixir designed to handle high-throughput, concurrent workloads with ease. It provides an abstraction for defining pipelines that consume data from sources like RabbitMQ, Kafka, Amazon SQS, or custom producers. Each pipeline is fault-tolerant and backpressure-aware, ensuring stable throughput even under load. The library integrates seamlessly with GenStage and OTP supervision trees, making it highly resilient in production. Developers can enrich pipelines with batching, concurrency control, and metrics reporting, simplifying the management of complex data ingestion and processing systems. ... -
19
Liftbridge
Lightweight, fault-tolerant message streams
Lightweight, fault-tolerant message streams. Extend NATS with a Kafka-like durable pub/sub log API. Use Liftbridge as a simpler and lighter alternative to systems like Kafka and Pulsar or to add streaming semantics to an existing NATS deployment. Stream replication provides high availability and durability of messages. Clustering and partitioning provide horizontal scalability for streams and their consumers. -
20
Zerocode
API automation and load testing framework
A community-developed, free, open source, microservices API automation and load testing framework built using JUnit core runners for Http REST, SOAP, Security, Database, Kafka and much more. Zerocode Open Source enables you to create, change, orchestrate and maintain your automated test cases declaratively with absolute ease. Zerocode makes it easy to create and maintain automated tests with absolute minimum overhead for REST,SOAP, Kafka Real Time Data Streams and much more. It has the best of best ideas and practices from the community to keep it super simple, and the adoption is rapidly growing among the developer/tester community. ... -
21
Pholcus
Distributed high-concurrency crawler software written in pure golang
...It supports three operating modes of stand-alone, server and client, and has three operating interfaces, Web, GUI, and command line; simple and flexible rules, concurrent batch tasks, and rich output methods (mysql/mongodb/kafka/csv/excel, etc.); In addition, it also supports horizontal and vertical grabbing modes, and a series of advanced functions such as simulated login and task suspension and cancellation. This software is only used for academic research, users need to abide by the relevant laws and regulations of their location, please do not use it for illegal purposes! ... -
22
Nestjs boilerplate Microservice API
Nestjs boilerplate microservice api
A boilerplate for building microservices with NestJS, this provides a template with essential components for microservice architecture. -
23
EaseProbe
A simple, standalone, and lightweight tool
...Connect to a Redis server and run the PING command. Memcache. Connect to a Memcache server and run the version command or validate a given key/value pair. MongoDB. Connect to a MongoDB server and perform a ping. Kafka. Connect to a Kafka server and perform a list of all topics. -
24
trench
Open-Source Analytics Infrastructure
Trench is an open-source analytics infrastructure designed for tracking events and performing real-time analysis of application data at scale. The system is built on top of high-performance data technologies including Apache Kafka and ClickHouse, which allows it to ingest and process very large volumes of events while maintaining fast query performance. It was originally developed to solve scaling challenges in product analytics systems where traditional relational databases become inefficient as event tables grow. The platform enables developers to collect events such as page views, user actions, and behavioral metrics while storing them in a column-oriented analytics database optimized for time-series workloads. ... -
25
Watermill
Building event-driven applications the easy way in Go
...It is intended for building event driven applications, enabling event sourcing, RPC over messages, sagas and basically whatever else comes to your mind. You can use conventional pub/sub implementations like Kafka or RabbitMQ, but also HTTP or MySQL binlog if that fits your use case.
