Search Results for "topology%20optimization"

Broadway is a data processing library for Elixir designed to handle high-throughput, concurrent workloads with ease. It provides an abstraction for defining pipelines that consume data from sources like RabbitMQ, Kafka, Amazon SQS, or custom producers. Each pipeline is fault-tolerant and backpressure-aware, ensuring stable throughput even under load. The library integrates seamlessly with GenStage and OTP supervision trees, making it highly resilient in production. Developers can enrich...

Horde provides a distributed, fault-tolerant Registry and DynamicSupervisor for Elixir applications, letting you run and manage processes across clustered nodes as if they lived on a single machine. It relies on conflict-free replicated data types (CRDTs) to converge membership and process ownership without a central leader, so cluster topology can change freely as nodes join or leave. With Horde.Registry you register processes globally and look them up anywhere, while Horde.DynamicSupervisor starts and migrates children across nodes, rebalancing as capacity or health changes. Because everything runs under OTP supervision, failures are isolated and recoveries are automatic, even during network partitions or rolling deploys. ...

...It ships with multiple pluggable strategies—such as DNS polling, Kubernetes integration, EC2 metadata/tag discovery, UDP gossip, and EPMD helpers—so you can adapt clustering to your runtime environment. The library supervises a topology process that continually discovers peers and joins or prunes nodes as they become available or disappear. Its design embraces OTP principles: strategies implement a behavior, can be composed, and run under supervision for resilience. libcluster also emits events on nodeup/nodedown, allowing applications to react (for example, rebalance work or update registries) when membership changes. ...