DownloadBenthos is a high performance and resilient stream processor, able to connect various sources and sinks in a range of brokering patterns and perform hydration, enrichments, transformations and filters on payloads. It comes with a powerful mapping language, is easy to deploy and monitor, and ready to drop into your pipeline either as a static binary, docker image, or serverless function, making it cloud native as heck. Delivery guarantees can be a dodgy subject. Benthos processes and acknowledges messages using an in-process transaction model with no need for any disk persisted state, so when connecting to at-least-once sources and sinks it's able to guarantee at-least-once delivery even in the event of crashes, disk corruption, or other unexpected server faults. This behaviour is the default and free of caveats, which also makes deploying and scaling Benthos much simpler.
Features
- Benthos exposes lots of metrics either to Statsd, Prometheus or for debugging purposes an HTTP endpoint that returns a JSON formatted object
- Benthos also emits tracing events to a tracer of your choice (currently only Jaeger is supported) which can be used to visualise the processors within a pipeline
- Benthos provides lots of tools for making configuration discovery, debugging and organisation easy
- Build with Go (any currently supported version)
- It's pretty easy to write your own custom plugins for Benthos in Go
- Benthos uses golangci-lint for linting
Project Samples
