Search Results for "pipeline" - Page 5
Sort By:
Showing 679 open source projects for "pipeline"
View related business solutions-
Our Free Plans just got better! | Auth0 by OktaYou asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your secuirty. Auth0 now, thank yourself later.
-
Bright Data - All in One Platform for Proxies and Web ScrapingBright Data offers the highest quality proxies with automated session management, IP rotation, and advanced web unlocking technology. Enjoy reliable, fast performance with easy integration, a user-friendly dashboard, and enterprise-grade scaling. Powered by ethically-sourced residential IPs for seamless web scraping.
-
1
Conduit
Conduit streams data between data stores. Kafka Connect replacement
Conduit is a data streaming tool written in Go. It aims to provide the best user experience for building and running real-time data pipelines. Conduit comes with batteries included, it provides a UI, common connectors, processors and observability data out of the box. Sync data between your production systems using an extensible, event-first experience with minimal dependencies that fit within your existing workflow. Eliminate the multi-step process you go through today. Just download the... -
2
PipeRider
Code review for data in dbt
PipeRider automatically compares your data to highlight the difference in impacted downstream dbt models so you can merge your Pull Requests with confidence. PipeRider can profile your dbt models and obtain information such as basic data composition, quantiles, histograms, text length, top categories, and more. PipeRider can integrate with dbt metrics and present the time-series data of metrics in the report. PipeRider generates a static HTML report each time it runs, which can be viewed... -
3
Covalent workflow
Pythonic tool for running machine-learning/high performance workflows
Covalent is a Pythonic workflow tool for computational scientists, AI/ML software engineers, and anyone who needs to run experiments on limited or expensive computing resources including quantum computers, HPC clusters, GPU arrays, and cloud services. Covalent enables a researcher to run computation tasks on an advanced hardware platform – such as a quantum computer or serverless HPC cluster – using a single line of code. Covalent overcomes computational and operational challenges inherent... -
4
Elementary
Open-source data observability for analytics engineers
Elementary is an open-source data observability solution for data & analytics engineers. Monitor your dbt project and data in minutes, and be the first to know of data issues. Gain immediate visibility, detect data issues, send actionable alerts, and understand the impact and root cause. Generate a data observability report, host it or share with your team. Monitoring of data quality metrics, freshness, volume and schema changes, including anomaly detection. Elementary data monitors are... -
Shift, the browser that merges all of your web apps into one powerful window.Streamline everything you do online when you install Shift and access thousands of apps without leaving your browser. Connect all of your Gmail, Outlook, and Office 365 accounts and manage everything from one centralized window. Build out your Shift browser with apps that integrate seamlessly so you have ultra-fast access to all the tools you use to stream, shop, work, browse, and stay connected. Shift brings it all together.
-
5
BitSail
BitSail is a distributed high-performance data integration engine
BitSail is ByteDance's open source data integration engine which is based on distributed architecture and provides high performance. It supports data synchronization between multiple heterogeneous data sources, and provides global data integration solutions in batch, streaming, and incremental scenarios. At present, it serves almost all business lines in ByteDance, such as Douyin, Toutiao, etc., and synchronizes hundreds of trillions of data every day. BitSail has been widely used and... -
6
memphis
Next-Generation Event Processing Platform
Memphis enables building modern queue-based applications that require large volumes of streamed and enriched data, modern protocols, zero ops, up to x9 faster development, up to x46 fewer costs, and significantly lower dev time for data-oriented developers and data engineers. Queues and brokers are a mission-critical component in the modern application architecture and should be highly available and stable as possible. Provide great performance while maintaining efficient resource... -
7
Apache SeaTunnel
SeaTunnel is a distributed, high-performance data integration platform
SeaTunnel is a very easy-to-use ultra-high-performance distributed data integration platform that supports real-time synchronization of massive data. It can synchronize tens of billions of data stably and efficiently every day, and has been used in the production of nearly 100 companies. There are hundreds of commonly-used data sources of which versions are incompatible. With the emergence of new technologies, more data sources are appearing. It is difficult for users to find a tool that can... -
8
hindent
Haskell pretty printer
Haskell pretty printer. hindent is used in a pipeline style. The default indentation size is 2 spaces. Create a hindent.yaml file in your project directory or in your ~/ home directory. By default, hindent preserves the newline or lack of newline in your input. With force-trailing-newline, it will make sure there is always a trailing newline. hindent can be forced to insert a newline before specific operators and tokens with line-breaks. This is especially useful when utilizing libraries like... -
9
lakeFS
lakeFS - Git-like capabilities for your object storage
Increase data quality and reduce the painful cost of errors. Data engineering best practices using git-like operations on data. lakeFS is an open-source data version control for data lakes. It enables zero-copy Dev / Test isolated environments, continuous quality validation, atomic rollback on bad data, reproducibility, and more. Data is dynamic, it changes over time. Dealing with that without a data version control system is error-prone and labor-intensive. With lakeFS, your data lake is... -
Free and Open Source HR SoftwareGive your HR team the tools they need to streamline administrative tasks, support employees, and make informed decisions with the OrangeHRM free and open source HR software.
-
10
Raster Vision
Open source framework for deep learning satellite and aerial imagery
..., creating predictions, evaluating models, and bundling the model files and configuration for easy deployment. The input to a Raster Vision pipeline is a set of images and training data, optionally with Areas of Interest (AOIs) that describe where the images are labeled. The output of a Raster Vision pipeline is a model bundle that allows you to easily utilize models in various deployment scenarios. -
11
BentoML
Unified Model Serving Framework
BentoML simplifies ML model deployment and serves your models at a production scale. Support multiple ML frameworks natively: Tensorflow, PyTorch, XGBoost, Scikit-Learn and many more! Define custom serving pipeline with pre-processing, post-processing and ensemble models. Standard .bento format for packaging code, models and dependencies for easy versioning and deployment. Integrate with any training pipeline or ML experimentation platform. Parallelize compute-intense model inference workloads... -
12
AutoGluon
AutoGluon: AutoML for Image, Text, and Tabular Data
AutoGluon enables easy-to-use and easy-to-extend AutoML with a focus on automated stack ensembling, deep learning, and real-world applications spanning image, text, and tabular data. Intended for both ML beginners and experts, AutoGluon enables you to quickly prototype deep learning and classical ML solutions for your raw data with a few lines of code. Automatically utilize state-of-the-art techniques (where appropriate) without expert knowledge. Leverage automatic hyperparameter tuning,... -
13
StarRocks
StarRocks is a next-gen sub-second MPP database for full analytics
StarRocks is the next generation of real-time SQL engines for enterprise analytics. Real-time analytics is notoriously difficult. Complex data pipelines and de-normalized tables have always been a necessary evil. Processing any updates or deletes once data arrives has not been possible- until now. StarRocks solves these challenges and makes real-time analytics easy. Get amazing query performance on Star or Snowflake Schemas directly. From canceled orders to updated items, your analytics... -
14
Brighter
A framework for building messaging apps with .NET and C#
This project is a Command Processor & Dispatcher implementation with support for task queues that can be used as a lightweight library. It can be used for implementing Ports and Adapters and CQRS (PDF) architectural styles in .NET. It can also be used in microservices architectures for decoupled communication between the services. Brighter supports Task Queues. As such it can be used to improve performance by introducing concurrency using a queue, and/or as an integration strategy between... -
15
tRPC
End-to-end typesafe APIs made easy
... type safety & autocompletion are inferred from your API paths, their input data, & outputs. No code generation, run-time bloat, or build pipeline. Zero dependencies & a tiny client-side footprint. Easy to add to your existing brownfield project with adapters for Connect/Express/Next.js. Currently, GraphQL is the dominant way to implement typesafe APIs in TypeScript (and it's amazing!). -
16
Great Expectations
Always know what to expect from your data
Great Expectations helps data teams eliminate pipeline debt, through data testing, documentation, and profiling. Software developers have long known that testing and documentation are essential for managing complex codebases. Great Expectations brings the same confidence, integrity, and acceleration to data science and data engineering teams. Expectations are assertions for data. They are the workhorse abstraction in Great Expectations, covering all kinds of common data issues. Expectations... -
17
Scrapy-Redis
Redis-based components for Scrapy
You can start multiple spider instances that share a single redis queue. Best suitable for broad multi-domain crawls. Scraped items gets pushed into a redis queued meaning that you can start as many as needed post-processing processes sharing the items queue. Scheduler + Duplication Filter, Item Pipeline, Base Spiders. Default requests serializer is pickle, but it can be changed to any module with loads and dumps functions. Note that pickle is not compatible between python versions. Version 0.3... -
18
Stanza
Stanford NLP Python library for many human languages
Stanza is a collection of accurate and efficient tools for the linguistic analysis of many human languages. Starting from raw text to syntactic analysis and entity recognition, Stanza brings state-of-the-art NLP models to languages of your choosing. Stanza is a Python natural language analysis package. It contains tools, which can be used in a pipeline, to convert a string containing human language text into lists of sentences and words, to generate base forms of those words, their parts... -
19
Concourse
Concourse is a container-based continuous thing-doer written in Go
Built on the simple mechanics of resources, tasks, and jobs, Concourse presents a general approach to automation that makes it great for CI/CD. Concourse is designed to be expressive, versatile, and safe, remaining intuitive as the complexity of your project grows. A Concourse pipeline is like a distributed, continuous Makefile. Each job has a build plan declaring the job's input resources and what to run with them when they change. Your pipeline is then visualized in the web UI, taking only... -
20
Gitness
Gitness is an Open Source developer platform with Source Control
Gitness is an open source development platform packed with the power of code hosting and automated DevOps pipelines. Gitness represents a massive investment in the next generation of drones. Where Drone focuses on continuous integration, Gitness adds source code hosting, bringing code management and pipelines closer together. The goal is for Gitness to eventually be at full parity with Drone in terms of pipeline capabilities, allowing users to seamlessly migrate from Drone to Gitness. -
21
sense2vec
Contextually-keyed word vectors
sense2vec (Trask et. al, 2015) is a nice twist on word2vec that lets you learn more interesting and detailed word vectors. This library is a simple Python implementation for loading, querying and training sense2vec models. For more details, check out our blog post. To explore the semantic similarities across all Reddit comments of 2015 and 2019, see the interactive demo. -
22
Elyra
Elyra extends JupyterLab with an AI centric approach
Elyra is a set of AI-centric extensions to JupyterLab Notebooks. The Elyra Getting Started Guide includes more details on these features. A version-specific summary of new features is located on the releases page. -
23
Numaflow
Kubernetes-native platform to run massively parallel data/streaming
Numaflow is a Kubernetes-native tool for running massively parallel stream processing. A Numaflow Pipeline is implemented as a Kubernetes custom resource and consists of one or more source, data processing, and sink vertices. Numaflow installs in a few minutes and is easier and cheaper to use for simple data processing applications than a full-featured stream processing platform. -
24
PipeCD
The One CD for All {applications, platforms, operations}
A GitOps style continuous delivery platform that provides consistent deployment and operations experience for any applications. Automatically trigger a new deployment when a defined event has occurred (e.g. container image pushed, helm chart published, etc). Support multiple application kinds on multi-cloud including Kubernetes, Terraform, Cloud Run, AWS Lambda, Amazon ECS. Support multiple analysis providers including Prometheus, Datadog, Stackdriver, and more. Easy to operate... -
25
omegaml
MLOps simplified. From ML Pipeline ⇨ Data Product without the hassle
omega|ml is the innovative Python-native MLOps platform that provides a scalable development and runtime environment for your Data Products. Works from laptop to cloud.