Search Results for "workflow management"
Sort By:
Showing 59 open source projects for "workflow management"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
1
Covalent workflow
Pythonic tool for running machine-learning/high performance workflows
Covalent is a Pythonic workflow tool for computational scientists, AI/ML software engineers, and anyone who needs to run experiments on limited or expensive computing resources including quantum computers, HPC clusters, GPU arrays, and cloud services. Covalent enables a researcher to run computation tasks on an advanced hardware platform – such as a quantum computer or serverless HPC cluster – using a single line of code. Covalent overcomes computational and operational challenges inherent... -
2
Alova.js
Workflow-Streamlined next-generation request tools
...Send requests immediately by watching state changes, useful in tab switching and condition querying. Global interceptor that supports silent token refresh, as well as providing unified management of token-based login, logout, token assignment, and token refresh. -
3
Dolphin Scheduler
A distributed and extensible workflow scheduler platform
Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available `out of the box`. Dedicated to solving the complex task dependencies in data processing, making the scheduler system out of the box for data processing. Decentralized multi-master and multi-worker, HA is supported by itself, overload processing. All process... -
4
Luigi
Python module that helps you build complex pipelines of batch jobs
Luigi is a Python (3.6, 3.7, 3.8, 3.9 tested) package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more. The purpose of Luigi is to address all the plumbing typically associated with long-running batch processes. You want to chain many tasks, automate them, and failures will happen. These tasks can be anything, but are typically long running things like Hadoop jobs, dumping data to/from databases, running machine learning algorithms, or anything else. ... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
pointblank
Data quality assessment and metadata reporting for data frames
With the pointblank package it’s really easy to methodically validate your data whether in the form of data frames or as database tables. On top of the validation toolset, the package gives you the means to provide and keep up-to-date with the information that defines your tables. For table validation, the agent object works with a large collection of simple (yet powerful!) validation functions. We can enable much more sophisticated validation checks by using custom expressions, segmenting... -
6
Orange Data Mining
Orange: Interactive data analysis
Open source machine learning and data visualization. Build data analysis workflows visually, with a large, diverse toolbox. Perform simple data analysis with clever data visualization. Explore statistical distributions, box plots and scatter plots, or dive deeper with decision trees, hierarchical clustering, heatmaps, MDS and linear projections. Even your multidimensional data can become sensible in 2D, especially with clever attribute ranking and selections. Interactive data exploration for... -
7
targets
Function-oriented Make-like declarative workflows for R
The targets package is a pipeline / workflow management tool in R, designed to coordinate multi‐step computational workflows in data science / statistics. It tracks dependencies between “targets” (computational steps), skips steps whose upstream data or code hasn’t changed, supports parallel computation, branching (dynamic generation of sub‐targets), file format abstractions, and encourages reproducible and efficient analyses. -
8
Kestra
Kestra is an infinitely scalable orchestration and scheduling platform
Build reliable workflows, blazingly fast, deploy in just a few clicks. Kestra is an open-source, event-driven orchestrator that simplifies data operations and improves collaboration between engineers and business users. By bringing Infrastructure as Code best practices to data pipelines, Kestra allows you to build reliable workflows and manage them with confidence. Thanks to the declarative YAML interface for defining orchestration logic, everyone who benefits from analytics can participate... -
9
XState
State machines and statecharts for the modern web
JavaScript and TypeScript finite state machines and statecharts for the modern web. Statecharts are a formalism for modeling stateful, reactive systems. This is useful for declaratively describing the behavior of your application, from the individual components to the overall application logic. XState is a library for creating, interpreting, and executing finite state machines and statecharts, as well as managing invocations of those machines as actors. The following fundamental computer... -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
10
ClearML
Streamline your ML workflow
ClearML is an open source platform that automates and simplifies developing and managing machine learning solutions for thousands of data science teams all over the world. It is designed as an end-to-end MLOps suite allowing you to focus on developing your ML code & automation, while ClearML ensures your work is reproducible and scalable. The ClearML Python Package for integrating ClearML into your existing scripts by adding just two lines of code, and optionally extending your experiments... -
11
Registrator.jl
Julia package registration bot
Registrator is a GitHub app that automates the creation of registration pull requests for your Julia packages to the General registry. -
12
Distributions.jl
A Julia package for probability distributions and associated functions
A Julia package for probability distributions and associated functions. -
13
ClickVisual
Log analytic and data visualize platform built on clickhouse
ClickVisual is a lightweight browser-based logs analytics and logs search platform for ClickHouse. -
14
Modin
Scale your Pandas workflows by changing a single line of code
Scale your pandas workflow by changing a single line of code. Modin uses Ray, Dask or Unidist to provide an effortless way to speed up your pandas notebooks, scripts, and libraries. Unlike other distributed DataFrame libraries, Modin provides seamless integration and compatibility with existing pandas code. Even using the DataFrame constructor is identical. It is not necessary to know in advance the available hardware resources in order to use Modin. Additionally, it is not necessary to... -
15
janitor
Simple tools for data cleaning in R
janitor provides simple, convenient tools for data cleaning, formatting, and exploration in R. It is especially useful for cleaning messy data frames, removing duplicates, formatting column names, and producing frequency tables in a tidy workflow. -
16
FinEtools.jl
Finite Element tools in Julia
FinEtools is a package for basic operations on finite element meshes: Construction, modification, selection, and evaluation of quantities defined on a mesh. Utilities are provided for maintaining mesh-based data (fields), for defining normals and loads, for working with physical units and coordinate systems, and for integrating over finite element meshes. -
17
Airbyte
Data integration platform for ELT pipelines from APIs, databases
We believe that only an open-source solution to data movement can cover the long tail of data sources while empowering data engineers to customize existing connectors. Our ultimate vision is to help you move data from any source to any destination. Airbyte already provides the largest catalog of 300+ connectors for APIs, databases, data warehouses, and data lakes. Moving critical data with Airbyte is as easy and reliable as flipping on a switch. Our teams process more than 300 billion rows... -
18
ggstatsplot
Enhancing {ggplot2} plots with statistical analysis
{ggstatsplot} is an extension of {ggplot2} package for creating graphics with details from statistical tests included in the information-rich plots themselves. In a typical exploratory data analysis workflow, data visualization and statistical modeling are two different phases: visualization informs modeling, and modeling in its turn can suggest a different visualization method, and so on and so forth. Bayesian hypothesis-testing. The central idea of {ggstatsplot} is simple: combine these... -
19
Best-of Python
A ranked list of awesome Python open-source libraries
This curated list contains 390 awesome open-source projects with a total of 1.4M stars grouped into 28 categories. All projects are ranked by a project-quality score, which is calculated based on various metrics automatically collected from GitHub and different package managers. If you like to add or update projects, feel free to open an issue, submit a pull request, or directly edit the projects.yaml. Contributions are very welcome! Ranked list of awesome python libraries for web... -
20
Pyper
Concurrent Python made simple
Pyper is a Python-native orchestration and scheduling framework designed for modern data workflows, machine learning pipelines, and any task that benefits from a lightweight DAG-based execution engine. Unlike heavier platforms like Airflow, Pyper aims to remain lean, modular, and developer-friendly, embracing Pythonic conventions and minimizing boilerplate. It focuses on local development ergonomics and seamless transition to production environments, making it ideal for small teams and... -
21
beautiful-mermaid
Render Mermaid diagrams as beautiful SVGs or ASCII art
beautiful-mermaid is a styling and rendering toolkit built to produce visually enhanced diagrams from Mermaid syntax, aiming to bridge the gap between simple technical diagrams and rich, presentation-ready visualizations, all while preserving the lightweight text-to-diagram workflow that Mermaid offers. Instead of plain, utilitarian shapes and lines, Beautiful Mermaid applies themes, typography enhancements, color palettes, and layout optimizations so diagrams look polished and professional... -
22
clusterProfiler
A universal enrichment tool for interpreting omics data
clusterProfiler is an R/Bioconductor package that provides a unified workflow for functional enrichment analysis to interpret high-throughput omics results. It supports both over-representation analysis and gene set enrichment analysis, letting you work with unranked gene lists or ranked statistics from differential pipelines. The package connects to multiple knowledge bases—such as Gene Ontology, KEGG, Reactome, Disease Ontology, MeSH and others—through a consistent interface so you can... -
23
Population Shift Monitoring
Monitor the stability of a Pandas or Spark dataframe
popmon is a package that allows one to check the stability of a dataset. popmon works with both pandas and spark datasets. popmon creates histograms of features binned in time-slices, and compares the stability of the profiles and distributions of those histograms using statistical tests, both over time and with respect to a reference. It works with numerical, ordinal, categorical features, and the histograms can be higher-dimensional, e.g. it can also track correlations between any two... -
24
Ridgepole
Ridgepole is a tool to manage DB schema. It defines DB schema
...Dumps can be generated from an existing database, making it easy to adopt the tool on legacy systems and then evolve via code reviews. Used alongside application frameworks or standalone, Ridgepole offers a tidy workflow for versioning and promoting schema changes safely across development, staging, and production. -
25
awesome-single-cell
Community-curated list of software packages and data resources
Community-curated list of software packages and data resources for single-cell, including RNA-seq, ATAC-seq, etc. List of software packages (and the people developing these methods) for single-cell data analysis, including RNA-seq, ATAC-seq, etc. Rapid, accurate and memory-frugal preprocessing of single-cell and single-nucleus RNA-seq data. Find bimodal, unimodal, and multimodal features in your data. Ascend is an R package comprised of fast, streamlined analysis functions optimized to...
