Search Results for "airflow"

Apache Airflow is a community-created platform for programmatically authoring, scheduling, and monitoring workflows. It’s much easier to do all these things when workloads are defined as code. They become more versionable, testable, maintainable and collaborative. With Airflow you can author workflows as directed acyclic graphs (DAGs) of tasks. This means that your tasks are executed on an array of workers while following the specified dependencies. And with rich command line utilities...

This repository provides a command-line interface (CLI) utility that replicates an Amazon Managed Workflows for Apache Airflow (MWAA) environment locally. The CLI builds a Docker container image locally that’s similar to an MWAA production image. This allows you to run a local Apache Airflow environment to develop and test DAGs, custom plugins, and dependencies before deploying to MWAA. To stop the local environment, Ctrl+C on the terminal and wait till the local runner and the postgres...

cronsun is a distributed cron-style job system. It's similar with crontab on stand-alone *nix. The goal of this project is to make it much easier to manage jobs on lots of machines and provides high availability. cronsun is different from Azkaban, Chronos, Airflow. cronsun has been tested in production for years on hundreds of servers. Although the current version is not released as an stable version, but we think it is completely available for the production environment. We encourage you...