Apache Airflow

Apache Airflow

The Apache Software Foundation
Flyte

Flyte

Union.ai
+
+

Related Products

  • ActiveBatch Workload Automation
    366 Ratings
    Visit Website
  • dbt
    239 Ratings
    Visit Website
  • JS7 JobScheduler
    1 Rating
    Visit Website
  • JAMS
    265 Ratings
    Visit Website
  • Stonebranch
    175 Ratings
    Visit Website
  • RunMyJobs by Redwood
    425 Ratings
    Visit Website
  • DataBuck
    6 Ratings
    Visit Website
  • Pipeliner CRM
    750 Ratings
    Visit Website
  • Teradata VantageCloud
    1,105 Ratings
    Visit Website
  • Wrike
    7,463 Ratings
    Visit Website

About

Airflow is a platform created by the community to programmatically author, schedule and monitor workflows. Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Airflow is ready to scale to infinity. Airflow pipelines are defined in Python, allowing for dynamic pipeline generation. This allows for writing code that instantiates pipelines dynamically. Easily define your own operators and extend libraries to fit the level of abstraction that suits your environment. Airflow pipelines are lean and explicit. Parametrization is built into its core using the powerful Jinja templating engine. No more command-line or XML black-magic! Use standard Python features to create your workflows, including date time formats for scheduling and loops to dynamically generate tasks. This allows you to maintain full flexibility when building your workflows.

About

The workflow automation platform for complex, mission-critical data and ML processes at scale. Flyte makes it easy to create concurrent, scalable, and maintainable workflows for machine learning and data processing. Flyte is used in production at Lyft, Spotify, Freenome, and others. At Lyft, Flyte has been serving production model training and data processing for over four years, becoming the de-facto platform for teams like pricing, locations, ETA, mapping, autonomous, and more. In fact, Flyte manages over 10,000 unique workflows at Lyft, totaling over 1,000,000 executions every month, 20 million tasks, and 40 million containers. Flyte has been battle-tested at Lyft, Spotify, Freenome, and others. It is entirely open-source with an Apache 2.0 license under the Linux Foundation with a cross-industry overseeing committee. Configuring machine learning and data workflows can get complex and error-prone with YAML.

Platforms Supported

Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook

Platforms Supported

Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook

Audience

Anyone who wants to to programmatically author, schedule and monitor workflows

Audience

Developers in need of a workflow automation platform to create concurrent, scalable, and maintainable workflows

Support

Phone Support
24/7 Live Support
Online

Support

Phone Support
24/7 Live Support
Online

API

Offers API

API

Offers API

Screenshots and Videos

Screenshots and Videos

Pricing

No information available.
Free Version
Free Trial

Pricing

Free
Free Version
Free Trial

Reviews/Ratings

Overall 0.0 / 5
ease 0.0 / 5
features 0.0 / 5
design 0.0 / 5
support 0.0 / 5

This software hasn't been reviewed yet. Be the first to provide a review:

Review this Software

Reviews/Ratings

Overall 0.0 / 5
ease 0.0 / 5
features 0.0 / 5
design 0.0 / 5
support 0.0 / 5

This software hasn't been reviewed yet. Be the first to provide a review:

Review this Software

Training

Documentation
Webinars
Live Online
In Person

Training

Documentation
Webinars
Live Online
In Person

Company Information

The Apache Software Foundation
Founded: 1999
United States
airflow.apache.org

Company Information

Union.ai
Founded: 2020
United States
flyte.org

Alternatives

JAMS

JAMS

JAMS Software

Alternatives

Metaflow

Metaflow

Netflix
Ray

Ray

Anyscale
dbt

dbt

dbt Labs
Amazon MWAA

Amazon MWAA

Amazon

Categories

Categories

Integrations

Great Expectations
Ray
AWS Batch
Apache Hive
CrateDB
Dask
Datakin
DoubleCloud
Google Cloud BigQuery
IRI FieldShield
Kubernetes
Metaphor
Microsoft Purview
Netdata
Polars
Sifflet
Stackable
Yandex Data Proc
dbt
pandas

Integrations

Great Expectations
Ray
AWS Batch
Apache Hive
CrateDB
Dask
Datakin
DoubleCloud
Google Cloud BigQuery
IRI FieldShield
Kubernetes
Metaphor
Microsoft Purview
Netdata
Polars
Sifflet
Stackable
Yandex Data Proc
dbt
pandas
Claim Apache Airflow and update features and information
Claim Apache Airflow and update features and information
Claim Flyte and update features and information
Claim Flyte and update features and information