Apache DataFusion

Apache DataFusion

Apache Software Foundation
+
+

Related Products

  • StarTree
    25 Ratings
    Visit Website
  • Cribl Stream
    8 Ratings
    Visit Website
  • Amazon EventBridge
    90 Ratings
    Visit Website
  • Tenzir
    3 Ratings
    Visit Website
  • AWS Glue
    674 Ratings
    Visit Website
  • Amazon EKS
    242 Ratings
    Visit Website
  • Amazon Bedrock
    72 Ratings
    Visit Website
  • CredentialStream
    161 Ratings
    Visit Website
  • TelemetryTV
    272 Ratings
    Visit Website
  • Fivetran
    726 Ratings
    Visit Website

About

Easily capture, transform, and load streaming data. Create a delivery stream, select your destination, and start streaming real-time data with just a few clicks. Automatically provision and scale compute, memory, and network resources without ongoing administration. Transform raw streaming data into formats like Apache Parquet, and dynamically partition streaming data without building your own processing pipelines. Amazon Data Firehose provides the easiest way to acquire, transform, and deliver data streams within seconds to data lakes, data warehouses, and analytics services. To use Amazon Data Firehose, you set up a stream with a source, destination, and required transformations. Amazon Data Firehose continuously processes the stream, automatically scales based on the amount of data available, and delivers it within seconds. Select the source for your data stream or write data using the Firehose Direct PUT API.

About

Apache DataFusion is an extensible, high-performance query engine written in Rust that utilizes Apache Arrow as its in-memory format. Designed for developers building data-centric systems such as databases, data frames, machine learning, and streaming applications, DataFusion offers SQL and DataFrame APIs, a vectorized, multi-threaded, streaming execution engine, and support for partitioned data sources. It natively supports formats like CSV, Parquet, JSON, and Avro, and allows for seamless integration with object stores including AWS S3, Azure Blob Storage, and Google Cloud Storage. The engine features a comprehensive query planner, a state-of-the-art optimizer with capabilities like expression coercion and simplification, projection and filter pushdown, sort and distribution-aware optimizations, and automatic join reordering. DataFusion is highly customizable, enabling the addition of user-defined scalar, aggregate, and window functions, custom data sources, query languages, etc.

Platforms Supported

Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook

Platforms Supported

Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook

Audience

Companies in search of a solution to load real-time streams into data lakes, warehouses, and analytics services

Audience

Professional developers and data engineers seeking a solution for building data-centric systems

Support

Phone Support
24/7 Live Support
Online

Support

Phone Support
24/7 Live Support
Online

API

Offers API

API

Offers API

Screenshots and Videos

Screenshots and Videos

Pricing

$0.075 per month
Free Version
Free Trial

Pricing

Free
Free Version
Free Trial

Reviews/Ratings

Overall 0.0 / 5
ease 0.0 / 5
features 0.0 / 5
design 0.0 / 5
support 0.0 / 5

This software hasn't been reviewed yet. Be the first to provide a review:

Review this Software

Reviews/Ratings

Overall 0.0 / 5
ease 0.0 / 5
features 0.0 / 5
design 0.0 / 5
support 0.0 / 5

This software hasn't been reviewed yet. Be the first to provide a review:

Review this Software

Training

Documentation
Webinars
Live Online
In Person

Training

Documentation
Webinars
Live Online
In Person

Company Information

Amazon
Founded: 1994
United States
aws.amazon.com/firehose/

Company Information

Apache Software Foundation
Founded: 2019
United States
datafusion.apache.org

Alternatives

Alternatives

Apache Doris

Apache Doris

The Apache Software Foundation
Apache Spark

Apache Spark

Apache Software Foundation
BigLake

BigLake

Google

Categories

Categories

Integrations

Apache Parquet
AWS Network Firewall
AWS WAF
Amazon CloudWatch
Amazon Kinesis
Amazon MSK
Amazon OpenSearch Service
Amazon Redshift
Amazon Simple Notification Service (SNS)
Amazon Web Services (AWS)
Apache Arrow
Apache Avro
Apache Iceberg
Google Cloud Storage
JSON
Microsoft Excel
Orchestra
Python
Rust
Splunk Cloud Platform

Integrations

Apache Parquet
AWS Network Firewall
AWS WAF
Amazon CloudWatch
Amazon Kinesis
Amazon MSK
Amazon OpenSearch Service
Amazon Redshift
Amazon Simple Notification Service (SNS)
Amazon Web Services (AWS)
Apache Arrow
Apache Avro
Apache Iceberg
Google Cloud Storage
JSON
Microsoft Excel
Orchestra
Python
Rust
Splunk Cloud Platform
Claim Amazon Data Firehose and update features and information
Claim Amazon Data Firehose and update features and information
Claim Apache DataFusion and update features and information
Claim Apache DataFusion and update features and information