Apache AvroApache Software Foundation
|
Apache DataFusionApache Software Foundation
|
|||||
Related Products
|
||||||
About
Apache Avro™ is a data serialization system. Avro provides rich data structures, a compact, fast, binary data format, a container file, to store persistent data, remote procedure call (RPC). Also, it provides simple integration with dynamic languages. Code generation is not required to read or write data files nor to use or implement RPC protocols. Code generation as an optional optimization, only worth implementing for statically typed languages. Avro relies on schemas. When Avro data is read, the schema used when writing it is always present. This permits each datum to be written with no per-value overheads, making serialization both fast and small. This also facilitates use with dynamic, scripting languages, since data, together with its schema, is fully self-describing. When Avro data is stored in a file, its schema is stored with it, so that files may be processed later by any program. If the program reading the data expects a different schema this can be easily resolved.
|
About
Apache DataFusion is an extensible, high-performance query engine written in Rust that utilizes Apache Arrow as its in-memory format. Designed for developers building data-centric systems such as databases, data frames, machine learning, and streaming applications, DataFusion offers SQL and DataFrame APIs, a vectorized, multi-threaded, streaming execution engine, and support for partitioned data sources. It natively supports formats like CSV, Parquet, JSON, and Avro, and allows for seamless integration with object stores including AWS S3, Azure Blob Storage, and Google Cloud Storage. The engine features a comprehensive query planner, a state-of-the-art optimizer with capabilities like expression coercion and simplification, projection and filter pushdown, sort and distribution-aware optimizations, and automatic join reordering. DataFusion is highly customizable, enabling the addition of user-defined scalar, aggregate, and window functions, custom data sources, query languages, etc.
|
|||||
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
|||||
Audience
Developers looking for a data serialization system for rich data structures, data storage, and remote procedure calls
|
Audience
Professional developers and data engineers seeking a solution for building data-centric systems
|
|||||
Support
Phone Support
24/7 Live Support
Online
|
Support
Phone Support
24/7 Live Support
Online
|
|||||
API
Offers API
|
API
Offers API
|
|||||
Screenshots and Videos |
Screenshots and Videos |
|||||
Pricing
No information available.
Free Version
Free Trial
|
Pricing
Free
Free Version
Free Trial
|
|||||
Reviews/
|
Reviews/
|
|||||
Training
Documentation
Webinars
Live Online
In Person
|
Training
Documentation
Webinars
Live Online
In Person
|
|||||
Company InformationApache Software Foundation
Founded: 1999
United States
avro.apache.org
|
Company InformationApache Software Foundation
Founded: 2019
United States
datafusion.apache.org
|
|||||
Alternatives |
Alternatives |
|||||
|
||||||
|
|
|||||
|
|
|||||
Categories |
Categories |
|||||
Integrations
Amazon S3
Apache Avro
Apache Hive
Apache Parquet
Azure Blob Storage
Beats
C
Data Sentinel
Google Cloud Storage
Google Sheets
|
Integrations
Amazon S3
Apache Avro
Apache Hive
Apache Parquet
Azure Blob Storage
Beats
C
Data Sentinel
Google Cloud Storage
Google Sheets
|
|||||
|
|