BitSail

BitSail is ByteDance's open source data integration engine which is based on distributed architecture and provides high performance. It supports data synchronization between multiple heterogeneous data sources, and provides global data integration solutions in batch, streaming, and incremental scenarios. At present, it serves almost all business lines in ByteDance, such as Douyin, Toutiao, etc., and synchronizes hundreds of trillions of data every day. BitSail has been widely used and supports hundreds of trillions of large traffic. At the same time, it has been verified in various scenarios such as the cloud native environment of the volcano engine and the on-premises private cloud environment.

Features

Global Data Integration, covering batch, streaming and incremental scenarios
Distributed and cloud-native architecture, supporting horizontal scaling
High maturity in terms of accuracy, stability and performance
Rich basic functions, such as type conversion, dirty data processing, flow control, data lake integration, automatic parallelism calculation , etc.
Task running status monitoring, such as traffic, QPS, dirty data, latency, etc.
Mass data synchronization in heterogeneous data sources

Project Samples

Project Activity

See All Activity >

License

Apache License V2.0

Follow BitSail

BitSail Web Site

Other Useful Business Software

MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.

Start Free

Rate This Project

User Reviews

Be the first to post a review of BitSail!

Additional Project Details

Programming Language

Java

Related Categories

Java Data Pipeline Tool

Registered

2023-06-12

Similar Business Software

dbt

dbt helps data teams transform raw data into trusted, analysis-ready datasets faster. With dbt, data analysts and data engineers can collaborate on version-controlled SQL models, enforce testing and documentation standards, lean on detailed metadata to troubleshoot and optimize pipelines, and...

See Software
Hevo

Hevo Data is a no-code, bi-directional data pipeline platform specially built for modern ETL, ELT, and Reverse ETL Needs. It helps data teams streamline and automate org-wide data flows that result in a saving of ~10 hours of engineering time/week and 10x faster reporting, analytics, and...

See Software
DataBuck

DataBuck is an AI-powered data validation platform that automates risk detection across dynamic, high-volume, and evolving data environments. DataBuck empowers your teams to: ✅ Enhance trust in analytics and reports, ensuring they are built on accurate and reliable data. ✅ Reduce maintenance...

See Software
Cribl Stream

Cribl Stream allows you to implement an observability pipeline which helps you parse, restructure, and enrich data in flight - before you pay to analyze it. Get the right data, where you want, in the formats you need. Route data to the best tool for the job - or all the tools for the job - by...

See Software
OpenSnowcat

OpenSnowcat is an open source fork of Snowplow under the Apache 2.0 License that delivers a full event data pipeline for collection, enrichment, routing, and loading, remaining fully compatible with Snowplow and Segment SDKs. It provides an end-to-end solution to collect behavioral data from web...

See Software
Airbyte

Airbyte is an open-source data integration platform designed to help businesses synchronize data from various sources to their data warehouses, lakes, or databases. The platform provides over 550 pre-built connectors and enables users to easily create custom connectors using low-code or no-code...

See Software