Open Source Linux ETL Tools
Sort By:
ETL Tools for Linux
View 25 business solutionsBrowse free open source ETL tools and projects for Linux below. Use the toggles on the left to filter open source ETL tools by OS, license, language, programming language, and project status.
-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
Grafana: The open and composable observability platformGrafana is the open source analytics & monitoring solution for every database.
-
1
Pentaho
Pentaho offers comprehensive data integration and analytics platform.
Pentaho couples data integration with business analytics in a modern platform to easily access, visualize and explore data that impacts business results. Use it as a full suite or as individual components that are accessible on-premise, in the cloud, or on-the-go (mobile). Pentaho enables IT and developers to access and integrate data from any source and deliver it to your applications all from within an intuitive and easy to use graphical tool. The Pentaho Enterprise Edition Free Trial can be obtained from https://pentaho.com/download/ -
2
HPCC Systems
End-to-end big data in a massively scalable supercomputing platform.
HPCC Systems® (www.hpccsystems.com) from LexisNexis® Risk Solutions is a proven, open source solution for Big Data insights that can be implemented by businesses of all sizes. With HPCC Systems, developers can design applications with Big Data at their core, enabling businesses to better analyze and understand data at scale, improving business time to results and decisions. HPCC Systems offers a consistent data-centric programming language, two processing platforms and a single, complete end-to-end architecture for efficient processing. Read our blog (http://hpccsystems.com/blog ), or connect with us on Twitter (@hpccsystems), Facebook (https://www.facebook.com/hpccsystems ) and LinkedIn (http://www.linkedin.com/company/hpcc-systems) HPCC Systems is available on AWS & can be configured through the Instant Cloud Solution. -
3
Logstash
Centralize, transform and stash your data
Logstash is a server-side data processing pipeline that dynamically ingests data from numerous sources, transforms it, and ships it to your favorite “stash” regardless of format or complexity. It supports and ingests data of all shapes, sizes and sources, dynamically transforms and prepares this data, and transports it to the output of your choice. Logstash is extensible, with over 200 plugins available to let you create and configure your pipeline how you choose. -
4
Datapipe
Real-time, incremental ETL library for ML with record-level depend
Datapipe is a real-time, incremental ETL library for Python with record-level dependency tracking. Datapipe is designed to streamline the creation of data processing pipelines. It excels in scenarios where data is continuously changing, requiring pipelines to adapt and process only the modified data efficiently. This library tracks dependencies for each record in the pipeline, ensuring minimal and efficient data processing. -
Total Network Visibility for Network Engineers and IT ManagersThis means every device on your network, and every interface on every device is automatically analyzed for performance, errors, QoS, and configuration.
-
5
MyDBF2MySQL
Extract, transform, and load DBF into MySQL
This is an ETL software which loads data from DBF/XBase files into MySQL. This utility has command line interface, designed to work without user interaction. -
6
Transporter
Sync data between persistence engines, like ETL only not stodgy
Compose Transporter helps with database transformations from one store to another. It can also sync from one to another or several stores. This version officially only supports the mongodb and postgresql adaptors. Support for other DBs will be added later on. Other adaptors may or may not work. You're encouraged to still use v0.5.2 for non mongo/postgres migrations. Transporter allows the user to configure a number of data adaptors as sources or sinks. These can be databases, files or other resources. Data is read from the sources, converted into a message format, and then send down to the sink where the message is converted into a writable format for its destination. The user can also create data transformations in JavaScript which can sit between the source and sink and manipulate or filter the message flow. Adaptors may be able to track changes as they happen in source data. This "tail" capability allows a Transporter to stay running and keep the sinks in sync. -
7
SQL*Plus Commander
Text-based user interface to query data on Oracle DB in a smart way
SQL*Plus Commander is Text-based user interface (TUI) / framework to query data on Oracle DB in a smart way. It consists in a fully customizable script shell for bash and ksh. It executes custom queries or procedures on DB with SQLPlus for Oracle. The results of queries can be browsed in a colorful text interface resulting data from a query can be selected and passed dinamically as parameters for others queries or procedures It may be useful for people who runs frequently a limited number of query and uses the results as parameters for other queries. suggested for DBA activities, log tables browsing. downloaded version contains a demo with HR data model from oracle.com Try it and let me know if you find it useful any idea or suggestion will be appreciated -
8
Talend Spatial Module (aka Spatial Data Integrator or SDI) is an ETL tool for geospatial. Based on Talend Open Studio, input, output and transform geocomponents are available. IO components read/write GIS formats(eg.PostGIS, GeoRSS). Transformers all
-
9
CloverDX
Design, automate, operate and publish data pipelines at scale
Please, visit www.cloverdx.com for latest product versions. Data integration platform; can be used to transform/map/manipulate data in batch and near-realtime modes. Suppors various input/output formats (CSV,FIXLEN,Excel,XML,JSON,Parquet, Avro,EDI/X12,HL7,COBOL,LOTUS, etc.). Connects to RDBMS/JMS/Kafka/SOAP/Rest/LDAP/S3/HTTP/FTP/ZIP/TAR. CloverDX offers 100+ specialized components which can be further extended by creation of "macros" - subgraphs - and libraries, shareable with 3rd parties. Simple data manipulation jobs can be created visually. More complex business logic can be implemented using Clover's domain-specific-language CTL, in Java or languages like Python or JavaScript. Through its DataServices functionality, it allows to quickly turn data pipelines into REST API endpoints. The platform allows to easily scale your data job across multiple cores or nodes/machines. Supports Docker/Kubernetes deployments and offers AWS/Azure images in their respective marketplace -
D&B Hoovers is Your Sales AcceleratorSpeed up sales prospecting with the rich audience targeting capabilities of D&B Hoovers so you can spend more sales time closing.
-
10
GeoKettle is a powerful, metadata-driven spatial ETL (Extract, Transform and Load) tool dedicated to the integration of different data sources for building and updating geospatial databases, data warehouses and services.
-
11
PDI Data Vault framework
Data Vault loading automation using Pentaho Data Integration.
A metadata driven 'tool' to automate loading a designed Data Vault. It consists of a set of Pentaho Data Integration and database objects. Thel Virtual Machine (VMware) is a 64 bit Ubuntu Server 14.04, with MySQL (Percona Server) and PostgreSQL 9.4 as the database flavours and PDI version 5.2 CE. NB: Directory version_2.4 contains the most recent Virtual Machine. The readme.txt contains info about that VM. -
12
Jaspersoft ETL is a data integration platform providing high performance data extract-transform-load (ETL) capabilities. Jaspersoft ETL is appropriate for all analytic and operational data integration needs. Activity on this project is located at jas
-
13
Metl ETL Data Integration
Simple message-based, web-based ETL integration
Metl is a simple, web-based ETL tool that allows for data integrations including database, files, messaging, and web services. Supports RDBMS, SOAP, HTTP, FTP, SFTP, XML, FIXLEN, CSV, JSON, ZIP, and more. Metl implements scheduled integration tasks without the need for custom coding or heavy infrastructure. It can be deployed in the cloud or in an internal data center, and it was built to allow developers to extend it with custom components. -
14
apache spark data pipeline osDQ
osDQ dedicated to create apache spark based data pipeline using JSON
This is an offshoot project of open source data quality (osDQ) project https://sourceforge.net/projects/dataquality/ This sub project will create apache spark based data pipeline where JSON based metadata (file) will be used to run data processing , data pipeline , data quality and data preparation and data modeling features for big data. This uses java API of apache spark. It can run in local mode also. Get json example at https://github.com/arrahtech/osdq-spark How to run Unzip the zip file Windows : java -cp .\lib\*;osdq-spark-0.0.1.jar org.arrah.framework.spark.run.TransformRunner -c .\example\samplerun.json Mac UNIX java -cp ./lib/*:./osdq-spark-0.0.1.jar org.arrah.framework.spark.run.TransformRunner -c ./example/samplerun.json For those on windows, you need to have hadoop distribtion unzipped on local drive and HADOOP_HOME set. Also copy winutils.exe from here into HADOOP_HOME\bin -
15
AWS Data Wrangler
Pandas on AWS, easy integration with Athena, Glue, Redshift, etc.
An AWS Professional Service open-source python initiative that extends the power of Pandas library to AWS connecting DataFrames and AWS data-related services. Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON, and EXCEL). Built on top of other open-source projects like Pandas, Apache Arrow and Boto3, it offers abstracted functions to execute usual ETL tasks like load/unload data from Data Lakes, Data Warehouses, and Databases. Convert the column name to be compatible with Amazon Athena and the AWS Glue Catalog. Run a query against AWS CloudWatchLogs Insights and convert the results to Pandas DataFrame. Get QuickSight dashboard ID given a name and fails if there is more than 1 ID associated with this name. List IAM policy assignments in the current Amazon QuickSight account. -
16
AvaSattva
Search replace files or pipe
See https://github.com/qualiu/msr/ Match/Search/Replace: msr.exe/msr-Win32.exe/msr.cygwin/msr.gcc**/msr-i386.gcc** Match/Search/Replace/Execute/* Files/Pipe Lines/Blocks. Filter/Load/Extract/Transform/Stats/* Files/Pipe Lines/Blocks. Not-IN-latter: nin.exe/nin-Win32.exe/nin.cygwin/nin.gcc**/nin-i386.gcc** Get Exclusive/Mutual Line-Set or Key-Set; Remove Line-Set or Key-Set matched in latter file/pipe; Get Unique/Mutual/Distribution/Stats/* Files/Pipe Line-Set or Key-Set. Match/Search/Replace files/pipe text with plain/Regex syntax. And for ETL alike work like Load and filter files -> Extract -> Transform output. For replacing files, you can preview and backup, in multiple directories and files or pipe, with plain text matching or using general Regex as C++, C#, Java, Scala; So msr is a good tool to learn and test Regex since it has different colors for matched groups captured by the Regex pattern. -
17
The BEE Project is a suite of tools supporting Business Intelligence project implementation including ETL tool and OLAP server and a thin client. The ROLAP server ensures multipass SQL generation and powerful cache management (utilizes MySQL RDBMS).
-
18
BIAutomationTool
Tool created to aggregate commands to disparate ETL tools
This project was created to allow executing ETL jobs/tasks from a single command line tool with the same syntax, no matter what tool you were executing in. As long as you have a command line client for the ETL tool, you can configure the BIAutomationTool to use it. -
19
Parse, analyze and -- most importantly -- use COBOL data definitions. This gives you access to COBOL data from Python programs. Write data analyzers, one-time data conversion utilities and Python programs that are part of COBOL systems. Really.
-
20
Civi Data Integration
This is a Pentaho Data Integration plugin for CiviCRM.
This is a Pentaho Data Integration plugin for CiviCRM. It allows you to take advantage of the power of Pentaho Data Integration tools and use it with your CiviCRM instance. -
21
DataStreamProcessor
TPL (transfer,parse,load) tool for batch files.
This is an enterprise-strength system for batch file processing, e.g. transfer, parse and load data using batch (text) files within and without the enterprise. The system is controlled through parameters and doesn't require any programming, code generation or code deployment. This is a heavy duty back end system with no GUI. Nonetheless it's very easy to use, easier than most GUI-based ETLs, even easier to install. It currently supports 4 major dbs: Oracle, Sybase, MySQL, MSSQL. The free community edition allows to process about 10 files a day depending on the setup. For support and licensing go to the www.datastreamprocessor.com -
22
Drop is a graphical interface for the GPL project Wets (hosted in SourceForge). Wets is an ETL software.
-
23
-
24
-
25
EplSite ETL
ETL Based on Perl With WEB Interface
EplSite ETL is a tool to do easy the data migrations, doing extraction, transformation, validation and load in a very fast way. It was built by people involved in data migrations so, it contains the necessary to do the migration(Extract Transformation, validation and load) and do it well.
