Open Source Java ETL Tools

Pentaho Community Edition can now be downloaded from https://www.hitachivantara.com/en-us/products/pentaho-platform/data-integration-analytics/pentaho-community-edition.html Join the Community at https://community.hitachivantara.com/communities/community-pentaho-home?CommunityKey=e0eaa1d8-5ecc-4721-a6a7-75d4e890ee0 Pentaho couples data integration with business analytics in a modern platform to easily access, visualize and explore data that impacts business results. Use it as a full suite or as individual components that are accessible on-premise, in the cloud, or on-the-go (mobile). Pentaho Kettle enables IT and developers to access and integrate data from any source and deliver it to your applications all from within an intuitive and easy to use graphical tool. The Pentaho Enterprise Edition Trialware can be obtained from https://www.hitachivantara.com/en-us/products/lumada-dataops/data-integration-analytics/download-pentaho.html

Logstash is a server-side data processing pipeline that dynamically ingests data from numerous sources, transforms it, and ships it to your favorite “stash” regardless of format or complexity. It supports and ingests data of all shapes, sizes and sources, dynamically transforms and prepares this data, and transports it to the output of your choice. Logstash is extensible, with over 200 plugins available to let you create and configure your pipeline how you choose.

Please, visit www.cloverdx.com for latest product versions. Data integration platform; can be used to transform/map/manipulate data in batch and near-realtime modes. Suppors various input/output formats (CSV,FIXLEN,Excel,XML,JSON,Parquet, Avro,EDI/X12,HL7,COBOL,LOTUS, etc.). Connects to RDBMS/JMS/Kafka/SOAP/Rest/LDAP/S3/HTTP/FTP/ZIP/TAR. CloverDX offers 100+ specialized components which can be further extended by creation of "macros" - subgraphs - and libraries, shareable with 3rd parties. Simple data manipulation jobs can be created visually. More complex business logic can be implemented using Clover's domain-specific-language CTL, in Java or languages like Python or JavaScript. Through its DataServices functionality, it allows to quickly turn data pipelines into REST API endpoints. The platform allows to easily scale your data job across multiple cores or nodes/machines. Supports Docker/Kubernetes deployments and offers AWS/Azure images in their respective marketplace

P.S. Dear friends. Repository migration to https://github.com/ascrus/getl . You can download jar file from this site or maven. GETL - based package in Groovy, which automates the work of loading and transforming data. His name is an acronym for «Groovy ETL». GETL is a set of libraries of pre-built classes and objects that can be used to solve problems unpacking, transform and load data into programs written in Groovy, or Java, as well as from any software that supports the work with Java classes. GETL taken into account when developing ideas and following requirements: 1. The simpler the class hierarchy, the easier solution; 2. The data structures tend to change over time, or not be known in advance, working with them must be maintained; 3. All routine work ETL should be automated wherever possible; 4. Compiling the code on the fly bail speed and reserve for the optimization; 5. Sophisticated class hierarchy guarantee easy connection of other open source solutions.

Talend Spatial Module (aka Spatial Data Integrator or SDI) is an ETL tool for geospatial. Based on Talend Open Studio, input, output and transform geocomponents are available. IO components read/write GIS formats(eg.PostGIS, GeoRSS). Transformers all

A Java utility that uses the Informatica Operations API allowing parameter inputs, trapping of suspended workflows and ability to send an email on failure. This utility extends the functionality of the pmcmd startworkflow and starttask command. If you pass in a parameter file and individual parameters on the command line, a temporary parameter file is created that has the values from the parameter file and appends the individual parameters. The e-mail sent is in HTML format using tables and looks similar to the workflow monitor output.

GeoKettle is a powerful, metadata-driven spatial ETL (Extract, Transform and Load) tool dedicated to the integration of different data sources for building and updating geospatial databases, data warehouses and services.

Dbmetadata is a Java utility that reads the metadata from table(s) in a specified database and creates the Informatica XML to import into the repository. I created this utility when we were migrating to a new platform and needed a quick way to create flatfile and relational sources and targets that matched the DDL of the table. I also needed to use shortcuts. If you use the import table list, it will create one XML file with all of the tables and shortcuts (if a shortcut folder is specified) for the requested output type and database/file type.

ETL Framework is a standalone Extract Transform Load engine written in Java. It includes executables for all major platforms and can be easily integrated into other applications. Key Features: * embeddable, open source and free * fast and scalable * uses target database features to do transformations and loads * manual and automatic data mapping * data streaming * bulk data loads * data quality features using SQL, JavaScript? and regex * data transformations Requirements * Java 1.6 and up * At least 4 MB of RAM New in 3.2 (01/18/2013) * Improved auto-update functionality * Bug fixes

Scriptella is an open source ETL (Extract-Transform-Load) and script execution tool written in Java. Its primary focus is simplicity.

Createinfactl is a Java utility that enables Administrators to fully automate Informatica deployments from the command line by creating thedeployment group control XML file to be used with the pmrep command “deploydeploymentgroup”. Default settings for the control file can be overridden at the command line and works with both static and dynamic deployment groups in the repository. Please review the “Using the Deployment Control File” section in the Informatica Command Reference guide for further help on the deployment control file. This utility supports Informatica PowerCenter versions 8.6.1 onwards and Java 1.6\. This utility also contains JDBC drivers for DB2, SQL Server, Oracle, and Teradata databases.

XIForge is a team of IT volunteer to explore new free open source technology framework and platform. We focus Pentaho and OpenBravo ERP. Our current hosted project includes Pentaho Data Integration Parse JSON String plugin. Team founder is Reid Lai.

iLastic is an open-source framework to query, integrate and manipulate any type of data in English. Extract, transform and merge information from the web, databases, files or any other data repository using a language you already know... English

Utility that performs bulk user import to Active Directory from selected data sources. It can perform data mapping and generate required fields using existing info( generate userPrincipleName from name, surname and patronymic of user for example). This is still a beta-release, so things can work not so well sometimes.

A collection of Java programs using JavaMail API to help automating the process of sending mails.

Automatic Report Generator is a mini-ETL API which allows to retrieve data through an SQL query towards a structured file, would it be a CSV, XLSX, or XML. This API also supports BIRT reports, and, in this case, its respective template is required. The project is available on Maven: https://mvnrepository.com/artifact/net.sf.automatic-report-generator Version 3: https://mvnrepository.com/artifact/net.sf.ennahdi.automatic-report-generator

This is a Pentaho Data Integration plugin for CiviCRM. It allows you to take advantage of the power of Pentaho Data Integration tools and use it with your CiviCRM instance.

"ETL Adaptor" is a java-based ETL tool that uses the OpenAdaptor framework to address the integration tasks. It realibly accomplishes the ETL process of data warehouses and/or data marts of small and medium size.

A simple Java ASN BER decoder and profiler A tool for easy manipulation of BER encoded files. An "awk" for ASN.1 BER (for Unix people) or maybe a "notepad" for ASN.1 BER (for Windows people). Jberd (Java BER decoder) is a lightweight BER decoder and associated tools for interpreting and processing BER encoded ASN.1 files. The following facilities are provided: • JBerd Profiler. A tool for profiling the contents of BER encoded files • JBerd Flattener. A tool for converting BER encoded files to flat files for processing by other facilities • JBerd Decoder objects. A set of Java facilities for writing BER applications that require BER decoding Go to the "files" section (link at the top of this page) to download a pdf of detailed documentation. Andrew Forsyth

Jipes provides open source Java APIs deeply integrated into the Oracle RDBMS, including an Ant task for building and exporting database objects. A Java Data Cartridge replacing database links is also in process.

KETL(tm) is a production ready ETL platform. The engine is built upon an open, multi-threaded, XML-based architecture. KETL's is designed to assist in the development and deployment of data integration efforts which require ETL and scheduling

Metl is a simple, web-based ETL tool that allows for data integrations including database, files, messaging, and web services. Supports RDBMS, SOAP, HTTP, FTP, SFTP, XML, FIXLEN, CSV, JSON, ZIP, and more. Metl implements scheduled integration tasks without the need for custom coding or heavy infrastructure. It can be deployed in the cloud or in an internal data center, and it was built to allow developers to extend it with custom components.

Simple and easy ETL tool useful for small data warehouse projects. Written in Java.