Search Results for "etl."

...Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON, and EXCEL). Built on top of other open-source projects like Pandas, Apache Arrow and Boto3, it offers abstracted functions to execute usual ETL tasks like load/unload data from Data Lakes, Data Warehouses, and Databases. Convert the column name to be compatible with Amazon Athena and the AWS Glue Catalog. Run a query against AWS CloudWatchLogs Insights and convert the results to Pandas DataFrame. Get QuickSight dashboard ID given a name and fails if there is more than 1 ID associated with this name. ...

...The parser handles standard CSV semantics including quoted fields, variable delimiters, escape sequences, and optional headers; this makes it robust for a variety of CSV dialects you might encounter. Because it works incrementally (row by row), it is well suited for ETL pipelines, data ingestion workflows, CSV-to-database imports, or any context where you need to process or transform large tabular data in Node.js efficiently. Using the .on('data') / .on('end') (or equivalent async patterns), you can accumulate, filter, transform, or stream data further downstream without waiting for the whole file.

Fast parallel async HTTP/SSH/TCP/UDP/Ping client java library on Akka. Aggregate 100,000 APIs & send results anywhere in 20 lines of code. View production use cases. Ping or HTTP calls 8000 servers with responses aggregated in 12 seconds. Parallec means Parallel Client (pronounced as "para-like"). Open Source from eBay Cloud. A convenient response context passes any object you need when handling a response. Process data anyway and send it anywhere. Intuitive builder pattern APIs make...

webStraktor is a programmable World Wide Web data extraction client. Its purpose is to scrape HTML based content via the HTTP protocol and extract relevant information. webStraktor features a scripting language to facilitate the collection, the extraction and the storage of information available on the web, including images. The scripting language uses elements of the Regular Expression and xPath syntax. The webStraktor scripting language has a small instruction set and its syntax is easy...

TopGun Twitter Analytics is an open source data warehouse for collecting and analyzing Twitter topics. A topic is made up of one or more keywords.

Concatenate successive lines within a text file, with an option to skip a number of subsequent line(s), and an option to insert a character or string between lines. Useful for turning multi-line log files into single line files (think CSV!)

It's a Java based Extract Transform Load(ETL) tool with following features -- 1. It can take data from any source to any destination, any thing you can think of - for example from a web crawler to a database or filesystem 2. It's multithreaded and

Pypes is a framework which allows users to break complex data processing logic down into a series of smaller less complex tasks. These tasks, referred to as components, can then be connected so that the output of one becomes the input to another.

A command line utility to read a text file containing lines of data, clean up any CR/LF anomalies, and output the lines of text with clean CR/LF terminators to standard output. The binary is a Windows 32 bit console app.

Comprehensive & high performance data distribution &transformation system. Features a simple, user-friendly event driven scripting interface transparently generates & execs highly efficient Perl/C code. Uses:ETL,datawarehousing,statistics,data-cleansing.

Wets is a very flexible ETL which can handle data coming from text file / XML / MySQL / postgreSQL and Zope server and because of the language associated there is no limitation concerning the transformation process. See the drop project for a GUI.

Document Management System created using JEE6